3.0 beta 2

Large dataset makes the grid really slow

Hi! I am trying the xml - simple.htm example with a large dataset(500+ rows). I notice some performance degradation and increased memory consumption. Is a some sort of a virtual mode available for this grid ?
Anything I can do to optimize for the large dataset ?

Thank you for your help
Mike
April 7,
Hey I was buying the library for commercial use, but If this is true, I need it for large datasets visualization (like 100+ rows and 20 or more columns).

Any thought about the slowdown ?
Maybe latest 1.0 version fixed this leeks ?

Regards
April 15,
I use a 200 row limit and have not had a problem. I limit the data that comes back to 200 rows. Takes under 4 seconds to load (including download data on a local intranet) and about 3 seconds to sort.

I give users the tools to filter their searches and figure if they max out the number of rows then that they can learn to filter better.

I don't think you will find another component that does what this does.
Ken Gregg
April 18,
I am planning to add 'virtual' loading at some point, but the current version does not have this feature. If your dataset exceeds 100 rows you have to implement some sort of paging otherwise the data loading and sorting will be quite slow (approx. 0.5-1 sec per 1000 cells). There is a simple paging example in /patches/paging1.htm which might help to start with paging implementation.

The memory leaks is a different issue. Every effort had been taken to eliminate possibility of memory leaks in ActiveWidgets library. So far I am not aware of any memory leaks in the current version. If anyone sees memory leaks related to ActiveWidgets code - please let me know.
Alex (ActiveWidgets)
April 18,
I use the paging - and right now am working with 30 columns, and about 700 rows

It takes maybe a second on my system right now to sort, which is fine with me (considering I have more things open than NASA right now)
Zach
July 2,
I think javascript native array sorting algorthm is very efficient. It usually takes more time to repaint (subset of) rows than sort them.
Alex (ActiveWidgets)
July 5,
I agree. I don't use Alex's grid but I have a 300+ row table which is sortable on primary, secondary and tertiary colums. It was having speed issues and investigation showed that the sort was near instantaneous while the rendering took forever.

Those timings are approximate. ;-)
Fergus
July 6,
If I have a data of say, 1000 rows and if I want to display 50 rows at a time, what would be the method to call which takes from and to values as parameters and prints the rows depending on the parameters?

Thanks in Advance.

Seen
July 12,
I'm evaluating the product and have data requirements of 2 to 8 columns and between 20 and 75,000 (!) rows. I'm loading the data from a client-side ActiveX control (no URL for the XML) which caches XML data on the client for ready access. So, the (significant) load time is definitely in these two calls:

awTable.setXML(xmlDomDoc);
-or-
theDiv.innerHTML = awObj;

Alex, This really is the best Web client grid I've seen in my years and I'd love to be able to purchase it for use in my application if I can get it to meet my needs. Do you have any suggestions for how to use your AW Grid with data requirements of such a large size?

Any help would be greatly appreciated...
Pete
August 11,
Here is a trick to help speed things up when you have a lot of rows. First, before displaying the grid, set the rowCount to 10. Display the grid then change the rowCount to the actual number of rows and do a refresh(). You are going to find that it SIGNIFICANTLY speeds up load/display time of the grid. I use a window.setTimeout to trigger a function to do the dirty work. I set the timeout to 500 ms and it seems to work great for us.
Jim Hunter
August 11,
Jim, would you be so kind as to post some code to this? It would help if I had an example (with and without your "trick") so that I can fully investigate the impact of this.

Would be great if what you say is true.
AcidRaZor
August 12,
The trick which Jim has described should work fine. What it does - it first renders the grid with a small subset of data (top 10 rows), so the user sees the result without delay. Then it renders the grid again with full data set which would take as much time as normally, but this is invisible for the user because the top rows are already on screen. The downside is that browser is still locked during the long second rendering.

I would say that the only real solution for large datasets is paging - but this trick can significantly improve 'perception of speed'.
Alex (ActiveWidgets)
August 12,
Alex,

If you do some testing you are actually going to find that the total time to load the page goes down using my trick. We brought a 20 second total load time down to about 7 seconds. It surprised us, but we did many tests and they all bore similar results.
Jim Hunter
August 12,
The paging patch is the way to go, I must agree. The only problem with the patch is when one select a colomn for sorting, the page label is not updated.
Neil Craig
August 17,
Guys,
I kinda dissagree with paging as the ultimate solution to the strategy. The two best ways to truly solve the problem are:

1) Virtual Mode: Don't render rows until they come into view. Look at the stats, it take 95% of the time to render the rows, and 5% to load and sort the data. Domapi (www.domapi.com) v4 has a listgrid that is very mature and supports this functionality. A quick test of Domapi's listgrid with Virtual Mode On vs. Off shows a DRAMATIC improvement speedwise from only rendering rows in view.

2) Seemless Paging: Use AJAX calls for data everytime the grid is loaded,scrolled or sorted. ive seen random implementations of this in custom grid components online, and its surprisingly effective, and enables the grid to work with infinite size data sets .

#2 Is complex to implement, but virtual mode is very implementable. Alex if you want some ideas how to do it, i recommend taking a look at the domapi source.
lee
August 27,
Guys, this widget is great for small recordsets, I large recordsets, it's better to use simple html tables, manipulating the DOM.
I get a 2000 rows 20 cols, render in les than 1 sec. suing simple DOM techniques.
blow
November 18,

This topic is archived.

See also:


Back to support forum

Search

Version 2

Looking for AW 2.6 ?
The old site is here.