My PyQt application pulls data from third party API calls. The dataset returned usually contains in the neighborhood of hundreds of items. On occasion, the dataset returned contains in the tens of thousands of items. On those occasions, the interface is painfully slow to display - too slow to be useful.
To speed things up, I would like to load less of the data during the initial load. I would like to be able to populate the interface based on the scrollbar handle position. I would prefer that the scrollbar have the correct range as soon as the widget is displayed, but as the user scrolls, the data that they should be seeing is populated into the widget (a QTreeWidget in this case). This is to say that I'd rather the user didn't have to scroll to the bottom of the widget to load more data at the bottom & therefore change the range of the scroll bar.
I believe QSqlTable behaves this way out of the box, but because I'm not relying on sql queries (and because some of the columns' data is calculated by the GUI), I don't believe I can use that module. Is this possible with QTreeWidget and w/o direct sql calls?
There is built-in functionality for this in Qt model-view framework. See QAbstractItemModel.canFetchMore() and QAbstractItemModel.fetchMore() here
Oh, I've just realised you aren't using MVF but stand-alone QTreeWidget instead. If you are dealing with large data and require such a functionality, a switch to MVF may be a right thing to do.
Related
I'm writing a GUI in python, and using tkinter. I'm having trouble settling on an approach and need guidance.
Background: there's a server (not a webserver) that wants to present a lot of information to users, and let them edit some of it. It needs to send down information that a (relatvely) dumb python client uses to fill the window. Read only fields are Labels. The fields are generally single line Entry widgets, but some are multiline Text. There are some buttons, checkboxes and dropdowns. Asynchronously, the server can also update widgets, add them and remove them. In some cases, there are tables presented, to which the user needs to be able to add and remove rows.
The real problem is, the layout is dense and chaotic. The first row might contain 3 dropdown fields. The next might be 20 short Labels. The next might be a single long Entry field, and then I might want two tables (of different lengths) side by side,and then etc.. Based on user input of external factors, widgets, rows or entire tables might have to be dyamically added, or vanish.
I considered Grid, but it's unusable. A row with a single, long entry widgit in it, makes the first column wide and thereby pushes 12 of the 13 columns in the next row right off the window.
I considered Place, but this app will run on 3 different operating systems and users will be able to select their own fonts, so I'll never get the positions right. If there was some way to ask a widget how big it was, I'd happily use that to compute my own layouts in pixels, but it's apparently impossible to ask the size of a widget until AFTER it's been laid out by a geometry manager, which of course is too late.
So what I think I'm left with is Pack, where each row is its own frame, and some of those rows have tables (grids) in them. But I'm concerned that that means lots and lots of frames to render, and some of the users are on old, slow hardware. Plus... it looks just plain complex.
Am I missing a better way? Grid would be fine if I could convince it to stop trying to make columns line up. Place would be crunchy, but ok, if I could get the size of each widget in advance. Is placing within a lot of frames really the best I have?
Short answer, there's no better way; and the frame count isn't high enough to cause performance problems; so generating a frame per row is what works.
I have a MultiGrid component with a single fixed row. I would like to print the result, but since multiple columns overflow on the x-axis, the print output gets truncated.
Is it possible to wrap each row in another element and then use display table-cell/table-row to get the desired, table-like behavior? The added benefit is that a table can easily stretch the entire page, even if the number of columns is low.
Is it possible to wrap each row in another element
Yes. It would be possible to wrap rows by injecting your own cellRangeRenderer property. That being said, I don't really recommend it. RV doesn't really have "rows" or "columns"- just positioned cells. Wrapping would add extra elements which could slow down scroll performance. (Probably not much, but every little bit counts for scrolling.)
and then use display table-cell/table-row to get the desired,
table-like behavior?
If you're using MultiGrid I assume you have enough columns to warrant windowing horizontal data as well as vertical? In which case, I don't think display:table would really work for you. I'm not sure what it would buy you.
Have you considered just rendering a non-RV layout specifically printing? I Haven't done this myself but maybe you could tap into beforeprint/afterprint and setState to render a different result? Alternately you could try setting overflowColumnCount really high when print mode is enabled to just render the entire horizontal axis.
I don't really have much experience with this unfortunately. It might require a little of experimentation on your part for the best performing solution. :)
Is it possible to use React Virtualized when there is no notion of a row index to get rowdata?
I would like to use React Virtualized to display data coming from a large (100k+ rows) database-table that is constantly being modified: rows are added/deleted/updated at random positions in the table.
I have no function that can get a row by using a row index because the position of every row is changing every few seconds.
The table is sorted and every row is guaranteed to have a unique content, so what I do have are the following functions:
getFirst/LastRow() => data : get the data content for the (currently) first/last row
getNext/PreviousRows(startData, nrRows) => data[] : get the data content for the (currently) next/previous nrRows, starting at row with content startData
findRow(data) => data : find the row that has content data
I also have an observer function that is tracking the table mutations in real-time, so I can get a callback for every insert/delete/update operation for the table.
Is there a way to map these available functions to a workable React Virtualized configuration ?
Is it possible to use React Virtualized when there is no notion of a row index to get rowdata?
No. Mapping an index (or indices) to data is core to how react-virtualized works. You would need to build/maintain some structure that allowed you to efficiently access data at an index (even if that index frequently changed) in order to benefit from the lib.
When the data changes, every few seconds, is it usually items being appended onto one end of the remote collection? Or could it be resorting, deletions, etc?
The linked-list style API you describe almost seems like it was meant more to work with a pagination control.
I think the answer as asked might be "Yes" if the semantics of
to use React Virtualized
can be understood to mean where RV is "a piece of the puzzle". I think this was essentially stated very well and accurately with the opposite answer in terms of RV itself having
no notion of a row index
RV does need to know about its own row indexes. But if RV is implemented, and simply handed data at whatever event interval (i.e. pagination) from a "parent"... it is possible to use RV w/o it having an understanding of the parent's row index being used
to get rowdata
You could think of your paginated app as the parent. The dataGrid component as the child. The state/render cycle in your parent React app just needs to re-render the dataGrid (and it's scroller) at the spot the user will find intuitive in relation to your data "pages".
I've been modeling a similar solution with a different "dataGrid" component, and following various mindshare on the issue. At the risk of [more] tl;dr, my current technique is like this:
feed a total rowcount to the dataGrid component, so it sets up as if I'm going to give it the whole api in one un-paginated shot
feed only the data from the currently available data page (say 500 records)
dataGrid's onScroll event calculates a conditional threshold for me to cross (scrollTop, etc.) indicating that the list has scrolled to the top or bottom row of the current page as my parent application understands it
onScroll calculates rows and pages, gets new data, and sets new state
new state has the same huge rowcount, but along with different data rows from a API call
use should/did update lifecycles in React to:
"scrollTo" and reposition the component grid to whichever row my
parent understands as the top of right "page", and
(warning... prototype) I concatenate my paged data array with padding (cringe) like [...Array({#ofRows*#ofpages})].concat(newData), so that the child component leaves the drag handle where the user expects just before I render. Ultimately, I am hoping this will be done by adjusting the height of the top or bottom-most divs in my list to compensate for the intended scrollbar position, but it's an evolving work in its priorities.
I've also maybe found a macrocosm of the same issues in my own pagination that the windowing libraries solve with over-scanning rows. It seems I must "overscan" in my API calls, so that I have a buffer outside the windowing component's own top/botom. This may well go away with better logic in the end. Fingers crossed.
References I am finding helpful:
Some more history for the discussion using this library is here.
Also, react-window-paginated looks to have solved the basics of the problem your comments describe using the newer react-window lib. This solution is very compelling to me as a code design of where I might end up in my final composition. RW doesn't leave you with RV's feature set, though, but it's easily extendable and maybe easier to start here if you don't need tons of RV features out of the gate.
If you were to tackle the pagination solution in RV, this might stimulate a prototype, though it does not solve the API pagination or mapping/threading issue itself.
I would like to implement a GUI handling a huge number of rows and I need to use GTK in Linux.
I started having a look at GTKTreeView with lists but I don't think that adding millions of lines directly to that widget will help in having a GUI that doesn't slow the application.
Do you know whether there is a GTK widget already in place for this problem or do I have to handle my self the window frame that that must display those lines? Eventually I would write the data directly using GtkDrawingArea (essentially writing a new widget).
Any suggestion about any GTK topic or project I can look as starting point for my research?
As suggested in the comments, you can use the Cell Data Func, and get the displayed data under contro. But I have another idea: Millions of lines are much much more than any amount of information a human user can see and understand. So maybe a better, more usable and user-friendly solution, is to diaplay the data in a way the users can more easily navigate in it.
Imagine opening a huge hierarchy, scrolling down, and forgetting what were the top-level items you opened.
Example for a possible solution: Have a combo box which allows to choose some filter or category, and this can reduce the amount of data to a reasonable amount the user can more easily navigate and make a mental model of it if necessary.
NOTE: As far as I know, GtkTreeView doesn't support sorting/filtering and drag-n-drop at the same time, so if you want to use both features, I suggest you use the existing drag-n-drop functionality (otherwise very complicated to implement by hand) and implement your own sorting/filtering.
I'm dealing with an old Motif application that needs to load and display a long list of entries (around 1500). It creates and manages an instance of xmFormWidgetClass via XtVaCreateManagedWidget() and then it stuffs it with a bunch of linear hierarchies xmFrameWidgetClass->xmFormWidgetClass->xmFormWidgetClass->xmPushButtonWidgetClass. Each PushButton contains a multi-line label. When this this thing is being populated, it takes a lot of CPU, which it spends doing some geometry calculations inside of X/Motif libraries. The pace at which new buttons are added, degrades very quickly. It looks like there is an O(N) algorithm being used inside of XtVaCreateManagedWidget().
The things get much much better if I do XtUnrealizeWidget() on the original instance of the xmFormWidgetClass. Entries are being added at almost constant speed but then I cannot find a way to display the whole thing that I built. XtRealizeWidget() for the original instance of the xmFormWidgetClass does not render it in the window.
What am I doing wrong? Is there a way to populate the hierarchy and then calculate the geometry and render it to the screen at once?
Redesigning the application is an option but it is a last resort type on an option.
Any advice that keeps me within Motif libraries will be highly appreciated!
Regards,
/Sergey
Try calling XtManageChild after XtRealizeWidget.
Try creating all widgets unmanaged and place them on a WidgetList, then call XtManageChildren(). Please see the following reference
http://www.s-and-b.su/syshlp/motif_guide/MotifProgGuide/Making_Widgets_Visible.html
Every time an individual widget is managed the parent changed_managed procedure is called.
XtManageChildren calls the changed_manage procedure only once. This may help.