I have been using JavaFx's tableview to display a huge amount of data from the database. The table is like 150+ columns and millions of rows. I can handle the rows by getting the data in chunks and implementing paginations(link). But the no of columns is also an area of concern. It takes a lot of time to paint the data and does not update the data when I change the values in the model(ref). I dont have a predefined data structure and so the method at this is not useful. I moved on to use JTable inside JavaFx but JFX2.0 has removed the support for Swing components inside a JavaFX scene. So being left with tableview is there some way to add pagination to columns and rows both without overutlizing the memory ?
Related
I have a fairly large excel Table (30x5000) that houses entries for all different types of equipment. I would like to be able to create custom views that only show the specific data for a given equipment type.
For example, if I were to select "Pumps" for custom views it would show meaningful description columns like Flow Rate, Capacity, etc. and hide those columns that apply to other equipment. Ideally this could be done on the table and not a separate tab as this would also make entry of these equipment items vastly simpler as right now I am scrolling through non-related columns when entering in new equipment.
My problem is that Custom Views don't work with excel tables. Any workaround I have found requires formulas that would separate the data for viewing purposes - I need to be able to edit in new views.
Any solutions to this would be GREATLY appreciated as I am truthfully stumped.
I am creating a table using Tabulator, which seems great and very powerful.
I want a way to save relevant data of the table so it can e recreated on the fly.
Currently, I think there are a few things I need...
The row data - I get this using table.getData();
The columns - I get this using table.getColumnDefinitions();
The row data seems perfect I can store that and use it. However, the column information I am saving doesnt appear to have the size of the columns if I have resized them?
Is there a way of getting ALL the relevant column info, so I can save and recreate it exactly?
Alternatively, if there's a single 1 function that saves everything (row data, columns (including order, size etc)) in one go as a JSON or something that may be handy
So you have a few options here.
Config Persistence
If you simply want the table to be the same way it was the last time the user used it on that computer, you could look at using the Peristent Configuration module. This will store a copy of the table column configuration on the browsers local storage so that next time they load the page it will be laid out the same.
Column Layout
If you want to store it externally then you are correct,
the column width is not updated in the definition after a user changes it.
If you want to get the current layout of the columns then you can use the getColumnLayout function to retrieve the current layout of columns:
var columnLayout = table.getColumnLayout();
Though this will only contain the key layout characteristics and not the full definition, you would need to merge them if you wanted to store them in one place.
More details on this method can be found in the Manual Column Layout Documentation
I have a table with LOTS of columns, however, to separate these columns into smaller tables would increase the manual processes.
What I would like to achieve is to have a table that is linked to 3 other views of the same data.
I created this on Coda however with the sort of numbers we are using it was too laggy and the company is already familiar with excel.
But to show the functionality, here is a link to view;
https://coda.io/d/LAP-Sheets_dGgLPS7V1d8/_sudZF
Basically, I have the first table, which shows every column.
Then, there are 'on market', 'referencing' and 'rent collection', which are all linked views of the first table, just with a lot of hidden columns so it is more manageable.
When you update one of the tables, it updates on the other 3, and if you press the button to change the status, it moves it to the next table.
This is perfect - except moving it across to excel is proving very difficult.
I have built out the first table - but I don't know how to achieve the different views of it.
Thanks.
I have Table A prompted on Year/Month and Table B. Table B also has a Year/Month column. Table A is the default data table (gets pulled in first). I have set up a relationship between Table A and B on the common Year/Month column.
The goal is to get Table B to only pull through data where the Year/Month matches the Year/Month on Table A (what the user entered). The purpose is to keep the user from entering the Year/Month multiple times.
The issue is Table B contains almost 35 million records. What I do not want to do is have Spotfire pull across all 35 Million records. What is currently happening is Spotfire is pulling all those records, then by setting filtering to include Filtered Rows Only on Table B, I am limiting what is seen in the visualization to under 200,000 rows. I would much rather just pull across 200,000 rows to start with.
The question: Is there a way to force Spotfire to filter the data table (Table B) by another data table (Table A) as it pulls the data table (Table B) across, thus only pulling a small number of records into memory?
I'm writing this off the basis that most people utilize information links to get data into Spotfire, especially large data sets where the data is not embedded in the analysis. With that being said, I prefer to handle as much if not all of the joining / filtering / massaging at the data source versus the Spotfire application. Here are my views on the best practices and why.
Tables / Views vs Procedures as Information Links
Most people are familiar with the Table / View structure and get data into Spotfire in one of 2 ways
Create all joins / links in information designer based off data relations defined by the author by selecting individual tables from the data sources avaliable
Create a view (or similar object) at the data source where all joining / data relations are done, thus giving Spotfire a single flat file of data
Personally, option 2 is much easier IF you have access to the data source since the data source is designed to handle this type of work. Spotfire just makes it available but with limited functionality (i.e. complex queries, Intellisense, etc aren't available. No native IDE). What's even better is Stored Procedures IMHO and here is why.
In options 1 and 2 above, if you want to add a column you have to change the view / source code at the data source, or individually add a column in the information designer. This creates dwarfed objects and clutters up your library. For example, when you create an information link there is a folder with all the elements associated with it. If you want to add columns later, you'll have another folder for any columns added, and this gets confusing and hard to manage. If you create a procedure at the data source to return the data you need, and later want to add some columns, you only have to change this at the data source. i.e. change the procedure. Everything else will be inherited by Spotfire... all you have to do is click the "reload data" button in Spotfire. You don't have to change anything in the information designer. Additionally, you can easily add new parameters, set default parameter properties or prompt the user, making this a very efficient method of data retrieval. This is perfect when the data source is an OLTP and not a data-mart/data-warehouse (i.e. the data isn't already aggregated / cleansed) but can also be powerful in data warehouse environments as well.
Ditch the GUI, Edit the SQL
I find managing conditions, parameters, join paths, etc a bit annoying--but that's me. Instead, when possible, I prefer to click "Edit SQL" next to all the elements in my Information Link and alter the SQL there. This will allow database guys to work in an environment which is more familiar.
My apologies if this question is posted elsewhere, but I believe part of my problem is that I'm not familiar with the terminology. Searching becomes rather difficult when you don't know the keywords to use...
I've inherited a PivotTable that pulls data from an SQL database. Three of the columns in this table are "grouped." By this I mean that all duplicate items in these columns are always next to each other. I want to removing these groupings so that sorting the table is easier. Does anybody know how to do this?
EDIT:My work computer has both snipping tool and paint disabled...So here is a picture of something similar. In my table, all of the data is pulled from the database. The left two columns (in this example) have repeated data, so it groups them like so. These sections are collapsible and expandable.
(source: datapigtechnologies.com)
There is an option on the Pivot Table menu Design->Report Layout->Repeat All Item Labels. This will repeat all the labels when combined with a Tabular or Outline style layout. You will also do better (more than likely) to remove the subtotals as well.
Menu and before
After
I had this issue as well. Remove ALL filtering so you ONLY have those two columns at once. Once all filtering is removed it will allow you to ungroup them and the extra column will go away.