I want to insert 1 million records in CLIENT side HSQLDB and i want to perform pivoting operation with that one
million records in client side to analyze the business growth in various ways.
Is there HSQLDB having this feature. If so kindly help me.
In general pivoting operations are implemented with aggregate functions together with CASE WHEN and GROUP BY.
HSQLDB has all the features of PostgreSQL and MySQL that are used for operating pivots. You can look at questions with the PIVOT tag on stackoverflow for examples.
I will add the PIVOT tag to this question.
Related
I have some tables in scheme of PostgreSQL. I need to find text in some tables at the same time. I need find the same phrases in different tables and also different phrases in different tables. In the end I need to join this tables and give id from the main table. Which solution is the best? P.S. Tables will update frequently.
I am trying to get a list of tables and columns in a database, so I could find which tables have a particular column, the best I could find is use separate queries like one to show all tables , and then one to show all columns in one table, e.g. SHOW TABLES FROM database_name, SHOW COLUMNS FROM databasename.tablename. It will not be ideal when you have many tables to go through. Any solution out there at all?
Unfortunately, there is no way to fetch all metadata in one call. You can only do show databases, show tables in ..., describe table .... There is also spark.catalog.listTables, etc., but they could be slower than corresponding SQL queries.
I answered to related question yesterday - you can find code there.
Building a BI system (dashboards) in Excel using imported tables (from excel files). We're using Excel 2016 query, data model, measures using DAX expressions, resulting in more pivot tables (some of which are reloaded into data model), etc.
My question: is there "best practice" on 1) naming these data elements and 2) documenting these bits to have a more complete system documentation.
Background: I'm the senior "hacker" munging these things together. But I need to move this towards being sustainable. I did some prototyping work and when I went back a week later it was challenging to reconstruct my thoughts and relationships...
I've seen folks refer to use of PowerBI flow diagrams to support documentation; but it seems to be more of the "icing on the cake" than the "cake" itself.
So what "bread and butter" documentation approaches have you, more experienced developers, taken to ensure that your systems are clearly documented so that others can pick up where you left off???
For naming, I follow the Kimball Group's advice for data warehouses/marts, e.g.
https://www.kimballgroup.com/2014/07/design-tip-168-whats-name/
I rename many/most Query steps to reference the column or table name, e.g. Added Custom => Added Customer Name, Append Queries => Append Customers. The idea is to be able to pick the right step first time when coming back for maintenance.
You can select all the Queries in the Query Editor window and copy their code, then paste it into Word etc as the starting point for your documentation. You can also screen-shot the Query Editor's Query Dependancies pop-up.
For the Power Pivot logic, try this solution:
https://powerpivotpro.com/2014/03/automatically-create-data-dictionary-for-your-power-pivot-model/
The objective I am trying to achieve is to have 2 slicers in PowerPivot, ClientID and CSQName. When a ClientID is selected only the CSQnames that are related to that ClientID show up ,and vice versa
Relationship diagram link: https://goo.gl/photos/PnCZrnsXXTx3oFGh8
I am having a problem linking a many to many relationship in PowerPivot. A brief background on the application I am trying to build...
I am trying to combine a SQL database (IDM) and Informix SQL database (Cisco Call Data). The IDM database includes the Client Data and TBAS Open Case Data. Each Client has a specific ClientID. The Cisco database includes Call Detail Info and CSQNames(queue names). A many to many relationship exists, for example, a clientid can have multiple CSQname (clientid 3 has CSQ names of "A" and "B"). Also a csqname can have multiple clientids (csqname "Z" includes clientids "99", "98" and "97". Therefore I created an innerjoin table to create the many to many relationship called "Clients_CSQ".
I am trying to use this innerjoin table for both the "TBAS Open Cases" and "Call Detail". When I use this table for my filters, PowerPivot is stating that no relationships exist. Are there any solutions? If this does not make sense please let me know and I will try to specify. I have ready many posts but am unable to grasp how to make the DAX many to many relationship work with the calculate function. If someone can shed some light on the issue I am having it would be greatly appreciated. Thank you.
This really depends upon the data you are looking to report on.
When you add two slicers to a PowerPivot table, the available selections in each slicer will be affected by the selection in the other slicer IF and ONLY IF all of the fields in the Values section of the Pivot Table are reliant on the entries in both of the slicer fields.
In your case, it is possible to make this work (as an example) by creating 3 measures:
[Call Total]=SUM('TBAS Open Cases'[Case duration])
[Number of Calls]=COUNTA('Call Detail'[appname])
[Calls by Duration]=SUMX('Clients_CSQ',DIVIDE([Call Total],[Number of Calls]))
Place the last of these 3 measures in a pivot table with the slicers set to use 'Clients_IDM'[ic_client_id] and 'CSQ Name'[csqname] and "Hey Presto!"
The first two measures are straightforward enough. The third one is cycling through each entry in the only table that these two slicer fields have in common (Clients_CSQ) and performing a calculation using the data from your FACT tables. I have no idea if the [Calls by Duration] measure that I've come up with makes any sense with your data set, but hopefully the example will help you reach the solution you want. Again depending on what data you want to show it doesn't really matter if this measure returns junk, the important thing is that it's pulling your two data sets together.
Remember that as soon as you add any raw field from either of the fact tables to this 'unifying pivot table', the inter-relationship between the slicers will break. !!!BUT!!! there is nothing to stop you from linking the csqname slicer to another pivot on the same sheet which contains fields from your Call Detail table and likewise linking the ic_client_id slicer to a pivot that contains TBAS Open Cases data. In fact, the 'unifying pivot table' could be on a different sheet from your slicers, so you only see the two sets of data that you are interested in.
And ignore that warning about no relationships existing!
In powerpivot, Related(Othertable[field]) retrieves the associated column from a related table.
I would like to import ALL such columns, doing the equivalent of a join.
Is it possible to do this ?
nicolas,
the smartest thing to do from my perspective is to merge your queries into one so that you can keep your original tables.
I would suggest using new PowerQuery Merge funcionality, which is very easy and works reliably (and also supports loading data directly into your PowerPivot data model).
Or you can write you custom Query in PowerPivot - if you use MSSQL (or any other) database as your source, you can actually use JOIN directly in the PowerPivot window with Table Import Wizard that makes things a bit easier.
So the answer is: keep your original data tables intact, and create a new one that will be merging them together just for the purpose of your desired report.
Hope this helps.