I have a very annoying problem when i try to merge two tables on power query excel. I use one column to match records from both tables and when i try to expand the second table it pops up the following message:
DataFormat.Error: We couldn't convert to Number.
Λεπτομέρειες:
ECS
I have no idea how to fix this. The columns that are matched have text, not value. There are no errors when i import data. Is there anyone that can help?
Try the following:
Delete the step #"Changed Type" in both queries
Make sure that the two merged columns have the same type (text ABC, in your case)
When you create a query from a table, Power Query try to guess (based on the first 200 lines) the type of each column. Now, the value "Λεπτομέρειες: ECS" is probably included in a different column (than the two merged) that has Number 123 as a type. It's kind of a mixed column (due to the source of data itself or to a delimiter issue).
Related
I'm working on achieving the following data transformation/wrangling within Power Query but can't seem to get there on my own. i have read a lof of different questions and answers on the forum but it seems just a bit beyond my grasp.
I have a table which has the ticker of a specific currency in the first column.
There is a second column with the date and time when a certain event, related to that specific currency, happens. This second column is basically the different 5-minute intervals which exist on any given day.
Finally there is a third column which describes the magnitude of the event.
The table therefore looks like this
What I would like to do in power Query is transpose the uniques name of the currencies as the first row of a new table. The first column of this table would be the largest time interval for any given currency. In this case, as you can see in the data I am attaching, the largest timeseries would be that of the currency ETH. Using the longest calendar as our first column I would then like to place the values described in item 3 above as rows in the new table.
The new layout would look like this
My steps to transform the raw data in the first table are detailed in this image. Basically just expanding a JSON file and getting all the data I need into that first format which I described previously.
What I then do is:
Pivot using the first column
Transpose
That gives me a whole bunch of new columns. Way more than I want. Any idea what I can do differently?
In powerquery,
click select pair column
Transform .. pivot column .. values column: basis advanced options: do not aggregate
code:
#"Pivoted Column6" = Table.Pivot(YourPriorStepName, List.Distinct(Source[pair]), "pair", "basis", List.Sum)
output:
I have a table with several column having the same name.
This columns is updated and provided regularly to power bi.
The columns has several columns with the same name such as "Result", "Result" and do so on...
However, Power BI adds each time in an automated way a number after my columns.
When I try to "force" power bi not to have a number, I get the following message
"The name "Result" is already used for a column..."
How could I change this?
The only way would be for the people using my file to extract the data and correct the name manually in excel...which is not great
You cannot edit this behavior, PowerBI needs an unique identifier to reference the data, therefore the column name must be unique within a table (the complete identifier is given by table + column), otherwise the tool won't be able to reference the data.
This rule usually applies to any tool that manages data and sometimes to the data themselves (that's up to the format though). How can the tool get data from "Result" if more than one column has this identifier? which is the right one? The tool does not know and based on the context can give you an error or will fix this issue itself by making the names unique.
Note that also excel will append numbers to the columns (with the same name names) if you put the data in a proper table (insert-> table), in fact, an excel sheet can be considered unstructured free data, meanwhile, an excel table will enforce the data structure.
Most tools (like PowerBI) will also enforce data types.
I have mismatched data lines in a power query so I am attempting to renumber/reorganize the data then merge the information to realign.
Here, I want the data in Column Answer 2 to go into column Answer, cells 6,7,11,12.
I've indexed each of my files and merged the queries. However, when I expand the merged queries, PQ seems to randomize my data.
I'm new to PQ so I don't really write the 'code', just use the user interface.
As you can see from the second image, the data comes out in the wrong order.
I merged two tables, then added index column and moved it to the beginning, then expanded the merged table and deleted index column. The order of rows has left the same, as in the source table.
I have a VBA script that generates a query string for a SAP HANA ODBC Connection in Excel. The query is determined by user inputs and can vary greatly in length. The query itself uses many versions of a similar query appended to one another using UNION ALL syntax.
The script sometimes throws a runtime error when trying to refresh. From my research, it has become clear that the reason for this is that the CommandText string exceeds a maximum allowed length of 32,767 (https://ask.sqlservercentral.com/questions/50819/too-long-sql-in-excel-vba.html).
I wondered whether there is a workaround for this, other than using a stored procedure (I am not against this if there is a way to create a stored procedure at runtime then execute it, but I cannot use a predefined stored procedure as my query is always different hence the need for VBA to create it)
Some more info about the dynamic query in VBA:
Column names, as well as parameters, are created dynamically and can be different every time
The query uses groups of lists of product numbers to generate an IN statement for each product group, then sums the sales for those products under the name of the group. These are then all UNION'd together to create one table with grouped records
Example of user input:
Example of resulting query:
WITH SOME_CTE (SOME_FIELDS) AS
(SELECT SOME_STUFF
FROM SOME_TABLE
WHERE SOME_STUFF_IS_GOING_ON)
SELECT GEND "Gender", 'Attribute 1' "Attribute", SUM(UNITS) "Units", SUM(VAL) "Value", SUM(MARGIN) "Margin"
FROM SOME_CTE
WHERE PRODUCT IN ('12345', '23456', '34567', '45678')
GROUP BY GEND
UNION ALL
SELECT GEND, 'Attribute 2' ATTR_NAME, SUM(UNITS), SUM(VAL), SUM(MARGIN)
FROM SOME_CTE
WHERE PRODUCT IN ('01234', '02345', '03456', '03567')
GROUP BY GEND
ORDER BY "Gender", "Attribute"
...and so on.
As you can see, with 2 attribute groups containing 4 products each there is no problem, but when we get to about 30 with several hundred each, it could be too long.
Note: I have tried things like shortening field references in the repeated parts of the query string to 1 character etc. which helps but does not solve the problem.
Any help would be greatly appreciated.
One workaround is to send multiple queries. Since you are using union all, you could execute every time single select statement, i.e.
create table in (for example) master database (don't create temporary tables! as they will be dropped after every query) - but before that, make sure you create new table, so delete old one if exists (also drop the table after you are done with it). Now every single select statement you'll change to insert statement, which will insert records to your so-called temporary table.
This way, you'll avoid lengthy queries, you'll just send single insert .. into.. select statements.
At the end, to get all results, you just need simple select query. After getting this data, you should drop that table, as it's no longer needed.
I am working with Excel 2010, Power Query, and PowerPivot.
I have a query named Database that consists of 60+ merged tables containing a total of 2m+ rows. I also have a separate query that consists of two columns PrimaryKey3 and Members (a count of members per month). The entries in PrimaryKey3 are unique, consisting of ID-MMM-YY.
Both queries have PrimaryKey3 in common, however in Database there can be multiple rows with the same PrimaryKey3.
In order to match a member amount to each row in Database, I tried a Left Outer join. There were no errors, but when I try to upload to PowerPivot it says there are only 169K rows. I then tried Full Outer join and Inner Join, and received an error "could not convert value to number," coming from a column already formatted as a text in Database. This column contains numbers and numbers proceeding with a letter: 1234, A234. Every non-blank row has a PrimaryKey3. Why is it trying to reformat my columns/ how do I get around that?
Should I be using a different type of join, or is there another way besides merging to do this?
Hope this makes sense, thank you for any help in advance!
I uploaded both queries to PowerPivot, and created a relationship through PrimaryKey3. I then created a new column in Database with =Related(Enrollment[Members]).