I've got an Excel 2016 spreadsheet set up with an Oracle db data source and I already have several queries set up in PowerQuery to get data from the tables in a specific schema and all is working well.
I now need to get some data from the data dictionary - I need to find the name of a trigger associated with a specific table in the schema - so I've set up a query to try to get data from user_triggers, but so far I've not been able to get it to work.
This is the query I have set up so far (SourceTableName is a reference to a named cell in the sheet to get the table name) :
let
STN = Excel.CurrentWorkbook(){[Name="SourceTableName"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(STN,{{"Column1", type text}}),
table_name = #"Changed Type"{0}[Column1],
Source = Oracle.Database("MY_DB", [HierarchicalNavigation=true]),
Schema = Source{[Schema="MY_SCHEMA"]}[Data],
USER_TRIGGERS = Schema{[Name="USER_TRIGGERS"]}[Data]
in
USER_TRIGGERS
This works perfectly fine for the other queries I already have set up as long as the table name is one of the tables in the schema, but referring to a data dictionary view as in the above example doesn't seem to work.
The error I get when trying to run the this query is:
Expression.Error: The key didn't match any rows in the table.
Details:
Key=Record
Table=Table
Does anyone know if it's actually possible to get data from the data dictionary using powerquery and if it is what do I need to change to get it to work?
Thanks in advance for any help with this!
Cheers,
Dave
I've figured it out! Answering my own question in case it's useful for anyone else in the future
It's actually possible to specify an SQL query directly in the db connection line and you can include variable names from other parts of the query in the SQL, like so:
let
STN = Excel.CurrentWorkbook(){[Name="SourceTableName"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(STN,{{"Column1", type text}}),
table_name = #"Changed Type"{0}[Column1],
upper_tn = Text.Upper(table_name),
Triggers = Oracle.Database("MY_DB", [HierarchicalNavigation=true, Query="select trigger_name from user_triggers where table_name = '" & upper_tn & "'"])
in
Triggers
Using the SQL query directly in this way seems to work fine for data dictionary views :)
Related
I have a list of elemental impurities in power query which I wish to filter according to whether or not they exist on another list known as the prop65 list.
The screenshot below shows a simplified example of what I am trying to achieve.
I appreciate that using formulas However I don't know how to achieve this using a Power query solution. If anyone know how to achieve this it would be appreciated.
Data shown:
Aluminium 33.885
Antimony 0.6777
Arsenic 3.5064
Barium 2.259
Boron 1.3554
Bromoform 0.555
Cadmium 3.18895
Chromium 0.33885
Cobalt 1.1295
Copper 0.4518
Indium 0.4518
Simplified Prop65 List
Arsenic
Bromoform
Cadmium
Furan
Lead
Nafenopin
Here is one way to do that:
Read in the two tables
Do an Inner Join
let
//get original data
Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
data = Table.TransformColumnTypes(Source,{{"Impurity", type text}, {"Amount (ppm)", type number}}),
//get Filter
Source2 = Excel.CurrentWorkbook(){[Name="Prop65"]}[Content],
filter = Table.TransformColumnTypes(Source2,{"Simplified Prop65 List", Text.Type}),
//Join them
filteredData = Table.Join(data,"Impurity", filter, "Simplified Prop65 List",JoinKind.Inner),
//Remove unneeded column
#"Removed Columns" = Table.RemoveColumns(filteredData,{"Simplified Prop65 List"})
in
#"Removed Columns"
Another method would be a filter (Table.SelectRows) method, but it may be slower with a large dataset. At least, in a single instance where I had an opportunity to compare, the Table.Join method was faster on a 100,000 row data set.
let
//get original data
Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
data = Table.TransformColumnTypes(Source,{{"Impurity", type text}, {"Amount (ppm)", type number}}),
//get Filter
Source2 = Excel.CurrentWorkbook(){[Name="Prop65"]}[Content],
filter = Table.TransformColumnTypes(Source2,{"Simplified Prop65 List", Text.Type})[#"Simplified Prop65 List"],
//filter the rows
filteredData = Table.SelectRows(data, each List.Contains(filter,[Impurity]))
in
filteredData
I'm new to node-postgres and trying to pass in parameters connected to a table alias, but it's not working. How can I parameterize the simple query below? Take something like
const pgQuery = 'SELECT t1.YEAR as year, t1.CODE as code FROM data t1;';
and instead do something like
const pgQuery = 'SELECT t1.$1 as year, t1.$2 as code FROM data t1;';
const values = ['YEAR', 'CODE'];
Per the docs here:
https://node-postgres.com/features/queries
PostgreSQL does not support parameters for identifiers. If you need to have dynamic database, schema, table, or column names (e.g. in DDL statements) use pg-format package for handling escaping these values to ensure you do not have SQL injection!
Which takes you here:
https://www.npmjs.com/package/pg-format
Repo:
https://github.com/datalanche/node-pg-format
I am trying to compare a string variable in my Python script with a varchar variable in SQL. Following is a snippet from my code:
todaysDate = datetime.now().date().strftime('%d-%m-%y')
read(conn, f'select cou_id from NewCourier where date_created = {todaysDate}'
Where:
todaysDate => variable in python storing current date
cou_id & date_created => columns in a relational DB table NewCourier
I tried:
read(conn, f'select cou_id from NewCourier where CAST(date_created AS INT) = {todaysDate}
I've also gone through a couple of questions and solutions on StackOverflow and other sites and found only this solution similar to my problem but I didn't get a clear idea of how to solve it.
Thanks!
Python 3.8.5
Microsoft SQL Server Management Studio 18
It seems like the column data type is integer and you are trying to compare it to a string. If you can find code in your program that inserts into the table you should be able to figure out how to convert your date time properly
Instead of creating a variable to store the current date in Python, I used the GETDATE() method of SQL and converted it to varchar using the CONVERT() method in SQL which solved my problem
(i.e., it allowed me to get the courier IDs of all the records generated on that particular day).
Here's the updated line:
read(conn, f'select cou_id from NewCourier where date_created = CONVERT(varchar, getdate(), 23)
I'm trying to setup tables in a spreadsheet as using query, from file, xlsx
On the source data there are 16,149 rows and after the import I only get 16,107 rows.
I've dummy'd down the import as much as possible I think:
let
Source = Excel.Workbook(File.Contents("C:\Users\myfile.xlsx"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet)
in
#"Promoted Headers"
Is there any way to debug this?
ok figured it out. problem was I was ignoring an error message when I ran the query that was a result of the new file missing one column that was present in the old data.
Net-net, the new file never got imported and that's why the row counts were different.
I've got a question regarding how to retrieve the auto-increment or identity value for a column in SQL Server 2005, when said column is not the first declared column in a table.
I can get the generated value for a table just by issuing the following code:
MyTable newRecord = new MyTable();
newRecord.SomeColumn = 2;
newRecord.Save();
return newRecord.MyIdColumn;
Which works fine regardles of how many other columns make up the primary key of that particular table, but the first column declared MUST be the identity column, otherwise this doesn't work.
My problem is that I have to integrate my code with other tables that are out of my reach, and they have identity columns which are NOT the first columns in those tables, so I was wondering if there is a proper workaround to my problem, or if I'm stuck using something along the lines of SELECT ##IDENTITY to manually get that value?
Many thanks in advance for all your help!
From the "ewww gross" department:
Here's my workaround for now, hopefully someone may propose a better solution to what I did.
MyTable newRecord = new MyTable();
newRecord.SomeColumn = 2;
newRecord.Save();
CodingHorror horror = new CodingHorror();
string SQL = "SELECT IDENT_CURRENT(#tableName)";
int newId = horror.ExecuteScalar<int>(SQL, "MyTable");
newRecord.MyIdColumn = newId;
newRecord.MarkClean();
return newRecord.MyIdColumn;