I'm fetching data through an SQL statement in PowerQuery:
let
Source = Oracle.Database("sampleDB", [Query="SELECT * FROM mySampleTable WHERE CustomerID in (1,2,3,4,5)"])
in
Source
I need the SQL filter to be dynamic. So I have another query with a single cell containing a text string. For the example above, the single cell looks like => "1,2,3,4,5". This is why I want to build a dynamic SQL statement in PowerQuery that references that single cell from my other query.
I've tried the code below and other variants but none work!:
let
Source = Oracle.Database("sampleDB", [Query="SELECT * FROM mySampleTable WHERE CustomerID IN (" & MyReferenceQuery["SingleCell"] & ")"]
in
Source
How can I reference this other query?
If the text is in a single cell in a table, you need to access the value by specifying a column and a row-index, like below:
= Oracle.Database("sampleDB", [Query = "SELECT " & MyReferenceQuery[Column1]{0} & " FROM Table"])
Related
I have a SQL server query that has 7 joins and pulls over 50 columns. I run this in SSMS, query results are stored in a local temp table, and then there are a dozen or so SELECT statements within the query that run based off the temp table. I have no control over the data itself, and only have read access to the tables. So basically:
SELECT *
INTO #myTempTable
FROM myTableAnd7Joins
SELECT <stuff>
FROM #myTempTable
SELECT <different stuff>
FROM #myTempTable
<like 10 more SELECT statements>
The temp table can easily be 500,000 rows and can take 5-10 minutes to return.
I run this query once, where it creates the single temp table and then multiple sets of results are returned. I then copy and paste each set of results to various tabs in an Excel file, and the rest of the Excel file is already populated with formulas and such which read the data from those tabs. I want to get rid of having to copy and paste from SSMS to Excel and do everything in Excel instead. I also want to use parameterized queries, so I can change some of the query variables on the fly. The parameters are stored in a range on a tab named 'Parameters'.
I currently have an Excel workbook saved that has the dozen individual Power Query queries, but each one has to do that initial query into #myTempTable first before performing it's specific query.
Each of my current dozen queries look like this:
LET
myParamTable = Excel.CurrentWorkbook(){[Name="tParams"]}[Content],
fieldList = myParamTable[#"Parameter"],
valueList = myParamTable[Value],
param1 = valueList{List.PositionOf(fieldList, "First Parameter")},
param2 = valueList{List.PositionOf(fieldList, "Second Parameter")},
param3 = valueList{List.PositionOf(fieldList, "Third Parameter")},
Source = Sql.Database("mySqlServer", "DatabaseName"),
GoQuery = Value.NativeQuery(Source,
"
DECLARE
# <a bunch of different variables>
BEGIN
SET <a bunch of different variables>
END
SELECT <50 different columns from all the tables>
INTO #myTempTable
FROM myTable t1
JOIN otherTable t2 on t1.thing = t2.thing
JOIN <6 or 7 other tables to t1>
SELECT <complex stuff that's more than just a filter or two>
FROM #myTempTable
WHERE <something> = p1
DROP TABLE #myTempTable",
[p1 = param1,
p2 = param2,
p3 = param3])
IN
GoQuery
Is it possible to create #myTempTable just once, and then have the dozen other queries use that as the source, instead of having to create #myTempTable each time? I tried to put #myTempTable into a table and then get the data From Table/Range but don't see how I can replicate my SQL as suggested in a previous question of mine (Power Query - Can a range be queried using SQL?)
I am trying to fetch data from my Access table into Excel sheet. The below mentioned SQL query is throwing an error "Data type mismatch in criteria expression" while the execution reaches Open query.
I googled for all possible fixes. All are saying the number might be passed in quotes. i checked for it still clueless, The query is pretty simple where i am trying select data from a userform text box in DD-MMM-YYYY format (03-OCT-2018) and comapring with date portion of timestamp field and order by my customer id field
SELECT * FROM Master_Intrpay_Cash where DateValue(LAST_UPDATE_TIMESTAMP)>=#" & Trim(startdate.value) & "# and DateValue(LAST_UPDATE_TIMESTAMP)<=#" & Trim(enddate.value) & "# ORDER BY CUSTOMER_ID
Below Shown is the msgbox showing the query being passed. if it helps.
Also the crazy part is the above query was copy pasted from an existing working query, just changed the table name and timestamp field name. Rest everything is exactly the same.
Try without DateValue:
SELECT *
FROM Master_Intrpay_Cash
WHERE LAST_UPDATE_TIMESTAMP >= # " & Trim(startdate.value) & " #
AND LAST_UPDATE_TIMESTAMP <= # " & Trim(enddate.value) & " #
ORDER BY CUSTOMER_ID
DateValue expects a String as an argument, your column as Date/Time
Also, a preferable format for the date is #mm/dd/yyyy# (US date format), otherwise you may have problems with different locales.
I have a MS Access database with about 1000 tables. The Tables are independent of each other, that is they are not linked to each other. Basically each table contains records of stock of a particular good. The Names of each table is the name of the item, for example 125 gm Soap, 200 ml Oil, etc. and contains the following columns : -
Date
Daily Opening Stock
Daily Sales
Daily Purchases
Daily Closing Stock
Each day, sales/purchases of all the goods are recorded in accounting software. The End of Day report summary is generated in Excel Format. This report is a consolidated report of daily Sales/Purchases of all the goods, row-wise. The Columns in Excel are :
Date
Item Name
Daily Opening Stock
Daily Sales
Daily Purchases
Daily Closing Stock
Presently I have to manually open each table in Access & copy-paste the respective daily sales/purchases data from the excel report. I want to automate this process so that the record is automatically appended to each of the respective table, basis the Item Name.
In summary, I want to update multiple tables in Ms Access database file, from data contained in 1 Ms Excel spreadsheet.
Is that possible ? Your help would be highly appreciated.
Thanks,
Thiru V. Klack
My first question to you is why can't the Access database match the data structure of the Excel spreadsheet? Updating 1 Access table from 1 spreadsheet is a lot simpler than what you're trying to do. Having 1,000 tables with the same data structure, where each table is named after a product, seems pretty fishy to me.
That being said, if you need to keep your 1,000 tables (I'm honestly surprised you're not hitting the DB maximum threshold for Access), there are ways you can make it work. I would suggest having a lookup table that matches the product (as it appears on the spreadsheet) to the table in Access it goes to. Then, you'd import the Excel data into a staging table, and run some VBA code to loop through the lookup table, running a query to append all the data from Excel into the appropriate Access table. Something like this:
private const STAGING_TABLE_NAME as string = "tDataFromExcel"
private Const LOOKUP_TABLE_NAME as String = "tGoodsToTableNames"
public sub AddData()
Dim db as DAO.Database
Dim rsItemLookup as DAO.Database
Dim strSQL as string
Dim GoodName as string, TableName as string
set db = CurrentDB
set rsItemLookup = db.OpenRecordset("select good_name, table_name from " & LOOKUP_TABLE_NAME)
do while not rsItemLookup.EOF
GoodName = rsItemLookup.fields("good_name").Value
TableName = rsItemLookup.fields("table_name").value
strSQL = _
"insert into " & TableName & " (" & _
"[date], " & _
"[Daily Opening Stock], " & _
"[Daily Sales], " & _
"[Daily Purchases], " & _
"[Daily Closing Stock]) "
"select " & _
"[date], " & _
"[Daily Opening Stock], " & _
"[Daily Sales], " & _
"[Daily Purchases], " & _
"[Daily Closing Stock]) " & _
"from " & STAGING_TABLE_NAME & " " & _
"where [Item Name] = """ & GoodName & """"
db.Execute strSQL
rsItemLookup.MoveNext
loop
end sub
Again, I'm certainly not advocating you take this approach: If you can, I would strongly implore you to refactor your database to not have 1,000 virtually identical tables.
Zack's answer provides a way to do what you're asking, but recommends that you do something else. I'm going to try to demonstrate that what he recommends would work for your situation, and show you how to achieve it.
You want to transfer data in an excel spreadsheet into a relational database that holds the same data - usually to give better access/manipulation. The data is in a big table of one day, with a separate line for each product, and you want to be able to view the entire history of a single product, so you have separated out the tables.
This is going to significantly slow down the automatic transfer process - that long loop that re-writes and processes the SQL over and over. You can look at the data in that way without this setup though. Lets call this tblProductDetails
One table has your product information - things like the name of the product, how much it costs... The other has the in/out records of what was bought and sold, keeping track of stock on hand at any point. This second table is going to have a LOT of records, as it's a new record for each product that was bought and/or sold each and every day. We might call this tblInOut.
With a Select query you can save in Access, you can look at the history of any one product at any time - you can even set this as a variable so that when you open the only query you save it prompts you for the name of the product you want to look up.
I've mocked up a 4 product sample schema for you with a query to show all the data, and another to show just the one product's history. You can use the second query exactly as is in Access' query builder (SQL view) where '125g Soap' (on the second last line) is mentioned in the query name so you know which query is which.
If you want to use just one query, do the same thing with one exception: on the second last line, where it says '125g Soap' replace the whole thing with Product_Name (without the inverted commas) and when you open the query, Access will give you a message box asking you to type in a product name.
This feature will only work if you type the name exactly right though, so you might prefer just to build a form so you can give the user a combo box with which to select the exact product name.
Background
I have a dataset with 10,000+ variables as column headers, which I want to reduce to the amount needed. I know how to select a sample of columns by listing columns that contain manually specified strings, say "glu" and "pep", that the columns must contain in order to be selected. This is the M code used to select the sample columns:
let
Source = Excel.CurrentWorkbook(){[Name="data"]}[Content],
ColumnsToSelect = List.Select(Table.ColumnNames(Source), each Text.Contains(_, "glu") or Text.Contains(_, "pep")),
SelectColumns = Table.SelectColumns(Source, ColumnsToSelect)
in
SelectColumns
This Power Query produces a table that i call "Data". Since I want to select columns based on multiple strings they must contain, I have made a dynamic list of strings that I have called "Outcomes". I want my Power Query to utilize this list of strings when choosing what columns to select.
Question
Is it possible to get my Power Query to utilize this dynamic list in the List.Select() or Table.SelectColumns() function or any other function, that will make my Power Query select only the columns that contain the strings on the list?
Use with this lines:
let
Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
Source2 = Excel.CurrentWorkbook(){[Name="Outcomes"]}[Content],
Outcomes = Source2[Outcomes],
UnpivotedColumns = Table.UnpivotOtherColumns(Source, {}, "ColumnNames", "Filters"),
FilteredRows = Table.SelectRows(UnpivotedColumns, each List.AnyTrue(List.Transform(Outcomes, (substring) => Text.Contains([Filters], substring)))),
ColumnNames = List.Sort(List.Distinct(FilteredRows[ColumnNames]),Order.Ascending),
SelectColumns = Table.SelectColumns(Source,ColumnNames)
in
SelectColumns
the magic is in this line:
FilteredRows = Table.SelectRows(UnpivotedColumns, each List.AnyTrue(List.Transform(Outcomes, (substring) => Text.Contains([Filters], substring)))),
The following line of code will create a variable of the type table with the content from a table in the spreadsheet.
Instead of referring to a table in excel spreadsheets, I want to refer to another query. — How do I do that?
mytable = Excel.CurrentWorkbook(){[Name="table_in_excel_spreadsheet"]}[Content]
If your query name is a text variable, use Record.Field(#shared, YourQuery)
If you're query is called Query1, it's:
mytable = Query1
For queries with more complicated names with spaces and special characters, like My Query, it's:
mytable = #"My Query"