I have created a query which takes data from a folder, and orders the data in such a way that I need..
The problem is that it won't work on MacOs.. The get from folder function does not exist in MacOs, I understood that I need to use VBA in order to do the same thing on MacOs..
I have no clue on how to use VBA
These are some of the steps in my power query:
= Folder.Files("C:\Users\location..")
= Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true)
= Table.AddColumn(#"Filtered Hidden Files1", "Transform File", each #"Transform File"([Content]))
And so on..
How could I mimic these in MacOs with VBA?
So here is a proposed solution, but unfortunately it runs into permission problems, which maybe someone else can help with.
First have three cells in your spreadsheet. Use the Excel name manager to name them OS, FilePath and FileName respectively. Their content is:
OS: =LET(vers,INFO("OSVERSION"),IFERROR(LEFT(vers,FIND(" ",vers)-1),vers))
FilePath: =IF(A1="Windows","C:\...\","/Users/.../")
FileName= report.csv
(obviously subsititute the actual windows & mac filepaths above)
As the start of your Power Query M code, use:
let
FilePath= Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FileName= Excel.CurrentWorkbook(){[Name="FileName"]}[Content]{0}[Column1],
Source = Csv.Document(File.Contents(FilePath & FileName), [Delimiter = ",", Columns = 12, Encoding = 65001, QuoteStyle = QuoteStyle.None])
in
Source
This works fine on the Windows side - I am having some problems on Mac, getting an error such as:
Query 'FetchReport (2)' (step 'AutoRemovedColumns1') is accessing data sources that have privacy levels which cannot be used together. Please rebuild this data combination.0:
Related
In our lab we have a system that many people uses and produces a oddly shaped txt file with the results. I created a power query that cleans the file and I would like to share this with others (not very computer savvy) so they can apply it to the files they will generate.
What can I do to make it as easy as possible for other users to select the file they want the query to be applied to? Example: is there an easy way to create button that opens a dialog requesting the file location? Right now I have to edit the query source to select the data, this approach is clunky and will be confusing for some of my colleagues.
let
Source = Table.FromColumns({Lines.FromBinary(File.Contents("X:\foo\foo.txt"), null, null, 1252)}),
#"Removed Top Rows2" = Table.Skip(Source,32),
#"Removed Bottom Rows" = Table.RemoveLastN(#"Removed Top Rows2",16),
#"Other Steps" = ...
Thanks!
You can directly grab a filepath from a range name cell without a function by
let
NameValue= Excel.CurrentWorkbook(){[Name="rangenamehere"]}[Content]{0}[Column1],
Source = Table.FromColumns({Lines.FromBinary(File.Contents(NameValue), null, null, 1252)}),
Or if you wanted the VBA route for file prompt
1 Create a range name, here aaa
2 Use VBA to populate it using a file prompt
Sub prompt()
Dim FName As Variant
FName = Application.GetSaveAsFilename("", "Data file (*.xl*),*.xl*", 1)
If FName = False Then
MsgBox "False"
Exit Sub
End If
Range("aaa").Value = FName
End Sub
3 Refer to the named range in powerquery you set up
let
NameValue= Excel.CurrentWorkbook(){[Name="aaa"]}[Content]{0}[Column1],
Source = Table.FromColumns({Lines.FromBinary(File.Contents(NameValue), null, null, 1252)}),
4 Tack on code at end of VBA to refresh all queries or specific query
ActiveWorkbook.RefreshAll
or
ActiveWorkbook.Queries("QueryNameHere").Refresh
I found this post from 2014 that works pretty well. You write a function on Query (fnGetParameter) that reads the file location from a table and then you feed it to the query that processes the data.
All the user needs to do is write the file location on the table and name and refresh.
I changed the first to lines on my PowerQuery code to look like this:
Fileloc = fnGetParameter("File Path"),
Source = Table.FromColumns({Lines.FromBinary(File.Contents(Fileloc), null, null, 1252)}),
Any suggestions to make it even better?
You can make the fnGetParameter into a one-liner:
= ( getValue as text ) => Excel.CurrentWorkbook(){[Name=”Parameters”]}[Content]{[Parameter=getValue]}?[Value]?
I have been trying to get current directory into Power Query. But it somehow doesn't work. How can I get current directory, to make the path dynamic for my Query, so that in case the file moved a new directory, I will have no issue with retrieving data into Power Query. Here is the code I tried with:
let
Source = Excel.CurrentWorkbook(){[Name="pathTable"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Path", type text}}),
Path = Table.ReplaceValue(#"Changed Type","\[PathForSummaryFiles.xlsx]Path","\summary.xlsx",Replacer.ReplaceText,{"Path"})
GetFilesFromFolder = Folder.Files(Path)
in
GetFilesFromFolder
the code above throws an error.
Is this what you are trying to do?
In excel, name a cell DirectoryRangeName using formulas ... name manager
Within that cell, put in a formula to capture the path of that file
=LEFT(CELL("filename",A1),FIND("[",CELL("filename",A1))-1)
or enter your own path such as:
c:\directory\subdirectory\
Then, once in powerquery, read that value, and combine however you want to reference it such as
let Directory = Excel.CurrentWorkbook(){[Name="DirectoryRangeName"]}[Content]{0}[Column1],
Source = Excel.Workbook(File.Contents(Directory & "abs.xlsx"), null, true)
in Source
Or
let Directory = Excel.CurrentWorkbook(){[Name="DirectoryRangeName"]}[Content]{0}[Column1],
GetFilesFromFolder = Folder.Files(Directory)
in GetFilesFromFolder
Instead of a formula in the range name, you could simply put a full filepath such as c:\temp\a.xlsx
and then read it within powerquery
let FilePath= Excel.CurrentWorkbook(){[Name="DirectoryRangeName"]}[Content]{0}[Column1],
Source = Excel.Workbook(File.Contents(FilePath), null, true)
in Source
I have a worksheet DATA with the table populated from json file through the Microsoft Query.
There're different json files so I need to create several connections to any of those files.
I also have a cell on another worksheet where I would like to indicate a parameter (for example Yesterday,Today,Tomorrow).
According to selected parameter the table in the DATA worksheet should be populated from the related data connection (yesterday.json, today.json, tomorrow.json).
Is it possible to do it? If yes, what would be the procedure?
I have an idea that it might be possible to do by changing the filename inside the query.
For example, this is my query:
let
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\today.json",
Source = Json.Document(File.Contents(FullPathToFile1)),
So am thinking if there's some way to "inject" filename in the above query based on value of some cell.
Will appreciate any help, links etc.
Thanks!
UPDATE:
I have created a named cell jsonPath and put the file name in it.
Then I have modified above query as follows, but it gives me an error.
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\" & [jsonPath],
Source = Json.Document(File.Contents(FullPathToFile1)),
I got it working by modifying my query as follows:
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FileName = Excel.CurrentWorkbook(){[Name="jsonPath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\" & FileName,
Source = Json.Document(File.Contents(FullPathToFile1)),
For weeks I've been searching the answer for this problem I have. I need to send by email query results from the day before in an excel sheet.
Requirements are:
Extract some data from database from the day before -- OK
Send this query result as Excel file (format xlsx) by email -- NOK
Problems:
Output is only in CSV even when I change the format in #query_attachment_filename. When I use xlsx the generated file cannot be opened.
Missing leading zeros in first 2 columns
My half-working code now:
use xxx
DECLARE #AWBPREFIX VARCHAR(255)
DECLARE #Query VARCHAR(MAX)
SET #AWBPREFIX = '[sep=,' + CHAR(13) + CHAR(10) + 'AWB.PREFIX]'
SET #Query = '
SET NOCOUNT ON
SELECT AWB.PREFIX AS ' + #AWBPREFIX + ',
AWB.SERIAL,
AWB.ORIGIN,
AWB.DEST,
AWB_MOVE.CARRIER,
AWB.PIECES,
AWB.WEIGHT,
AWB_MOVE.PIECES,
AWB_MOVE.WEIGHT,
AWB.NATURE_GOODS,
AWB_MOVE.CARRIER,
AWB_MOVE.FLIGHT_NUMBER,
AWB_MOVE.FLIGHT_DATE,
AWB_MOVE.ORIGIN,
AWB_MOVE.DEST,
AWB_MOVE.ULD_NUMBER,
AWB_MOVE.SHC_LIST
FROM
xxx..AWB
INNER JOIN
AWB_MOVE ON AWB.SEQ = AWB_MOVE.AWB_SEQ
GROUP BY
AWB.PREFIX,
AWB.SERIAL,
AWB.ORIGIN,
AWB.DEST,
AWB_MOVE.CARRIER,
AWB.PIECES,
AWB.WEIGHT,
AWB_MOVE.PIECES,
AWB_MOVE.WEIGHT,
AWB.NATURE_GOODS,
AWB_MOVE.CARRIER,
AWB_MOVE.FLIGHT_NUMBER,
AWB_MOVE.FLIGHT_DATE,
AWB_MOVE.ORIGIN,
AWB_MOVE.DEST,
AWB_MOVE.ULD_NUMBER,
AWB_MOVE.SHC_LIST,
AWB_MOVE.action_status
HAVING AWB_MOVE.FLIGHT_DATE = dateadd(dd,-1,cast(getdate() as date)) AND AWB_MOVE.DEST ="BRU" AND AWB_MOVE.action_status <>"NOT"'
EXEC msdb.dbo.sp_send_dbmail
#profile_name = 'Batch',
#recipients = 'xxx',
#execute_query_database = 'xxx',
#subject ='Import extracts',
#query =#Query,
#query_result_header = 1,
#query_result_separator=',',
#query_result_no_padding=1,
#query_result_width=32767,
#attach_query_result_as_file = 1,
#query_attachment_filename = 'import_batch.csv'
I cannot use SSIS due to office restrictions.
Can someone help me please?
Thanks
Michael
sp_send_dbmail will not make a .xlsx file for you. However, you can use the method described here to make the .csv from sp_send_dbmail readable in excel.
If you have SSRS in your environment, you might be able to create a report subscription which publishes to excel.
I am able to create .XLS files as an attachment, but not .XLSX. I get a warning when opening it, but click "Yes" and it opens.
Try that.
There may be a better way that I have not found.
EXEC msdb.dbo.sp_send_dbmail
#profile_name = 'My_SQL_MailProfile',
#recipients = 'me#myemail.com',
#query = 'select * from My_Table' ,
#subject = 'Test email as XLS',
#attach_query_result_as_file = 1,
#query_attachment_filename='filename.xls', --- XLS worked. XLSX shows as "corrupted"
#query_result_separator=' ' -- tab
I have an excel dokument (tab delimitted). Every time before i can insert the excel in my program, i need to map the clients column names to the ones in my app.
So before i begin developing a mapping tool - it would be better if such already existed. But, i don't know i didn't find it.
This tool would actually read an excel or txt file, allow me to name all the names of the new columns on the right and drag and drop (for example).
Maybe this tool has an xml or something where i can define my custom columns, and then it would show op on the right side.
I hope you know what i mean and that someone also had the need for this.
Thanks
info update
I wanted to mention a few things as an update to my question if it's not to late: I have about 50 headers/columns (example: First Name, Middle Name, Street1, Street2,..). So what i always need do every time a client gives me his excel file (contacts backup) is manually copy data for each of his column to the one on my side. And the problem is, every client has different column names, and also some of the columns of the client can go to multiple columns on my side.
So i think, if i can't find a solution, i would make a c++/qt app, which takes an excell and lets me to assign (dragdrop,etc..) every column of his side to on or more columns of my side.
I haven't done any vb programming, so if you can be a bit more detailed about how to aproach the solution i vb that would be great.
Firs part of answer: You can achieve it with a simple vbscript. Copy this code in a vbs extension text file and double click.
Dim oCN As Connection
Dim fs As Scripting.FileSystemObject
Set oCN = New Connection
Set fs = New Scripting.FileSystemObject
sCSVFileName = "C:\Temp\Test1.csv"
sSourceSql = "SELECT field1 as f1, field2 as f2, ... FROM " &
fs.GetFileName(sCSVFileName )
sDestinationTable = "yourAppTable"
With oCN
.Provider = "Microsoft.Jet.OLEDB.4.0"
.Properties("Extended Properties").Value =
"TEXT;HDR=YES;FMT=TabDelimited;MAXSCANROWS=0"
.Open fs.GetFile(sCSVFileName).ParentFolder
End With
sSql = "INSERT INTO [ODBC;DRIVER={SQL Server};Server=" &
ServerName & ";Database=" & DBName & _
IIf(IntegratedSecurity, ";Trusted_Connection=Yes;", _
";UID=" & UID & ";PWD=" & PWD & ";") & _
"]." & sDestinationTable & " " & sSourceSql
oCN.Execute sSql, , adExecuteNoRecords
More info at microsoft social forums
Also, you can parametrize mapping (source and destination tables and mapping fields) with a external xml file.
Second part of answer: You ask if somebody else have this needed and if this is a good idea. Well, this is a very good proposal. And for this reason they are some solutions with this functionality.
First of all, this kind of technology is named ETL. Extract - Transform - Load.
Each database has its own tool
SQL Server Import and Export Wizard
Oracle data pump and loader
etc.
Also exists specific technologies:
SSIS from microsoft.
IBM WebSphere DataStage
etc.
All this tools have mapping columns capabilities.
i use a generic mapping mechanism for that, configured by a couple arrays where you put the names used in your app (aDsNames) with these in your source, the csv file in your case (aDbNames)
I walk the fields in the source, check if the getDbName(name) is in aDsNames and if so write the value in the insertstring to the database.
You can do this both ways, usually my app requests a field to the databasemodule on the server, this module translates to the databasename and does the select.
Hope this is helpfull..
Cheers
select case store
case "store1Midoc"
aDbNames = array("id" , "beheerder", "datumlijst", "rnr13" , "datvan", "dattot", "opmerking", "status" , "waarde", "kode" , "type")
aDsNames = array("id" , "persnr ", "datum ", "rnr13" , "datvan", "dattot", "opmerking", "status" , "waarde", "kode" , "type")
aTypes = array("number", "string", "date" , "string", "date" , "date" , "string" , "number", "number", "string", "string")
case .....
end select
Function getDbName(dsName)
Dim a
getDbName = "undefined"
If instr(join(aDsNames,","),dsName) Then
For a = 0 to UBound(aDbNames)
If aDsNames(a) = dsName Then
getDbName = aDbNames(a)
End If
Next
End If
End Function