If I have created a PowerQuery function that imports XML from a folder, how in the same excel file do I reuse the query when there are new files in the folder to only include data from those new files and append the data to the current table?
If you start a Power Query using From File / From Folder and browse to your folder, you will see each file represented as a row in a table, with columns such as Date modified. You can filter that list using Date/Time filters on Date modified or by something more complicated if necessary (post your specific requirements and I'll try to steer you in the right direction).
Once you have filtered the query to just the "new files", you can pass the [Content] column into your Function.
Finally Append a new query based on the saved Excel Table output from your pre-existing query together with the "new files" query above to get your combined output. The new query would be set to Load To / Only Create Connection.
you can watch a folder for file changes with a simple vba script that uses WMI to poll the directory contents every n seconds.
Something similar to this ...
Sub WatchDirectory(dir as string, every as integer)
Set wmisvc = GetObject("winmgmts:\\.\root\cimv2")
let query = "SELECT * FROM __InstanceOperationEvent " _
& "WITHIN " & every _
& " WHERE Targetinstance ISA 'CIM_DirectoryContainsFile' and " _
& "TargetInstance.GroupComponent='Win32_Directory.Name=" _
& Chr(34) & dir & Chr(34) & "'"
Set events = wmisvc.ExecNotificationQuery(query)
Do While True
Set event = events.NextEvent()
if event.Class = "__InstanceCreationEvent" then
....
end if
Loop
For more info on wmi see https://sites.google.com/site/beyondexcel/project-updates/exposingsystemsecretswithvbaandwmiapi
For more details on file watching with WMI, see https://blogs.technet.microsoft.com/heyscriptingguy/2005/04/04/how-can-i-monitor-for-different-types-of-events-with-just-one-script/
Related
I have a worksheet DATA with the table populated from json file through the Microsoft Query.
There're different json files so I need to create several connections to any of those files.
I also have a cell on another worksheet where I would like to indicate a parameter (for example Yesterday,Today,Tomorrow).
According to selected parameter the table in the DATA worksheet should be populated from the related data connection (yesterday.json, today.json, tomorrow.json).
Is it possible to do it? If yes, what would be the procedure?
I have an idea that it might be possible to do by changing the filename inside the query.
For example, this is my query:
let
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\today.json",
Source = Json.Document(File.Contents(FullPathToFile1)),
So am thinking if there's some way to "inject" filename in the above query based on value of some cell.
Will appreciate any help, links etc.
Thanks!
UPDATE:
I have created a named cell jsonPath and put the file name in it.
Then I have modified above query as follows, but it gives me an error.
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\" & [jsonPath],
Source = Json.Document(File.Contents(FullPathToFile1)),
I got it working by modifying my query as follows:
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FileName = Excel.CurrentWorkbook(){[Name="jsonPath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\" & FileName,
Source = Json.Document(File.Contents(FullPathToFile1)),
This is a continuity question of the below link. https://stackoverflow.com/a/56649098?noredirect=1
I need to do two things
1. Copy all text from PDF and paste it to the excel
2. Copy multiple lines and run a loop to find the text I need
The background of my project - I am opening a webpage with user credentials, after couples of clicks a PDF is opened in the browser. So now I need to get a particular statement from it. The problem is that the PDF is dynamic and the statement I require keeps changing (the line sometimes it is in 6th and sometimes in 9th or 10th and 11th) so the 2 things which I mentioned above will help me I am aware both sounds the same but it is different. Below is the code I used to get a particular statement, but how do I create a loop through all the statement or get multiple statements.
Const statext As String = _
"addEventListener('message',function(e){" & _
" if(e.data.type=='getSelectedTextReply'){" & _
" var txt=e.data.selectedText;" & _
" callback(txt && txt.match(/[^\r\n]+/g)[7]);" & _
" }" & _
"});" & _
"plugin.postMessage({type:'initialize'},'*');" & _
"plugin.postMessage({type:'selectAll'},'*');" & _
"plugin.postMessage({type:'getSelectedText'},'*');"
Casestatus = bot.ExecuteAsyncScript(statext)
I am very new to programming and automation so I may be missing a basic thing. Kindly pardon me.
Good Afternoon,
I have an access query that contains a list of all my customers lets call that CUS
I have another query that has a list of ORDERS
I would like to write some VBS that cycles through the customer list and exports a csv file containing all orders that belong to that customer.
The vba would then move on to the next customer on the list and perform the same action.
Any help would be great.
Snippet of code below
almost there cant get the WHERE condition working it keeps displaying a popup for me to populate however the same string is feeding the msgbox fine here is a snippet below tht is within the loop
strcustcode = rs!OCUSTCODE
ordercount = rs!orders
TIMEFILE = Format$(Time, "HHMM")
MsgBox ([strcustcode] & " has " & [ordercount] & " orders")
StrSQL = "Select * From [24-ND_Cus] where [24-ND_Cus].[OCUSTCODE] = strcustcode "
Set qd = db.CreateQueryDef("tmpExport", StrSQL)
DoCmd.TransferText acExportDelim, , "tmpExport", "c:file.csv" db.QueryDefs.Delete "tmpExport" –
Don't use [ ] around VBA variables. Don't use parens for the MsgBox when you just want to give user a message. The parens make it a function that requires a response by user to set a variable.
MsgBox strcustcode & " has " & ordercount & " orders"
Concatenate the variable into the SQL statement. If OCUSTCODE is a text type field, use apostrophe delimiters for the parameter.
StrSQL = "Select * From [24-ND_Cus] Where [OCUSTCODE] = '" & strcustcode & "'"
I don't advise code that routinely modifies design and changing a query SQL statement is changing design. If the only change is filter criteria and a dynamic parameterized query won't work, I suggest a 'temp' table - table is permanent, data is temporary. Delete and write records to the table and export the table.
I am stymied by an SQL mediated import of a CSV file using VBA code. I am using a Third EXCEL macro/spreadsheet, to analyze a LEFT JOIN of 2 files, one as an XLXS and the other as a CSV.
I suspect that part of the problem may be how the SQL command is used, for a FROM reference to an excel file. I am using Excel VBA, 2010, The 14 Database Access Engine.
I want to end with an SQL statement that pulls from an external comma delimited CSV file
I anticipate heading the macro with this pseudo code, in a stand-alone macro enabled excel file:
dbEngine = CreateObject(DAO.engine ... )
set DB = dbEngine.OpenDatabase(theNormalExternalExcellFile,....)
For the SQL statement, in pseudo-code, I want this:
SELECT fields
FROM [Table$] ' a normal external excel file
LEFT JOIN [an external CSV, comma delimited file]
ON...
GROUP...
I can successfully import an XLXS, or the CSV, independently, in a simple SQL statement, yet when I place the outside file references within an SQL's FROM clause, I get one of two errors, depending on how I play with the code: an Invalid File Path, or an error in the FROM Clause. The path is -not- invalid.
The error is shown, below, where it occurs, at the recordset instruction.
I also provide alternative SQL strings, which I had played with to test where in the code the error is generated.
'the Seating Chart
strPathSource = ThisWorkbook.Worksheets("Logic").Range("rngPathSource")
'strFileNameSource = ThisWorkbook.Worksheets("Logic").Range("rngFileNameSource")
'strFileNameSourceWOExt = Left(strFileNameSource, Len(strFileNameSource) - 4)
'the attendance
strPathAttendance = ThisWorkbook.Worksheets("Logic").Range("rngPathAttendance")
strFileNameAttendance = ThisWorkbook.Worksheets("Logic").Range("rngFileNameAttendance")
strFolderAttendance = ThisWorkbook.Worksheets("Logic").Range("rngFolderAttendance")
strFileNameAttendanceWOExt = Left(strFileNameAttendance, Len(strFileNameAttendance) - 4)
Set dbE = CreateObject("Dao.DBEngine.120")
Set db = dbe.OpenDatabase(strPathSource, True, False, "Excel 12.0;HDR=Yes")
''Set db = DAO.OpenDatabase(strFolderAttendance, True, False, "text;HDR=Yes;FMT=Delimited(,)")
'[Master$] is a tab on the spreadsheet at strPathSource
'[Attendance#csv]
' This reference to the table at strPathAttendance which otherwise works: [Attendance#csv]
' when not inside the FROM clause
strSQL = _
"SELECT tM.Job, Count(tA.Name) AS CountOfName" _
& " FROM [Master$] tM" _
& " LEFT JOIN" _
& " (SELECT * FROM [text;HDR=Yes;FMT=Delimited(,);Database='" _
& strPathAttendance & "'].[" & strFileNameAttendanceWOExt & "#csv]) tA" _
& " ON (tM.GivenName = tA.GivenName) AND (tM.SurName = tA.SurName)" _
& " GROUP BY tM.Job" _
& " ORDER BY tM.Job, Count(tA.Name)"
'Debug.Print strSQL
' This is the reported value for the string, strSQL, particularly the FROM clause:
' SELECT tM.Job, Count(tA.Name) AS CountOfName FROM [Master$] tM LEFT JOIN
' (SELECT * FROM
' [text;HDR=Yes;FMT=Delimited(,);Database=T:\Solutions Team Shared Folder\Seats -
' Attendance\Attendance.csv].[Attendance#csv]) tA
' ON (tM.GivenName = tA.GivenName) AND (tM.SurName = tA.SurName)
' GROUP BY tM.Job ORDER BY tM.Job, Count(tA.Name)
'' putting a single or double quote, around the database path, does not change the error
Set rstR = db.OpenRecordset(strSQL)
'Error:
' 'T:\...\...\Attendance.csv' is not a valid path. Make sure that
' the path name is spelled correctly and that you are connected to the server
' on which the file resides.
' ALT SQL strings, to test what's going on.
'strSQL = _
' "Select * FROM [Attendance#csv]"
'strSQL = _
' "Select * FROM (Select * FROM [Excel 12.0;HDR=Yes;Database=" & strPathSource & "].[Master$])"
'strSQL = _
' "SELECT * FROM [text;HDR=Yes;FMT=Delimited(,);Database=" _
' & strPathAttendance & "].[" & strFileNameAttendanceWOExt & "#csv]"
'strSQL = _
' "Select * FROM [Excel 12.0;HDR=Yes;Database=" & strPathSource & "].[Master$]"
When connected to text files with Jet/ACE SQL, the database parameter needs to reference the directory path not any specific text file. The period qualifier will then specify the individual file.
Therefore, simply remove the file name and extension from strPathAttendance (without quotes). So query should look like the below:
SELECT tM.Job, Count(tA.Name) AS CountOfName
FROM [Master$] tM
LEFT JOIN
(SELECT * FROM
[text;HDR=Yes;FMT=Delimited(,);Database=T:\Solutions Team Shared Folder\Seats -
Attendance].[Attendance#csv]) tA
ON (tM.GivenName = tA.GivenName) AND (tM.SurName = tA.SurName)
GROUP BY tM.Job
ORDER BY tM.Job, Count(tA.Name)
I have an excel dokument (tab delimitted). Every time before i can insert the excel in my program, i need to map the clients column names to the ones in my app.
So before i begin developing a mapping tool - it would be better if such already existed. But, i don't know i didn't find it.
This tool would actually read an excel or txt file, allow me to name all the names of the new columns on the right and drag and drop (for example).
Maybe this tool has an xml or something where i can define my custom columns, and then it would show op on the right side.
I hope you know what i mean and that someone also had the need for this.
Thanks
info update
I wanted to mention a few things as an update to my question if it's not to late: I have about 50 headers/columns (example: First Name, Middle Name, Street1, Street2,..). So what i always need do every time a client gives me his excel file (contacts backup) is manually copy data for each of his column to the one on my side. And the problem is, every client has different column names, and also some of the columns of the client can go to multiple columns on my side.
So i think, if i can't find a solution, i would make a c++/qt app, which takes an excell and lets me to assign (dragdrop,etc..) every column of his side to on or more columns of my side.
I haven't done any vb programming, so if you can be a bit more detailed about how to aproach the solution i vb that would be great.
Firs part of answer: You can achieve it with a simple vbscript. Copy this code in a vbs extension text file and double click.
Dim oCN As Connection
Dim fs As Scripting.FileSystemObject
Set oCN = New Connection
Set fs = New Scripting.FileSystemObject
sCSVFileName = "C:\Temp\Test1.csv"
sSourceSql = "SELECT field1 as f1, field2 as f2, ... FROM " &
fs.GetFileName(sCSVFileName )
sDestinationTable = "yourAppTable"
With oCN
.Provider = "Microsoft.Jet.OLEDB.4.0"
.Properties("Extended Properties").Value =
"TEXT;HDR=YES;FMT=TabDelimited;MAXSCANROWS=0"
.Open fs.GetFile(sCSVFileName).ParentFolder
End With
sSql = "INSERT INTO [ODBC;DRIVER={SQL Server};Server=" &
ServerName & ";Database=" & DBName & _
IIf(IntegratedSecurity, ";Trusted_Connection=Yes;", _
";UID=" & UID & ";PWD=" & PWD & ";") & _
"]." & sDestinationTable & " " & sSourceSql
oCN.Execute sSql, , adExecuteNoRecords
More info at microsoft social forums
Also, you can parametrize mapping (source and destination tables and mapping fields) with a external xml file.
Second part of answer: You ask if somebody else have this needed and if this is a good idea. Well, this is a very good proposal. And for this reason they are some solutions with this functionality.
First of all, this kind of technology is named ETL. Extract - Transform - Load.
Each database has its own tool
SQL Server Import and Export Wizard
Oracle data pump and loader
etc.
Also exists specific technologies:
SSIS from microsoft.
IBM WebSphere DataStage
etc.
All this tools have mapping columns capabilities.
i use a generic mapping mechanism for that, configured by a couple arrays where you put the names used in your app (aDsNames) with these in your source, the csv file in your case (aDbNames)
I walk the fields in the source, check if the getDbName(name) is in aDsNames and if so write the value in the insertstring to the database.
You can do this both ways, usually my app requests a field to the databasemodule on the server, this module translates to the databasename and does the select.
Hope this is helpfull..
Cheers
select case store
case "store1Midoc"
aDbNames = array("id" , "beheerder", "datumlijst", "rnr13" , "datvan", "dattot", "opmerking", "status" , "waarde", "kode" , "type")
aDsNames = array("id" , "persnr ", "datum ", "rnr13" , "datvan", "dattot", "opmerking", "status" , "waarde", "kode" , "type")
aTypes = array("number", "string", "date" , "string", "date" , "date" , "string" , "number", "number", "string", "string")
case .....
end select
Function getDbName(dsName)
Dim a
getDbName = "undefined"
If instr(join(aDsNames,","),dsName) Then
For a = 0 to UBound(aDbNames)
If aDsNames(a) = dsName Then
getDbName = aDbNames(a)
End If
Next
End If
End Function