For weeks I've been searching the answer for this problem I have. I need to send by email query results from the day before in an excel sheet.
Requirements are:
Extract some data from database from the day before -- OK
Send this query result as Excel file (format xlsx) by email -- NOK
Problems:
Output is only in CSV even when I change the format in #query_attachment_filename. When I use xlsx the generated file cannot be opened.
Missing leading zeros in first 2 columns
My half-working code now:
use xxx
DECLARE #AWBPREFIX VARCHAR(255)
DECLARE #Query VARCHAR(MAX)
SET #AWBPREFIX = '[sep=,' + CHAR(13) + CHAR(10) + 'AWB.PREFIX]'
SET #Query = '
SET NOCOUNT ON
SELECT AWB.PREFIX AS ' + #AWBPREFIX + ',
AWB.SERIAL,
AWB.ORIGIN,
AWB.DEST,
AWB_MOVE.CARRIER,
AWB.PIECES,
AWB.WEIGHT,
AWB_MOVE.PIECES,
AWB_MOVE.WEIGHT,
AWB.NATURE_GOODS,
AWB_MOVE.CARRIER,
AWB_MOVE.FLIGHT_NUMBER,
AWB_MOVE.FLIGHT_DATE,
AWB_MOVE.ORIGIN,
AWB_MOVE.DEST,
AWB_MOVE.ULD_NUMBER,
AWB_MOVE.SHC_LIST
FROM
xxx..AWB
INNER JOIN
AWB_MOVE ON AWB.SEQ = AWB_MOVE.AWB_SEQ
GROUP BY
AWB.PREFIX,
AWB.SERIAL,
AWB.ORIGIN,
AWB.DEST,
AWB_MOVE.CARRIER,
AWB.PIECES,
AWB.WEIGHT,
AWB_MOVE.PIECES,
AWB_MOVE.WEIGHT,
AWB.NATURE_GOODS,
AWB_MOVE.CARRIER,
AWB_MOVE.FLIGHT_NUMBER,
AWB_MOVE.FLIGHT_DATE,
AWB_MOVE.ORIGIN,
AWB_MOVE.DEST,
AWB_MOVE.ULD_NUMBER,
AWB_MOVE.SHC_LIST,
AWB_MOVE.action_status
HAVING AWB_MOVE.FLIGHT_DATE = dateadd(dd,-1,cast(getdate() as date)) AND AWB_MOVE.DEST ="BRU" AND AWB_MOVE.action_status <>"NOT"'
EXEC msdb.dbo.sp_send_dbmail
#profile_name = 'Batch',
#recipients = 'xxx',
#execute_query_database = 'xxx',
#subject ='Import extracts',
#query =#Query,
#query_result_header = 1,
#query_result_separator=',',
#query_result_no_padding=1,
#query_result_width=32767,
#attach_query_result_as_file = 1,
#query_attachment_filename = 'import_batch.csv'
I cannot use SSIS due to office restrictions.
Can someone help me please?
Thanks
Michael
sp_send_dbmail will not make a .xlsx file for you. However, you can use the method described here to make the .csv from sp_send_dbmail readable in excel.
If you have SSRS in your environment, you might be able to create a report subscription which publishes to excel.
I am able to create .XLS files as an attachment, but not .XLSX. I get a warning when opening it, but click "Yes" and it opens.
Try that.
There may be a better way that I have not found.
EXEC msdb.dbo.sp_send_dbmail
#profile_name = 'My_SQL_MailProfile',
#recipients = 'me#myemail.com',
#query = 'select * from My_Table' ,
#subject = 'Test email as XLS',
#attach_query_result_as_file = 1,
#query_attachment_filename='filename.xls', --- XLS worked. XLSX shows as "corrupted"
#query_result_separator=' ' -- tab
Related
In our lab we have a system that many people uses and produces a oddly shaped txt file with the results. I created a power query that cleans the file and I would like to share this with others (not very computer savvy) so they can apply it to the files they will generate.
What can I do to make it as easy as possible for other users to select the file they want the query to be applied to? Example: is there an easy way to create button that opens a dialog requesting the file location? Right now I have to edit the query source to select the data, this approach is clunky and will be confusing for some of my colleagues.
let
Source = Table.FromColumns({Lines.FromBinary(File.Contents("X:\foo\foo.txt"), null, null, 1252)}),
#"Removed Top Rows2" = Table.Skip(Source,32),
#"Removed Bottom Rows" = Table.RemoveLastN(#"Removed Top Rows2",16),
#"Other Steps" = ...
Thanks!
You can directly grab a filepath from a range name cell without a function by
let
NameValue= Excel.CurrentWorkbook(){[Name="rangenamehere"]}[Content]{0}[Column1],
Source = Table.FromColumns({Lines.FromBinary(File.Contents(NameValue), null, null, 1252)}),
Or if you wanted the VBA route for file prompt
1 Create a range name, here aaa
2 Use VBA to populate it using a file prompt
Sub prompt()
Dim FName As Variant
FName = Application.GetSaveAsFilename("", "Data file (*.xl*),*.xl*", 1)
If FName = False Then
MsgBox "False"
Exit Sub
End If
Range("aaa").Value = FName
End Sub
3 Refer to the named range in powerquery you set up
let
NameValue= Excel.CurrentWorkbook(){[Name="aaa"]}[Content]{0}[Column1],
Source = Table.FromColumns({Lines.FromBinary(File.Contents(NameValue), null, null, 1252)}),
4 Tack on code at end of VBA to refresh all queries or specific query
ActiveWorkbook.RefreshAll
or
ActiveWorkbook.Queries("QueryNameHere").Refresh
I found this post from 2014 that works pretty well. You write a function on Query (fnGetParameter) that reads the file location from a table and then you feed it to the query that processes the data.
All the user needs to do is write the file location on the table and name and refresh.
I changed the first to lines on my PowerQuery code to look like this:
Fileloc = fnGetParameter("File Path"),
Source = Table.FromColumns({Lines.FromBinary(File.Contents(Fileloc), null, null, 1252)}),
Any suggestions to make it even better?
You can make the fnGetParameter into a one-liner:
= ( getValue as text ) => Excel.CurrentWorkbook(){[Name=”Parameters”]}[Content]{[Parameter=getValue]}?[Value]?
I have created a query which takes data from a folder, and orders the data in such a way that I need..
The problem is that it won't work on MacOs.. The get from folder function does not exist in MacOs, I understood that I need to use VBA in order to do the same thing on MacOs..
I have no clue on how to use VBA
These are some of the steps in my power query:
= Folder.Files("C:\Users\location..")
= Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true)
= Table.AddColumn(#"Filtered Hidden Files1", "Transform File", each #"Transform File"([Content]))
And so on..
How could I mimic these in MacOs with VBA?
So here is a proposed solution, but unfortunately it runs into permission problems, which maybe someone else can help with.
First have three cells in your spreadsheet. Use the Excel name manager to name them OS, FilePath and FileName respectively. Their content is:
OS: =LET(vers,INFO("OSVERSION"),IFERROR(LEFT(vers,FIND(" ",vers)-1),vers))
FilePath: =IF(A1="Windows","C:\...\","/Users/.../")
FileName= report.csv
(obviously subsititute the actual windows & mac filepaths above)
As the start of your Power Query M code, use:
let
FilePath= Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FileName= Excel.CurrentWorkbook(){[Name="FileName"]}[Content]{0}[Column1],
Source = Csv.Document(File.Contents(FilePath & FileName), [Delimiter = ",", Columns = 12, Encoding = 65001, QuoteStyle = QuoteStyle.None])
in
Source
This works fine on the Windows side - I am having some problems on Mac, getting an error such as:
Query 'FetchReport (2)' (step 'AutoRemovedColumns1') is accessing data sources that have privacy levels which cannot be used together. Please rebuild this data combination.0:
I have a worksheet DATA with the table populated from json file through the Microsoft Query.
There're different json files so I need to create several connections to any of those files.
I also have a cell on another worksheet where I would like to indicate a parameter (for example Yesterday,Today,Tomorrow).
According to selected parameter the table in the DATA worksheet should be populated from the related data connection (yesterday.json, today.json, tomorrow.json).
Is it possible to do it? If yes, what would be the procedure?
I have an idea that it might be possible to do by changing the filename inside the query.
For example, this is my query:
let
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\today.json",
Source = Json.Document(File.Contents(FullPathToFile1)),
So am thinking if there's some way to "inject" filename in the above query based on value of some cell.
Will appreciate any help, links etc.
Thanks!
UPDATE:
I have created a named cell jsonPath and put the file name in it.
Then I have modified above query as follows, but it gives me an error.
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\" & [jsonPath],
Source = Json.Document(File.Contents(FullPathToFile1)),
I got it working by modifying my query as follows:
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FileName = Excel.CurrentWorkbook(){[Name="jsonPath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\" & FileName,
Source = Json.Document(File.Contents(FullPathToFile1)),
Hello and good day to all,
I am trying to import some excel sheets having multiple tabs into the matlab. For this I have written small loop but after many tries and adjustments still cant get all the data into matlab. In the source directory I have 15 excel sheets with 8 tabs each containing data in the 52 x 102 cells. The data is in signed form meaning containing positive and negative values. Here below is the code I was working on and I applied different changes which I found on the internet but no success.
srcdir = 'path to the folder';
srcfiles = dir(fullfile(srcdir, '*.xls'));
for i = 1:length(srcfiles)
[status,sheets] = xlsfinfo(srcfiles(i));
for s = 1:numel(sheets)
[data,titles]=xlsread(srcfiles(i).name,sheets{s});
end
end
Right now I am getting this error " Filename must be a string". I even tried to change it to the char to string but still didn't work.
Only once it worked When I instead of giving the path in the source directory i.e srcdir, gave the name of the file directly in xlsread().
Can anybody help where am I doing mistakes?. Thank you
You likely need to specify the full path to the file using fullfile. Also, in the outer for loop you'll need to use srcfiles(i).name instead of srcfiles(i)
srcdir = 'path to the folder';
srcfiles = dir(fullfile(srcdir, '*.xls'));
for k = 1:numel(srcfiles)
filename = fullfile(srcdir, srcfiles(k).name);
[status,sheets] = xlsfinfo(filename);
for s = 1:numel(sheets)
[data,titles] = xlsread(filename, sheets{s});
end
end
Currently I have an application that takes information from a SQLite database and puts it to Excel. However, I'm having to take each DataRow, iterate through each item, and put each value into it's own cell and determine highlighting. What this is causing is 20 minutes to export a 9000 record file into Excel. I'm sure it can be done quicker than that. My thoughts are that I could use a data source to fill the Excel Range and then use the column headers and row numbers to format only those rows that need to be formatted. However, when I look online, no matter what I seem to type, it always shows examples of using Excel as a database, nothing about importing into excel. Unless I'm forgetting a key word or to. Now, this function has to be done in code as it's part of a bigger application. Otherwise I would just have Excel connect to the DB and pull the information itself. Unfortunately that's not the case. Any information that could assist me in quick loading an excel sheet would be appreciated. Thanks.Additional Information:Another reason why the pulling of the information from the DB has to be done in code is that not every computer this is loaded on will have Excel on it. The person using the application may simply be told to export the data and email it to their supervisor. The setup app includes the needed dlls for the application to make the proper format.Example Code (Current):
For Each strTemp In strColumns
excelRange = worksheet.Cells(1, nCounter)
excelRange.Select()
excelRange.Value2 = strTemp
excelRange.Interior.Color = System.Drawing.Color.Gray.ToArgb()
excelRange.BorderAround(Excel.XlLineStyle.xlContinuous, Excel.XlBorderWeight.xlThin, Excel.XlColorIndex.xlColorIndexAutomatic, Type.Missing)
nCounter += 1
Next
Now, this is only example code in terms of the iteration I'm doing. Where I'm really processing the information from the database I'm iterating through a dataTable's Rows, then iterating through the items in the dataRow and doing essentially the same as above; value by value, selecting the range and putting the value in the cell, formatting the cell if it's part of a report (not always gray), and moving onto the next set of data. What I'd like to do is put all of the data in the excel sheet (A2:??, not a row, but multiple rows) then iterate through the reports and format each row then. That way, the only time I iterate through all of the records is when every record is part of a report.
Ideal Code
excelRange = worksheet.Cells("A2", "P9000")
excelRange.DataSource = ds 'ds would be a queried dataSet, and I know there is no excelRange.DataSource.
'Iteration code to format cells
Update:
I know my examples were in VB, but it's because I was also trying to write a VB version of the application since my boss prefers VB. However, here's my final code using a Recordset. The ConvertToRecordset function was obtained from here.
private void CreatePartSheet(Excel.Worksheet excelWorksheet)
{
_dataFactory.RevertDatabase();
excelWorksheet.Name = "Part Sheet";
string[] strColumns = Constants.strExcelPartHeaders;
CreateSheetHeader(excelWorksheet, strColumns);
System.Drawing.Color clrPink = System.Drawing.Color.FromArgb(203, 192, 255);
System.Drawing.Color clrGreen = System.Drawing.Color.FromArgb(100, 225, 137);
string[] strValuesAndTitles = {/*...Column Names...*/};
List<string> lstColumns = strValuesAndTitles.ToList<string>();
System.Data.DataSet ds = _dataFactory.GetDataSet(Queries.strExport);
ADODB.Recordset rs = ConvertToRecordset(ds.Tables[0]);
excelRange = excelWorksheet.get_Range("A2", "ZZ" + rs.RecordCount.ToString());
excelRange.Cells.CopyFromRecordset(rs, rs.RecordCount, rs.Fields.Count);
int nFieldCount = rs.Fields.Count;
for (int nCounter = 0; nCounter < rs.RecordCount; nCounter++)
{
int nRowCounter = nCounter + 2;
List<ReportRecord> rrPartReports = _lstReports.FindAll(rr => rr.PartID == nCounter).ToList<ReportRecord>();
excelRange = (Excel.Range)excelWorksheet.get_Range("A" + nRowCounter.ToString(), "K" + nRowCounter.ToString());
excelRange.Select();
excelRange.NumberFormat = "#";
if (rrPartReports.Count > 0)
{
excelRange.Interior.Color = System.Drawing.Color.FromArgb(230, 216, 173).ToArgb(); //Light Blue
foreach (ReportRecord rr in rrPartReports)
{
if (lstColumns.Contains(rr.Title))
{
excelRange = (Excel.Range)excelWorksheet.Cells[nRowCounter, lstColumns.IndexOf(rr.Title) + 1];
excelRange.Interior.Color = rr.Description.ToUpper().Contains("TAG") ? clrGreen.ToArgb() : clrPink.ToArgb();
if (rr.Description.ToUpper().Contains("TAG"))
{
rs.Find("PART_ID=" + (nCounter + 1).ToString(), 0, ADODB.SearchDirectionEnum.adSearchForward, "");
excelRange.AddComment(Environment.UserName + ": " + _dataFactory.GetTaggedPartPrevValue(rs.Fields["POSITION"].Value.ToString(), rr.Title));
}
}
}
}
if (nRowCounter++ % 500 == 0)
{
progress.ProgressComplete = ((double)nRowCounter / (double)rs.RecordCount) * (double)100;
Notify();
}
}
rs.Close();
excelWorksheet.Columns.AutoFit();
progress.Message = "Done Exporting to Excel";
Notify();
_dataFactory.RestoreDatabase();
}
Can you use ODBC?
''http://www.ch-werner.de/sqliteodbc/
dbName = "c:\docs\test"
scn = "DRIVER=SQLite3 ODBC Driver;Database=" & dbName _
& ";LongNames=0;Timeout=1000;NoTXN=0;SyncPragma=NORMAL;StepAPI=0;"
Set cn = CreateObject("ADODB.Connection")
cn.Open scn
Set rs = CreateObject("ADODB.Recordset")
rs.Open "select * from test", cn
Worksheets("Sheet3").Cells(2, 1).CopyFromRecordset rs
BTW, Excel is quite happy with HTML and internal style sheets.
I have used the Excel XML file format in the past to write directly to an output file or stream. It may not be appropriate for your application, but writing XML is much faster and bypasses the overhead of interacting with the Excel Application. Check out this Introduction to Excel XML post.
Update:
There are also a number of libraries (free and commercial) which can make creating excel document easier for example excellibrary which doesn't support the new format yet. There are others mentioned in the answers to Create Excel (.XLS and .XLSX) file from C#
Excel has the facility to write all the data from a ADO or DAO recordset in a single operation using the CopyFromRecordset method.
Code snippet:
Sheets("Sheet1").Range("A1").CopyFromRecordset rst
I'd normally recommend using Excel to pull in the data from SQLite. Use Excel's "Other Data Sources". You could then choose your OLE DB provider, use a connection string, what-have-you.
It sounds, however, that the real value of your code is the formatting of the cells, rather than the transfer of data.
Perhaps refactor the process to:
have Excel import the data
use your code to open the Excel spreadsheet, and apply formatting
I'm not sure if that is an appropriate set of processes for you, but perhaps something to consider?
Try this out:
http://office.microsoft.com/en-au/excel-help/use-microsoft-query-to-retrieve-external-data-HA010099664.aspx
Perhaps post some code, and we might be able to track down any issues.
I'd consider this chain of events:
query the SQLite database for your dataset.
move the data out of ADO.NET objects, and into POCO objects. Stop using DataTables/Rows.
use For Each to insert into Excel.