I am trying to get the last modification date of database content, to know when to refresh some data. I thought this would work:
Static ADOCat As ADOX.Catalog
If Not ADOCat Is Nothing Then Set ADOCat = New ADOX.Catalog
If ADOCat.ActiveConnection Is Nothing Then ADOCat.ActiveConnection = "Provider=Microsoft.ace.OLEDB.12.0; Data Source=X:\MANUAL_ANALYSIS.accdb; Mode=Read"
If LastModificationDate <> ADOCat.Tables("ANALYSES").DateModified Then
bRefreshData = True
LastModificationDate = ADOCat.Tables("ANALYSES").DateModified
End If
However, the .DataModified returns the date when the table structure was changed, such as removing/adding fields, not the database content modification. Is there another method that can find this out for me?
/J
Related
Wow, my first stack question despite using the answers for years. Very exciting.
I'm fairly new to VBA and Excel and entirely new to Access, full disclosure. So Im trying to create a core database of lab reports, and I have a form for entering the information about a new report which adds info about the report to a master table of all reports, including assigning it a unique label. After entering the info, I then have a button which allows the user to select the Excel .csv file accompanying the report and imports it into the DB as a new table. It returns a success or error message. And it works! (And the code came from somewhere on here)
The problem is I'd like to then add a field to the new table that adds the label assigned to the new report to all records so it can be referenced by queries through the use of that label. I'd also like to add an index field to the new table if possible as it doesn't seem like importing the .csv as a table creates an index. I figure I'll make another sub that gets passed the new report name as a name for the new field (which will also be the value of the field through all records) and the table to append that to.
How do I pass this sub the newly imported table if I just imported it? I need this all to work from the button as it will mostly be my manager using this form/button to import new files, and they won't be able to just manually go into the tables as they are created and add fields (yes, I know that's the obvious solution, but trust me...this must be a button)
Heres the code I'm using (yes, I know lots of it could be done differently but it works!)
Public Function ImportDocument() As String
On Error GoTo ErrProc
Const msoFileDIalogFilePicker As Long = 3
Dim fd As Object
Set fd = Application.FileDialog(msoFileDIalogFilePicker)
With fd
.InitialFileName = "Data Folder"
.Title = "Enthalpy EDD Import"
With .Filters
.Clear
.Add "Excel documents", "*.xlsx; *.csv", 1
End With
.ButtonName = " Import Selected "
.AllowMultiSelect = False 'Manual naming currently requires one file at a time be imported
'If aborted, the Function will return the default value of Aborted
If .Show = 0 Then GoTo Leave 'fb.show returns 0 if 'cancel' is pressed
End With
Dim selectedItem As Variant
Dim NewTableName As String
NewTableName = InputBox(Prompt:="Enter the Report Name", _
Title:="Report Name")
For Each selectedItem In fd.SelectedItems 'could later be adapted for multiple imports
DoCmd.TransferText acImportDelim, , NewTableName, selectedItem, True 'Imports csv file selected, true is 'has headers'
Next selectedItem
'Return Success
ImportDocument = "Success"
'Append report label and index
AppendReportLabelField(NewTableName, #What to put here as the table to append to?)
'error handling
Leave:
Set fd = Nothing
On Error GoTo 0
Exit Function
ErrProc:
MsgBox Err.Description, vbCritical
ImportDocument = "Failure" 'Return Failure if error
Resume Leave
End Function
The AppendReportLabelField would get passed the name (and value) of the field and the name of the (newly imported) table. How do I pass it the table? NewTableName is just a string currently. If I can pass the new sub the table I'm sure the rest will be simple.
Thanks in advance for the help!
Consider storing all user input data in a single master table with all possible fields and use a temp, staging table (a replica of master) to migrate CSV table to this master table. During the staging, you can update the table with needed fields.
SQL (save as stored queries)
(parameterized update query)
PARAMETERS [ParamReportNameField] TEXT;
UPDATE temptable
SET ReportNameField = [ParamReportNameField]
(explicitly reference all columns)
INSERT INTO mastertable (Col1, Col2, Col3, ...)
SELECT Col1, Col2, Col3
FROM temptable
VBA
...
' PROCESS EACH CSV IN SUBSEQUENT SUBROUTINE
For Each selectedItem In fd.SelectedItems
Call upload_process(selectedItem, report_name)
Next selectedItem
Sub upload_process(csv_file, report_name)
' CLEAN OUT TEMP TABLE
CurrentDb.Execute "DELETE FROM myTempStagingTable"
' IMPORT CSV INTO TEMP TABLE
DoCmd.TransferText acImportDelim, , "myTempStagingTable", csv_file, True
' RUN UPDATES ON TEMP TABLE
With CurrentDb.QueryDefs("myParameterizedUpdateQuery")
.Parameters("ParamReportNameField").Value = report_name
.Execute dbFailOnError
End With
' RUNS APPEND QUERY (TEMP -> MASTER)
CurrentDb.Execute "myAppendQuery"
End Sub
If CSV uploads vary widely in data structure, then incorporate an Excel cleaning step to standardize all inputs. Alternatively, force users to use a standardized template. Staging can be used to validate uploads. Databases should not be a repository of many, dissimilar tables but part of a relational model in a pre-designed setup. Running open-ended processes like creating new tables by users on the fly can cause maintenance issues.
I have a worksheet DATA with the table populated from json file through the Microsoft Query.
There're different json files so I need to create several connections to any of those files.
I also have a cell on another worksheet where I would like to indicate a parameter (for example Yesterday,Today,Tomorrow).
According to selected parameter the table in the DATA worksheet should be populated from the related data connection (yesterday.json, today.json, tomorrow.json).
Is it possible to do it? If yes, what would be the procedure?
I have an idea that it might be possible to do by changing the filename inside the query.
For example, this is my query:
let
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\today.json",
Source = Json.Document(File.Contents(FullPathToFile1)),
So am thinking if there's some way to "inject" filename in the above query based on value of some cell.
Will appreciate any help, links etc.
Thanks!
UPDATE:
I have created a named cell jsonPath and put the file name in it.
Then I have modified above query as follows, but it gives me an error.
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\" & [jsonPath],
Source = Json.Document(File.Contents(FullPathToFile1)),
I got it working by modifying my query as follows:
FilePath = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
FileName = Excel.CurrentWorkbook(){[Name="jsonPath"]}[Content]{0}[Column1],
FullPathToFile1 = FilePath & "\json\" & FileName,
Source = Json.Document(File.Contents(FullPathToFile1)),
Is there any way to read all the data from excel and put it in the datatable or any other container so that i can filter the data based on the conditions required. As shown in attached image i want to get the CuValue of a Partnumber whose status is Success and i want the latest record based on the Calculation date(Latest calculation date). In the below example i want the CuValue 11292 as it is the latest record with status Success..lue.
Thanks in advance
Your question seems very broad, but you're right to ask because there are many different possibilities and pitfalls.
As you don't provide any sample code, i assume you are looking for a strategy, so here is it.
In short: create a database, a table and a stored procedure. Copy the
data you need in this table, and then query the table to get the
result.
You may use ADO for this task. If it is not available on your machine you can download and install the MDAC redistributable from the Microsoft web site.
The advantage vs. OLE Automation is that you doesn't need to install Excel on the target machine where the import shall be executed, so you can execute the import also server-side.
With ADO installed, you will need to create two Connection objects, a Recordset object to read the data from the Excel file and a Command object to execute a stored procedure which will do the INSERT or the UPDATE of the subset of the source fields in the destination table.
Following is a guideline which you should expand and adjust, if you find it useful for your task:
Option Explicit
Dim PartNo as String, CuValue as Long, Status as String, CalcDate as Date
' objects you need:
Dim srcConn As New ADODB.Connection
Dim cmd As New ADODB.Command
Dim rs As New ADODB.Recordset
Dim dstConn As New ADODB.Connection
' Example connection with your destination database
dstConn.Open *your connection string*
'Example connection with Excel - HDR is discussed below
srcConn.Open "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=C:\Scripts\Test.xls;" & _
"Extended Properties=""Excel 8.0; HDR=NO;"";"
rs.Open "SELECT * FROM [Sheet1$]", _
srcConn, adOpenForwardOnly, adLockReadOnly, adCmdText
' Import
Do Until rs.EOF
PartNo = rs.Fields.Item(0);
CuValue = rs.Fields.Item(1);
CalcDate = rs.Fields.Item(6);
Status = rs.Fields.Item(7);
If Status = "Success" Then
'NumSuccess = NumSuccess + 1
' copy data to your database
' using a stored procedure
cmd.CommandText = "InsertWithDateCheck"
cmd.CommandType = adCmdStoredProc
cmd(1) = PartNo
cmd(2) = CuValue
cmd(3) = CalcDate
cmd.ActiveConnection = dstConn
cmd.Execute
Else
'NumFail = NumFail + 1
End If
rs.MoveNext
Loop
rs.Close
Set rs = Nothing
srcConn.Close
Set srcConn = Nothing
dstConn.Close
Set dstConn = Nothing
'
By using a stored procedure to check the data and execute the insert or update in your new table, you will be able to read from Excel in fast forward-only mode and write a copy of the data with the minimum of time loss, delegating to the database engine half the work.
You see, the stored procedure will receive three values. Inside the stored procedure you should insert or update this values. Primary key of the table shall be PartNo. Check the Calculation Date and, if more recent, update CuValue.
By googling on the net you will find enough samples to write such a stored procedure.
After your table is populated, just use another recordset to get the data and whatever tool you need to display the values.
Pitfalls reading from Excel:
The provider of your Excel file shall agree to remove the first two or three rows, otherwise you will have some more work for the creation of a fictitious recordset, because the intelligent datatype recognition of Excel may fail.
As you know, Excel cells are not constrained to the same data type per-column as in almost all databases.
If you maintain the field names, use HDR=YES, without all the first three rows, use HDR=NO.
Always keep a log of the "Success" and "Fail" number of records read
in your program, then compare these values with the original overall
number of rows in Excel.
Feel free to ask for more details, anyway i think this should be enough for you to start.
There are lots ways you can do this.
1. You can create an access DB table and import by saving your sheet as can file first, into the access table. Then you can write queries.
2. You can create a sql DB and a table, write some code to import the sheet into that table.
3. You can Write some code in VBA and accomplish that task if your data is not very big.
4. You can write c# code to access the sheet using excel.application and office objects, create a data table and query that data table
Depends on what skills you want to employ to accomplish your task.
I need to import data from SAS to excel via VBA. The import needs to run eg. on workbookOpen or Worksheet_BeforeDoubleClick or it can be called in any macro. This is solved in the below code:
Sub GetSASdata()
Dim obConnection As ADODB.Connection
Dim obRecordset As ADODB.Recordset
Dim i As Integer
Set obConnection = New ADODB.Connection
' Do not get stuck on the choice of connection provider.
obConnection.Provider = "sas.LocalProvider"
obConnection.Properties("Data Source") = "C:\path\"
obConnection.Open
Set obRecordset = New ADODB.Recordset
obRecordset.Open "MySAStable", obConnection, adOpenDynamic, adLockReadOnly, ADODB.adCmdTableDirect
'add header row
Cells(1, 1).Select
For i = 0 To obRecordset.Fields.Count - 1
ActiveCell.Offset(0, i).Value = obRecordset.Fields(i).Name
Next i
obRecordset.MoveFirst
obRecordset.Filter = "Weight > 0"
Cells(2, 1).Select
ActiveCell.CopyFromRecordset obRecordset, 100
obRecordset.Close
Set obRecordset = Nothing
obConnection.Close
Set obConnection = Nothing
End Sub
In this example I have restricted the output to be only the first 100 rows. However, the original data set is 1.4 m rows and 150 columns, and I want to be able to restrict the data import to only take columns that I define and rows which meet certain criteria. In sql terms:
select col1, col2, col10, col11 from MySAStable where code = MyCode and Date > MyDate
But I cannot find a way to do it. The first criteria is that the code should run entirely from Excel.
I have experimented some with obRecordset.Filter but the performance is poor. It takes forever. So idealy I would like to import only the data that I need. Is there a way to do this?
The
obConnection.Provider = "sas.LocalProvider"
is arbitrary. I found an example online, tested it and it worked. If someone has an answer to my problem that involves a different connection type, i am still interested to know. Very idealy the code can also be run by users who do not have SAS installed on their computer (but have access to the folder where data is placed.)
Thank you for any help
I have used two methods to read SAS data from within Excel.
The first uses SAS Add-In to MS Office. Do you have this product?
You can define the source with filters, and when the user opens the workbook, it will automatically refresh agains the datasource. You can also automate the refresh task with VBA code.
Secondly, I have done it with a Stored Process. If you have a stored process server, you can set up a web query in Excel and read the Stored Process that way, using any filter you need.
Currently I have an application that takes information from a SQLite database and puts it to Excel. However, I'm having to take each DataRow, iterate through each item, and put each value into it's own cell and determine highlighting. What this is causing is 20 minutes to export a 9000 record file into Excel. I'm sure it can be done quicker than that. My thoughts are that I could use a data source to fill the Excel Range and then use the column headers and row numbers to format only those rows that need to be formatted. However, when I look online, no matter what I seem to type, it always shows examples of using Excel as a database, nothing about importing into excel. Unless I'm forgetting a key word or to. Now, this function has to be done in code as it's part of a bigger application. Otherwise I would just have Excel connect to the DB and pull the information itself. Unfortunately that's not the case. Any information that could assist me in quick loading an excel sheet would be appreciated. Thanks.Additional Information:Another reason why the pulling of the information from the DB has to be done in code is that not every computer this is loaded on will have Excel on it. The person using the application may simply be told to export the data and email it to their supervisor. The setup app includes the needed dlls for the application to make the proper format.Example Code (Current):
For Each strTemp In strColumns
excelRange = worksheet.Cells(1, nCounter)
excelRange.Select()
excelRange.Value2 = strTemp
excelRange.Interior.Color = System.Drawing.Color.Gray.ToArgb()
excelRange.BorderAround(Excel.XlLineStyle.xlContinuous, Excel.XlBorderWeight.xlThin, Excel.XlColorIndex.xlColorIndexAutomatic, Type.Missing)
nCounter += 1
Next
Now, this is only example code in terms of the iteration I'm doing. Where I'm really processing the information from the database I'm iterating through a dataTable's Rows, then iterating through the items in the dataRow and doing essentially the same as above; value by value, selecting the range and putting the value in the cell, formatting the cell if it's part of a report (not always gray), and moving onto the next set of data. What I'd like to do is put all of the data in the excel sheet (A2:??, not a row, but multiple rows) then iterate through the reports and format each row then. That way, the only time I iterate through all of the records is when every record is part of a report.
Ideal Code
excelRange = worksheet.Cells("A2", "P9000")
excelRange.DataSource = ds 'ds would be a queried dataSet, and I know there is no excelRange.DataSource.
'Iteration code to format cells
Update:
I know my examples were in VB, but it's because I was also trying to write a VB version of the application since my boss prefers VB. However, here's my final code using a Recordset. The ConvertToRecordset function was obtained from here.
private void CreatePartSheet(Excel.Worksheet excelWorksheet)
{
_dataFactory.RevertDatabase();
excelWorksheet.Name = "Part Sheet";
string[] strColumns = Constants.strExcelPartHeaders;
CreateSheetHeader(excelWorksheet, strColumns);
System.Drawing.Color clrPink = System.Drawing.Color.FromArgb(203, 192, 255);
System.Drawing.Color clrGreen = System.Drawing.Color.FromArgb(100, 225, 137);
string[] strValuesAndTitles = {/*...Column Names...*/};
List<string> lstColumns = strValuesAndTitles.ToList<string>();
System.Data.DataSet ds = _dataFactory.GetDataSet(Queries.strExport);
ADODB.Recordset rs = ConvertToRecordset(ds.Tables[0]);
excelRange = excelWorksheet.get_Range("A2", "ZZ" + rs.RecordCount.ToString());
excelRange.Cells.CopyFromRecordset(rs, rs.RecordCount, rs.Fields.Count);
int nFieldCount = rs.Fields.Count;
for (int nCounter = 0; nCounter < rs.RecordCount; nCounter++)
{
int nRowCounter = nCounter + 2;
List<ReportRecord> rrPartReports = _lstReports.FindAll(rr => rr.PartID == nCounter).ToList<ReportRecord>();
excelRange = (Excel.Range)excelWorksheet.get_Range("A" + nRowCounter.ToString(), "K" + nRowCounter.ToString());
excelRange.Select();
excelRange.NumberFormat = "#";
if (rrPartReports.Count > 0)
{
excelRange.Interior.Color = System.Drawing.Color.FromArgb(230, 216, 173).ToArgb(); //Light Blue
foreach (ReportRecord rr in rrPartReports)
{
if (lstColumns.Contains(rr.Title))
{
excelRange = (Excel.Range)excelWorksheet.Cells[nRowCounter, lstColumns.IndexOf(rr.Title) + 1];
excelRange.Interior.Color = rr.Description.ToUpper().Contains("TAG") ? clrGreen.ToArgb() : clrPink.ToArgb();
if (rr.Description.ToUpper().Contains("TAG"))
{
rs.Find("PART_ID=" + (nCounter + 1).ToString(), 0, ADODB.SearchDirectionEnum.adSearchForward, "");
excelRange.AddComment(Environment.UserName + ": " + _dataFactory.GetTaggedPartPrevValue(rs.Fields["POSITION"].Value.ToString(), rr.Title));
}
}
}
}
if (nRowCounter++ % 500 == 0)
{
progress.ProgressComplete = ((double)nRowCounter / (double)rs.RecordCount) * (double)100;
Notify();
}
}
rs.Close();
excelWorksheet.Columns.AutoFit();
progress.Message = "Done Exporting to Excel";
Notify();
_dataFactory.RestoreDatabase();
}
Can you use ODBC?
''http://www.ch-werner.de/sqliteodbc/
dbName = "c:\docs\test"
scn = "DRIVER=SQLite3 ODBC Driver;Database=" & dbName _
& ";LongNames=0;Timeout=1000;NoTXN=0;SyncPragma=NORMAL;StepAPI=0;"
Set cn = CreateObject("ADODB.Connection")
cn.Open scn
Set rs = CreateObject("ADODB.Recordset")
rs.Open "select * from test", cn
Worksheets("Sheet3").Cells(2, 1).CopyFromRecordset rs
BTW, Excel is quite happy with HTML and internal style sheets.
I have used the Excel XML file format in the past to write directly to an output file or stream. It may not be appropriate for your application, but writing XML is much faster and bypasses the overhead of interacting with the Excel Application. Check out this Introduction to Excel XML post.
Update:
There are also a number of libraries (free and commercial) which can make creating excel document easier for example excellibrary which doesn't support the new format yet. There are others mentioned in the answers to Create Excel (.XLS and .XLSX) file from C#
Excel has the facility to write all the data from a ADO or DAO recordset in a single operation using the CopyFromRecordset method.
Code snippet:
Sheets("Sheet1").Range("A1").CopyFromRecordset rst
I'd normally recommend using Excel to pull in the data from SQLite. Use Excel's "Other Data Sources". You could then choose your OLE DB provider, use a connection string, what-have-you.
It sounds, however, that the real value of your code is the formatting of the cells, rather than the transfer of data.
Perhaps refactor the process to:
have Excel import the data
use your code to open the Excel spreadsheet, and apply formatting
I'm not sure if that is an appropriate set of processes for you, but perhaps something to consider?
Try this out:
http://office.microsoft.com/en-au/excel-help/use-microsoft-query-to-retrieve-external-data-HA010099664.aspx
Perhaps post some code, and we might be able to track down any issues.
I'd consider this chain of events:
query the SQLite database for your dataset.
move the data out of ADO.NET objects, and into POCO objects. Stop using DataTables/Rows.
use For Each to insert into Excel.