Is there any way to read all the data from excel and put it in the datatable or any other container so that i can filter the data based on the conditions required. As shown in attached image i want to get the CuValue of a Partnumber whose status is Success and i want the latest record based on the Calculation date(Latest calculation date). In the below example i want the CuValue 11292 as it is the latest record with status Success..lue.
Thanks in advance
Your question seems very broad, but you're right to ask because there are many different possibilities and pitfalls.
As you don't provide any sample code, i assume you are looking for a strategy, so here is it.
In short: create a database, a table and a stored procedure. Copy the
data you need in this table, and then query the table to get the
result.
You may use ADO for this task. If it is not available on your machine you can download and install the MDAC redistributable from the Microsoft web site.
The advantage vs. OLE Automation is that you doesn't need to install Excel on the target machine where the import shall be executed, so you can execute the import also server-side.
With ADO installed, you will need to create two Connection objects, a Recordset object to read the data from the Excel file and a Command object to execute a stored procedure which will do the INSERT or the UPDATE of the subset of the source fields in the destination table.
Following is a guideline which you should expand and adjust, if you find it useful for your task:
Option Explicit
Dim PartNo as String, CuValue as Long, Status as String, CalcDate as Date
' objects you need:
Dim srcConn As New ADODB.Connection
Dim cmd As New ADODB.Command
Dim rs As New ADODB.Recordset
Dim dstConn As New ADODB.Connection
' Example connection with your destination database
dstConn.Open *your connection string*
'Example connection with Excel - HDR is discussed below
srcConn.Open "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=C:\Scripts\Test.xls;" & _
"Extended Properties=""Excel 8.0; HDR=NO;"";"
rs.Open "SELECT * FROM [Sheet1$]", _
srcConn, adOpenForwardOnly, adLockReadOnly, adCmdText
' Import
Do Until rs.EOF
PartNo = rs.Fields.Item(0);
CuValue = rs.Fields.Item(1);
CalcDate = rs.Fields.Item(6);
Status = rs.Fields.Item(7);
If Status = "Success" Then
'NumSuccess = NumSuccess + 1
' copy data to your database
' using a stored procedure
cmd.CommandText = "InsertWithDateCheck"
cmd.CommandType = adCmdStoredProc
cmd(1) = PartNo
cmd(2) = CuValue
cmd(3) = CalcDate
cmd.ActiveConnection = dstConn
cmd.Execute
Else
'NumFail = NumFail + 1
End If
rs.MoveNext
Loop
rs.Close
Set rs = Nothing
srcConn.Close
Set srcConn = Nothing
dstConn.Close
Set dstConn = Nothing
'
By using a stored procedure to check the data and execute the insert or update in your new table, you will be able to read from Excel in fast forward-only mode and write a copy of the data with the minimum of time loss, delegating to the database engine half the work.
You see, the stored procedure will receive three values. Inside the stored procedure you should insert or update this values. Primary key of the table shall be PartNo. Check the Calculation Date and, if more recent, update CuValue.
By googling on the net you will find enough samples to write such a stored procedure.
After your table is populated, just use another recordset to get the data and whatever tool you need to display the values.
Pitfalls reading from Excel:
The provider of your Excel file shall agree to remove the first two or three rows, otherwise you will have some more work for the creation of a fictitious recordset, because the intelligent datatype recognition of Excel may fail.
As you know, Excel cells are not constrained to the same data type per-column as in almost all databases.
If you maintain the field names, use HDR=YES, without all the first three rows, use HDR=NO.
Always keep a log of the "Success" and "Fail" number of records read
in your program, then compare these values with the original overall
number of rows in Excel.
Feel free to ask for more details, anyway i think this should be enough for you to start.
There are lots ways you can do this.
1. You can create an access DB table and import by saving your sheet as can file first, into the access table. Then you can write queries.
2. You can create a sql DB and a table, write some code to import the sheet into that table.
3. You can Write some code in VBA and accomplish that task if your data is not very big.
4. You can write c# code to access the sheet using excel.application and office objects, create a data table and query that data table
Depends on what skills you want to employ to accomplish your task.
Related
Very frustrating results from:
ws.Range(myExcelTable).CopyFromRecordset rs
' myExcelTable is a named table with assigned range
The pull for rs (the recordset, is a select all, doesn't matter if it is done this way or with field names, it is the same result)
cmdSQL2 = "SELECT * FROM " & myAccessTable
rs.Open cmdSQL2, conn
For an unknown reason it copies records into excel in this order (using pID to show ordering):
3-8
0-2
9-number of records
I have tried several range methods and the result is the same.
I will "hand bomb" them here to cut out the extra work I performed in building range strings.
ws.Range("S4").CopyFromRecordset rs
ws.Range("S4:BP258").CopyFromRecordset rs
This behavior does not appear if the table you are copying from Access has only one Field (column) and the Table in Excel has a single header (column), but it is appearing for me now in this multi-field table.
I feel this is happening when the recordset is actually created somehow and not on the "paste" or assignment side (Excel).
'*************************************************************************
Problem Solved by this SO Contributor (will leave here until it is placed as an answer):
https://stackoverflow.com/users/7296893/erik-a
Using ORDER BY in the sql sommand:
cmdSQL2 = "SELECT * FROM " & myAccessTable & " ORDER BY " & primaryKey
From immediate window (removes the variables so you can see the command better for those new to such things:
SELECT * FROM County_Directory ORDER BY [pID]
'*************************************************************************
Preface
I'm making this question specific to conform to SO asking guidelines, but feel free to suggest wholesale redesign if you wish. I may be using some bad practices.
Basic Question
I use ADO to execute a multi-step SQL Server query that takes several minutes to execute. I use Raiserror in my tsql queries to let myself know more verbosely which steps have finished. Is it possible to pass these messages to VBA before the complete query finishes, while still continuing with the query?
Details and Code
I use the vba below to execute the t-SQL query underneath. As you can see, there are two errors raised in the t-SQL that display "Step 1 complete" and "Step 2 complete". Could I pass these messages (or alternately use error numbers and pass those) back to VBA in a way that would allow me to detect them and update a progress bar while continuing to execute the query?
VBA used to execute the query:
Set cmd = New ADODB.Command
cmd.ActiveConnection = cnn
cmd.CommandTimeout = 0
cmd.CommandText = strQuery
Set rst = New ADODB.Recordset
rst.Open cmd
'Go to the second to last recordset of the multi-step query
String1 = Replace(strQuery, ";", "")
For Loop2 = 1 To (Len(strQuery) - (Len(String1) + 1))
Set rst = rst.NextRecordset
Next Loop2
'Copy results
If Not rst.EOF Then
(snip - actions)
Else
MsgBox "Error: No records returned."
End If
Stripped-down piece of multi-step tSQL query:
--#DRS1: The numbers being researched
select distinct numbers
into #DRS1
from Table1 (nolock)
where numbers in ()
--#DRS1: Index
create nonclustered index Idx_DRS1
on #DRS1(numbers);
Raiserror(“Step 1 complete”,1,1) with nowait;
--#DRS2: Table2 for numbers being researched
select distinct
DRS1.numbers
,a.ID
into #DRS2
from #DRS1 DRS1
join Table2 (nolock) a
on DRS1.numbers = a.numbers
Raiserror(“Step 2 complete”,1,1) with nowait;
--MORE STEPS
(more steps)
(more raiserror statements)
Clarification
I am not interested in:
A method that doesn't allow me to update a progress bar until the query is completely done.
A method that uses Progress/MaxProgress, because as I understand it that would return separate numbers for each of the steps in my query, rather than one progress measure for the entire query.
I am less interested in:
Using # records affected messages to determine progress, because some steps may return equal numbers of records to previous steps.
Research
The closest thing I have found to what I'm looking for is here, but as the discussion of that solution here says:
This approach would only work for stored procedures that are not intended to return results, say procs that insert data into tables. Another approach would be needed if your stored proc returns a result set.
Since I return results in the final step of my query to be manipulated in Excel I don't think this would work for me.
External link code for reference
SQL:
CREATE PROCEDURE dbo.updTesting As
Declare #RetVal integer
Exec #RetVal = updTesting2
Return #RetVal
GO
CREATE PROCEDURE dbo.updTesting2 As
raiserror('Error From Testing 2 procedure',16,1)
Return -2
GO
VBA:
Private Sub Command1_Click()
On Error GoTo ErrorHandler
Dim db As ADODB.Connection
Dim cmd As ADODB.Command
Set db = New ADODB.Connection
db.CursorLocation = adUseClient
db.Open "provider=sqloledb;data source=handel;initial catalog=northwind;integrated security=sspi"
Set cmd = New ADODB.Command
With cmd
Set .ActiveConnection = db
.CommandText = "updTesting"
.CommandType = adCmdStoredProc
.Parameters.Append .CreateParameter("#RetVal", adInteger, adParamReturnValue)
.Execute , , adExecuteNoRecords
End With
ExitPoint:
On Error Resume Next
Set cmd.ActiveConnection = Nothing
Set cmd = Nothing
db.Close
Set db = Nothing
Exit Sub
ErrorHandler:
MsgBox "Error # " & Err.Number & vbNewLine & vbNewLine & Err.Description
Resume ExitPoint
End Sub
There are several possibilities to work out a solution for your problem:
(1) Capture the error messages as they occur while the query is running. That's the requested approach.
(2) Break-down the big, long query into several smaller chunks and run them one after the other. Like this you know which part is completed and you can update your progress bar based on that information just before sending the next chunk to the server.
(3) Update the big, long query to log its progress on the server in a temp table and then read out this log while the other query is still running.
While I'd recommend to use errors only when errors occur and not to "abuse" them for logging, tracking, or feedback, both options (1 & 2) are quite feasible with events:
Similar to Worksheet events Worksheet_Change, Worksheet_Activate, or Worksheet_BeforeDoubleClick there are also ADODB events for ADODB.Connection and ADODB.Recordset. Both are well documented and can be easily viewed within the VBE by (1) adding a reference to Microsoft ActiveX Data Objects x.x Library (2) pressing F2 (3) selecting the ADODB library in the drop-down menu at the top (4) and finally looking up Recordset or Connection within the classes. Here are the available events for Connection:
As you can see, all events are marked with a lightning. To capture / use these events you need to create a Class Module in the VBE and add the following line to it:
Dim WithEvents adoConnection As ADODB.Connection
Afterwards, you can make use of the newly created ADODB.Connection event and select the required event from the top of the list:
The applicable event for option (1) is the InfoMessage event which occurs "[...] whenever a warning occurs during a ConnectionEvent operation." The import part here is during a connection. So, this event fires automatically whenever an ADODB connection "gets" an error.
Of course, this means that the original query to the server must be sent without waiting for an answer. Instead you should use the above event to capture any errors while the query is executing and create yet another event to automatically fire when the entire query completed.
For some more help in respect to asynchronous ADODB connection and possible problems with them you might want to have a look at the following two posts here:
ExecuteComplete ADODB Connection event not fired with adAsyncExecute parameter
Running multiple async queries with ADODB - callbacks not always firing
A similar approach can be used with option (3) as described above and asynchronous ADODB connections.
Let me know if this solves your problems or if you have any further questions.
All available ADODB events can be reviewed here https://msdn.microsoft.com/en-us/library/ms675083%28v=vs.85%29.aspx
I need to import data from SAS to excel via VBA. The import needs to run eg. on workbookOpen or Worksheet_BeforeDoubleClick or it can be called in any macro. This is solved in the below code:
Sub GetSASdata()
Dim obConnection As ADODB.Connection
Dim obRecordset As ADODB.Recordset
Dim i As Integer
Set obConnection = New ADODB.Connection
' Do not get stuck on the choice of connection provider.
obConnection.Provider = "sas.LocalProvider"
obConnection.Properties("Data Source") = "C:\path\"
obConnection.Open
Set obRecordset = New ADODB.Recordset
obRecordset.Open "MySAStable", obConnection, adOpenDynamic, adLockReadOnly, ADODB.adCmdTableDirect
'add header row
Cells(1, 1).Select
For i = 0 To obRecordset.Fields.Count - 1
ActiveCell.Offset(0, i).Value = obRecordset.Fields(i).Name
Next i
obRecordset.MoveFirst
obRecordset.Filter = "Weight > 0"
Cells(2, 1).Select
ActiveCell.CopyFromRecordset obRecordset, 100
obRecordset.Close
Set obRecordset = Nothing
obConnection.Close
Set obConnection = Nothing
End Sub
In this example I have restricted the output to be only the first 100 rows. However, the original data set is 1.4 m rows and 150 columns, and I want to be able to restrict the data import to only take columns that I define and rows which meet certain criteria. In sql terms:
select col1, col2, col10, col11 from MySAStable where code = MyCode and Date > MyDate
But I cannot find a way to do it. The first criteria is that the code should run entirely from Excel.
I have experimented some with obRecordset.Filter but the performance is poor. It takes forever. So idealy I would like to import only the data that I need. Is there a way to do this?
The
obConnection.Provider = "sas.LocalProvider"
is arbitrary. I found an example online, tested it and it worked. If someone has an answer to my problem that involves a different connection type, i am still interested to know. Very idealy the code can also be run by users who do not have SAS installed on their computer (but have access to the folder where data is placed.)
Thank you for any help
I have used two methods to read SAS data from within Excel.
The first uses SAS Add-In to MS Office. Do you have this product?
You can define the source with filters, and when the user opens the workbook, it will automatically refresh agains the datasource. You can also automate the refresh task with VBA code.
Secondly, I have done it with a Stored Process. If you have a stored process server, you can set up a web query in Excel and read the Stored Process that way, using any filter you need.
I have an Excel Spreadsheet that calculates a risk (of perioperative mortality after aneurysm repair) based on various test results.
The user inputs the test results into the spreadsheet (into cells) and then out comes a set of figures (about 6 results) for the various models that predict mortality. The spreadsheet acts as a complex function to produce the results one patient at a time.
I also have a (separate) access database holding data on multiple patients - including all the data on test results that go into the spreadsheet. At the moment I have to manually input this data into the spreadsheet, get the results out and then manually enter them onto the database.
Is there a way of doing this automatically. Ie can I export data1, data2, data3... from Access into the spreadsheet to the cells where the data needs to be input and then get the results (result1, result2, result3...) from the cells where the results are displayed ported back into access.
Ideally this could be done live.
I suppose I could try to program the functionality of the spreadheet into a complex function in access, but if I'm honest, I am not really sure how the algorithm in the spreadsheet works. It was designed by anaesthetists who are much cleverer than me....
Hope this makes sense. Any help much appreciated.
Chris Hammond
It's possible to automate Excel from Access.
Const cstrFile As String = "C:\SomeFolder\foo.xls"
Dim xlApp As Object
Dim xlWrkBk As Object
Dim xlWrkSt As Object
Set xlApp = CreateObject("Excel.Application")
xlApp.Workbooks.Open cstrFile, ReadOnly:=True
Set xlWrkBk = xlApp.Workbooks(1)
Set xlWrkSt = xlWrkBk.Worksheets(1)
With xlWrkSt
.Range("A1") = 2
.Range("A2") = 19
Debug.Print .Range("A3")
End With
xlWrkBk.Close SaveChanges:=False
However, that seems like it would be cumbersome to repeat for each row of an Access table and I'm uncertain whether doing that live is reasonable.
I would try to adapt the Excel calculations to Access VBA functions and use those custom functions in an Access query. But I don't know how big of a task that would be. I suggest you shouldn't be scared off the the anaesthetists' cleverness; that doesn't mean they actually know much more about VBA than you. At least look to see whether you can tackle it.
To push the data back to Access, you can insert data from within the Excel VBA as follows:
dim val as variant
dim db as DAO.Database
val=thisworkbook.range("a1").value
set db=OpenDatabase("c:\myAccessDB.accdb")
db.execute "insert into patientData (someField) values (" & val & ")",dbFailOnError
db.Close
You'll need to add a reference to the Microsoft Office Access Database Engine Object Library.
Not sure to perfectly understand what you want, but if you just want to export the results of a query to a spreadsheet, you could use the following:
Private Sub ExportAccessDataToExcel()
Dim SqlString As String
SqlString = "CREATE TABLE testMeasurements (TestName TEXT, Status TEXT)"
DoCmd.RunSQL (SqlString)
SqlString = "INSERT INTO testMeasurements VALUES('Average Power','PASS')"
DoCmd.RunSQL (SqlString)
SqlString = "INSERT INTO testMeasurements VALUES('Power Vs Time','FAIL')"
DoCmd.RunSQL (SqlString)
SqlString = "SELECT testMeasurements.TestName, testMeasurements.Status INTO exportToExcel "
SqlString = SqlString & "FROM testMeasurements "
SqlString = SqlString & "WHERE (((testMeasurements.TestName)='Average Power'));"
DoCmd.RunSQL (SqlString)
DoCmd.TransferSpreadsheet acExport, acSpreadsheetTypeExcel7, "exportToExcel", "C:\TestMeasurements.xls", True, "A1:G12"
End Sub
Source: http://www.ehow.com/how_7326712_save-access-query-excel-vba.html
This could be done either directly from the database or from Excel (you would need to open the database with Excel VBA to do so, but most of the Office Suite products interact well with each other).
If you want to push the data of your spreadsheet into an Access database, that's different. You just have to open the database and loop through INSERT query. Here is a quick example, you just need to add the loop:
Dim db as DAO.Database
Set db = OpenDatabase(myDataBase.mdb)
Call db.Execute("INSERT INTO myTable (Field1, Field2) VALUES('Value1', 'Value2')")
I got stuck in the problem beneath, because I don´t use Access or Excel much and I have some basic programming language. So here's the deal:
I just made a fairly simple database in MS Access (2007) with a nice query to retrieve data, depending on which parameters you pass. In Excel (2007), I have this big 'template' which basically has parameters for the query. These parameters change per column & per row!
Perhaps superfluously, e.g.
column A contains paramA (10 different options)
column B contains paramB (8 different options)
column C contains paramC (2 different options)
What I'd like to do is to fill this template with dynamic data from Access, minding the continously changing parameters.
e.g.
column D contains Query (ParamA, ParamB, ParamC)
Best way to go I think is to make a (inline?) function that retrieves results from the query, also passing the parameters depending on the relative cell position. And this function is then copied as a normal inline excel function (like: SUM()).
I just don't know how to call /execute an MS Access query from inside an Excel Macro function.
Could someone help me with it? Thank you very much in advance!
A few notes.
Dim cn As Object
Dim rs As Object
''See: http://www.connectionstrings.com/access
strFile = "C:\Docs\AccessDB.mdb"
strCon = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & strFile _
& ";User Id=admin;Password=;"
Set cn = CreateObject("ADODB.Connection")
Set rs = CreateObject("ADODB.Recordset")
cn.Open strCon
strSQL = "SELECT SomeField, OtherField FROM SomeTable " _
& "WHERE SomeText='" & Range("A1") & "'"
rs.Open strSQL, cn
s = rs.GetString
MsgBox s
'' Or
Sheets("Sheet2").Cells(2, 1).CopyFromRecordset rs
To add to Remou's answer also see
Modules: Sample Excel Automation - cell by cell which is slow and
Modules: Transferring Records to Excel with Automation
Late binding means you can safely remove the reference and only have an error when the app executes lines of code in question. Rather than erroring out while starting up the app and not allowing the users in the app at all. Or when hitting a mid, left or trim function call.
This also is very useful when you don't know version of the external application will reside on the target system. Or if your organization is in the middle of moving from one version to another.
For more information including additional text and some detailed links see the "Late Binding in Microsoft Access" page