Using Excel & Access Together passing a variable from excel to access - excel

In excel I have a linked table to a access table "tbl_Output"
Currently there is a manual step that before I run a excel macro I have to go into the database and open up a create table query and manual enter a criteria and run. Call it Field "Vendor Name"
This vendor name exists in the excel document. Is it possible to declare that variable in excel, pass it to access and run the create table query using that variable as its criteria.
The task gets run for many vendors so if I can automate this step I can create a loop to go through all vendors.
I have tried a workaround by having a linked pivot table to the data source that the create table query is based off then filtering in the excel pivot table itself but due to the large amount of data the refresh takes too long.
Apologies if this is something obvious. Coding vba with access is something im not familiar with.

Not 100% on the question that is being asked but I'm gonna take a shot at it. The code below will take a list of Vendor Names [Vendor Ids] and will loop through the list executing a create table query for each of the Vendor Ids that contains the information for that specific Vendor
Sub umCreateDBTables()
Dim DBPath As String ' Path to the database with the information
Dim DBPath2 As String ' Path to the database to store the newly created tables in
Dim xlCell As Range
Dim sSQL As String
Dim DB As DAO.Database
Dim VenID As String
Dim i As Integer
DBPath = "C:\DB_Temp\AccessDB_A.accdb"
DBPath2 = "C:\DB_Temp\AccessDB_B.accdb"
Set DB = OpenDatabase(DBPath, True, False)
Set xlCell = Range("A2") ' Assumes that this is the beginning of the column with your vendor ids
Do Until xlCell.Value = "" ' Loops through the list of vendor ids until it gets to an empty cell
VenID = "v_" & xlCell.Value ' would be best to feed this through a function to strip out any invalid db field name characters
sSQL = "SELECT T1.F1, T1.F2, T1.F3, INTO " & VenID & " IN '" & DBPath2 & "' FROM T1 WHERE (((T1.F1)='" & xlCell.Value & "'));"
For i = DB.TableDefs.Count - 1 To 0 Step -1 ' Cycle through the list of database objects [tables, queries, etc....]
If DB.TableDefs(i).Name = VenID Then ' If the vendor table already exists in the DB, delete it so it can be recreated
DB.TableDefs.Delete (VenID)
End Select
Next i
DB.Execute sSQL ' Run the SQL to create the vendor table
Set xlCell = xlCell.Offset(1, 0) ' move down to the next row
Loop
DB.Close
Set DB = Nothing
Set xlCell = Nothing
End Sub
Hope this helps

Thank you so much Glenn G
The code you provided was extremely helpful and put me in the right direction.
I was having run-time issues with the DAO even with references added though but worked around it.
Code I got to work was:
Sub UpdateDatabase()
Dim DBPath As String
Dim xlcell As Range
Dim sSQL As String, stProvider As String
Dim DB As New ADODB.Connection
DBPath = "C:\Temp\Data.accdb"
stProvider = "Microsoft.ACE.OLEDB.12.0"
With DB
.ConnectionString = DBPath
.Provider = stProvider
.Open
End With
Set xlcell = Sheet3.Range("X2")
Do Until xlcell.Value = ""
venid = xlcell.Value
sSQL = "SELECT ALL * INTO tbl_Output FROM qry_Data WHERE (((qry_Data.VendorName)='" & xlcell.Value & "'));"
DB.Execute "DROP TABLE tbl_Output;"
DB.Execute sSQL
Set xlcell = xlcell.Offset(1, 0)
Loop
DB.Close
Set DB = Nothing
Set xlcell = Nothing
Thank you again for your help.
Regards
Richard

Related

Pivot full text instead of counts - Excel

I'm trying to determine the most efficient/effective way to display specific rows from a central table on sheet1 as text on sheet2.
I have setup a table that contains numerous events that is continually being used by multiple people. This sheet acts as a central database and is shared with multiple people who are all using it in real time.
I want to provide a table on sheet2 that allows other users to view specific events from the central database. So I need to export specific values from sheet1 to sheet2. I understand this can easily be done using a filter but the table on sheet1 is constantly being used and it can't be disrupted.
I can't just do a sort or filter on the sheet1 table because it needs to be used by other parties at all times
I really only need to view specific values from sheet1 for the last month. I've got code the exports all rows based off a specific value entered into a designated column on Sheet1. But due to the size of the file Excel constantly crashes.
I then thought a pivot table may be easier and I wouldn't have to use VBA. Is it possible to pivot out specific rows as text, which can be grouped by date, e.g. month?
For instance, if I want to view all ['A's'] and ['X's'] from Column B from the last month as full text it would look like the following:
Central Database table Sheet1
A B C D
0 11/1 A Big Dog
1 10/1 X 1 2
2 11/1 Y Y Y
3 1/2 A Big Cat
4 1/2 X 3 4
5 1/2 Y Y Y
Output table Sheet2
A B C D
1 1/2 A Big Cat
2 1/2 X 3 4
As others have mentioned in the comments, using SQL with ADODB is likely a better approach than using PivotTables. I'd also recommend separating your data (Sheet1) from your presentation layer (Excel). E.g. store your data in an actual database like Access, SQL Server etc.
However, as you are looking for a stop gap, I figured I could give you an approach that might fill the need temporarily. The code is commented, but feel free to ask questions. You will need to add a reference to Microsoft Active X Data Object 2.8 or greater to get this working. How to add a reference?
Early Binding Approach
Option Explicit
Public Sub DisplayView(StartDate As Date, EndDate As Date)
'Add a reference to Microsoft Active X Data Object 2.8 or greater
Dim dbConnection As ADODB.Connection
Dim dbRecordset As ADODB.Recordset
Dim dbCommand As ADODB.Command
Dim OutputSheet As Excel.Worksheet
Dim dbField As Variant
Dim fieldCounter As Long
Set dbConnection = New ADODB.Connection
Set dbRecordset = New ADODB.Recordset
Set dbCommand = New ADODB.Command
Set OutputSheet = ThisWorkbook.Worksheets("Sheet2")
'Do a quick check to determine the correct connection string
'if one of these don't work, have a look here --> https://www.connectionstrings.com/excel/
If Left$(ThisWorkbook.FullName, 4) = "xlsm" Then
dbConnection.connectionstring = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & _
ThisWorkbook.FullName & ";Extended Properties='Excel 12.0 Macro;HDR=YES';"
Else
dbConnection.connectionstring = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & _
ThisWorkbook.FullName & ";Extended Properties='Excel 12.0;HDR=YES';"
End If
'Open the connection and parameterize the query
dbConnection.Open
With dbCommand
.ActiveConnection = dbConnection
.CommandType = adCmdText
'A in B in the text below are the field names in your Sheet 1
'I wasn't sure what the names of the fields are so I named them as they appeared
'That being Column A is called A, Column B is called B etc
.CommandText = "Select * from [Sheet1$] where B in ('A','X') and A >= #StartDate and A < #EndDate"
.Parameters.Append .CreateParameter("#StartDate", adDate, adParamInput, , StartDate)
.Parameters.Append .CreateParameter("#EndDate", adDate, adParamInput, , EndDate)
Set dbRecordset = .Execute
End With
'Clear the Output Sheet
OutputSheet.Cells.Clear
'Add Headers to output
For Each dbField In dbRecordset.Fields
fieldCounter = fieldCounter + 1
OutputSheet.Cells(1, fieldCounter).Value2 = dbField.Name
Next
'Dump the found records
OutputSheet.Range("A2").CopyFromRecordset dbRecordset
If dbConnection.State = adStateOpen Then dbConnection.Close
End Sub
'Run from here
Public Sub ExampleRunner()
'Supply the dates you want to filter for
DisplayView #1/1/2019#, #1/20/2019#
End Sub
As requested, here is the Late Binding Approach that doesn't require an explicit reference to Microsoft Active X Data Object.
Option Explicit
Private Const adCmdText As Long = 1
Private Const adDate As Long = 7
Private Const adParamInput As Long = 1
private const adStateOpen as long = 1
Public Sub DisplayView(StartDate As Date, EndDate As Date)
'Add a reference to Microsoft Active X Data Object 2.8 or greater
Dim dbField As Variant
Dim fieldCounter As Long
Dim dbConnection As Object
Dim dbRecordset As Object
Dim dbCommand As Object
Dim OutputSheet As Excel.Worksheet
Set dbConnection = CreateObject("ADODB.Connection")
Set dbRecordset = CreateObject("ADODB.Recordset")
Set dbCommand = CreateObject("ADODB.Command")
Set OutputSheet = ThisWorkbook.Worksheets("Sheet2")
'Do a quick check to determine the correct connection string
'if one of these don't work, have a look here --> https://www.connectionstrings.com/excel/
If Left$(ThisWorkbook.FullName, 4) = "xlsm" Then
dbConnection.connectionstring = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & _
ThisWorkbook.FullName & ";Extended Properties='Excel 12.0 Macro;HDR=YES';"
Else
dbConnection.connectionstring = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & _
ThisWorkbook.FullName & ";Extended Properties='Excel 12.0;HDR=YES';"
End If
'Open the connection and parameterize the query
dbConnection.Open
With dbCommand
.ActiveConnection = dbConnection
.CommandType = adCmdText
'A in B in the text below are the field names in your Sheet 1
'I wasn't sure what the names of the fields are so I named them as they appeared
'That being Column A is called A, Column B is called B etc
.CommandText = "Select * from [Sheet1$] where B in ('A','X') and A >= #StartDate and A < #EndDate"
.Parameters.Append .CreateParameter("#StartDate", adDate, adParamInput, , StartDate)
.Parameters.Append .CreateParameter("#EndDate", adDate, adParamInput, , EndDate)
Set dbRecordset = .Execute
End With
'Clear the Output Sheet
OutputSheet.Cells.Clear
'Add Headers to output
For Each dbField In dbRecordset.Fields
fieldCounter = fieldCounter + 1
OutputSheet.Cells(1, fieldCounter).Value2 = dbField.Name
Next
'Dump the found records
OutputSheet.Range("A2").CopyFromRecordset dbRecordset
If dbConnection.State = adStateOpen Then dbConnection.Close
End Sub
'Run from here
Public Sub ExampleRunner()
'Supply the dates you want to filter for
DisplayView #1/1/2019#, #1/20/2019#
End Sub
Here are some screenshots of the Results of the below mentioned Power Query. I selected (In Excel 2003) Data->Import External Data->New Database Query
Then, I chose "Excel Files" and added the data I wanted. Make sure to select "is not null" in the query options. I then added the auto sort feature.
You could use a Power Query in the Sheet2 Excel Workbook. Another link here. This way you could update the data whenever you need to. Then, use SQL to query what you need.
It is pretty simple to use and doesn't take any coding (unless you want to use SQL).
This can be done and then in your other workbook you can do the filtering and sorting.
#ryan-wildry 's post is pretty great (and so is his command text for SQL), but if you do not want to use vba or a database then, you can use this (as well as his SQL text).
An example would be:
SELECT * FROM [Sheet1$] WHERE Column2='X';
The only problem would be if you have mixed datatypes in the same column. As can be seen with rows 1 & 4 (they were not able to come across with the query because the values are not the same type. (This is using Excel 2003 so if you have a newer version then, you may be able to check it with that).
I've been doing some research and found that Excel's Power Query will handle mixed data types so, you should be set if you are using Power Query.

How to transfer an excel spreadsheet to an access database

I'm making a program to track my weight, calories I eat in a day, and the date, to help me lose weight. I'm manually putting in these values into a spreadsheet with those three columns (date, calories, weight). I want to transfer the information in these three columns into an access database.
Code so far:
Sub transferdata()
Dim cn As New ADODB.Connection
Dim rs As New ADODB.Recordset
connStr = "C:\Users\sachu\Desktop\Assignment 5\CalorieDatabase.mdb"
providerStr = "Microsoft.ACE.OLEDB.12.0"
With cn
.ConnectionString = connStr
.Provider = providerStr
.Open
End With
rs.Open sqlStr, cn
rs.Close
cn.Close
End Sub
So far my code is only starting the connection between access and excel
There are many ways to do this. Let's look at a couple of case studies.
Export data from Excel to Access (ADO)
If you want to export data to an Access table from an Excel worksheet, the macro example below shows how this can be done.
Sub ADOFromExcelToAccess()
' exports data from the active worksheet to a table in an Access database
' this procedure must be edited before use
Dim cn As ADODB.Connection, rs As ADODB.Recordset, r As Long
' connect to the Access database
Set cn = New ADODB.Connection
cn.Open "Provider=Microsoft.Jet.OLEDB.4.0; " & _
"Data Source=C:\FolderName\DataBaseName.mdb;"
' open a recordset
Set rs = New ADODB.Recordset
rs.Open "TableName", cn, adOpenKeyset, adLockOptimistic, adCmdTable
' all records in a table
r = 3 ' the start row in the worksheet
Do While Len(Range("A" & r).Formula) > 0
' repeat until first empty cell in column A
With rs
.AddNew ' create a new record
' add values to each field in the record
.Fields("FieldName1") = Range("A" & r).Value
.Fields("FieldName2") = Range("B" & r).Value
.Fields("FieldNameN") = Range("C" & r).Value
' add more fields if necessary...
.Update ' stores the new record
End With
r = r + 1 ' next row
Loop
rs.Close
Set rs = Nothing
cn.Close
Set cn = Nothing
End Sub
Also . . .
Export data from Excel to Access (DAO)
If you want to export data to an Access table from an Excel worksheet, the macro example below illustrates another way to do this.
Sub DAOFromExcelToAccess()
' exports data from the active worksheet to a table in an Access database
' this procedure must be edited before use
Dim db As Database, rs As Recordset, r As Long
Set db = OpenDatabase("C:\FolderName\DataBaseName.mdb")
' open the database
Set rs = db.OpenRecordset("TableName", dbOpenTable)
' get all records in a table
r = 3 ' the start row in the worksheet
Do While Len(Range("A" & r).Formula) > 0
' repeat until first empty cell in column A
With rs
.AddNew ' create a new record
' add values to each field in the record
.Fields("FieldName1") = Range("A" & r).Value
.Fields("FieldName2") = Range("B" & r).Value
.Fields("FieldNameN") = Range("C" & r).Value
' add more fields if necessary...
.Update ' stores the new record
End With
r = r + 1 ' next row
Loop
rs.Close
Set rs = Nothing
db.Close
Set db = Nothing
End Sub
Also . . .
Browse to a single EXCEL File and Import Data from that EXCEL File via TransferSpreadsheet (VBA)
Here's yet another way . . .
Sub TryThis()
Dim strPathFile As String
Dim strTable As String, strBrowseMsg As String
Dim strFilter As String, strInitialDirectory As String
Dim blnHasFieldNames As Boolean
' Change this next line to True if the first row in EXCEL worksheet
' has field names
blnHasFieldNames = False
strBrowseMsg = "Select the EXCEL file:"
' Change C:\MyFolder\ to the path for the folder where the Browse
' window is to start (the initial directory). If you want to start in
' ACCESS' default folder, delete C:\MyFolder\ from the code line,
' leaving an empty string as the value being set as the initial
' directory
strInitialDirectory = "C:\MyFolder\"
strFilter = ahtAddFilterItem(strFilter, "Excel Files (*.xls)", "*.xls")
strPathFile = ahtCommonFileOpenSave(InitialDir:=strInitialDirectory, _
Filter:=strFilter, OpenFile:=False, _
DialogTitle:=strBrowseMsg, _
Flags:=ahtOFN_HIDEREADONLY)
If strPathFile = "" Then
MsgBox "No file was selected.", vbOK, "No Selection"
Exit Sub
End If
' Replace tablename with the real name of the table into which
' the data are to be imported
strTable = "tablename"
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel9, _
strTable, strPathFile, blnHasFieldNames
' Uncomment out the next code step if you want to delete the
' EXCEL file after it's been imported
' Kill strPathFile
End Sub
I know this looks like a dead thread but I wanted to revive for Office 360 users. I had to compile an answer from several sources to make something workable. See below.
First- note that you need at least the 2 following references active in your Tools>References Menu.
Microsoft Access 16.0 Object Library &
Microsoft Office 16.0 Access Database Engine Object
You may also need to have:
Visual Basic for Applications//
Microsoft Excel 16.0 Object Library//
OLE Automation//
Microsoft Forms 2.0 Object Library//
Microsoft Outlook 16.0 Object Library//
Microsoft Office 16.0 Object Library
Sub rtnExporttoAccess()
Dim oDAO As DAO.DBEngine, oDB As DAO.Database, oRS As DAO.Recordset
Dim oSelect As Range, sPath As String, sRecordSet As String
Set sheet = ActiveWorkbook.Worksheets("YourSheet") 'excel sheet
Set table = sheet.ListObjects.Item("YourTable") 'excel table
Set oSelect = table.ListRows(table.ListRows.Count).Range 'set your sheet and range however you want
sPath = "your database file path"
sRecordSet = "the title of the table in that database" 'access table
Set oDAO = New DAO.DBEngine
Set oDB = oDAO.OpenDatabase(sPath)
Set oRS = oDB.OpenRecordset(sRecordSet)
oRSct = oRS.Fields.Count
xclFieldCt = table.ListColumns.Count
If oRSct > xclFieldCt Then
intTargetCt = oRSct
Else
intTargetCt = xclFieldCt
End If
For i = 2 To oSelect.Rows.Count
oRS.AddNew
'finds the correct fields to add data to
For j = 0 To intTargetCt - 1 'access is base 0 so the end is always -1
oRSHeaderName = oRS.Fields(j).Name 'gets database table variable header name
For col = 1 To intTargetCt 'excel is base 1
lastRowHeaderName = table.HeaderRowRange(1, col) ' gets excel table variable header name
If oRSHeaderName = lastRowHeaderName Then 'this verifies both headers are the same
oRS.Fields(j) = Now
Exit For
End If
If oRSHeaderName = "Pass/Failed" And lastRowHeaderName = "Pass/Failed" Then 'this verifies the you are putting the data where you want it if headers arent the same.
oRS.Fields(j) = cbxPF
Exit For
End If
Next col
Next j
oRS.Update
Next i
oDB.Close
End Sub
ASH's second DAO option is basically what this code is as well. I included the references and gave some extra options for you to see some more examples.
Creat a stand alone Acces DB then link the Excel in it. The Access has tools to import data from Excel with live communication.
Follow this:
Open MS Access
Creat new blank database (in this step you have to give name to the database, and set the save location)
In the new database on External Data tab choose the correct type to add based on what you want to import (in this case you have to select Excel)
in the earlier MS Access version the popular insertable things were stretched
in the 2016 version, and O365 the options is more compact so there is one option called New Data Source which contain all possibilities
The import progress consits of several steps.
you have to select the source and set how you want to import data. You can import data into a new table in Access as a copy, or you can connect the source of data to the Access database. Select connect source data for live communication.
select inner data source (for example which sheet, or range you want to import)
set if the first row contain headers
give a name to the linked table
Finally data from Excel linked into Access and it will update when you use it.

Connect access query to excel with dynamic input parameter

I've got some queries I have connected to an excel sheet that I refresh in order to get the latest data.
Is it possible to make excel "ask" for input parameters for one column lets say?
I am looking to get data by company so wondering if it's possible to do it as in access (type [company]:) in the field criteria
If I save the query like that in access it won't let me connect it to excel
Thanks
Alright another edit.
Here it is, I created a database, its simple.
The database is called "Database1.accdb"
There are more records than shown in the screen shot.
I created a workbook with one sheet, its name is "AccessDBtest.xlsm"
I created a button on Sheet1 and entered the field parameter I wanted in the cell beside it, C3 or (3,3) in (row,col) format.
This is the code that works returning the data set (without field names) based on the input criteria. I made a msgBox before the SQL query execution so that I could look at it first. You don't need that if you do not want it, good for testing.
Private Sub CommandButton1_Click()
Dim inputSheet As Worksheet
Dim fieldSTR As String
Dim placementRange As Range
Dim rs As Object 'record set
Dim conn As Object
Dim strQuery As String
Dim myDB As String
Set inputSheet = ThisWorkbook.Worksheets("Sheet1")
Set placementRange = inputSheet.Range("E2")
fieldSTR = CStr(inputSheet.Cells(3, 3).Value) 'C3 cell
myDB = "C:\Users\Documents\0_Excel Projects\Testing\Database1.accdb"
Set conn = CreateObject("ADODB.Connection")
With conn
.Provider = "Microsoft.ACE.OLEDB.12.0" 'For *.ACCDB Databases
.ConnectionString = myDB
.Open
End With
strQuery = "SELECT * FROM " & _
"tbl_test WHERE tbl_test.Color = " & "'" & fieldSTR & "'" & ";"
'The below gives the same result as * but you could limit the fields returned as well
'tbl_test.ID, tbl_test.Color, tbl_test.number
'just using Color also works you do not need to reference the table directly
MsgBox (strQuery)
Set rs = conn.Execute(strQuery)
placementRange.CopyFromRecordset rs
rs.Close
Set rs = Nothing
conn.Close
Set conn = Nothing
End Sub
See if you can map what you want to do using that.
I wanted to capture a string from a cell and then use that string in the query.
Here is the MsgBox before the query executes (what access will see):
I tested it with * for all fields and below you can see that I called up specific fields, they both work.
And here is what happens after I clear the msgBox, the recordset is pulled and pasted starting in the range I specified Range.("E3") on the sheet I specified (Sheet1)
As you can see, we can pull queries from a database using input that is found on a current sheet.
Let me know if this helps your situation.
We can dig deeper if required.
-WWC

Excel VBA - Refresh selected queries / connections

Situation
I have 5 Workbook connections set up that query data from an MS Access database, and I have given them the names qry_1, qry_2, ... , qry_5
I have a selection of these query names listed in a table called qry_Table on a worksheet:
Query Name
----------
qry_1
qry_4
qry_5
and I can loop through the table (using .listObjects) to get each query name as a string
Question
I can loop through all the queries in the workbook and refresh them all, but I can't seem to work out how to select and refresh only those queries listed in the table, namely qry_1, qry_4 and qry_5.
I would like to be able to set the query object so that I can define the .Connection string too.
Is this possible?
Notes
The code here shows how I get each row in the column Query Name of the table qry_Table
dim wksControl As worksheet
dim objList As ListObject
set wksControl = worksheets("Control") 'the worksheet that contains 'qry_Table'
Set objList = wksControl.ListObjects("qry_Table")
With objList.ListColumns("Query Name").DataBodyRange
For i = 1 To .Rows.count
str = .Rows(i) 'query name to refresh
'***Required: Define the qry to refresh
Set qry = .QueryTable(str) '<~~ this code fails
'code to .Refresh BackgroundQuery:=False
Next i
End With
And to refresh all the queries in the workbook I can use
'code for generic query connections
For Each objList In wks.ListObjects
If objList.SourceType = xlSrcQuery Then 'only refresh if it's the right query type
'ensure it's using the right connection/database as specified on the Control sheet
strConnection = "ODBC;DSN=MS Access Database;DBQ=" & Range("dbFilePath") & _
Range("dbName") & ";DefaultDir=" & Range("dbFilePath") & _
";DriverId=25;FIL=MS Access;MaxBufferSize=2048;PageTimeout=5;"
With objList.QueryTable
.Connection = strConnection
.BackgroundQuery = False
.Refresh
End With
count = count + 1
End If
Next
You need to refer to the ListObject by name, rather than the connection, and then access its Querytable property:
With wks.Listobjects(str).QueryTable
by using str = .Rows(i) you are assigning whole row's range to str, replacing the line with str = .Cells(i,1).Value should work as it will return value in first column of the i row.

Fastest way to transfer Excel table data to SQL 2008R2

Does anyone know the fastest way to get data from and Excel table (VBA Array) to a table on SQL 2008 without using an external utility (i.e. bcp)?
Keep in mind my datasets are usually 6500-15000 rows, and about 150-250 columns; and I end up transferring about 20-150 of them during an automated VBA batch script.
I have tried several methods for getting large amounts of data from an Excel table (VBA) to SQL 2008. I have listed those below:
Method 1. Pass table into VBA Array and send to stored procedure (ADO)
-- Sending to SQL is SLOW
Method 2. Create disconnected RecordSet load it, then sync.
-- Sending to SQL VERY SLOW
Method 3. Put table into VBA array, loop though the array and concatenate(using delimiters) then send to stored procedure.
-- Sending to SQL SLOW, but faster than Method 1 or 2.
Method 4. Put table into VBA array, loop though the array and concatenate(using delimiters) then place each row with ADO recordset .addnew command.
--Sending to SQL very FAST (about 20 times faster than methods 1-3), but now I will need to split that data using a separate procedure, which will add significant wait time.
Method 5. Put table in VBA array, serialize into XML, send to stored procedure as VARCHAR and specify XML in stored procedure.
--Sending to SQL INCREDIBLY SLOW (about 100 times slower than methods 1 or 2)
Anything I am missing?
There is no single fastest way, as it's dependent on a number of factors. Make sure the indexes in SQL are configured and optimized. Lots of indexes will kill insert/update performance since each insert will need to update the index. Make sure you only make one connection to the database, and do not open/close it during the operation. Run the update when the server is under minimal load. The only other method you haven't tried is to use a ADO Command object, and issue a direct INSERT statement. When using the 'AddNew' Method of the recordset object, be sure to issue only one 'UpdateBatch' Command at the end of the inserts. Short of that, the VBA can only run as fast as the SQL server accepting the inputs.
EDIT:
Seems like you've tried everything. There is also what is known as 'Bulk-Logged' recovery mode in SQL Server, that reduces the overhead of writting so much to the transaction log. Might be something worth looking into. It can be troublesome since it requires fiddling with the database recovery model a bit, but it could be useful for you.
The following code will transfer the thousands of data in just few seconds(2-3 sec).
Dim sheet As Worksheet
Set sheet = ThisWorkbook.Sheets("DataSheet")
Dim Con As Object
Dim cmd As Object
Dim ServerName As String
Dim level As Long
Dim arr As Variant
Dim row As Long
Dim rowCount As Long
Set Con = CreateObject("ADODB.Connection")
Set cmd = CreateObject("ADODB.Command")
ServerName = "192.164.1.11"
'Creating a connection
Con.ConnectionString = "Provider=SQLOLEDB;" & _
"Data Source=" & ServerName & ";" & _
"Initial Catalog=Adventure;" & _
"UID=sa; PWD=123;"
'Setting provider Name
Con.Provider = "Microsoft.JET.OLEDB.12.0"
'Opening connection
Con.Open
cmd.CommandType = 1 ' adCmdText
Dim Rst As Object
Set Rst = CreateObject("ADODB.Recordset")
Table = "EmployeeDetails" 'This should be same as the database table name.
With Rst
Set .ActiveConnection = Con
.Source = "SELECT * FROM " & Table
.CursorLocation = 3 ' adUseClient
.LockType = 4 ' adLockBatchOptimistic
.CursorType = 0 ' adOpenForwardOnly
.Open
Dim tableFields(200) As Integer
Dim rangeFields(200) As Integer
Dim exportFieldsCount As Integer
exportFieldsCount = 0
Dim col As Integer
Dim index As Integer
index = 1
For col = 1 To .Fields.Count
exportFieldsCount = exportFieldsCount + 1
tableFields(exportFieldsCount) = col
rangeFields(exportFieldsCount) = index
index = index + 1
Next
If exportFieldsCount = 0 Then
ExportRangeToSQL = 1
GoTo ConnectionEnd
End If
endRow = ThisWorkbook.Sheets("DataSheet").Range("A65536").End(xlUp).row 'LastRow with the data.
arr = ThisWorkbook.Sheets("DataSheet").Range("A1:CE" & endRow).Value 'This range selection column count should be same as database table column count.
rowCount = UBound(arr, 1)
Dim val As Variant
For row = 1 To rowCount
.AddNew
For col = 1 To exportFieldsCount
val = arr(row, rangeFields(col))
.Fields(tableFields(col - 1)) = val
Next
Next
.UpdateBatch
End With
flag = True
'Closing RecordSet.
If Rst.State = 1 Then
Rst.Close
End If
'Closing Connection Object.
If Con.State = 1 Then
Con.Close
End If
'Setting empty for the RecordSet & Connection Objects
Set Rst = Nothing
Set Con = Nothing
End Sub
By far the fastest way to do this is via T-SQL's BULK INSERT.
There are a few caveats.
You will likely need to export your data to a csv first (you may be able to import directly from Excel; my experience is in going from Access .mdbs to SQL Server which requires the interim step to csv).
The SQL Server machine needs to have access to that csv (when you run the BULK INSERT command and specify a filename, remember that the filename will be resolved on the machine where SQL Server is running).
You may need to tweak the default FIELDTERMINATOR and ROWTERMINATOR values to match your CSV.
It took some trial and error for me to get this set up initially, but the performance increase was phenomenal compared to every other technique I had tried.
works pretty fine, on the other hand to improve speed we may still modify the query:
Instead: Source = "SELECT * FROM " & Table
We can use: Source = "SELECT TOP 1 * FROM " & Table
Here is we only need column names. So no need to maka a query for entire table, which is extending the process as long as new data imported.
As far as I remember, you can create a linked server to the Excel file (as long as the server can find the path; it's best to put the file on the server's local disk) and then use SQL to retrieve data from it.
Having just tried a few methods, I came back to a relatively simple but speedy one. It's fast because it makes the SQL server do all the work, including an efficient execution plan.
I just build a long string containing a script of INSERT statements.
Public Sub Upload()
Const Tbl As String = "YourTbl"
Dim InsertQuery As String, xlRow As Long, xlCol As Integer
Dim DBconnection As New ADODB.Connection
DBconnection.Open "Provider=SQLOLEDB.1;Password=MyPassword" & _
";Persist Security Info=false;User ID=MyUserID" & _
";Initial Catalog=MyDB;Data Source=MyServer"
InsertQuery = ""
xlRow = 2
While Cells(xlRow, 1) <> ""
InsertQuery = InsertQuery & "INSERT INTO " & Tbl & " VALUES('"
For xlCol = 1 To 6 'Must match the table structure
InsertQuery = InsertQuery & Replace(Cells(xlRow, xlCol), "'", "''") & "', '" 'Includes mitigation for apostrophes in the data
Next xlCol
InsertQuery = InsertQuery & Format(Now(), "M/D/YYYY") & "')" & vbCrLf 'The last column is a date stamp, either way, don't forget to close that parenthesis
xlRow = xlRow + 1
Wend
DBconnection.Execute InsertQuery 'I'll leave any error trapping to you
DBconnection.Close 'But do be tidy :-)
Set DBconnection = Nothing
End Sub

Resources