excel vba run sql query on entire array - excel

Question

I have an sql query that is running in a loop in excel. It works while the array is small. However the array is now up to around 4000 items and it causes excel to crash. Is there a better way to do this, am I missing some easy way to do this
For k = 0 To ubound(sqlarray)
Dim conn As ADODB.Connection
Dim recst As ADODB.Recordset
Dim sqlstring As String
Set conn = New ADODB.Connection
Set recst = New ADODB.Recordset
conn.ConnectionString = "Driver={Client Access ODBC Driver (32-
bit)};System=mysys ;NAM=1;CurrentSchema=myschema;"
conn.Open
Sql = "Select ACCOUNT_NBR, ASSIGNED_TO from prodlib.gr_exp_account where
account_nbr in ('" & sqlarray(k) & "' )"
Set recst.ActiveConnection = conn
recst.Open Sql, conn
Sheets("recst").Range("A" & k).CopyFromRecordset recst
conn.Close
Application.ScreenUpdating = True
Next k
End Sub

Answer 1

As noted in my comment above you should be submitting a single SQL statement and getting a single result set back. Looping through every customer and submitting SQL for it is bad.
Consider something like the following:
Sub whatever(sqlArray as string)
'initialize these outside your loop next time too
Dim conn As ADODB.Connection
Dim recst As ADODB.Recordset
Dim sqlstring As String
Set conn = New ADODB.Connection
Set recst = New ADODB.Recordset
conn.ConnectionString = "Driver={Client Access ODBC Driver (32-
bit)};System=mysys ;NAM=1;CurrentSchema=myschema;"
conn.Open
'All the accounts in one list in one statement. Use array function 'Join'
'to make the array a comma delimited string (or comma with quotes as a delim)
Sql = "Select ACCOUNT_NBR, ASSIGNED_TO from prodlib.gr_exp_account where
account_nbr in ('" & Join(sqlArray, "','") & "' )"
Set recst.ActiveConnection = conn
recst.Open Sql, conn
'Drop the entire recordset in a sheet starting at "A1"
Sheets("recst").Range("A1").CopyFromRecordset recst
conn.Close
Application.ScreenUpdating = True
End Sub
If JOIN() has a problem with the size of the array, or your database is complaining about the number of accounts you are sending in at a time, you could still loop, but using bigger chunks:
Sub whatever(sqlArray As String)
'initialize these outside your loop next time too
Dim conn As ADODB.Connection
Dim recst As ADODB.Recordset
Dim sqlstring As String
Set conn = New ADODB.Connection
conn.ConnectionString = "Driver={Client Access ODBC Driver (32-bit)};System=mysys ;NAM=1;CurrentSchema=myschema;"
conn.Open
Dim k As Integer
Dim chunk As Integer
'Set this to how many accout numbers you can squeeze through in a single request safely
chunk = 400
For k = 1 To UBound(sqlArray) Step chunk
'reset this object in the loop
Set recst = New ADODB.Recordset
'All the accounts in one list in one statement. Use array function 'Join'
'to make the array a comma delimited string (or comma with quotes as a delim)
Sql = "Select ACCOUNT_NBR, ASSIGNED_TO from prodlib.gr_exp_account where account_nbr in ('" & Join_With_Limit(sqlArray, "','", chunk, k) & "' )"
Set recst.ActiveConnection = conn
recst.Open Sql, conn
Sheets("recst").Range("A" & k).CopyFromRecordset recst
'close this for the next loop
recst.close
Next k
conn.Close
Application.ScreenUpdating = True
End Sub
Function Join_With_Limit(inArray As Variant, strDelim As String, intLimit As Integer, intOffset As Offset) As String
Dim i As Integer
For i = intOffset To intOffset + intLimit
'make sure we are below the ubound of the array
if i > uBound(inArray) Then Exit For
'If this isn't the first thing we are adding, then stick the delimiter
'before dropping in the value
If i <> intOffset Then Join_With_Limit = Join_With_Limit + strDelim
'Add the next item
Join_With_Limit = Join_With_Limit + inArray(i)
Next i
End Function