I get Query timeout expired when I run this code.
I have tried putting in timeout limits on the Conn and the Command from Query timeout expired when trying to run a short procedure but not successful.
You can see my comments in CAPS in the code for the timeout clauses. In SSMS it executes in about 45sec. My workbook has multiple queries using the same connection. This is the heaviest query (returning about 7k rows), so I'm pretty sure the problem is a query timeout, not connection.
Sub Units()
Application.ScreenUpdating = False
Dim rs4 As Object
Set rs4 = CreateObject("ADODB.Recordset") '04Unit2
Dim sqlstr04 As String
sqlstr04 = "select * from dbo.[04Units]"
Sheet17.Cells.Clear
Call connectDatabase
rs4.Open sqlstr04, DBCONT
DBCONT.commandtimeout = 120 'CONNECTION TIMEOUT
rs4.commandtimeout = 120 'RECORDSET QUERY TIMEOUT
'Debug.Print sqlstrledger03
For intColIndex = 0 To rs4.Fields.Count - 1
Sheet17.Range("A1").Offset(0, intColIndex).Value =
rs4.Fields(intColIndex).Name
Next
Sheet17.Range("A2").CopyFromRecordset rs4
End Sub
Error
DBCont is defined in a separate module:
The Open-method of your recordset already triggers the execution of the SQL-command. Your TimeOut-commands must be set before the command is executed.
Simply change the order of your code and set the Timeout before you execute the SQL command should do the trick (I think a recordset object has no timeout method)
DBCONT.commandtimeout = 120 'CONNECTION TIMEOUT
rs4.Open sqlstr04, DBCONT
Related
I get Operation is not allowed when the object is closed error.
Why I do get this error?
[Error location][2]
There is nothing wrong with the connection.
I can do SQL queries on the same cmd connection and have no problems with them.
Only this query messes me up.
I usually don't use Set search path or table short names, but just incase got
into very detail this time.
When I run the same query in PostgreSQL I get the result:
[postgres output for the same SQL query][1]
I have been digging around google for hours and I can't find any answers.
Does it have to do something with drivers?
How do I know what driver I use for excel?
What driver I should use?
How I change driver?
[References I use][3]
https://i.stack.imgur.com/R4nLe.png
https://i.stack.imgur.com/k4uJo.png
https://i.stack.imgur.com/Ytrlk.png
CODE
Public Function getDBArray(cmd, strSQL)
Dim recordID As Long, recordAmount As Long, totalFields As Integer, fieldID As Integer
Dim rs As New ADODB.Recordset
Set rs = CreateObject("ADODB.Recordset")
cmd.CommandText = strSQL '' << this is the strSQL i copy pasted directly to pgadmin and it worked
Set rs = cmd.Execute
totalFields = getTableAmount(rs.Fields)
recordAmount = getTableAmount(rs) ' << error in this function
recordID = 0
Apparently I was not suppose to used SET SEARCH PATH
There are a few answers about this problem, but my question is about the particular code I have.
I'm trying to get the last inserted ID of this query executing on VBA code.
Public Function Execute(cQry As excfw_dbQuery) As ADODB.Recordset
If pConn.State = 0 Then
OpenConnection
End If
qry = "INSERT INTO [some really long query, which actually works]; SELECT SCOPE_IDENTITY()"
On Error Resume Next
Dim rs As ADODB.Recordset
Set rs = New ADODB.Recordset
rs.Open qry, pConn 'also tried with adOpenKeyset, adLockOptimistic
'some error handling code which is not related to the issue
Set rs = rs.NextRecordset() 'also tried without moving onto the next recordset
pInsertedId = rs.Fields(0).Value
Set Execute = rs 'this is just to pass the query result for SELECT queries
End Function
This should save the last inserted ID on the pInsertedId variable, but instead I get 0 each time I insert a row. The weird thing is, when I copy and paste the same code into the SSMS, it works.
I might just get away with inserting some unique data to some unused column of the database and querying through that.
--UPDATE--
I've just noticed that when running a SELECT query, rs object remains open until it goes out of scope. Here is a screenshot of the watch section:
on an insert statement instead, it gets closed as soon as the query gets executed:
You can explicitly save the results of the insert statement by using an output clause and return the results with a select:
qry =
"declare #Ids as ( Id Int );" +
"insert into MyTable ( Name ) " + ' Assuming Id is an identity column.
"output Inserted.Id into #Ids " +
"values ( #Name );" +
"select Id from #Ids;"
From the documentation for output:
INSERTED Is a column prefix that specifies the value added by the
insert or update operation. Columns prefixed with INSERTED reflect the
value after the UPDATE, INSERT, or MERGE statement is completed but
before triggers are executed.
You can use an output clause to get any data from the rows (Note plural.), e.g. identity column values for newly inserted rows. Output can be used with insert, update, delete and merge and provides access to both before and after values in the case of update. A tool well worth having in your pocket.
As it turns out, the table that I'm trying to insert to has multiple triggers attached to it. So, the query which includes the SELECT SCOPE_IDENTITY(); is actually 4th query. I had to move 4 queries forward in order to get the correct scope. How I pulled that off programatically is as follows. Not very clean (and possibly not the best way to do it), but does the job for me.
I basically go ahead to next recordset until there is none left, which I detect by checking the error number 91 (Object variable or with block variable not set)
Public Function Execute(cQry As excfw_dbQuery) As ADODB.Recordset
If pConn.State = 0 Then
OpenConnection
End If
qry = "INSERT INTO [some really long query, which actually works]; SELECT SCOPE_IDENTITY()"
On Error Resume Next
Dim rs As ADODB.Recordset
Set rs = New ADODB.Recordset
rs.Open cQry.Query, pConn, adOpenKeyset, adLockOptimistic
'some error handling code which is not related to the issue
On Error Resume Next
'begin loop
Do
'go to next recordset
Set rs = rs.NextRecordset()
'if we're getting error n. 91, it means
'recordsets are exhausted, hence we're getting
'out of the loop
If Err.Number = 91 Then
Err.Clear
Exit Do
End If
'if we are not out of recordsets, check the
'result of the query. If it is bigger then zero,
'it means we hit the jackpot.
If rs.Fields(0).Value > 0 Then
pInsertedId = rs.Fields(0).Value
End If
Loop
On Error GoTo 0
End Function
Again, not the cleanest, nor the most correct way to do it, but did the trick. I'm open for any improvements or suggestions.
I am using a set of scripts that pull a lot of different data from iSeries via ODBC.
ActiveWorksheets.RefreshAll does not work as it doesn't leave enough time to run the background queries
I've tried the below,but to no avail
Dim qry As Connections
'Set qry =
For Each qry In ActiveWorksheets.Connections
qry.BackgroundQuery = False
qry.RefreshAll
DoEvents
Next qry
This gives me the expected Error 424 Object expected.
I don't expect to use Set qry = here, as I need to run through 30 different connections
Let's just call them connection1, connection2 etc for now, as their names are all over the place
Is the simplest option to stop the background query, refresh, activate background query - before the data import, or is there a better way?
I've looked all over SO - but can't find info on multiple ODBC connections
EDIT:
Dim qry As WorkbookConnection
For Each qry In ActiveWorkbook.Connections
qry.Refresh
DoEvents
Next qry
I believe your
Dim qry As Connections
should read
Dim qry As WorkbookConnection
The ActiveWorksheets.Connections.Item property returns an object of type WorkbookConnection. If you are trying to refresh the connections one at a time as it seems from your For Each statement, that object represents a single connection with methods like Refresh rather than the collection of all connections.
I managed to work this out. So all who may need this in the future can see:
Dim qry As WorkbookConnection
For Each qry In ActiveWorkbook.Connections
qry.ODBCConnection.BackgroundQuery = False
qry.Refresh
qry.ODBCConnection.BackgroundQuery = True
Next qry
Although it doesn't look like BackgroundQuery = True/False doesn't look like it's vital here. Turning it off means when you qry.Refresh, it pulls the data and refreshes it.
Also using For Each qry means that rather than writing out out 20 times, I can just look every connection and turn them off, refresh, and turn them back on
In Excel 2016 VBA, I'm refreshing several queries like this:
MyWorkbook.Connections(MyConnectionName).Refresh
After the code is done, and no errors are encountered, I see that the hourglass icons for most of the queries are still spinning for several seconds.
Is it possible to check for success AFTER all the refreshes are completed? I'm concerned that my code isn't going to know if an error happens after the code finishes but before the queries are done refreshing.
BTW I don't want to do a RefreshAll, because some of the queries are dependent on others (uses them as a source). I refresh them in a certain sequence so that dependent queries are refreshed after the queries they are dependent on.
UPDATE:
I see that the Connection objects have a read-only RefreshDate property, which at first glance looked like it could be used to do this check:
MyWorkbook.Connections(MyConnectionName).OLEDBConnection.RefreshDate
HOWEVER, it doesn't seem to be getting set. I get an error trying to check it. If I set a Variant variable to that RefreshDate property, the variable shows as "Empty". The source is a SQL server database.
The QueryTable object exposes two events: BeforeRefresh and AfterRefresh.
You need to change your paradigm from procedural/imperative to event-driven.
Say you have this code in ThisWorkbook (won't work in a standard procedural code module, because WithEvents can only be in a class):
Option Explicit
Private WithEvents table As Excel.QueryTable
Private currentIndex As Long
Private tables As Variant
Private Sub table_AfterRefresh(ByVal Success As Boolean)
Debug.Print table.WorkbookConnection.Name & " refreshed. (success: " & Success & ")"
currentIndex = currentIndex + 1
If Success And currentIndex <= UBound(tables) Then
Set table = tables(currentIndex)
table.Refresh
End If
End Sub
Public Sub Test()
tables = Array(Sheet1.ListObjects(1).QueryTable, Sheet2.ListObjects(1).QueryTable)
currentIndex = 0
Set table = tables(currentIndex)
table.Refresh
End Sub
The tables variable contains an array of QueryTable objects, ordered in the order you wish to refresh them; the currentIndex variable points to the index in that array, for the QueryTable you want to act upon.
So when Test runs, we initialize the tables array with the QueryTable objects we want to refresh, in the order we want to refresh them.
The implicit, event-driven loop begins when table.Refresh is called and the QueryTable fires its AfterRefresh event: then we report success, and update the event-provider table object reference with the next QueryTable in the array (only if the refresh was successful), and call its Refresh method, which will fire AfterRefresh again, until the entire array has been traversed or one of them failed to update.
Just found this solution at Execute code after a data connection is refreshed
The bottom line is: Excel refreshes data connection in the background and thus the rest of the code is executed without interruption.
Solution: set BackgroundQuery property to False
Example:
For Each cnct In ThisWorkbook.Connections
cnct.ODBCConnection.BackgroundQuery = False
Next cnct
Possible problem: don't know which connection it is...
Remedy: case... when...
Dim cnct as WorkbookConnection ' if option explicit
' ODBC and OLE DB
For Each cnct In ThisWorkbook.Connections
Select case cnct.type
case xlconnectiontypeodbc
cnct.ODBCConnection.BackgroundQuery = False
case xlconnectiontypeoledb
cnct.OledbConnection.BackgroundQuery = False
end select
Next cnct
As you can see, code above only deals with ODBC and OLE DB. Depending on what types of data connection you are using, you can expand the select case clause. Unless changed, once run, connection's BackgroundQuery will remain off.
Preface
I'm making this question specific to conform to SO asking guidelines, but feel free to suggest wholesale redesign if you wish. I may be using some bad practices.
Basic Question
I use ADO to execute a multi-step SQL Server query that takes several minutes to execute. I use Raiserror in my tsql queries to let myself know more verbosely which steps have finished. Is it possible to pass these messages to VBA before the complete query finishes, while still continuing with the query?
Details and Code
I use the vba below to execute the t-SQL query underneath. As you can see, there are two errors raised in the t-SQL that display "Step 1 complete" and "Step 2 complete". Could I pass these messages (or alternately use error numbers and pass those) back to VBA in a way that would allow me to detect them and update a progress bar while continuing to execute the query?
VBA used to execute the query:
Set cmd = New ADODB.Command
cmd.ActiveConnection = cnn
cmd.CommandTimeout = 0
cmd.CommandText = strQuery
Set rst = New ADODB.Recordset
rst.Open cmd
'Go to the second to last recordset of the multi-step query
String1 = Replace(strQuery, ";", "")
For Loop2 = 1 To (Len(strQuery) - (Len(String1) + 1))
Set rst = rst.NextRecordset
Next Loop2
'Copy results
If Not rst.EOF Then
(snip - actions)
Else
MsgBox "Error: No records returned."
End If
Stripped-down piece of multi-step tSQL query:
--#DRS1: The numbers being researched
select distinct numbers
into #DRS1
from Table1 (nolock)
where numbers in ()
--#DRS1: Index
create nonclustered index Idx_DRS1
on #DRS1(numbers);
Raiserror(“Step 1 complete”,1,1) with nowait;
--#DRS2: Table2 for numbers being researched
select distinct
DRS1.numbers
,a.ID
into #DRS2
from #DRS1 DRS1
join Table2 (nolock) a
on DRS1.numbers = a.numbers
Raiserror(“Step 2 complete”,1,1) with nowait;
--MORE STEPS
(more steps)
(more raiserror statements)
Clarification
I am not interested in:
A method that doesn't allow me to update a progress bar until the query is completely done.
A method that uses Progress/MaxProgress, because as I understand it that would return separate numbers for each of the steps in my query, rather than one progress measure for the entire query.
I am less interested in:
Using # records affected messages to determine progress, because some steps may return equal numbers of records to previous steps.
Research
The closest thing I have found to what I'm looking for is here, but as the discussion of that solution here says:
This approach would only work for stored procedures that are not intended to return results, say procs that insert data into tables. Another approach would be needed if your stored proc returns a result set.
Since I return results in the final step of my query to be manipulated in Excel I don't think this would work for me.
External link code for reference
SQL:
CREATE PROCEDURE dbo.updTesting As
Declare #RetVal integer
Exec #RetVal = updTesting2
Return #RetVal
GO
CREATE PROCEDURE dbo.updTesting2 As
raiserror('Error From Testing 2 procedure',16,1)
Return -2
GO
VBA:
Private Sub Command1_Click()
On Error GoTo ErrorHandler
Dim db As ADODB.Connection
Dim cmd As ADODB.Command
Set db = New ADODB.Connection
db.CursorLocation = adUseClient
db.Open "provider=sqloledb;data source=handel;initial catalog=northwind;integrated security=sspi"
Set cmd = New ADODB.Command
With cmd
Set .ActiveConnection = db
.CommandText = "updTesting"
.CommandType = adCmdStoredProc
.Parameters.Append .CreateParameter("#RetVal", adInteger, adParamReturnValue)
.Execute , , adExecuteNoRecords
End With
ExitPoint:
On Error Resume Next
Set cmd.ActiveConnection = Nothing
Set cmd = Nothing
db.Close
Set db = Nothing
Exit Sub
ErrorHandler:
MsgBox "Error # " & Err.Number & vbNewLine & vbNewLine & Err.Description
Resume ExitPoint
End Sub
There are several possibilities to work out a solution for your problem:
(1) Capture the error messages as they occur while the query is running. That's the requested approach.
(2) Break-down the big, long query into several smaller chunks and run them one after the other. Like this you know which part is completed and you can update your progress bar based on that information just before sending the next chunk to the server.
(3) Update the big, long query to log its progress on the server in a temp table and then read out this log while the other query is still running.
While I'd recommend to use errors only when errors occur and not to "abuse" them for logging, tracking, or feedback, both options (1 & 2) are quite feasible with events:
Similar to Worksheet events Worksheet_Change, Worksheet_Activate, or Worksheet_BeforeDoubleClick there are also ADODB events for ADODB.Connection and ADODB.Recordset. Both are well documented and can be easily viewed within the VBE by (1) adding a reference to Microsoft ActiveX Data Objects x.x Library (2) pressing F2 (3) selecting the ADODB library in the drop-down menu at the top (4) and finally looking up Recordset or Connection within the classes. Here are the available events for Connection:
As you can see, all events are marked with a lightning. To capture / use these events you need to create a Class Module in the VBE and add the following line to it:
Dim WithEvents adoConnection As ADODB.Connection
Afterwards, you can make use of the newly created ADODB.Connection event and select the required event from the top of the list:
The applicable event for option (1) is the InfoMessage event which occurs "[...] whenever a warning occurs during a ConnectionEvent operation." The import part here is during a connection. So, this event fires automatically whenever an ADODB connection "gets" an error.
Of course, this means that the original query to the server must be sent without waiting for an answer. Instead you should use the above event to capture any errors while the query is executing and create yet another event to automatically fire when the entire query completed.
For some more help in respect to asynchronous ADODB connection and possible problems with them you might want to have a look at the following two posts here:
ExecuteComplete ADODB Connection event not fired with adAsyncExecute parameter
Running multiple async queries with ADODB - callbacks not always firing
A similar approach can be used with option (3) as described above and asynchronous ADODB connections.
Let me know if this solves your problems or if you have any further questions.
All available ADODB events can be reviewed here https://msdn.microsoft.com/en-us/library/ms675083%28v=vs.85%29.aspx