Recordset empty issue - excel

I have issue with filling up recordset.
I stored this query into txt file:
DROP TABLE IF EXISTS "temp_invoice_id_amount_1";
CREATE TEMPORARY TABLE "temp_invoice_id_amount_1" AS
SELECT
ROW_NUMBER() OVER() "RN",
c."ID" "CaseID",
p."ID" "PackageID",
trim(concat(d."LastName", ' ', d."FirstName")) "FullName",
c."ContragentCaseID",
i."ID" "InvoiceID",
i."AccountNum",
i."FromDate" "InvoiceFromDate",
i."ToDate" "InvoiceToDate",
i."InvoiceCurrency",
(coalesce(sum(td."Amount" * er."Value"), 0) * (-1)):: NUMERIC(12,2) "NumActualInvoiceAmount",
trim(replace(to_char((coalesce(sum(td."Amount" * er."Value"), 0) * (-1)):: NUMERIC(12,2), '99999990D00'),'.',',')) "ActualInvoiceAmount"
FROM "Package" p
JOIN "Case" c ON p."ID" = c."PackageID"
JOIN "Debtor" d ON d."ID" = c."DebtorID"
JOIN "Invoice" i ON c."ID" = i."CaseID" AND i."IsDeleted" = 0
LEFT JOIN "Transaction" t ON i."ID" = t."InvoiceID" AND t."IsDeleted" = 0 AND t."IsPayment" = 0
LEFT JOIN "TransactionDetails" td ON t."ID" = td."TransactionID"
LEFT JOIN "ExchangeRate" er ON
--t."PaymentDate" BETWEEN er."FromDate" AND er."ToDate" AND
er."SourceCurrency" = t."Currency" AND
er."DestinationCurrency" = i."InvoiceCurrency"
WHERE p."ID" in ($package)
AND i."AccountNum" not like 'OP%'
AND p."Inserted" + interval '1 minute' > i."Inserted"
group by c."ID", trim(concat(d."LastName", ' ', d."FirstName")), c."ContragentCaseID", i."FromDate", i."ToDate",
i."InvoiceCurrency", i."ID", p."ID";
CREATE UNIQUE INDEX "idx_temp_invoice_id_amount_1" ON temp_invoice_id_amount_1("InvoiceID");
DROP TABLE IF EXISTS "temp_case_id_amount_1";
CREATE TEMPORARY TABLE "temp_case_id_amount_1" AS
SELECT t1."CaseID",
trim(replace(to_char((coalesce(sum(t1."NumActualInvoiceAmount" * er_eur."Value"), 0)):: NUMERIC(12,2), '99999990D00'),'.',',')) "ActualCaseAmount_EUR",
trim(replace(to_char((coalesce(sum(t1."NumActualInvoiceAmount"), 0)):: NUMERIC(12,2), '99999990D00'),'.',',')) "ActualCaseAmount_HRK"
FROM temp_invoice_id_amount_1 t1
LEFT JOIN "ExchangeRate" er_eur ON
current_date BETWEEN er_eur."FromDate" AND er_eur."ToDate" AND
er_eur."SourceCurrency" = t1."InvoiceCurrency" AND
er_eur."DestinationCurrency" = 'EUR'
LEFT JOIN "ExchangeRate" er_hrk ON
current_date BETWEEN er_hrk."FromDate" AND er_hrk."ToDate" AND
er_hrk."SourceCurrency" = t1."InvoiceCurrency" AND
er_hrk."DestinationCurrency" = 'HRK'
GROUP BY t1."CaseID";
CREATE UNIQUE INDEX "idx_temp_case_id_amount_1" ON temp_case_id_amount_1("CaseID");
SELECT ti1."RN",
ti1."CaseID",
--ti1."PackageID",
ti1."FullName",
'ZIPPY',
'CITTY',
'STREETY',
ti1."ContragentCaseID",
--ti1."InvoiceID",
-- ti1."AccountNum",
ti1."InvoiceFromDate",
ti1."InvoiceToDate",
ti1."InvoiceCurrency",
ti1."ActualInvoiceAmount",
'0',
--tc1."ActualCaseAmount_EUR",
tc1."ActualCaseAmount_HRK",
'0'
FROM temp_invoice_id_amount_1 ti1
JOIN "temp_case_id_amount_1" tc1 ON ti1."CaseID" = tc1."CaseID";
Then i used this part to read from txt file:
Dim strSQL2 As String
fileSpec = "\\192.168.0.7\...\Reports\confirmation.txt"
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objTS = objFSO.OpenTextFile(fileSpec, ForReading)
strSQL2 = objTS.ReadAll
strSQL2 = Replace(strSQL2, "$package", package)
objTS.Close
And this part for executing the query:
cmd.CommandType = ADODB.CommandTypeEnum.adCmdText
cmd.ActiveConnection = Conn
cmd.CommandText = strSQL
cmd2.CommandType = ADODB.CommandTypeEnum.adCmdText
cmd2.ActiveConnection = Conn
cmd2.CommandText = strSQL2
Set rs = New ADODB.Recordset
Set rs = cmd.Execute
Set rs2 = New ADODB.Recordset
Set rs2 = cmd2.Execute
But for some reason recordset rs2 is totally empty while rs1 is normally filled with data.
Does anyone knows what could be reason why recordset rs2 is empty?

I don't know for sure, but I suspect the issue is you are executing everything at once. It's likely doing everything but not returning the results the way you expect.
I think what you may want to do is execute each code block in sequence rather than all at once.
Something like this:
Dim rs2 as ADODB.Recordset
For Each codeBlock In Split(strSQL2, ";")
cmd2.CommandText = codeBlock
rs2 = cmd2.Execute
Next codeBlock
You can also put a breakpoint on each "execute" to check the results at each step.
You'll probably also need to omit the final semicolon so it doesn't try to execute blank (or otherwise handle that the command text has to contain some non-whitespace).
Out of curiosity, why the temp tables? Why not simply run the query? Have you considered functions/views?

I didn't find exact solution but workaround solution.
I splitted my query into 2 separate txt files.
First file contains part where i am dropping and creating temp tables,
and second file contains only this:
SELECT ti1."RN",
ti1."CaseID",
--ti1."PackageID",
ti1."FullName",
'ZIPPY',
'CITTY',
'STREETY',
ti1."ContragentCaseID",
--ti1."InvoiceID",
-- ti1."AccountNum",
ti1."InvoiceFromDate",
ti1."InvoiceToDate",
ti1."InvoiceCurrency",
ti1."ActualInvoiceAmount",
'0',
--tc1."ActualCaseAmount_EUR",
tc1."ActualCaseAmount_HRK",
'0'
FROM temp_invoice_id_amount_1 ti1
JOIN "temp_case_id_amount_1" tc1 ON ti1."CaseID" = tc1."CaseID";
Then i executed these 2 files separately.

Related

Dropdown list with Adodb query

I would like to create the dropdown list with the results from my query. I'm looking for help please because I don't know how to display this results on the list.
The exemple of the list:
My query is :
'DROPDOWN LIST
Private Sub cb_gest_Change()
If Not FSD.cb_gest.MatchFound And FSD.cb_gest <> "" Then
MsgBox "Saisie impossible, le gestionnaire n'existe pas !", , "ContrĂ´le
Gestionnaire"
FSD.cb_gest = ""
Else
FSD.Cells(29, COL_DATA) = FSD.cb_gest
End If
End Sub
'DROPDOWN LIST
Sub init_combo()
Dim Resultat As ADODB.Recordset
Dim Requete As String
FSD.cb_gest.Clear
Requete = "select lb_gestion from DB_GESTIONNAIRE "
Requete = Requete + "WHERE (d_deb_valid <= TRUNC(SYSDATE) OR d_deb_valid IS
NULL) AND (d_fin_valid >= TRUNC(SYSDATE) OR d_fin_valid IS NULL)"
Requete = Requete + " ORDER BY LB_GESTION"
Set Resultat = New ADODB.Recordset
Resultat.ActiveConnection = oBase
Resultat.Source = Requete
Resultat.Open
While Not Resultat.EOF
FSD.cb_gest.AddItem Resultat!lb_gestion
Resultat.MoveNext
Wend
If FSD.Cells(29, COL_DATA).Value <> "" Then
FSD.cb_gest = FSD.Cells(29, COL_DATA).Value
Else
FSD.Cells(29, COL_DATA).Value = ""
End If
End Sub
Thank you for your help !
Consider a different, codeless approach:
Add a new sheet to the host document / workbook
Import the external data from the "Data" Ribbon tab (select "From SQL Server")
Excel creates a ListObject table backed with a QueryTable object that uses a WorkbookConnection that can be configured to automatically refresh on open, or left alone as a one-time pull.
Select the produit column in the ListObject/table; Excel highlights the entire column content and leaves the heading un-selected.
From the "Formulas" Ribbon tab, click the "Define Name" command in the "Defined Names" group.
Name the range ProductsList, verify it refers to TableName[produit] so that it automatically grows and shrinks to fit the column contents.
Change the data validation list to =ProductsList.
Hide the worksheet housing the query and table, if needed.
No code needed, and the validation list will always keep up with the query results as they are refreshed.
Side note, the query appears to be making inefficient cross-joins, and at least one of them is a where-join that can be expressed as an inner join. Are you sure the query is yielding the expected records (I'm suspecting it's yielding a ton of duplicates, depending on how many records exist in the cross-joined tables)?
SELECT prod.cd_produit AS produit
FROM db_dossier sousc, db_produit prod, db_protocole proto, db_tiers tiers, db_personne pers
WHERE sousc.cd_dossier = 'SOUSC' AND sousc.lp_etat_doss NOT IN ('ANNUL','A30','IMPAY') AND sousc.is_produit = prod.is_produit
Instinct would be to remove the tables we're not selecting or filtering anything from - if this query produces the same expected output, then assuming primary and foreign keys are defined I believe its execution plan would be more efficient:
SELECT prod.cd_produit AS produit
FROM db_dossier AS sousc
INNER JOIN db_produit AS prod ON sousc.is_produit = prod.is_produit
WHERE sousc.cd_dossier = 'SOUSC' AND sousc.lp_etat_doss NOT IN ('ANNUL','A30','IMPAY')

SELECT SCOPE_IDENTITY() to return the last inserted ID

There are a few answers about this problem, but my question is about the particular code I have.
I'm trying to get the last inserted ID of this query executing on VBA code.
Public Function Execute(cQry As excfw_dbQuery) As ADODB.Recordset
If pConn.State = 0 Then
OpenConnection
End If
qry = "INSERT INTO [some really long query, which actually works]; SELECT SCOPE_IDENTITY()"
On Error Resume Next
Dim rs As ADODB.Recordset
Set rs = New ADODB.Recordset
rs.Open qry, pConn 'also tried with adOpenKeyset, adLockOptimistic
'some error handling code which is not related to the issue
Set rs = rs.NextRecordset() 'also tried without moving onto the next recordset
pInsertedId = rs.Fields(0).Value
Set Execute = rs 'this is just to pass the query result for SELECT queries
End Function
This should save the last inserted ID on the pInsertedId variable, but instead I get 0 each time I insert a row. The weird thing is, when I copy and paste the same code into the SSMS, it works.
I might just get away with inserting some unique data to some unused column of the database and querying through that.
--UPDATE--
I've just noticed that when running a SELECT query, rs object remains open until it goes out of scope. Here is a screenshot of the watch section:
on an insert statement instead, it gets closed as soon as the query gets executed:
You can explicitly save the results of the insert statement by using an output clause and return the results with a select:
qry =
"declare #Ids as ( Id Int );" +
"insert into MyTable ( Name ) " + ' Assuming Id is an identity column.
"output Inserted.Id into #Ids " +
"values ( #Name );" +
"select Id from #Ids;"
From the documentation for output:
INSERTED Is a column prefix that specifies the value added by the
insert or update operation. Columns prefixed with INSERTED reflect the
value after the UPDATE, INSERT, or MERGE statement is completed but
before triggers are executed.
You can use an output clause to get any data from the rows (Note plural.), e.g. identity column values for newly inserted rows. Output can be used with insert, update, delete and merge and provides access to both before and after values in the case of update. A tool well worth having in your pocket.
As it turns out, the table that I'm trying to insert to has multiple triggers attached to it. So, the query which includes the SELECT SCOPE_IDENTITY(); is actually 4th query. I had to move 4 queries forward in order to get the correct scope. How I pulled that off programatically is as follows. Not very clean (and possibly not the best way to do it), but does the job for me.
I basically go ahead to next recordset until there is none left, which I detect by checking the error number 91 (Object variable or with block variable not set)
Public Function Execute(cQry As excfw_dbQuery) As ADODB.Recordset
If pConn.State = 0 Then
OpenConnection
End If
qry = "INSERT INTO [some really long query, which actually works]; SELECT SCOPE_IDENTITY()"
On Error Resume Next
Dim rs As ADODB.Recordset
Set rs = New ADODB.Recordset
rs.Open cQry.Query, pConn, adOpenKeyset, adLockOptimistic
'some error handling code which is not related to the issue
On Error Resume Next
'begin loop
Do
'go to next recordset
Set rs = rs.NextRecordset()
'if we're getting error n. 91, it means
'recordsets are exhausted, hence we're getting
'out of the loop
If Err.Number = 91 Then
Err.Clear
Exit Do
End If
'if we are not out of recordsets, check the
'result of the query. If it is bigger then zero,
'it means we hit the jackpot.
If rs.Fields(0).Value > 0 Then
pInsertedId = rs.Fields(0).Value
End If
Loop
On Error GoTo 0
End Function
Again, not the cleanest, nor the most correct way to do it, but did the trick. I'm open for any improvements or suggestions.

Truncate String to first wildcard in VBS

I have the following expression, which goes to a SQL table (SQL Server 2008 R2) and grabs the file path to open (usually a PDF or photo) which is formatted to work as a hyperlink in our Access front-ends.
Here's what I'm doing in my VBS file...
Dim Input
Input = InputBox("Enter your WO#", "PDF Search", "ie. 40900500")
set conn2 = createobject("ADODB.Connection")
conn2.open "Driver={SQL Server}; Server=server; UID=user; PWD=password; Database=db;"
dim sql2
msgbox("Testing fixture connection...")
dim rs2
set rs2 = createobject("ADODB.Recordset")
sql2 = "select Photo from Fixtures where Application = '" & input & "'"
rs2.open sql2, conn2
msgbox("It worked! Your photo is here: " & rs2.fields(0).value)
...which eventually returns this...
...and all I need is the file path prior to the first number sign (this message box was just to test the connection and show what data I'm working with).
How would I left truncate the field value I'm returning from SQL up to the first number sign so I can open the file via running a powershell object within my .vbs file?
Any assistance is always welcome! :)
1) You could use following expression to extract file path before first #:
sql2 = "select LEFT(Photo, NULLIF(CHARINDEX('#', Photo), 0) - 1) from Fixtures where ..."
If some rows doesn't contain # then you could use:
sql2 = "select LEFT(Photo, ISNULL(NULLIF(CHARINDEX('#', Photo), 0) - 1, 260)) from Fixtures where ..."
Test:
DECLARE #Photo VARCHAR(260)
SET #Photo = 'J:\DRAWINGS\Folder1\File1.jpg#aaa#'
SELECT LEFT(#Photo, NULLIF(CHARINDEX('#', #Photo), 0) - 1) AS T1
-- J:\DRAWINGS\Folder1\File1.jpg
SET #Photo = 'J:\DRAWINGS\Folder1\File1.jpg'
SELECT LEFT(#Photo, ISNULL(NULLIF(CHARINDEX('#', #Photo), 0) - 1, 260)) AS T2
-- J:\DRAWINGS\Folder1\File1.jpg
2) You should avoid string concatenation in above line
sql2 = "select Photo from Fixtures where Application = '" & input & "'"
and you should use parameterized queries => see SQL Injections.

ADODB Command.Execute Timeout

I am seeing a timeout error when running a query from Excel vba
The query takes <2 seconds from the SQL Server Management Studio, but from vba the timeout happens at 2 minutes with nothing returned
Is there something i am not doing right setting up the Command object? I have noticed that ADODB seems to be slower but never anything like this
The query joins several tables and does some other calculations, but going from 1.5 seconds to >2 minutes must mean something in the vba that I have missed
This is my vba connection string code:
If svrCon Is Nothing Then
Set svrCon = New ADODB.Connection
End If
If Not CheckServerConnectionState Then
conStr = "Provider=SQLOLEDB;Data Source=ussantapps332;" & _
"Initial Catalog=Global_OEE_Data_Capture_Dev;User Id=sqluser;Password=*****;"
' Open the connection
svrCon.ConnectionTimeout = 0
svrCon.Open conStr
End If
This is my vba SELECT code:
Dim cmd As ADODB.Command
Dim par As ADODB.Parameter
Dim rst As ADODB.Recordset
' Create command object
Set cmd = New ADODB.Command
cmd.CommandTimeout = 120
cmd.ActiveConnection = svrCon
cmd.CommandText = sql
' Create parameter object
If IsArrayInitialized(params) Then
For x = 0 To UBound(params)
If IsNull(params(x, 1)) Then
Set par = cmd.CreateParameter(Type:=params(x, 0), Size:=1)
Else
Set par = cmd.CreateParameter(Type:=params(x, 0), Size:=Len(params(x, 1)) + 1)
End If
par.Value = params(x, 1)
cmd.Parameters.Append par
Set par = Nothing
DoEvents
Next
End If
' Open recordset object
On Error GoTo ExecuteError
Debug.Print Format(Now, "hh:mm:ss")
Set rst = cmd.Execute
Debug.Print Format(Now, "hh:mm:ss")
On Error GoTo 0
The sql string and parameters are passed into the function, the connection is opened from another method
The query is:
SELECT U.UnitsID, L.LineName, V.VSName, O.OperatorShift, O.LineLeader, O.CotyOps, O.TempOps, U.WorkOrder, U.ProductCode,
S.ProdDesc, U.TimeLineStart, U.TimeLineEnd, U.UnitsProduced, U.ActLineSpeed, U.TgtLineSpeed, SUM(CASE WHEN C.DTIncludedInOEE = 0 THEN D.DowntimeLength ELSE 0 END),
U.OfflineTaskID, R.Rate, S.LabHrsPerThou, S.PHeads, T.TgtOEE, T.TgtEff, T.TgtProd
FROM dataUnits U
LEFT JOIN dataOperatorNames O ON O.OperatorID = U.OperatorNameID
INNER JOIN setupLines L ON U.LineID = L.LineID
INNER JOIN setupValueStreams V on V.VSID = L.VSID
INNER JOIN setupPUs P ON V.PUID = P.PUID
LEFT JOIN dataDowntimes D ON U.UnitsID = D.UnitsID
LEFT JOIN setupDowntimes sD ON D.DTID = sD.DTID
LEFT JOIN setupDowntimeCats C ON sD.DTCatID = C.DTCatID
LEFT JOIN (SELECT VSID, AVG(RateVal) Rate
FROM dataRates WHERE FYStart >= '2014-07-01' AND FYStart < '2015-07-01'
GROUP BY VSID) R ON R.VSID = L.VSID
LEFT JOIN dataStandards S ON S.ProdCode = U.ProductCode
LEFT JOIN (SELECT LineID, AVG(TgtOEE) TgtOEE, AVG(TgtEff) TgtEff, AVG(TgtProd) TgtProd
FROM dataTargets WHERE TgtMonth >= '2015-03-01' AND TgtMonth < '2015-04-01'
GROUP BY LineID) T ON L.LineID = T.LineID
WHERE (S.SAPVersion = (SELECT MIN(SAPVersion) FROM dataStandards s2 WHERE s2.ProdCode = S.ProdCode)
OR S.SAPVersion IS NULL)
AND P.SiteID = 2 AND U.TimeLineStart >= '2015-03-05 23:00' AND U.TimeLineStart < '2015-03-31 23:00'
GROUP BY U.UnitsID, L.LineName, V.VSName, O.OperatorShift, O.LineLeader, O.CotyOps, O.TempOps, U.WorkOrder, U.ProductCode, S.ProdDesc, U.TimeLineStart,
U.TimeLineEnd, U.UnitsProduced, U.ActLineSpeed, U.TgtLineSpeed, U.OfflineTaskID, R.Rate, S.LabHrsPerThou, S.PHeads, T.TgtOEE, T.TgtEff, T.TgtProd
ORDER BY U.TimeLineStart ASC
The discussion on the chat showed that:
The problem occurs only while connecting to the remote database.
The old SQLOLEDB provider is used.
I suggested to give a chance to a newer provider SQLNCLI which should be more effective while communicating with MSSQL. When a connection string was modified the execution time dropped from 2 minutes to 3 seconds.

Excel connect to Access with low performance

My code want to query access very frequently, I use "for" for every row and check a cell's value if existed in access table. But I feel the performance is very bad. Now I'm using adodb.connection to connect to access. Sorry I cannot put code because its not in my hand. Anyone could help me about how to query a table very frequently from excel vba and with fast performance?
EDIT:
For rowNum = 2 To 1000000
'check if title exists,if yes, get ppid, if not, insert one, get ppid, and make relation in r-table
ppID = isTitleExistReturnID(ppTitle)
If ppID = "0" Then
ppID = addPpReturnID(ppTitle, ppDate, ppJournal)
paperAddedCount = paperAddedCount + 1
isPpAdded = True
Else
isPpAdded = False
End If
Next rowNum
Function isTitleExistReturnID(title As String) As String
Dim r As New ADODB.Recordset
sqlstr = "select * from paper where title = '" & title & " '"
'MsgBox sqlstr
dbConnection.Open
r.Open sqlstr, dbConnection, adOpenKeyset, adLockOptimistic, adCmdText
If r.RecordCount < 1 Then
dbConnection.Close
isTitleExistReturnID = "0"
Else
aidi = r.Fields(0).Value
dbConnection.Close
isTitleExistReturnID = aidi
End If
End Function
Function addPpReturnID(title As String, pubDate As String, journaL As String) As String
Dim r As New ADODB.Recordset
sqlstr = "select * from paper where (1=0)"
'MsgBox sqlstr
dbConnection.Open
r.Open sqlstr, dbConnection, adOpenKeyset, adLockOptimistic, adCmdText
r.AddNew
r.Fields(1) = title
r.Fields(2) = pubDate
r.Fields(3) = journaL
r.Update
maxid = CStr(r.Fields(0).Value)
dbConnection.Close
addPpReturnID = maxid
End Function
The above is part of my code:
check if the item is in access table
if YES, return its ID
if NO, add this item and return ID
Do it for more than 100,000 times, very low performance
Any advice will be so appreciated, thanks in advance.
Any RBAR (row by agonizing row) approach has the potential to be a performance challenge --- that's why it's called agonizing.
And yours is definitely RBAR because you do stuff separately for each of a million (For rowNum = 2 To 1000000) spreadsheet rows. Compounding the problem is the fact that you're opening and closing ADODB objects (connections and recordsets) at least one, but sometimes twice, for each of those rows.
Try to find a set-based approach instead. For example, if you can drive this operation from Access ...
Create a link to the spreadsheet.
Create an "unmatched" query (there is a query wizard to guide you) to select those spreadsheet rows whose ppTitle does not exist in the Access destination table.
Create an "append" query to add those unmatched rows to the Access table.
I don't know whether that outline is a suitable fit for your situation or whether it's close enough that you can adapt it. But the more important point is to find a set-based method instead of RBAR.

Resources