For complex reasons, I want to automate the calling of a Bloomi BQL query in VBA.
I am changing the inputs to a BQL.Query formula in the Excel sheet from a VBA script, and calling Application.Calculate to run the query. The display changes to "N/A Requesting Data ...". In the VBA I wait a bit (using Wait()) and even throw in a DoEvents() for good measure.
While rngOS.Value < 0
Application.Calculate
Sleep 2000 'Waits 2000 ms
DoEvents
Wend
Trouble is, the BQL.Query just doesnt return. Only when I end the VBA script, does the actual data come back into the worksheet.
Ideally I would have a synchronous way to just call BQL.Query from VBA and wait for the return.
All suggestions welcome!
Here is a pointer: https://stackoverflow.com/a/33667663/829571
In substance: schedule your function to run a bit later, check if BQL is done (for example by counting the number of "N/A Requesting Data") and if not done, schedule the function a bit later again, etc. Once the count of N/As is down to 0, you know the update is finished and you can proceed with the rest of your code.
Related
In an Excel spreadsheet/VBA script I'm making, I need to call data from a database, and refresh the values every 5 minutes. The program starts from the push of a button, and should run continuously until the user breaks the execution. I'm currently not sure how to make Excel/VBA 'wait' 5 minutes without pausing the spreadsheet and, ideally, without being computationally inefficient.
I've tried using the "Application.Wait" and "Sleep" functions, but both of those pause the spreadsheet during the 5 minute wait.
My current solution is to use a "While" loop with "DoEvents" inside it, as shown in the code below. This makes the program just run the "While" loop for 5 minutes, and it does not pause the spreadsheet thanks to "DoEvents". However, while the spreadsheet is usable, this is computationally inefficient, since the program execution isn't technically paused, it's just running the "While" loop continuously, and some of the slower computers that may end up using my program might be significantly slowed by this.
My current solution is as follows:
Sub MainProgram()
'dimension variables, open database connection, etc.
Do While 1 < 2 'ad infinitum
'get database data, write to spreadsheet, etc.
WasteTime()
Loop
End Sub
Sub WasteTime()
EndTime = Now + TimeSerial(0,5,0)
While Now < EndTime
DoEvents
Wend
End Sub
The problem with this, as mentioned above, is the computational inefficiency. CPU utilization is fairly high throughout the WasteTime loop. So I'm wondering, is there any way to pause the script without pausing the spreadsheet and without running the code continuously, thus burdening the CPU?
As BigBen mentions in comments Application.OnTime is the best option for this. It avoids the overhead you describe by scheduling a second macro to be called at a future time.
Below is an example. You can modify the wait time with the constant variable. These should be within the same Module (or change theCalculation macro to not be private).
Sub TheTimerMac()
'enter in timevalue (hour:Minute:Second)
Const DelayTime As String = "00:05:00"
Dim nextRunTime As Date
nextRunTime = Now + TimeValue(DelayTime)
'Schedules application to execute below macro at set time.
Application.OnTime nextRunTime, "TheCalculation"
End Sub
Private Sub TheCalculation()
'whatever you use for your calc here
Application.CalculateFull
'This will restart the timer portion.
Call TheTimerMac
End Sub
I have an Access (2013) database that I use to store/process all of our data received from a Qualtrics online survey. The raw data downloaded from Qualtrics is in a csv file that's poorly formatted for importing to Access, so I've got a fairly complex Macro in Excel (2013) that I use to pre-process the data before importing to Access.
In Access, I use the following code to open the excel file that contains the macro, run the macro, save the workbook, and close it. This had been working well several for several months, but now when I run it, it stops near the end of the Excel Macro with the run time error: -2147417856 Automation error System call failed.
ActivateOrOpenWorkbook WbkName & ".xlsm", strWbkPath
appExcel.Run ProcName, ProcArg
appExcel.Workbooks(WbkName).Save
If appExcel.Workbooks.Count = 1 Then
appExcel.Quit
Else
appExcel.Workbooks(WbkName).Close True
End If
ActivateOrOpenWorkbook is just a custom function to do exactly what the name implies, appExcel is the Excel Application. The workbook always opens fine, and the macro begins to run, but it never actually reaches the point where control returns to the Access VBA and saves the workbook.
It does run fine if I open the Workbook before running the Access procedure, insert breakpoints at every major VBA step (in both Access and Excel), and step through the whole thing one Sub at a time. It just fails if I try to let VBA run it all from start to finish on it's own.
Based on that evidence plus stories of similar problems that I've seen online, I suspect that the error is occurring because the Excel macro is taking too long to run (we recently added some new variables to the Qualtrics survey), and Access is trying to take back control before Excel is finished. I just haven't found any viable way to solve that suspected problem or investigate it further.
I did try inserting this makeshift Wait routine into my ErrorHandling for the Access Sub, but it didn't work at all, because the error message still popped up in the same amount of time as before.
If Err = -2147417856 Then
TWait = Time
TWait = DateAdd("s", 15, TWait)
Do Until TNow >= TWait
TNow = Time
Loop
Resume Next
Any help will be appreciated!
Have you tried using DoEvents in the pre-processing macro?
If you have a loop where you're processing the CSV file line by line then use DoEvents every so often to give any pending events a chance to run. It would be overkill to call it on every line so maybe start with every 100 lines and adjust from there.
Given that the macro works when you step through the Subs, it would appear that it isn't the overall execution time that's the problem. If Access is throwing an error because Excel appears to be unresponsive then DoEvents might be enough to stop Access from giving up
I have an excel sheet that has dde links to real time market data. I use a timer to watch the dde prices every second. Then submit orders when certain conditions are met. I tried an infinite loop with DoEvent in the middle which works for 5 seconds then freezes the workbook.
Is there a way to respond to changes in dde updates? Change event doesn't detect them. It just detects when user makes manual change.
I was told that if I have conditional formatting there's a way to pick up that event. So I can create a cell formula to turn true when my condition is met and then conditional format that cell to some formatting when it's true and then pick up the format change event. Is that possible? If so how. Any suggestions would be appreciated.
To Clarify: I want to pick up an event IN VBA that would submit an order to trade a stock. The way i'm doing this right now is with a timer which loops over all the rows looking for true cell in the trigger column. Once found it shuts off the flag for that row (sets the true condition to false) and submits the order.
The problem is that one second is an eternity for fast moving stocks. So I need an event to be thrown in VBA when a cell in trigger column turns true so I can respond immediately and not wait for the second interval of the timer class.
As far as I know, you can't call on the timer with a value of less than a second. If I could use milliseconds my problem would be solved. I'd just loop over the list every 10 milliseconds.
As far as I know I can't create another thread in VBA. If I could I would make an infinite loop and put it to sleep after every iteration for 10 milliseconds or so.
As far as I know, I can't pull dde directly into VBA or even .net for that matter since MSDN says it's no longer supported.
I hope this clarifies. All suggestion are appreciated.
If you create a dummy function which has your DDE output cells as parameters then you should be able to respond to the Worksheet_Calculate event?
I'm guessing that might work, but I've no experience with DDE: the DDE update may even trigger the Calculate event directly.
If you are asking if you can use conditional formatting to trigger an event, yes that is possible. I am not familiar with the DDE model myself, however, and mixing conditional formatting to trigger an event from a data condition does seem like an extra step, as Stepan1010 notes.
You may want to consider the discussion in Mr Excel in this link, as the issue centers around changing a cell value based changes made to cells from a DDE connection: http://www.mrexcel.com/forum/showthread.php?176508-Comments-VBA-amp-Min-Max
You may also consider using a DoEvent with a loop set for a time period based on how long you will actually implement the macro, if that applies to your application. This SO article is focused on a status bar, but I think the same logic applies in terms of an event execution, say based on a conditional within a loop: Force a screen update in Excel VBA
Hopefully this is helpful to you =)
~JOL
Why don't you just recreate whatever conditional formatting logic you have in that cell into a seperate cell?
For example your conditional formatting logic might highlight a cell when it is above a certain number - you could just put that logic in another cell- eg. =if(A1>100,TRUE,FALSE)
So I guess my question is - Why pick up the format change event when you can just pick up the event itself?
Edit for your clarification:
If you want to run a macro continuously in VBA you don't need a timer - you can just do a continuous loop like this:
Sub macro1()
Dim i As Double
With Sheet1
Do
'.Cells(5, 4).Value = i
i = i + 1
.Cells(1, 1).Value = i
' you are going to want to comment this out if you want to don't need to do other things
DoEvents
If Sheets("Sheet1").Range("A2").Value = True Then
' put your code here.
End If
Loop
End With
End Sub
So I must still be having trouble understanding your situation.
I have a rather large workbook that takes a really long time to calculate. It used to be quite a challenge to get it to calculate all the way, since Excel is so eager to silently abort calculation if you so much as look at it.
To help alleviate the problem, I created some VBA code to initiate the the calculation, which is initiated by a form, and the result is that it is not quite as easy to interrupt the calculation process, but it is still possible. (I can easily do this by clicking the close X on the form, but I imagine there are other ways)
Rather than taking more steps to try and make it harder to interrupt calculation, I'd like to have the code detect whether calculation is complete, so it can notify the user rather than just blindly forging on into the rest of the steps in my code. So far, I can't find any way to do that.
I've seen references to Application.CalculationState, but the value is xlDone after I interrupt calculation, even if I interrupt the calculation after a few seconds (it normally takes around an hour).
I can't think of a way to do this by checking the value of cells, since I don't know which one is calculated last. I see that there is a way to mark cells as "dirty" but I haven't been able to find a way to check the dirtiness of a cell. And I don't know if that's even the right path to take, since I'd likely have to check every cell in every sheet.
The act of interrupting calculation does not raise an error, so my ON ERROR doesn't get triggered.
Is there anything I'm missing? Any ideas?
Any ideas?
I think the trick you need to implement (if you're application runs in Excel 2007 or later) is to handle this with the Application.AfterCalculate event, which is raised after both calculation is complete and there are no outstanding queries.
If you've never worked with events in VBA before, there is a good overview from cpearson.com.
The (MSDN) solution by Charles Williams above worked for me where I had 1000's of VLOOKUP's that neeeded to recalculate as the code was changing the lookup value because of an iteration loop. Results were skewed as calculations were not running to 100% completion.
At the beginning of my subroutine the code executes
Application.Calculation = xlManual
This eliminated unnecessary calculations by Excel until I was ready.
Now at the critical point the code executes
Application.Calculation = xlAutomatic
ThisWorkbook.ForceFullCalculation = True
Application.Calculate
Having forced Excel to perform a full calculation, the code could then saved the result and move onto the next iteration ... but before doing so
ThisWorkbook.ForceFullCalculation = False
Application.Calculation = xlManual
Remembering at the very end
Application.Calculation = xlAutomatic
I've never actually used it but I think this might work to prevent calculation from being interrupted.
Application.CalculationInterruptKey = xlNoKey
I think I'm hearing that you need a way to monitor whether each step within the calculations being performed was executed.
Assuming that you're not interested in re-engineering the workbook to use methods that are easier to track than spreadsheet calculations (such as volatile calculations within VBA or Pivot Tables), this may work for you:
Within VB, you can utilize .EnableCalculation and .Calculate to set an entire worksheet as "Dirty" (needing calculation) and then recalculate. The key difference between this and your current process is that we will perform these actions one worksheet at a time in manual mode. By initiating the calculations one worksheet at a time from within VBA, you will be able to perform additional intermediate actions that can be used to track how far you got in the calculation process.
Please note that this approach assumes a fairly linear workbook structure such that your workbook will produce the correct results if we first recalculate Sheet1, then Sheet2, Sheet3, and so on, in whatever order you wish. If your formula dependencies are more "spaghetti" than linear, this probably won't work for you. It also assumes you are working in Excel 2000 or later.
For example, you could write a VBA routine that accomplishes the following steps.
You will need to know your dependencies in order to know which calculations must come before others, and start with the worksheet in a "clean" state where no calculations are currently pending.
Step 1: Set the active sheet to the first worksheet where recalculation is needed
Step 2: Set the calculation mode to manual as follows:
Application.Calculation = xlCalculationManual
Step 3: "Dirty" the entire active sheet as follows:
With ActiveSheet
.EnableCalculation = False
.EnableCalculation = True
Step 4: Initiate a recalculation for this worksheet only (not the entire workbook) using:
.Calculate
End With
Note that if the calculation mode were set to automatic, Step 3 would initiate a re-calculation across the entire workbook. By using manual mode and With, we are constraining that calculation to the current sheet.
Now you have dirtied and re-calculated the first sheet (hurray!). Now, by embedding Steps 3 and 4 above into a For/Each or For/Next loop, you can repeat the process for each worksheet in your workbook. Again, make sure you know the order in which your worksheets need to be calculated (if an order is needed).
Now for the big finish - by creating a counter variable within your loop, you can track how far you got in the calculations by updating your counter variable value each time you complete a worksheet calculation. For example, after you recalculate a worksheet, you can set the counter value to current value + 1 and store the results either in a global variable (so that it will persist even after your VBA routine ends), or in a cell within your worksheet. That way, you can check this value later to see how many worksheets were updated before the calculations finished or were interrupted.
If you have relatively few worksheets in your workbooks, the same approach could be applied to one range at a time rather than a sheet.
I won't go into detail about how to construct a "counter", loops, or global variables here, but if needed, this information can be easily found using your favorite search engine. I would also highly recommend re-enabling automatic calculations once you are done as it is easy to forget that it's been set to manual mode.
I hope this works for you - for more information on calculation modes and recalculation, this is a helpful link:
http://msdn.microsoft.com/en-us/library/bb687891.aspx
Perhaps the following would work:
Do Until Application.CalculationState = xlDone
DoEvents
Loop
Can't say I've tested it, nor that I know how robust the functionality of Application.CalculationState really is to determine whether 'complete' calculation occurred, as opposed to something interrupting the process and flagging the calculation state as done.
Private sub SomeCodeThatGeneratesFormulas
Application.Calculation = xlCalculation.xlCalculationManual
'...Some formulas are copied here'
Application.OnTime DateTime.DateAdd ("s",.01,DateTime.Now), "Module1.CalculateFullRebuildAndSubsequentSteps" 'By using Application.OnTime, this method will be called in a way that locks the end-user out of providing inputs into Excel until the calculation itself is complete.
end sub
public sub CalculateFullRebuildAndSubsequentSteps
Application.CalculateFullRebuild
'...Do next steps, i.e. paste as values'
end sub
On the status bar, right hand side, it will say Calculating (N processors) X% (where N is the number of processors on your computer and X% is how much it has completed) when recalculating. If you don't see text there, it's not recalculating.
I'm using Office 2010, but it should be there in all versions. It's just kinda subtle so it's easy to miss.
Arrays in Excel can be a bit stupid. That is that in order to accomplish some tasks people avoid to use intermediate columns/rows to store (temporary) data, so arrays have to recalculate staff from the beginning every time, thus getting really slow. My Suggestion would be:
fix arrays to avoid multiple searches. Use hidden cells or even hidden sheets
Avoid using A:A and rather use A1:A1000 specially in excel 2007 or later
use formulas to equal zero or error (ex: NA()) while previous items aren't calculated, so you can clearly see if an operation is done at all.
some VBA could be used to inject formulas in place one step at a time, perform calculations, then proceed to next step, but this could mean lots of work...
I'm using COM in LotusScript (Lotus Notes) to make Excel print several sheets in one workbook to PDFCreator, then make PDFCreator combine then into one PDF.
The problem is that calling Excel's PrintOut method immediately followed by PDFCreator's cCombineAll method results in one or more sheets being omitted from the PDF. It seems like Excel's PrintOut method returns before printing is complete.
Putting a Sleep in my code works, but may not be reliable as the printing time varies, so...
Is there any Excel property or method I can call to determine whether printing has finished?
Alternately, is there a way make the PrintOut method block until printing is finished?
I haven't been able to find an answer in Excel's VBA Help.
This sample code from excelguru.ca suggestst you need to use the cCountOfPrintJobs property of PDFCreator to monitor when the job starts and stops printing. Once printing is complete, you can perform whatever other operations you need
Set pdfjob = New PDFCreator.clsPDFCreator
' ...missing out various initialisation steps
'Print the document to PDF
ActiveSheet.PrintOut copies:=1, ActivePrinter:="PDFCreator"
'Wait until the print job has entered the print queue
Do Until pdfjob.cCountOfPrintjobs = 1
DoEvents
Loop
pdfjob.cPrinterStop = False
'Wait until PDF creator is finished then release the objects
Do Until pdfjob.cCountOfPrintjobs = 0
DoEvents
Loop
pdfjob.cClose
I think the other method is to set the "Background" argument of "PrintOut" to false.