I'm currently using VBA to automate pulling of large data-sets. However, it often happens that either
the sub simply gets stuck and does not update any more/connection breaks (application is still responsive) or
the application becomes totally unresponsive.
Is there a way to implement a time-out function which automatically closes the application after say 15 minutes elapse?
In other programming languages, I would have solved this using multi-threading. But not sure if something similar is feasible in VBA given that it is entirely single-threaded.
The basic algorithm I employed to automate the data looks like this:
Sub automateFetchData()
Dim requestTable As Workbook
Set requestTable = Workbooks.Open(requestTablePath)
Run ("'" & requestTable.Name & "'" & "!fetchData")
' This is where the application starts fetching data and occasionally gets stuck
debug.print "Data retrieval complete"
End Sub
For those familiar with Thomson Reuters Datastream, I am using request tables and the pre-programmed macro within the request table file to connect and fetch data from the datastream server.
Therefore, I unfortunately do not know/understand in detail what is happening when I run:
Run ("'" & requestTable.Name & "'" & "!fetchData")
I just know that once I call it, a progress bar appears looking like this:
Which then transitions to:
If the data retrieval is successful, the progress bar disappears, output is pasted to a worksheet and the code above reaches the
debug.print "Data retrieval complete"
Statement.
If it is not successful, it just freezes with the progress bar from the second screenshot still open and running. At this point, I would want the time-out to kick-in and exit the function which called
Run ("'" & requestTable.Name & "'" & "!fetchData")
Please excuse me for being so vague in the description of the issue. I'd be glad if someone could nonetheless help me out! Always happy to share more info where I can. Cheers!
Related
Cleanly cancelling a long API-Ridden procedure is hellish, and I'm attempting to work out the best way to navigate the hellishness.
I'm using excel 2016 (with manual calculations and no screen updates) - I may take some time to attempt to run the procedure on 2010 to see if any issues resolve in the coming days (I'm aware of the slowdowns).
Over time, my procedure LongProcedure has lost its ability to successfully use its cancel feature (presumably due to increasing complexity). It was initially inconsistent and required significant spam-clicking to cancel, and now it fails altogether
Here's the setup:
First, LongProcedure is in a class module LongClass with a public property checked against for early cancelling, allowing it to clean up.
Public Sub LongProcedure()
' [Set up some things] '
For Each ' [Item In Some Large Collection (Est. 300 Items)] '
' [Some Code (ETA 5 Seconds) Sprinkled with 3-4 DoEvents] '
' [Export workbook (ETA 10 Seconds)] '
If (cancelLongProcedure) Then Exit For
Next
' [Clean up some things] '
GeneratorForm.Reset ' Let the UserForm know we're finished
End Sub
Second, I have a UserForm shown from a macro, which instantiates the procedure class, and runs the procedure. It contains a run button, a status label, and a cancel button.
Private MyLong As LongClass
Public Sub ButtonRunLongProcedure_Click()
Set myLong = New LongClass
myLong.LongProcedure()
End Sub
So the issue overall is twofold.
The ExportAsFixedFormat call opens a "Publishing..." progress bar which freezes excel for around ten seconds - fine. In all of my efforts, I haven't found a single way to process user input while this is happening.
On top of this, the DoEvents calls seemingly no longer do anything to allow the cancel button to be clicked. The process inconsistently freezes excel, tabs into other open programs, and (when not freezing) updates the status label.
I've Tried:
Appending DoEvents to the SetStatusLabel method instead of sprinkling - while the form still often freezes, it otherwise updates the status label consistently (while still not allowing the cancel button)
Using winAPI Sleep in place of, and in addition to DoEvents with delays of 1, 5, 10, 50, and 250ms - The form simply stopped updating at all without doevents, and with both it froze more.
Using a Do While loop to run DoEvents constantly for one second (Froze)
Overriding QueryClose to cancel the form. This one helped significantly. For some reason, the close [x] button can be clicked far more consistently than the userform buttons - Still not as consistently as I'd like. The problem? during publishing, Excel stops responding, and as such, modern windows will end the process if you click the close button twice... without cleanup.
Using Application.OnTime to regularly call DoEvents. Didn't seem to improve the situation overall
Alt-Tabbing. No, really. for some reason, while alt-tabbing occasionally just makes the UserForm freeze harder, sometimes it makes it stop freezing and update.
This is an issue I'm willing to do significant refactor work for, including smashing up the idea of the long procedure into separate methods, performing setup initially, and cleanup on class termination. I'm looking for something that provides consistent results. - I'll accept anything from excel versions to excel settings to refactors to winAPI calls.
Thanks for any insight into this one.
As it turns out simply combining together some of the useful improvements, along with a new one, made all the difference.
QueryClose is up to personal preference. Leave it in to catch more terminations, leave it out to ensure users use the new solution
Stick to sprinkling doEvents in places you feel are logical (not just when the status bar updates - like before and after an Application.Calculate call)
Optimize the long-running process as best you can, avoiding excel calls
And, most significantly
The integrated cancel key feature (CTRL+Break by default) is significantly more responsive than UserForm buttons and the form close button, without the chance of accidentally ending the excel task.
Here's the process to polish that for a finished product
First, set up a debugMode, or the inverse handleErrors, module-level variable to control whether to implement break-to-cancel and error handling. (Error handling will make your code harder to debug, so you'll appreciate the switch)
If your process is handling errors, you'll set Application.EnableCancelKey to xlErrorHandler, and On Error GoTo [ErrorHandlingLabel]. The error handling label should be directly before cleanup, and immediately set EnableCancelKey to xlDisabled to avoid bugs. Your handler should check the stored Err.Number and act accordingly, before continuing on to the cleanup steps.
Ensure that if you defer to any other complex vba in your script (such as using Application.Calculate on a sheet with UDFs), you set On Error GoTo 0 beforehand, and On Error GoTo [ErrorHandlingLabel] after, to avoid catching cellbound errors.
Unfortunately, the drawback is that for the UX to be consistently readable, you'll have to leave the cancel key on xlDisabled until the form is closed.
And in code:
Public Sub LongProcedure()
If handleErrors Then
On Error GoTo ErrorHandler
Application.EnableCancelKey = xlErrorHandler
End If
' [Set up some things] '
For Each ' [Item In Some Large Collection (Est. 300 Items)] '
' [Some Code (ETA 5 Seconds) Sprinkled with 3-4 DoEvents] '
' [Export workbook (ETA 10 Seconds)] '
Next
ErrorHandler:
If handleErrors Then
Application.EnableCancelKey = xlDisabled
If (Err.Number <> 0 And Err.Number <> 18) Then
MsgBox Err.Description, vbOKOnly, "Error " & CStr(Err.Number)
End If
Err.Clear
On Error GoTo 0
End If
' [Clean up some things] '
GeneratorForm.Reset ' Let the UserForm know we're finished
End Sub
and in the UserForm
Private Sub UserForm_QueryClose(Cancel As Integer, CloseMode As Integer)
If MyLong.handleErrors Then Application.EnableCancelKey = xlInterrupt
End Sub
A small note that this method will likely generate a few bugs you weren't expecting to encounter because the execution jumps directly to the specified label. Your cleanup code will need to have required variables instantiated from the start.
Overall, once these issues are fixed, this setup ensures that the user can click CTRL+Break as many times as they could possibly want without causing crashes or popups.
I am trying to make a macro that returns at least some partial control to the user when it runs. The macro behavior cannot be changed dangerously if the user edits cells, as there is few dependent content.
I am using this method to allow the user to occupy the execution thread and commit their cell changes before the macro gains control back and proceeds to the next statement:
Sub retCtrl(Optional ms As Long = 350)
Dim l As Long: l = Timer
Do While Timer - l < (0# + ms / 1000)
DoEvents
Loop
End Sub
I put this after some long statements and inside loops.
It's choppy, but functional, with only one problem:
if the user takes to long to edit the cell, the macro silently stops.
Can anyone explain why this happens? Is it because the timer elapses the threshold? I thought all macro execution stops when the workbook is in edit mode?
Any advice on how to handle this otherwise?
You can't & shouldn't return control to the user while code is running, and your issue is an example of why.
Excel expects certain elements to be in certain states, and when the user starts changing things, unexpected things can happen. For example, what would happen if the user tries to change data the same time Excel needs it? There is no multi-processing built in, "this isn't that kind of programming..."
So Excel ceases running the macro when it notices something happening. If you need to run a different process simultaneously, do it in a separate instance of Excel.
More Information:
Running Macros in the Background
Stack Overflow : How can I execute a long running process in VBA without making pc crawl?
Mr. Excel : Can VBA Run in the Background?
A little background:
I am writing some very basic error handling for a macro that runs some pretty important company processes.
As nobody here at work is a true software engineer, the time we can devote to writing error handling routines is not significant.
Anyway, there's a routine that runs every night on a schedule and it processes several reports (in excel) that were completed throughout the day. This routine runs fine for the most part, but since it occurs in the background I have no way of knowing if something went wrong during the previous night's update unless I check on it every morning.
So the error handling I have in place at the moment is that on error - any error - it sends me an email saying "hey something went wrong" (or something to that effect). This lets me aware of any intermittent errors that pop up before the other departments see them, without having to "helicopter parent" the system, so to speak.
My question is:
I have this fantasy in my head that within the email, it would have the text of the line where the error occurred. It's not the biggest deal not to have this, but it would speed up the process of troubleshooting a little bit.
Googling this is proving a little frustrating... does anyone know if this is possible, and how I would go about doing this?
You can send yourself a line number so that you could find the offending line
The line numbers can be added easily using Mz-Tools
Sub ErrTest()
10 On Error GoTo ErrHandler
20 Range("Derp").Select
30 On Error GoTo 0
40 Exit Sub
ErrHandler:
50 Email "example#email.com", _
Subject:="Error " & Err.Number, _
Body:="Error " & Err.Number & "(" & Err.Description & ") occured in ErrTest on line " & Erl & "."
End Sub
You can retrieve a line of code if you have "Trust access to the VBA project object model" checked in the trust center.
Sub Test()
Dim LineCount As Long
LineCount = ThisWorkbook.VBProject.VBComponents("Module1").CodeModule.CountOfLines
'Print every line in Module1 (lines 1 to LineCount)
Debug.Print ThisWorkbook.VBProject.VBComponents("Module1").CodeModule.Lines(1, LineCount)
End Sub
This could be used to email yourself the specific line number. You would have to add line numbers to every line of code and use Erl in order for this to work.
In order to head off a storm of "comment it out" replies, here is my situation:
I have a process is normally run 1 iteration by 1 iteration. A user manually hits a button that calls a macro which, upon completion, pops up a message box that reports the total time length the macro ran for. It's pretty handy for diagnosing issues. This code is locked down and I cannot modify it.
I am trying to do this at scale. Because the code in the main spreadsheet and workbook are locked, I have a separate workbook open in the same instance of excel with a macro that operates the locked down workbook. Rather than 1 by 1, I've got a set of 300 I'm trying to run through. Right now I have to babysit the thing and hit space to get past the MsgBox. Does anyone know of any tricks to prevent me having to monitor the thing? Either disabling the pop-ups or some way to make them non-modal. Maybe a trick to make the mouse click?
You're right in knowing that the best way to fix the issue is to correct the code. In which case you would probably make the pop-ups toggle-able.
However, I wrote this for you which could be used as a potential work around. It utilizes VBScript to "sort-of" simulate multithreading so that you can send a key to the modal Msgbox. Assuming you can do what you want to do via code, simply call SendDelayedKeys before the action that will cause a Msgbox. You may have to tinker with the Delay based upon your circumstances as 100 milliseconds may not be enough. To change the Delay, just call like this: SendDelayedKeys 500 for 500 milliseconds.
Sub SendDelayedKeys(Optional Delay As Long = 100, Optional keys As String = """ """)
Dim oFSO As Object
Dim oFile As Object
Dim sFile As String
sFile = "C:\SendKeys.vbs" 'Make this a valid path to which you can write.
'Check for the .vbs file.
If Not Len(Dir$(sFile)) Then
'Create the vbs file.
Set oFSO = CreateObject("Scripting.FileSystemObject")
Set oFile = oFSO.CreateTextFile(sFile)
oFile.WriteLine "Set WshShell = WScript.CreateObject(""WScript.Shell"")"
oFile.WriteLine "WScript.Sleep CLng(WScript.Arguments(0))"
oFile.WriteLine "WshShell.SendKeys WScript.Arguments(1)"
oFile.Close
End If
Shell "wscript C:\SendKeys.vbs " & Delay & " " & keys
End Sub
Sub ProofOfConcept()
'Using default parameters which sends a space after 100 milliseconds
SendDelayedKeys
MsgBox "I disappear on my own!"
End Sub
A word of warning: Any solution that utilizes SendKeys is a fragile solution and should be avoided when possible. However, when your options are limited and you need to avoid a manual process, sometimes it's your only option.
Since SiddhartRout rightly pointed out that this could be solved using API calls: here's a link with C# code that would close your msgbox every second.
The problem here really isn't strictly a problem more code can (or indeed should) solve.
There are a great many things to consider and any solution will be more complex AND less reliable than the problem it is initially trying to solve. But lets look at your options...
SendKeys is not reliable for that kind of use, what happens if the dialogue says "would you like me to save this workbook?" just after making a change that was meant to be temporary or "would you like to play global thermonuclear war?" Plus with a batch process like that you want to get on with something else while you wait, even if it's only to come here to downvote trolls. If nothing else you may not be in control of this code so what kind of mess will it cause when the maintainers realise msgbox is bad UX and kill it?
FindWindow API calls would let you check the content in the window to make sure it says what you're expecting but then you're potentially asking some bit of quick & dirty vbscript to go into a race condition until the right window comes up. Can you guarantee that the threads won't lock up?. What about platform issues - what happens if someone wants to run your code on their shiny new surface? What happens when your 64 bit modal Excel dialogue window can't be seen by the 32-bit api calls you were making? What about a new version of office that doesn't present modal dialogues in the same way? None of those problems are insurmountable but each adds complexity (aka opportunity for failure.)
The best solution is fix the actual problem you have identified from the outset which is that the original code throws up an unnecessary modal dialogue. Someone needs to fix that - it doesn't have to be you but if you estimate how much time that modal dialogue wastes in lost productivity that should get you a solid business case for getting it sorted.
we have a third party program which uses OLE to write Data to Excel. Unfortunately, the programmers of that application do not understand that making excel invisible and hiding updates speed up excel enormously.
At this moment, large jobs take up to 15 minutes to complete which is quite annoying.
Therefore I want to write an Addin, which shuts down visibillity etc. when the program writes to excel. Is there a way to do that? It is the only application which uses this technique, so detecting COM / OLE Interaction would be sufficient.
Rather that have excel check to see if it is in the hands of OLE automation, it is possible to check windows for the name of the process that is runnning the OLE
something like
Function isRunning(ProcessName) As Boolean
Dim objWMIcimv2 As Object
Dim objList As Object
Set objWMIcimv2 = GetObject("winmgmts:" _
& "{impersonationLevel=impersonate}!\\.\root\cimv2") 'Connect to CIMV2 Namespace
Set objList = objWMIcimv2.ExecQuery _
("select * from win32_process where name='" & ProcessName & "'")
If objList.Count > 0 Then
isRunning = True
End If
End Function
From there you can suppress screen updating pretty easily. I am making a guess that the third party application doesn't usually sit running idle on the users desktop. if it does then obviously this approach won't help you.
Code inspired from this link on terminating programs