Too long for a comment on the question that prompted this, so I'll ask here.
Is it safe (at least from the problems I talk about below) to use callback function pointers in VBA code; given its slightly unusual relationship to the host application and the quirks of the interpreter?
This question about using WinAPI to register callbacks discusses potential pitfalls which lead to crashing when the OS invokes the callback function. Specifically, it asks why interacting with the worksheet occasionally leads to problems in the TIMERPROC callback. My immediate thought upon reading the description was that the Excel Object Model doesn't work well with asynchronous programming.
The question includes an extract of code which "works", I won't include it here though because I think this answer to a related question actually illustrates it much more succinctly:
I was able to solve it with
On Error Resume Next
[...]
Sub TimerProc(ByVal HWnd As Long, ByVal uMsg As Long, ByVal nIDEvent As Long, ByVal dwTimer As Long)
On Error GoTo BeforeExit
Range("A1") = "test"
BeforeExit:
End Sub
...that confirmed my idea that the problem was probably coming from interacting with the object model (Range("A1") = "test") asynchronously whilst it was in some locked state, and could be solved by catching the error with OERN.
In that original question, #NigelHeffernan gave a great answer about what might be causing a crash when Windows invokes the callback function, which he chalks down to 3 reasons:
[developers using the API] are assumed to
have put the error-handling and contingency management in place in the
calling code.
Those assumptions are:
There will definitely be a valid function behind that pointer;
It definitely be available when it is called;
...And it will raise no errors to the caller.
Point 3 is (I think) most relevant to the specific question, and what I assume to be the cause of OP's crashes.
What I'm concerned about is that some of the comments (1,2) imply (to my mind at least) that point 2 might be to blame; that Excel being in "edit mode" somehow suspends VBA code from running and means the OS can't run the callback, but instead gets an "Excel is busy" error.
I know that VBA code often runs in the host's main/UI thread, which means that if Excel is busy running other UI stuff then VBA can't execute. However the message loop which reads the WM_TIMER message and ultimately runs the callback necessarily lives in the same thread as my VBA code; so the callback won't be called when Excel is actually busy processing other UI messages. Sure the Excel Object Model may be in some state that means trying to access it will raise errors, but then the issue is raising an error to the caller, not that the callback is unavailable.
Meanwhile another comment from the OP and some explanation given later on imply that their problem was actually the third factor (point 1 - the function pointer was invalid). What they've written seems to suggest that the TimerProc behind the function pointer is being Garbage-Collected and invalidated at some point during execution, OERN keeps it alive by holding a reference to it.
This whole idea seems really weird to me though; functions are not 1st class citizens in VBA and I don't think they follow COM reference counting (what would hold the reference to them anyway, a pointer is just a number not a reference?). I assumed that function pointers were set at compile-time, and they seem to remain valid even between End statements and reset buttons, so I'm not sure what would make them go out of scope as OP suggests. I also think that if they did it would have consequences for VBA code in general, not just when using APIs. So I'm not sure whether this is actually a problem to worry about.
TL;DR
When using callbacks in asynchronous single threaded code (such as with the SetTimer API), is there any risk of:
Callbacks falling out of scope between calls so that the function pointers provided to APIs become invalid
Excel/ the host application being in a "busy state" when the callbacks are invoked so that the calls fail
I can imagine why these might be problems across threads, but I thought the whole message queue system that Windows is built on would prevent this from being a problem for single threaded VBA code.
Or in other words, am I right in thinking that:
The host application should have no influence on the "availability" of VBA code on that thread
There is no reason for a function pointer to become invalid (un-callable) after I hit F5 - except perhaps for some manual memory mischief
Oh and please do read those comments in context, it's perfectly possible that I've totally misinterpreted them:) (I don't have much experience in this area at all)
Related
We have been facing a very strange issue with one of our RPGLE programs that bombs intermittently with the subjected error.
This happens specifically at a line where a write operation is performed to a subfile record format. I have debugged and checked all the values assigned to variables during runtime and could not find absolutely no issues. As per the https://www.ibm.com/support/pages/node/644069 IBM page, I can only assume that this might be related to the parameter definitions of the programs called within the RPG. But I have checked the parameters of each and every prototyped program call and everything seems to be in sync.
Can some one please guide on the direction to go to find out the root cause of this problem?
But I have checked the parameters of each and every prototyped program
call
Assuming you're using prototypes properly, ie. there is one prototype defined in a separate source member and it is /INCLUDE into BOTH the caller and the callee...
Then prototype calls aren't the problem, as long as you're properly handling any *OMIT and *NOPASS parameters.
Look at any old style CALL or CALLB calls and anyplace you're not using prototypes properly...meaning there's a explicit PR coded in both caller & callee.
Note that you it's not just old-style calls made by the program that bombs, it's calls made anywhere down the call chain.
And if the program is repeatedly called with LR=*OFF or without reclaiming resources, then it could be any old style calls up the call chain also.
Lastly, old style calls include any made by CL or CLLE programs.
Good luck!
Can someone explain to me what this error I'm seeing is?
Current thread must be set to single thread apartment (STA) mode before OLE calls can be made.
Specifically, I'm trying to open the SaveFileDialog/OpenFileDialog within C++/CLI on a form.
SaveFileDialog^ saveFileDialog1 = gcnew SaveFileDialog;
saveFileDialog1->ShowDialog();
if (saveFileDialog1->ShowDialog() == System::Windows::Forms::DialogResult::OK)
{
s = saveFileDialog1->OpenFile();
}
s->Close();
}
The error that is throwing is
An unhandled exception of type 'System.Threading.ThreadStateException' occurred in System.Windows.Forms.dll
Additional information: Current thread must be set to single thread apartment (STA) mode before OLE calls can be made. Ensure that your Main function has STAThreadAttribute marked on it. This exception is only raised if a debugger is attached to the process.
I'm not really familiar with what this error is saying. I know just a bit about threading, but I'm not sure how threading would be an issue here. I've seen some people reference things like STAThread without providing a clear explanation as to what it does, and Microsoft's documentation makes no mention of having this exception thrown when calling SaveFileDialog/OpenFileDialog, or how to handle it.
Thanks!
When you use OpenFileDialog then a lot of code gets loaded into your process. Not just the operating system component that implements the dialog but also shell extensions. Plugins that programmers write to add functionality to Windows Explorer. They work in that dialog as well. There are many, one you are surely familiar with is the extension that makes a .zip file look like a folder.
One thing Microsoft did when they designed the plug-in interface is to not force an extension to be thread-safe. Because that is very hard to do and often a major source of bugs. They made the promise that the thread that creates the plugin instance is also the thread on which any call to the plugin is made. Thus ensuring that the plugin is always used in a thread-safe manner.
That however requires a little help from you. You have to make a promise that your thread, the one that calls OpenFileDialog::Show(), observes the requirements of a single-threaded apartment. STA for short. You make the promise with the [STAThread] attribute on your program's Main() entrypoint. Or if it is a thread that you created yourself then by calling Thread::SetApartmentState() before you start it.
That's just a promise however, you also have to implement what you promised. Takes two things, you promise to never block the thread and you promise to pump a message loop. Application::Run() in a .NET program. The never-block promise ensures that you won't cause deadlock. And the message loop promise says that you implement a solution to the producer-consumer problem.
This should never be a problem, it is very unclear how this got fumbled in your project. Another implicit requirement for a dialog is that it must have an owner. Another window on which it can be on top of. If it doesn't have one then there are very high odds that the user never sees the dialog. Covered by another program's window, the user can only ever find it back by accident. When you create windows then you always also must call Application::Run() so the windows can respond to user input. Use the boilerplate code in a C++/CLI app so this is done correctly.
I'm writing a Haskell binding to some the library and there is a function void foo(), which calls select() inside. When i call this function from Haskell, that select() call starts to constantly return EINTR. This confuses library code and it starts looping forever.
In the #haskell IRC channel i've been told to run foo() from a bound thread. I've used runInBoundThread for this and now everything seems to work. But in some rare cases i'm getting Alarm clock message in the console (Ok, i've found it means that app catches SIGALRM).
I'm not sure it's proper way to handle this problem and i don't want to depend on Control.Concurrency. What should i do?
The cause of SIGALRM was GHC's runtime using old codepath for managing timer stuff. This old codepath was turning on because GHC's configure script had sort of linuxism in its check for create_timer() function. Fixing it made GHC use the same mechanism that is used on all platforms, and eliminated the error in question.
Relevant commit: https://gitlab.haskell.org/ghc/ghc/-/commit/edc059a425068f9bf4a60520e8d8906bc764e2b5
n.m.'s comment is correct: the code in withRTSSignalsBlocked will hide signals from your ffi'd code: http://hackage.haskell.org/packages/archive/HDBC-mysql/0.6.6.1/doc/html/Database-HDBC-MySQL.html#v:withRTSSignalsBlocked
This should also eliminate the need for runInBoundThread, I think.
I'm using 3.9.7 cURL library to download files from the internet, so I created a dynamic bibioteca of viculo. dll written in C using VC + + 6.0 the problem is that when either I call my function from within my vb6 application window locks and unlocks only after you have downloaded the file how do I solve this problem?
The problem is that when you call the function from your DLL, it "blocks" your app's execution until it gets finished. Basically, execution goes from the piece of code that makes the function call, to the code inside of the function call, and then only comes back to the next line after the function call after the code inside of the function has finished running. In fact, that's how all function calls work. You can see this for yourself by single-stepping through your code in the VB 6 development environment.
You don't normally notice this because the code inside of a function being called doesn't take very long to execute before control is returned to the caller. But in this case, since the function you're calling from the DLL is doing a lot of processing, it takes a while to execute, so it "blocks" the execution of your application's code for quite a while.
This is a good general explanation for the reason why your application window appears to be frozen. A bit more technically, it's because the message pump that is responsible for processing user interaction with on-screen elements is not running (it's part of your code that has been temporarily suspended until the function that you called finishes processing). This is a bit more difficult for a VB programmer to appreciate, since none of this nitty-gritty stuff is exposed in the world of VB. It's all happening behind the scenes, just like it is in a C program, but you don't normally have to deal with any of it. Occasionally, though, the abstraction leaks, and the nitty-gritty rears its ugly head. This is one of those cases.
The correct solution to this general problem, as others have hinted at, is to run lengthy operations on a background thread. This leaves your main thread (right now, the only one you have, the one your application is running on) free to continue processing user input, while the other thread can process the data and return that processed data to the main thread when it is finished. Of course, computers can't actually do more than one thing at a time, but the magic of the operating system rapidly switching between one task and another means that you can simulate this. The mechanism for doing so involves threads.
The catch comes in the fact that the VB 6 environment does not have any type of support for creating multiple threads. You only get one thread, and that's the main thread that your application runs on. If you freeze execution of that one, even temporarily, your application freezes—as you've already found out.
However, if you're already writing a C++ DLL, there's no reason you can't create multiple threads in a VB 6 app. You just have to handle everything yourself as if you were using another lower-level language like C++. Run the C++ code on a background thread, and only return its results to the main thread when it is completely finished. In the mean time, your main thread is free.
This is still quite a bit of work, though, especially if you're inexperienced when it comes to Win32 programming and the issues surrounding multiple threads. It might be easier to find a different library that supports asynchronous function calls out-of-the-box. Antagony suggests using VB's AsyncRead method. That is probably a good option; as Karl Peterson says in the linked article, it keeps everything in pure VB 6 code, which can be a real time saver as well as a boon to future maintenance programmers. The only problem is that you'll still have to process the data somehow once you obtain it. And if that's slow, you're right back where you started from…
Check out this article, which demonstrates how to asynchronously transfer large files using a little-known method in user controls.
This is a bit of a long question, but here we go. There is a version of FormatDateTime that is said to be thread safe in that you use
GetLocaleFormatSettings(3081, FormatSettings);
to get a value and then you can use it like so;
FormatDateTime('yyyy', 0, FormatSettings);
Now imagine two timers, one using TTimer (interval say 1000ms) and then another timer created like so (10ms interval);
CreateTimerQueueTimer
(
FQueueTimer,
0,
TimerCallback,
nil,
10,
10,
WT_EXECUTEINTIMERTHREAD
);
Now the narly bit, if in the call back and also the timer event you have the following code;
for i := 1 to 10000 do
begin
FormatDateTime('yyyy', 0, FormatSettings);
end;
Note there is no assignment. This produces access violations almost immediatley, sometimes 20 minutes later, whatever, at random places. Now if you write that code in C++Builder it never crashes. The header conversion we are using is the JEDI JwaXXXX ones. Even if we put locks in the Delphi version around the code, it only delays the inevitable. We've looked at the original C header files and it all looks good, is there some different way that C++ uses the Delphi runtime? The thread safe version of FormatDatTime looks to be re-entrant. Any ideas or thoughts from anyone who may have seen this before.
UPDATE:
To narrow this down a bit, FormatSettings is passed in as a const, so does it matter if they use the same copy (as it turns out passing a local version within the function call yeilds the same problem)? Also the version of FormatDateTime that takes the FormatSettings doesn't call GetThreadLocale, because it already has the Locale information in the FormatSettings structure (I double checked by stepping through the code).
I made mention of no assignment to make it clear that no shared storage is being accessed, so no locking is required.
WT_EXECUTEINTIMERTHREAD is used to simplify the problem. I was under the impression you should only use it for very short tasks because it may mean it'll miss the next interval if it is running something long?
If you use a plain old TThread the problem doesn't occur. What I am getting at here I suppose is that using a TThread or a TTimer works but using a thread created outside the VCL doesn't, that's why I asked if there was a difference in the way C++ Builder uses the VCL/Delphi RTL.
As an aside this code as mentioned before also fails (but takes longer), after a while, CS := TCriticalSection.Create;
CS.Acquire;
for i := 1 to LoopCount do
begin
FormatDateTime('yyyy', 0, FormatSettings);
end;
CS.Release;
And now for the bit I really don't understand, I wrote this as suggested;
function ReturnAString: string;
begin
Result := 'Test';
UniqueString(Result);
end;
and then inside each type of timer the code is;
for i := 1 to 10000 do
begin
ReturnAString;
end;
This causes the same kinds of failiures, as I said before the fault is never in the same place inside the CPU window etc. Sometimes it's an access violation and sometimes it might be an invalid pointer operation. I am using Delphi 2009 btw.
UPDATE 2:
Roddy (below) points out the Ontimer event (and unfortunately also Winsock, i.e. TClientSocket) use the windows message pump (as an aside it would be nice to have some nice Winsock2 components using IOCP and Overlapping IO), hence the push to get away from it. However does anyone know how to see what sort of thread local storage is setup on the CreateQueueTimerQueue?
Thanks for taking the time to think and answer this problem.
I am not sure if it is good form to post an "Answer" to my own question but it seemed logical, let me know if that is uncool.
I think I have found the problem, the thread local storage idea lead me to follow a bunch of leads and I found this magical line;
IsMultiThread := True;
From the help;
"IsMultiThread is set to true to indicate that the memory manager should support multiple threads. IsMultiThread is set to true by BeginThread and class factories."
This of course is not set by using a single Main VCL thread using a TTimer, However it is set for you when you use TThread. If I set it manually the problem goes away.
In C++Builder, I do not use a TThread but it appears by using the following code;
if (IsMultiThread) {
ShowMessage("IsMultiThread is True!");
}
that is it set for you somewhere automatically.
I am really glad for peoples input so that I could find this and I hope vainly it might help someone else.
As DateTimeToString which FormatDateTime calls uses GetThreadLocale, you may wish to try having a local FormatSettings variable for each thread, maybe even setting up FormatSettings in a local variable before the loop.
It may also be the WT_EXECUTEINTIMERTHREAD parameter which causes this. Note that it states it should only be used for very short tasks.
If the problem persists the problem may actually be elsewhere, which was my first hunch when I saw this but I don't have enough information about what the code does to really determine that.
Details about where the access violation occurs may help.
Are you sure this actually has anything to do with FormatDateTime? You made a point of mentioning that there is no assignment statement there; is that an important aspect of your question? What happens if you call some other string-returning function instead? (Make sure it's not a constant string. Write your own function that calls UniqueString(Result) before returning.)
Is the FormatSettings variable thread-specific? That's the point of having the extra parameter for FormatDateTime, so each thread has its own private copy that is guaranteed not to be modified by any other thread while the function is active.
Is the timer queue important to this question? Or do you get the same results when you use a plain old TThread and run your loop in the Execute method?
You did warn that it was a long question, but it seems there are a few things you could do to make it smaller, to narrow down the scope of the issue.
I wonder if the RTL/VCL calls you're making are expecting access to some thread-local storage (TLS) variables that aren't correctly set up whn you invoke your code via the timer queue?
This isn't the answer to your problem, but are you aware that TTimer OnTimer events just run as part of the normal message loop in the main VCL thread?
You found your answer - IsMultiThread. This has to be used anytime to revert to using the API and create threads. From MSDN: CreateTimerQueueTimer is creating a thread pool to handle this functionality so you have an outside thread working with the main VCL thread with no protection. (Note: your CS.acquire/release doesn't do anything at all unless other parts of the code respect this lock.)
Re. your last question about Winsock and overlapping I/O: You should look closely at Indy.
Indy uses blocking I/O, and is a great choice when you want high performance network IO regardless of what the main thread is doing. Now you've cracked the multi-threading issue, you should just create another thread (or more) to use indy to handle your I/O.
The problem with Indy is that if you need many connections it's not effiecient at all. It requires one thread per connection (blocking I/O) which doesn't scale at all, hence the benefit of IOCP and Overlapping IO, it's pretty much the only scalable way on Windows.
For update2 :
There is a free IOCP socket components : http://www.torry.net/authorsmore.php?id=7131 (source code included)
"By Naberegnyh Sergey N.. High
performance socket server based on
Windows Completion Port and with using
Windows Socket Extensions. IPv6
supported. "
i've found it while looking better components/library to rearchitecture my little instant messaging server. I haven't tried it yet but it looks good coded as a first impression.