Developing with RAD Studio (Delphi) v10.2.1 (Tokyo release 1)
on Windows 10 "Creators Update" 64bit, but 32bit development.
The application is VCL with multiple background threads, each using Indy TidHTTP to fetch network resources. Synchronisation between the main thread and the background threads is implemented using message queues (PostThreadMessage calls). This is complicated enough that offering direct code here would be difficult and messy, so I'm starting with a verbal description.
What is supposed to happen: Open a file with links to external resources, this generates the HTTP requests and hands them out to background processing, then waits for incoming messages on the application message queue to say the resources have been downloaded. The application messages are matched in event code assigned to TApplication.OnMessage (which, I suspect, is where my problem lies).
It works. Everything goes smoothly most of the time. But if I open a TSaveDialog - even if I cancel the dialog rather than actually doing anything - then messages go missing off the application message queue.
Through a process of writing log messages (it's impossible to debug directly because that upsets the timing required to cause the problem) I've worked out that the background threads are indeed posting the messages (and getting a positive response from PostThreadMessage), but they never appear on my TApplication.OnMessage event code.
I've seen that some sneaky code in various libraries will set up their own PeekMessage/TranslateMessage/DispatchMessage loops, but not all of them remember to check if there is a TApplication.OnMessage event. But I've just searched through the VCL code and the dozen or so third party libraries I'm using and have not found any instances of this that would be getting hit in this case (as far as I can tell).
Note: I am using madExcept, Indy, FastReport, AddictSpell, SynEdit, VclStyleUtils (amongst a few other less known libraries)
Note 2: I am wondering if it might be related to with Delphi 10.2.1 or Windows 10 Creator's update, since I am also seeing some other odd behaviour (long delays with the first exception or first TOpenDialog - but only on some applications) that definitely didn't happen with 10.1 (I didn't use 10.2.0). ... But this may be (probably is) something different.
So my question is: What can I do about this?
Any suggestions on how to find/verify that there is some other code stealing the application messages? Anything else I should search for other than PeekMessage?
Is there another way to intercept the application message queue messages that might let me avoid the problem?
If no better options show themselves I may have to abandon using application thread messages and implement my own messaging/synchronisation system, which I'd rather not do after getting this other working very well the rest of the time.
You mention PostThreadMessage. Look no further. You can't use this unless you control all message loops that might pull off thread messages, and you don't. The modal file dialog message loop is out of your control. It will pull off thread messages intended for a different message loop, and not know what to do with them.
The solution is simple enough. Post messages to a window rather than a thread. That way all sane message loops (the modal file dialog's message loop is sane) will dispatch the messages to the intended recipient window.
In Delphi terms this would involve using AllocateHWnd or similar to create a hidden window to receive the messages.
Raymond Chen has covered this topic here: Why do messages posted by PostThreadMessage disappear?
Related
I'd like to use Publish/Subscribe in our XPages application, in Java, e.g. with Jedis. The application runs in a multi-user setting, and when one user makes some changes to a document we'd like to see those changes reflected on other users' screens. In theory that could be done using PubSub: when the change is applied, a "document modified" message is published and sent to the party or parties that subscribed to this message. The subscriber part I'd like to put in a Thread, so that the object that subscribed can react immediately when the message is received.
The scope for most objects that use subscribe is viewscope, they should be destroyed when viewscope is destroyed. But what happens when the object is abandoned and the subscriber thread is still there? For instance, how can I tell the JVM that the Thread can safely be stopped and scrapped by the garbage collector?
I have yet to try this, so I have no code that I can show, but here's the questions I have:
am I right, will the Thread continue to run even when viewscope is destroyed?
is there a way to create a garbage-collectable Thread?
or maybe: is there some API that does PubSub in the multi-threaded XPages environment?
I did something like this years ago. You can find a Pub/Sub example with Guava here: http://hasselba.ch/blog/?p=2158
I am not sure what you are planning with the subscriber thread
respectively what the benefit is for your idea.
But to answer your questions:
Yes, a thread continues to run until you stop it. You should use an ExecutorService because it helps you with management a lot. If you want to have it "automatically" removed, you just have to do a "shutdown", which then processes all jobs left and stops the ThreadPool.
If you need a server-wide Pub/Sub system, think about an OSGi Plugin which starts / stops automatically with the Domino HTTP task. All objects of this plugin can be used by any application.
I am in the process of moving a newsletter service from a Windows server running Microsoft.NET 4.5 to a Linux server running Mono 3.0.3. The service uses Amazon's "Simple Email Service" (SES) to deliver the emails, via the official .NET SDK (wrapping a REST interface).
While sending emails via SES sequentially from Mono turns out to be slightly faster than Microsoft.NET using similar hardware, I am running into serious performance trouble when attempting to deliver multiple mails in parallel. Below is a chart showing the time required to send 128 emails on both platforms using a varying number of threads. As you can see, performance on Mono degrades rapidly after 8 threads, and with 128 threads I get only HTTP timeouts – not a single email is delivered.
Profiling via console output, it turns out that the first "batch" of mails is the source of the slowdown. With two threads, sending one email in each, both threads finish in around 2200 ms. With four threads, sending one email in each, they all finish in around 4400 ms. Eight threads, around 8800 ms, etc. It seems as if the web service, while spawned simultaneously, are run sequentially and are required to wait for one another before returning.
Any ideas what might be triggering this behavior? The source code for the Amazon SDK is available on GitHub, but I have not been able to pinpoint anything suspiciously. Maybe the use of the async methods on HttpWebRequest?
Yes, stop using async HttpWebRequest* for now because there is a bug being discussed in the Mono list. A patch has been provided, but apparently is not good enough and has been reverted from master.
If you're good with low level code, it would be nice that you contribute a patch.
* The fastest way to stop using the async infrastructure is calling mono witht eh environment variable MONO_DISABLE_AIO=1. By the way, if you're using more than one thread anyway, maybe a Parallel.For would be enough but keeping the code non-asynchronous? The best use-case of async is actually to avoid threading and still manage to achieve parallelization (or rather, avoid blocking waits).
HI
I have a control that accesses a database using proprietary datasets. The database is an old ISAM bases database.
The control uses a background thread to query the database using the proprietary datasets.
A form will have several of these controls on it, each using their own thread to access the data as they all need to load simultaneously.
The proprietary datasets handle concurrency by displaying a VCL TForm notifying the user that the table being opened is locked by another user and that the dataset is waiting for the lock to be released.
The form has a cancel button on it which lets the user cancel the lock wait.
The problem:
When using the proprietary datasets from within a thread, the application will crash, hang or give some error if the lock wait form it displayed. I suspect this is to do with the VCL not being thread safe.
I have solved the issue by synchronizing Dataset.Open however this holds up the main thread until the dataset.open returns, which can take a considerable amount of time depending on the complexity of the query.
I have displayed a modal progress bar which lets to user know that something it happening but I don't like this idea as the user will be sitting waiting for the progress bar to complete.
The proprietary dataset code is compiled into the main application, i.e. its not stored in a separate DLL. We are not allowed to change how the locking works or whether a form is displayed or not at this stage of the development process as we are too close to release.
Ideally I would like to have Dataset.open run in the controls thread as well instead of having the use the main thread, however this doesn't seem likely to work.
Can anyone else suggest a work around? please.
Fibers won't help you one bit, because they are in the Windows API solely to help ease porting old code that was written with cooperative multitasking in mind. Fibers are basically a form of co-routines, they all execute in the same process, have their own stack space, and the switching between them is controlled by the user code, not by the OS. That means that the switching between them can be made to occur only at times that are safe, so no synchronization issues. OTOH that means that only one fiber can be running within one thread at the same time, so using fibers with blocking code has the same characteristics as calling blocking code from within one thread - the application becomes unresponsive.
You could use fibers together with multiple threads, but that can be dangerous and doesn't bring any benefit over using threads alone.
I have used fibers successfully within VCL applications, but only for specific purposes. Forget about them if you want to deal with potentially blocking code.
As for your problem - you should make a control that is used for display purposes only, and which uses the standard inter-process communication mechanisms to exchange data with another process that accesses your database.
COM objects can run in out-of-process mode. May be in delphi it will be a bit easier to use them, then another IPC mechanisms.
I want to know whenever any application starts. Is there a windows message that I can set a hook for to know exactly when that happens?
If polling is not a problem you could use one of the approaches described in the answers to a related question:
C# Process Monitor
The suggested solutions use WMI or Windows audit process tracking mechanism.
The first message sent to new windows is WM_NCCREATE. But this has nothing to do with the process itself, which is what I suspect you're asking? By definition 'window messages' will start to arrive only after you create a window (using CreateWindowEx or whatever), but that can happen long after the process has started.
You don't say what language/framework you're using. In VC++ and the like you can just use whatever passes for the WinMain function. For VB it would be a Main function in a module.
Even with a poor network connection?
Specifically, I've written code which launches a separate thread (from the UI) that attempts to upload a file via HTTP POST. I've found, however, that if the connection is bad, the processor gets stuck on outputstream.close() or httpconnection.getheaderfield() or any read/write which forces data over the network. This causes not only the thread to get stuck, but steals the entire processor, so even the user interface becomes unresponsive.
I've tried lowering the priority of the thread, to no avail.
My theory is that there is no easy way of avoiding this behavior, which is why all the j2me tutorial instruct developers to create a ‘sending data over the network…’ screen, instead of just sending everything in a background thread. If someone can prove me wrong, that would be fantastic.
Thanks!
One important aspect is you need to have a generic UI or screen that can be displayed when the network call in background fails. It is pretty much a must on any mobile app, J2ME or otherwise.
As Honza said, it depends on the design, there are so many things that can be done, like pre-fetching data on app startup, or pre-fetching data based on the screen that is loaded (i.e navigation path), or having a default data set built in into the app etc.
Another thing that you can try is a built-in timer mechanism that retries data download after certain amount of time, and aborting after say 5 tries or 1-2 minutes and displaying generic screen or error message.
Certain handsets in J2ME allow detection of airplane mode, if possible you can detect that and promptly display an appropriate screen.
Also one design that has worked for me is synchronizing UI and networking threads, so that they dont lock up each other (take this bit of advice with heavy dose of salt as I have had quite a few interesting bugs on some samsung and sanyo handsets because of this)
All in all no good answer for you, but different strategies.
It pretty much depends on how you write the code and where you run it. On CLDC the concept of threading is pretty limited and if any thread is doing some long lasting operation other threads might be (and usualy are) blocked by it as well. You should take that into account when designing your application.
You can divide your file data into chunks and then upload with multiple retries on failure. This depends on your application strategy . If your priority is to upload a bulk data with out failure. You need to have assemble the chunks on server to build back your data . This may have the overhead for making connections but the chance is high for your data will get uploaded . If you are not uploading files concurrently this will work with ease .