Hy,
I tried to modelize a threads pool usage with the following scenario:
Request a db to retrieve a set of instances
Create a set of threads, each one doing the same calls set
start a loop for all retrieved instances
wait that one thread is availablle
Provides the instance to the available thread that executes its two tasks
enf of loop
I see this concurrency question but it does not modelize a pool.
Any idea ? A fork is like to create a thread so i can not figure how to modelize my goal.
Maybe something like, but this is not correct because StructuredAction does not get a isSynchronous attribute (i did not find one in StructuredAction generalizations)
The schema provided in the question is in fact my answer.
Related
I need to design a thread pool system, in Python in this case, but I'm more interested in the general methodology.
It has to be something along the lines of https://www.metachris.com/2016/04/python-threadpool/, where threads wait idling until some jobs are pushed into the pool. How that works, using condition variables, is clear to me.
I have one additional requirement though: the jobs I'm pushing into the pool cannot run all in parallel. Each of them has a class (i don't mean the object class here, just a simple integer that somehow classifies the job) and only one job per class can be running at the same time. If a job is pushed having the same class of a job that is currently running, it has to wait in the queue until the latter is done.
I have already modified the mentioned class to do this, but what I achieved is pretty messy and I'm not sure it's reliable, so I would ask what modifications would be suggested or whether I should use a totally different approach. Again: I don't need the code, but rather a description.
Thanks.
I Have a Singleton class called FILELOGGER and property called number_of_lines.
I will make sure only one object is able to create for FILELOGGER class which makes singleton.
Throughout my application, my object is able to write it to a file and update number_of_lines property for each write.
What if i use this design pattern in multi threaded environment. How it behaves and i feel like number_of_lines property should be locked when other threads are trying to update. And I might to loose logging of data with delay and performance will be loosing.
Say for example thread T1 is logging at time 10:10:10 and T2 is also logging at the same exact time and both trying to update number_of_lines property.
How to solve this problem? Is there any alternative design pattern to solve this. Thanks for your time.
You can either synchronize access to the whole file as you've already done, or there's an alternative with some cons: snapshotting.
N threads write file contents to a string variable.
A dedicated thread does snapshots of in-memory data to disk and updates number_of_lines. number_of_lines will be synchronized when this dedicated thread needs to update it. Snapshotting may occur in time intervals like 10 seconds, 1 minute, 1 hour...
The main issue with this approach is an application/system crash will mean losing the data that wasn't persisted to disk since the last snapshot, but since your application works with in-memory data, it should increase overall performance.
Also you have to implement the singleton pattern in a thread safe way. I think the best approach is with an inner class to guarantee the purpose of the singleton in a multi threaded application.
Let's have a worker thread which is accessed from a wide variety of objects. This worker object has some public slots, so anyone who connects its signals to the worker's slots can use emit to trigger the worker thread's useful tasks.
This worker thread needs to be almost global, in the sense that several different classes use it, some of them are deep in the hierarchy (child of a child of a child of the main application).
I guess there are two major ways of doing this:
All the methods of the child classes pass their messages upwards the hierarchy via their return values, and let the main (e.g. the GUI) object handle all the emitting.
All those classes which require the services of the worker thread have a pointer to the Worker object (which is a member of the main class), and they all connect() to it in their constructors. Every such class then does the emitting by itself. Basically, dependency injection.
Option 2. seems much more clean and flexible to me, I'm only worried that it will create a huge number of connections. For example, if I have an array of an object which needs the thread, I will have a separate connection for each element of the array.
Is there an "official" way of doing this, as the creators of Qt intended it?
There is no magic silver bullet for this. You'll need to consider many factors, such as:
Why do those objects emit the data in the first place? Is it because they need to do something, that is, emission is a “command”? Then maybe they could call some sort of service to do the job without even worrying about whether it's going to happen in another thread or not. Or is it because they inform about an event? In such case they probably should just emit signals but not connect them. Its up to the using code to decide what to do with events.
How many objects are we talking about? Some performance tests are needed. Maybe it's not even an issue.
If there is an array of objects, what purpose does it serve? Perhaps instead of using a plain array some sort of “container” class is needed? Then the container could handle the emission and connection and objects could just do something like container()->handle(data). Then you'd only have one connection per container.
Hi I need to send a WMI query to each system in a domain (potentially thousands), and WMI queries seem to take a long time to return. So I am reviewing the best ways to send multiple requests using multiple threads, so the process can run in the background and the calls can overlap.
I like the features that BackgroundWorker offers, and I read HERE that it uses the ThreadPool under the covers. I dont really understand though how I would leverage this to serve my purposes. It seems that if I had to send 1000 queries, I could do a loop in which I invoke a new BG worker for each query, and the threadpool will use up to 25(?) threads at one time, and the remaining 975 requests are queued. Is that what happens?
If this is right, I imagine the process of queuing up 1000 requests will itself freeze the UI, so should the queuing loop itself be running in another BG worker?
Is there a problem with invoking other worker threads from a worker thread?
Should I be only creating say 20 BG worker threads and manually launching another when one completes?
Am I understanding this right? Any advice would be much appreciated!
I use the Parallel.ForEach method found in the System.Threading.Tasks namespace.
Make a List<string> containing all the host names you want to query. Then make a method that takes a string as it's input and queries it and does whatever it is you're wanting to do with that data.
Put them in a ForEach method like this
Parallel.ForEach(ComputerList, QueryAComputer);
and let it rip. Be sure to call the Dispose() method on ManagementObjects as soon as you don't need them. I think there's some kind of issue that causes WMI to break when too many queries are performed at once. Dispose() should help release those resources and prevent deadlock.
I have a worker thread in a class that is owned by a ChildView. (I intend to move this to the Doc eventually.) When the worker thread completes a task I want all the views to be updated. How can I make a call to tell the Doc to issue an UpdateAllViews()? Or is there a better approach?
Thank you.
Added by OP: I am looking for a simple solution. The App is running on a single user, single CPU computer and does not need network (or Internet) access. There is nothing to cause a deadlock.
I think I would like to have the worker thread post (or send) a message to cause the views to update.
Everything I read about threading seems way more complicated than what I need - and, yes, I understand that all those precautions are necessary for applications that are running in multiprocessor, multiuser, client-server systems, etc. But none of those apply in my situation.
I am just stuck at getting the right combination of getting the window handle, posting the message and responding to the message in the right functions and classes to compile and function at all.
UpdateAllViews is not thread-safe, so you need to marshal the call to the main thread.
I suggest you to signal a manual-reset event to mark your thread's completion and check the event's status in a WM_TIMER handler.
suggested reading:
First Aid for the Thread-Impaired:
Using Multiple Threads with MFC
More First Aid for the Thread
Impaired: Cool Ways to Take Advantage
of Multithreading