I need a suggestion - I'm writing an application that will work as kind of TaskScheduler and I want each task to be a separate Thread. I have a
System.Collections.Concurrent.ConcurrentBag<Thread>
with my threads/tasks but - I can only start a thread once from this "Bag", once it is finished - I can't restart it again (it's a scheduler application so one thread may be launched once per day/hour/minute... every few seconds...).
Should I create new Thread in my "Bag" each time a task is supposed to run?
Also - I can't really remove objects from ConcurrentBag (?)
It would be easier with:
List<Thread>
... but I read List is not a good approach to multithreading.
One more thing - it will be required to kill tasks that take too long to finish. So I will need to do Abort on a thread - a thing I see people don't reccomend as well.
Does anyone have some tips for managing a list of threads/tasks as I described? thanks in advance for suggestions.
nevermind - I gave up with "threads" and will use separate "processes" instead.
Related
I am running a Rust app with Tokio in prod. In the last version i had a bug, and some requests caused my code to go into an infinite loop.
What happened is while the task that got into the loop was stuck, all the other task continue to work well and processing requests, that happened until the number of stalling tasks was high enough to cause my program to be unresponsive.
My problem is took a lot of time to our monitoring systems to identify that something go wrong. For example, the task that answer to Kubernetes' health check works well and I wasn't able to identify that I have stalled tasks in my system.
So my question is if there's a way to identify and alert in such cases?
If i could find way to define timeout on task, and if it's not return to the scheduler after X seconds/millis to mark the task as stalled, that will be a good enough solution for me.
Using tracing might be an option here: following issue 2655 every tokio task should have a span. Alongside tracing-futures this means you should get a tracing event every time a task is entered or suspended (see this example), by adding the relevant data (e.g. task id / request id / ...) you should then be able to feed this information to an analysis tool in order to know:
that a task is blocked (was resumed then never suspended again)
if you add your own spans, that a "userland" span was never exited / closed, which might mean it's stuck in a non-blocking loop (which is also an issue though somewhat less so)
I think that's about the extent of it: as noted by issue 2510, tokio doesn't yet use the tracing information it generates and so provide no "built-in" introspection facilities.
I need to design a thread pool system, in Python in this case, but I'm more interested in the general methodology.
It has to be something along the lines of https://www.metachris.com/2016/04/python-threadpool/, where threads wait idling until some jobs are pushed into the pool. How that works, using condition variables, is clear to me.
I have one additional requirement though: the jobs I'm pushing into the pool cannot run all in parallel. Each of them has a class (i don't mean the object class here, just a simple integer that somehow classifies the job) and only one job per class can be running at the same time. If a job is pushed having the same class of a job that is currently running, it has to wait in the queue until the latter is done.
I have already modified the mentioned class to do this, but what I achieved is pretty messy and I'm not sure it's reliable, so I would ask what modifications would be suggested or whether I should use a totally different approach. Again: I don't need the code, but rather a description.
Thanks.
I want to run a job on 10 threads to process 100 files. Each thread is supposed to work on separate file. When a thread is done it is supposed to pick the next file.
What I'm doing right now is basically going on a loop, kick off the job and make it run in the background (using &), wait for any process to end if the count of processes is greater than 10 and pick up the next file. It is working but is there a better to way to achieve this?
I don't see any better solution, as long as each file has to be processed separately.
You'd be better off having each thread process its file, then try to pick up the next file itself. Maybe it won't matter in your current application, but the starting and teardown of threads is relatively expensive. Common practice is to keep a thread alive if it is otherwise just going to be replaced by a clone of itself.
There is a very common task I face again. I have already solved this a couple of times, but now I am looking for a more "elegant" way - can you deliver some input?
Situation:
I have a Method which I would like to run "semi async". In other words: Start it and wait a given time x. If the method is not finished by then ("timed out"), I want to continue my code with some cleanup procedures.
Solutions so far:
Use an AutoResetEvent (or
ManualResetEvent) combined with an
annonymus method using
.WaitOne(x).
Use a Thread/BackgroundWorker
combined with a Timer. If the timer
hits its handler before the thread stops it, the therad is timed out.
Both appraochs work fine but I imagine there is a better way with 4.0.
Suggestions?
Does Task.Wait(Timeout) from the Task Parallel Library do what you want? (You may wish to combine this with cancellation tokens to cancel the task after the timeout occurs.)
I have a worker thread in a class that is owned by a ChildView. (I intend to move this to the Doc eventually.) When the worker thread completes a task I want all the views to be updated. How can I make a call to tell the Doc to issue an UpdateAllViews()? Or is there a better approach?
Thank you.
Added by OP: I am looking for a simple solution. The App is running on a single user, single CPU computer and does not need network (or Internet) access. There is nothing to cause a deadlock.
I think I would like to have the worker thread post (or send) a message to cause the views to update.
Everything I read about threading seems way more complicated than what I need - and, yes, I understand that all those precautions are necessary for applications that are running in multiprocessor, multiuser, client-server systems, etc. But none of those apply in my situation.
I am just stuck at getting the right combination of getting the window handle, posting the message and responding to the message in the right functions and classes to compile and function at all.
UpdateAllViews is not thread-safe, so you need to marshal the call to the main thread.
I suggest you to signal a manual-reset event to mark your thread's completion and check the event's status in a WM_TIMER handler.
suggested reading:
First Aid for the Thread-Impaired:
Using Multiple Threads with MFC
More First Aid for the Thread
Impaired: Cool Ways to Take Advantage
of Multithreading