I'm trying to create a timed scheduler that can execute tasks in parallel. For example:
Let's say I'm trying to create a function that will do something after 10 seconds of being called. After calling Process_1(), it will be expected to run its intended functionality after 10 seconds.
But at the 5 second mark while Process_1() is waiting to be executed at the halfway point, I'm now calling Process_2() midway. So at the 10 seconds mark, Process_1() will execute its function and at the 15 seconds mark, Process_2() will execute its function.
I've tried using node-cron for this but it doesn't seem like it can schedule things in parallel. Thanks in advance!
Nodejs runs your Javascript in a single thread unless you explicitly create a WorkerThread and run some code in that. True parallel execution where both jobs are running code that uses the CPU will only be accomplished if you either run each task in a WorkerThread or child process to get it out of the main thread.
Let me repeat, true parallel execution requires more than one thread or process in nodejs and nodejs does not do that by default so you will have to create a WorkerThread or child_process.
So, if you have code that takes more than a few ms to do its work and you want it to run at a fairly precise time, then you can't count on the main Javascript thread to do that because it might be busy at that precise time. Timers in Javascript will run your code no earlier than the scheduled time, and when that scheduled time comes around, the event loop is ready to run them, but they won't actually run until whatever was running before finishes and returns control back to the event loop so the event loop can run the code attached to your timer.
So, if all you're mostly doing is I/O kind of work (reading/writing files or network), then your actual Javascript execution time is probably only milliseconds and nodejs can be very, very responsive to run your timers pretty close to "on time". But, if you have computationally expensive things that keep the CPU busy for much longer, then you can't count on your timers to run "on time" if you run that CPU-heavy stuff in the main thread.
What you can do, is start up a WorkerThread, set the timer in the WorkerThread and run your code in the worker thread. As long as you don't ask that WorkerThread to run anything else, it should be ready to run that timer pretty much "on time".
Now WorkerThreads do share some resources with the main thread so they aren't 100% independent (though they are close to independent). If you want 100% independence, then you can start a nodejs child process that runs a node script, sets its own timers and runs its own work in that other process.
All that said, the single threaded model works very, very well at reasonably high scale for code that is predominantly I/O code because nodejs uses non-blocking I/O so while it's waiting to read or write from file or network, the main thread is free and available to run other things. So, it will often give the appearance of running things in parallel because progress is being made on multiple fronts. The I/O itself inside the nodejs library is either natively non-blocking (network I/O) or is happening in an OS-native thread (file I/O) and the programming interface to Javascript is callback or promise based so it is also non-blocking.
I mention all this because you don't say what your two operations that you want to run in parallel are (including your actual code allows us to write more complete answers). If they are I/O or even some crypto, then they may already be non-blocking and you may achieve desired parallelism without having to use additional threads or processes.
Related
I would like to be able to tune the Node event loop to abort or throw an exception if ever a piece of code listening for an event takes too long to execute.
Using the Async Hooks API, it is possible to monitor the time that a callback runs for. However, I cannot find a way to take control in any way.
Ideally I would like to be able to tune Node so that no synchronous code runs for too long. Code running for too long will cause the application to become blocked and unresponsive. (see example). So if I could set a limit to how long any callback is allowed to run for, that would be good.
From my research I think that what I want is not possible, but I would love to be wrong :)
EDIT: Any documentation about what tuning options are available would close this issue. So far I have only heard of garbage collection and heap size.
Node.js is single-threaded by design. For this reason it's not a good choice for CPU-intensive apps in the first place.
It's generally unsafe to interrupt a running thread asynchronously, regardless of the programming language, and JavaScript is no exception. A thread might be in a state when it locked some resources, allocated memory, is modifying global data, etc. so interrupting it can lead to deadlock, memory leaks, data corruption, etc.
A process that can be safely interrupted needs to implement that explicitly, for example by periodically checking for some flag and then stopping in a safe manner. Games often use time-based compute loops in which the elapsed time is checked periodically and the loop exits when it reaches e.g. 20ms, for the process to be continued later during the next invocation.
When possible, break down a long-running computation into a series of shorter steps/callbacks.
Would it make sense to use both asyncio and threading in the same python project so that code runs in different threads where is some of them asyncio is used to get a sequentially looking code for asynchronous activities?
or would trying to do this mean that I am missing some basic concept on the usage of either threading or asyncio?
I didn't understand what you're asking (part about "sequentially looking code for asynchronous activities"), but since there's no answers I'll write some thoughts.
Let's talk why we need asyncio/threads at all. Imagine we have a task to make two requests.
If we will use plain one-thread non-async code, only option for us
is to make request for one url and only after it's done - for
another:
request(url1)
request(url2)
Problem here is that we do job ineffective: each function most time of it's execution do nothing just waiting for network results. It would be cool if we somehow would be able to use CPU for second request while first one stuck with network stuff and don't need it.
This problem can be solved (and usually solves) by running functions in different threads:
with ThreadPoolExecutor(max_workers=2) as e:
e.submit(request, url1)
e.submit(request, url2)
We would get results faster this way. While first request is stuck with network, CPU would be able to do something useful for second request in another thread.
This is however not ideal solution: switching between threads have some cost, executing flow is more complex than in the first example.
There should be way better.
Use one function idle period to start executing another function is what asyncio in general about:
await asyncio.gather(
async_request(url1),
async_request(url2),
)
Event loop manages execution flow: when first coroutine reaches some I/O operation and CPU can be used to do job elsewhere, second coroutine starts. Later event loop returns to remain executing of first coroutine.
We get "parallel" requests and clean understandable code. Since we have parallelization in single thread, we just don't need another.
Actually, when we use asyncio threads still can be useful. If we ready to pay for them, they can help us to cast synchronous I/O functions to asynchronous very quickly:
async def async_request(url):
loop = asyncio.get_event_loop()
return (await loop.run_in_executor(None, request, url))
But again, it's optional and we usually can find module to make requests (and other I/O tasks) asynchronously without threads.
I didn't face with any other tasks when threads can be useful in asynchronous programs.
Sure it may make sense.
Asynchronous code in principle runs a bunch of routines in the same thread.
This means that the moment one routine has to wait for input or output (I/O) it will halt that routine temporarily and simply starts processing another routine until it encounters a wait there, etc.
Multi-threaded (or "parallelized" code) runs in principle at the same time on different cores of your machine. (Note that in Python parallel processing is achieved by using multiple processes as pointed out by #Yassine Faris below).
It may make perfect sense to use both in the same program. Use asyncio in order to keep processing while waiting for I/O. Use multi-threading (multi processing in Python) to do, for example, heavy calculations in parallel in another part of your program.
I understand that the event loop runs on a single process (thus, on a single core) but I'd like to know what happens when I launch threads from there.
If I launch multiple threads with uv_thread_create or uv_queue_work, will they run on multiple cores (if available)?
I'd hope they do, because (on Unix at least) everything is based on pthreads which is able to scale to multiple cores, but I don't know.
Also, I can't write code to check for that right now because I'm on mobile. I just posted to see if any of you guys already know the answer.
Happy holidays.
Edit: I've made the test code and everything runs on a single core.
That's really up to the kernel, but in general, it will schedule each thread to a different core (if you have multiple) so you can run your code in parallel. Note however, that libuv is not thread-safe (except for uv_async_send) so if you have multiple threads executing at once, once being the event loop, it's not safe to call libuv functions from outside the loop thread.
I believed any kind of asynchronous execution makes a thread in invisible area. But if so,
Async codes does not offer any performance gain than threaded codes.
But I can't understand why so many developers are making many features async form.
Could you explain about difference and cost of them?
The purpose of an asynchronous execution is to prevent the code calling the asynchronous method (the foreground code) from being blocked. This allows your foreground code to go on doing useful work while the asynchronous thread is performing your requested work in the background. Without asynchronous execution, the foreground code must wait until the background task is completed before it can continue executing.
The cost of an asynchronous execution is the same as that of any other task running on a thread.
Typically, an async result object is registered with the foreground code. The async result object can either raise an event when the background task is completed, or the foreground code can periodically check the async result object to see if its completion flag has been set.
Concurrency does not necessarily require threads.
In Linux, for example, you can perform non-blocking syscalls. Using this type of calls, you can for instance start a number of network reads. Your code can keep track of the reads manually (using handles in a list or similar) and periodically ask the OS if new data is available on any of the connections. Internally, the OS also keeps a list of ongoing reads. Using this technique, you can thus achieve concurrency without any (extra) threads, neither in your program nor in the OS.
If you use threads and blocking IO, you would typically start one thread per read. In this scenario, the OS will instead have a list of ongoing threads, which it parks when the tread tries to read data when there is none available. Threads are resumed as data becomes available.
Having the OS switch between threads might involve slightly more overhead in the form of context switching - switching program counter and register content. But the real deal breaker is usually stack allocation per thread. This size is a couple of megabytes by default on Linux. If you have a lot of concurrency in your program, this might push you in the direction of using non-blocking calls to handle more concurrency per thread.
So it is possible to do async programming without threads. If you want to do async programming using only blocking OS-calls you need to dedicate a thread to do the blocking while you continue. But if you use non-blocking calls you can do a lot of concurrent things with just a single thread. Have a look at Node.js, which have great support for many concurrent connections while being single-threaded for most operations.
Also check out Golang, which achieve a similar effect using a sort of green threads called goroutines. Multiple goroutines run concurrently on the same OS thread and they are restrictive in stack memory, pushing the limit much further.
Async codes does not offer any performance gain than threaded codes.
Asynchornous execution is one of the traits of multi-threaded execution, which is becoming more relevant as processors are packing in more cores.
For servers, multi-core only mildly relevant, as they are already written with concurrency in mind and will scale natrually, but multi-core is particularly relevant for desktop apps, which traditionally do only a few things concurrently - often just one foreground task with a background thread. Now, they have to be coded to do many things concurrently if they are to take advantage of the power of the multi-core cpu.
As to the performance - on single-core - the asynchornous tasks slow down the system as much as they would if run sequentially (this a simplication, but true for the most part.) So, running task A, which takes 10s and task B which takes 5s on a single core, the total time needed will be 15s, if B is run asynchronously or not. The reason is, is that as B runs, it takes away cpu resources from A - A and B compete for the same cpu.
With a multi-core machine, additional tasks run on otherwise unused cores, and so the situation is different - the additional tasks don't really consume any time - or more correctly, they don't take away time from the core running task A. So, runing tasks A and B asynchronously on multi-core will conume just 10s - not 15s as with single core. B's execution runs at the same time as A, and on a separate core, so A's execution time is unaffected.
As the number of tasks and cores increase, then the potential improvements in performance also increase. In parallel computing, exploiting parallelism to produce an improvement in performance is known as speedup.
we are already seeing 64-core cpus, and it's esimated that we will have 1024 cores commonplace in a few years. That's a potential speedup of 1024 times, compared to the single-threaded synchronous case. So, to answer your question, there clearly is a performance gain to be had by using asynchronous execution.
I believed any kind of asynchronous execution makes a thread in invisible area.
This is your problem - this actually isn't true.
The thing is, your whole computer is actually massively asynchronous - requests to RAM, communication via a network card, accessing a HDD... those are all inherently asynchronous operations.
Modern OSes are actually built around asynchronous I/O. Even when you do a synchronous file request, for example (e.g. File.ReadAllText), the OS sends an asynchronous request. However, instead of giving control back to your code, it blocks while it waits for the response to the asynchronous request. And this is where proper asynchronous code comes in - instead of waiting for the response, you give the request a callback - a function to execute when the response comes back.
For the duration of the asynchronous request, there is no thread. The whole thing happens on a completely different level - say, the request is sent to the firmware on your NIC, and given a DMA address to fill the response. When the NIC finishes your request, it fills the memory, and signals an interrupt to the processor. The OS kernel handles the interrupt by signalling the owner application (usually an IOCP "channel") the request is done. This is still all done with no thread whatsoever - only for a short time right at the end, a thread is borrowed (in .NET this is from the IOCP thread pool) to execute the callback.
So, imagine a simple scenario. You need to send 100 simultaneous requests to a database engine. With multi-threading, you would spin up a new thread for each of those requests. That means a hundred threads, a hundread thread stacks, the cost of starting a new thread itself (starting a new thread is cheap - starting a hundred at the same time, not so much), quite a bit of resources. And those threads would just... block. Do nothing. When the response comes, the threads are awakened, one after another, and eventually disposed.
On the other hand, with asynchronous I/O, you can simply post all the requests from a single thread - and register a callback when each of those is finished. A hundred simultaneous requests will cost you just your original thread (which is free for other work as soon as the requests are posted), and a short time with threads from the thread pool when the requests are finished - in "worst" case scenario, about as many threads as you have CPU cores. Provided you don't use blocking code in the callback, of course :)
This doesn't necessarily mean that asynchronous code is automatically more efficient. If you only need a single request, and you can't do anything until you get a response, there's little point in making the request asynchronous. But most of the time, that's not your actual scenario - for example, you need to maintain a GUI in the meantime, or you need to make simultaneous requests, or your whole code is callback-based, rather than being written synchronously (a typical .NET Windows Forms application is mostly event-based).
The real benefit from asynchronous code comes from exactly that - simplified non-blocking UI code (no more "(Not Responding)" warnings from the window manager), and massively improved parallelism. If you have a web server that handles a thousand requests simultaneously, you don't want to waste 1 GiB of address space just for the completely unnecessary thread stacks (especially on a 32-bit system) - you only use threads when you have something to do.
So, in the end, asynchronous code makes UI and server code much simpler. In some cases, mostly with servers, it can also make it much more efficient. The efficiency improvements come precisely from the fact that there is no thread during the execution of the asynchronous request.
Your comment only applies to one specific kind of asynchronous code - multi-threaded parallelism. In that case, you really are wasting a thread while executing a request. However, that's not what people mean when saying "my library offers an asynchronous API" - after all, that's a 100% worthless API; you could have just called await Task.Run(TheirAPIMethod) and gotten the exact same thing.
Using the code from the following article, I implemented an own ThreadPool:
http://www.developer.com/net/article.php/3783756
This is what I want to achieve:
Triggered through a Timer, a service should query a database every 5 seconds for new jobs to execute. A Job is basically only the information about a commandline program that needs to be run with arguments.
Up to 50 or more of these programs should be able to be executed at the same time. A program can be running a couple of seconds, minutes and also hours. The service needs to be in control of these programs at all times, i.e. it must be able to kill a program on request for instance.
Using the ThreadPool implementation from above, I startet to queue the programs to be executed and could see when the service indeed executed them. No problem so far. However, the mechanism here works like this:
The ThreadPool creates a workerthread and starts it. Whenever a program is queued, the workerthread notices this and calls a delegate that essentially instantiates a System.Diagnostics.Process object and starts the external program. The thread then is finished with its work and would be able to start further programs. However... when theres no program to start, an idle timer makes the threadmanager kill the thread and thus interrupt the Process that has been started.
This is not what I need. Does anyone here have an idea, how to handle the scenario I described better?
1) Why is the death of a thread in this process resulting in the death of the other, started process? If you answer that question, you'll have solved your problem.
2) That looks like a pretty lousy, and fairly naive, ThreadPool article. Check out Joe Duffy's series on a custom thread pool (part 1, part 2, and part 3). Code such as that is surprisingly intricate, and a significant maintenance burden on its own.
3) (The real answer) Why are you using a threadpool to do this at all? Your threadpool only ever uses one thread at a time, why not just use a timer and trigger your main thread? Does your app do other stuff besides in a UI that you need to keep responsive?
Get rid of the threadpool, there's no need for it, it's making your life difficult, and threadpools in general are not designed to host long-running tasks. That's what individual threads are for. If you must trigger the timer on a separate thread, simply create a Thread to handle it and use this one, identical thread to spawn all your processes. You can then track your process state in one, central, sensible location. :)
Are you sure that a thread pool is the optimal way of handling this? Spawning a new thread for each process which will be mostly idle but has to be present until the process terminates seems like a waste of a thread to me.
I would implement all this with a single thread and a dictionary of processes. The thread would periodically query the database and all the processes to see what actions need to be done.
AFAIK, processes spawned by Process.Start will continue to run even if the thread calling Process.Start exits. The following code illustrates this. LongRunningApp.exe will continue to run after the main program exits:
static void Main(string[] args)
{
Process p = new Process();
ProcessStartInfo psi = new ProcessStartInfo(#"C:\LongRunningApp.exe");
psi.CreateNoWindow = true;
psi.UseShellExecute = false;
p.StartInfo = psi;
p.Start();
Console.ReadLine();
}