Can we re-purpose the completion port thread (for async I/O operation) as worker thread in CLR process ThreadPool ?
If this is naïve then can someone suggest me how to maximize the use of thread pool threads in order to reduce number of work item stacked in the worker queue.
The IOCP threads are already sorta 'workers' - they take input from a queue and act on the received items. If you wish to avoid using another thread pool for processing items other than 'normal' IOCP completion objects rx from the network drivers, there is nothing stopping you from 'manually' queueing up objects to the IOCP queue that ask the IOCP pool threads to perform other actions. I forget the actual APIs now, but AFAIK there should be no problem.
I remember using such a mechanism for server tuning - reducing the number of IOCP threads by queueing an item that instructed the receiving IOCP pool thread to terminate.
That said, I'm not sure that such a mechansim will improve throughput significantly - the work has to be done somewhere and it may be that avoiding an extra thread pool would not help much. Empirically, as a general-purpose inter-thread comms mechanism, an IOCP queue has a worse performance than Windows message queues, (useless for thread pools anyway since only one thread can wait), and user-space CS/semaphore-based P-C queues.
Rgds,
Martin
Related
I'm just wondering if there would be any reason I might want to lock a queue. I am working on an application that has several threads that reads and writes to a database. In order to reduce traffic, I want to reduce the amount of calls to that database at any given point (I know many databases can handle some traffic already). Would it make any sense to make a queue for the read/write requests and only the request at the top executes and then protect the queue's push and pop commands with a lock? Is having a lock on each read/write call enough? Isn't a lock implemented as a "queue" by the OS anyways? Could size of this "queue" be an issue or would there be any other reason I wouldn't use a lock by itself?
Thanks!
You could limit the number of threads that are engaged in database requests or if that's not feasible due to the nature of your app, you could use a more granular approach to limit access to the shared resource. In python, you can use the built-in semaphore objects for inter-thread synchronization. For inter-process synchronization (or inter-thread), you'd use posix_ipc. It depends what your service's execution model is.
Most database clients wouldn't require any application-level throttling. In a typical system, the database connections would be pooled and the connection manager would be responsible for acquiring an available connection. Internally this usually involves a queue of some sort with timeouts to prevent waiting indefinitely. The database itself would then handle the scheduling of individual operations made by each connection.
However, a semaphore is a signalling primitive that can be used to limit the number of concurrent operations: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Semaphore.html
Tasks can also be modeled as a producer-consumer problem which involves a shared queue, however you'll have to deal with the added complexity of managing the consumer threads in addition to the producers.
Have some questions regarding NodeJs Architecture:
It says although NodeJs is single threaded, internally it uses libuv library's thread pool? Is it right?
All non-blocking requests handled by main thread and all blocking requests handled by libuv thread pool? While some says there is nothing like main thread. Right or misconception?
If yes, then what happen if thread pool size is 4 and blocking requests are
Request no 5 have to be wait until thread is available? Is it right?
if point 3 is the case, then how NodeJs is different from Java if blocking requests count exceeds thread pool size?
1.
In general, both libuv and v8 are allowed to use (and in fact use) threads.
As a rule of thumb, note that a single threaded JavaScript runtime environment doesn't mean that the underlying libraries cannot use threads.
2.
You can refer to the documentation of libuv to know what will be dispatched on threads.
I cite it:
File system operations
DNS functions
User specified code via uv_queue_work()
3.
It is said that you can queue work on the thread pool.
So, yes, if you queue more work that what you can schedule, requests are going to wait their turn to run.
4.
A thread pool is a concept that abstracts away from the language.
At the end of the day, libuv and thus node are well targeted for I/O bound applications where you do a lot of networking and the API clearly states it.
I've read tons of articles and stackoverflow questions, and I saw a lot of information about thread pool, but no one talks about physical CPU core usage. I believe this question is not duplicated.
Given that I have a quad-core computer and libuv thread pool size of 4, will Node.js utilize all those 4 cores when processing lots of i/o requests(maybe more than thousands)?
I'm also curious that which i/o request uses thread pool. No one gives clear and full list of request. I know that Node.js event loop is single threaded but uses a thread pool to handle i/o such as accessing disk and db.
I'm also curious that which i/o request uses thread pool.
Disk I/O uses the thread pool.
Network I/O is async from the beginning and does not use threads.
With disk I/O, the individual disk I/O calls still present to Javascript as non-blocking and asynchronous even though they use threads in their native code implementation. When you exceed more disk I/O calls in process than the size of the thread pool, the disk I/O calls are queued and when one of the threads frees up, the next disk I/O call in the queue will run using that now available thread. Since the Javascript for the disk I/O is all non-blocking and assumes a completion callback will get called sometime in the future, the queuing of requests when the thread pool is all busy just means it will take longer to get to the later I/O requests, but otherwise the Javascript programming interface is not affected.
Given that I have a quad-core computer and libuv thread pool size of 4, will Node.js utilize all those 4 cores when processing lots of i/o requests(maybe more than thousands)?
This is not up to node.js and is hard to answer in the absolute for that reason. The first referenced article below says that on Linux, the I/O thread pool will use multiple cores and offers a small demo app that shows that.
This is up to the specific OS implementation and the thread scheduler that it uses. node.js just happily creates the threads and uses them and the OS then decides how to make use of the CPU given what it is being asked to do overall on the system. Since threads in the same process often have to communicate with one another in some way, using a separate CPU for different threads in the same process is a lot more complicated.
There are a couple node.js design patterns that are guaranteed to take advantage of multiple cores (in any modern OS)
Cluster your app and create as many clusters as you have processor cores. This also has the advantage that each cluster has its own I/O thread pool that can work independently and each can execute it's own Javascript independently. With only one node.js process and multiple cores, you never get more than one thread of Javascript execution (this is where node.js is referred to as single threaded - even though it does use threads in its library implementations). But, with clustering, you get independent Javascript execution for each clustered server process.
For individual tasks that might be CPU-intensive (for example, image processing), you can create a work queue and a pool of child worker processes that you hand work off to. This has some benefits in common with clustering, but it is more special purpose where you know exactly where the CPU bottleneck is and you want to attack it specifically.
Other related answers/articles:
how libuv threads in nodejs utilize multi core cpu
Node.js on multi-core machines
Taking Advantage of Multi-Processor Environments in node.js
When is the thread pool used?
I know that Vibe.D implementation is based on Fibers.
But I don't know how high load scenarios are handled by Vibe.D. Is it the scheduler in Vibe.D allocating fibers on multiple threads or just one thread for all fibers?
This consideration is very important because even with the high efficiency of Fibers a lot of CPU time is wasted is no more than one thread is used to attend all incoming requests.
Their front page says yes:
http://vibed.org/
this page has the details
http://vibed.org/features#multi-threading
Distributed processing of incoming connections
The HTTP server (as well as any other TCP based server) can be instructed to process incoming connections across the worker threads of the thread pool instead of in the main thread. For applications that don't need to share state across different connections in the process, this can increase the maximum number of requests per second linearly with the number of cores in the system. This feature is enabled using the HTTPServerOption.distribute or TCPListenOptions.distribute settings.
Does a thread that is blocked cause the process to become blocked? Why and How? Thanks to all experts for answering.
A process cannot be blocked because the concept of "blocked" only applies to a thread of execution. The only meaningful sense in which you could say that a process was blocked is if the process only had one thread and that thread was blocked.
A thread is a flow of execution through the process code, with its own program counter, system registers and stack. A thread is also called a light weight process. Threads provide a way to improve application performance through parallelism. Threads represent a software approach to improving performance of operating system by reducing the overhead thread is equivalent to a classical process.
Each thread belongs to exactly one process and no thread can exist outside a process. Each thread represents a separate flow of control.Threads have been successfully used in implementing network servers and web server. They also provide a suitable foundation for parallel execution of applications on shared memory multiprocessors.
So, as you may have guessed, No ! A thread cannot block a process.