I have a project that is actor-based and for one part of it I must use some actors that receive message after that one actor assigns to each request separately and each actor is responsible for doing its message request, so I need something like a thread pool for actors of my project, are there any features in Scala that is useful for my necessity?
I have another question: My project has a great amount of requests and also these requests must be done as soon as possible so I thought that an actor-pool is necessary for handling them, is there any feature in Scala that is related to my demand?
tanks a lot for your attention!
Actors are [already] executed on a thread pool. Initially, there are 4 worker threads. The thread pool grows if all worker threads are blocked but there are still remaining tasks to be processed. Ideally, the size of the thread pool corresponds to the number of processor cores of the machine.
http://www.scala-lang.org/node/242
This is described in detail in this talk. Seventh slide counting from the end describes event driven actors running on a thread pool. This is without any third party library i believe. All built in.
Related
Have some questions regarding NodeJs Architecture:
It says although NodeJs is single threaded, internally it uses libuv library's thread pool? Is it right?
All non-blocking requests handled by main thread and all blocking requests handled by libuv thread pool? While some says there is nothing like main thread. Right or misconception?
If yes, then what happen if thread pool size is 4 and blocking requests are
Request no 5 have to be wait until thread is available? Is it right?
if point 3 is the case, then how NodeJs is different from Java if blocking requests count exceeds thread pool size?
1.
In general, both libuv and v8 are allowed to use (and in fact use) threads.
As a rule of thumb, note that a single threaded JavaScript runtime environment doesn't mean that the underlying libraries cannot use threads.
2.
You can refer to the documentation of libuv to know what will be dispatched on threads.
I cite it:
File system operations
DNS functions
User specified code via uv_queue_work()
3.
It is said that you can queue work on the thread pool.
So, yes, if you queue more work that what you can schedule, requests are going to wait their turn to run.
4.
A thread pool is a concept that abstracts away from the language.
At the end of the day, libuv and thus node are well targeted for I/O bound applications where you do a lot of networking and the API clearly states it.
I know the term "Load Balancing" can be very broad, but the subject I'm trying to explain is more specific, and I don't know the proper terminology. What I'm building is a set of Server/Client applications. The server needs to be able to handle a massive amount of data transfer, as well as client connections, so I started looking into multi-threading.
There's essentially 3 ways I can see implementing any sort of threading for the server...
One thread handling all requests (defeats the purpose of a thread if 500 clients are logged in)
One thread per user (which is risky to create 1 thread for each of the 500 clients)
Pool of threads which divide the work evenly for any number of clients (What I'm seeking)
The third one is what I'd like to know. This consists of a setup like this:
Maximum 250 threads running at once
500 clients will not create 500 threads, but share the 250
A Queue of requests will be pending to be passed into a thread
A thread is not tied down to a client, and vice-versa
Server decides which thread to send a request to based on activity (load balance)
I'm currently not seeking any code quite yet, but information on how a setup like this works, and preferably a tutorial to accomplish this in Delphi (XE2). Even a proper word or name to put on this subject would be sufficient so I can do the searching myself.
EDIT
I found it necessary to explain a little about what this will be used for. I will be streaming both commands and images, there will be a double-socket setup where there's one "Main Command Socket" and another "Add-on Image Streaming Socket". So really one connection is 2 socket connections.
Each connection to the server's main socket creates (or re-uses) an object representing all the data needed for that connection, including threads, images, settings, etc. For every connection to the main socket, a streaming socket is also connected. It's not always streaming images, but the command socket is always ready.
The point is that I already have a threading mechanism in my current setup (1 thread per session object) and I'd like to shift that over to a pool-like multithreading environment. The two connections together require a higher-level control over these threads, and I can't rely on something like Indy to keep these synchronized, I'd rather know how things are working than to learn to trust something else to do the work for me.
IOCP server. It's the only high-performance solution. It's essentially asynchronous in user mode, ('overlapped I/O in M$-speak), a pool of threads issue WSARecv, WSASend, AcceptEx calls and then all wait on an IOCP queue for completion records. When something useful happens, a kernel threadpool performs the actual I/O and then queues up the completion records.
You need at least a buffer class and socket class, (and probably others for high-performance - objectPool and pooledObject classes so you can make socket and buffer pools).
500 threads may not be an issue on a server class computer. A blocking TCP thread doesn't do much while it's waiting for the server to respond.
There's nothing stopping you from creating some type of work queue on the server side, served by a limited size pool of threads. A simple thread-safe TList works great as a queue, and you can easily put a message handler on each server thread for notifications.
Still, at some point you may have too much work, or too many threads, for the server to handle. This is usually handled by adding another application server.
To ensure scalability, code for the idea of multiple servers, and you can keep scaling by adding hardware.
There may be some reason to limit the number of actual work threads, such as limiting lock contention on a database, or something similar, however, in general, you distribute work by adding threads, and let the hardware (CPU, redirector, switch, NAS, etc.) schedule the load.
Your implementation is completely tied to the communications components you use. If you use Indy, or anything based on Indy, it is one thread per connection - period! There is no way to change this. Indy will scale to 100's of connections, but not 1000's. Your best hope to use thread pools with your communications components is IOCP, but here your choices are limited by the lack of third-party components. I have done all the investigation before and you can see my question at stackoverflow.com/questions/7150093/scalable-delphi-tcp-server-implementation.
I have a fully working distributed development framework (threading and comms) that has been used in production for over 3 years now across more than a half-dozen separate systems and basically covers everything you have asked so far. The code can be found on the web as well.
I've been reading bunch of articles regarding new TPL in .NET 4. Most of them recommend using Tasks as a replacement for Thread.QueueUserWorkItem. But from what I understand, tasks are not threads. So what happens in the following scenario where I want to use Producer/Consumer queue using new BlockingCollection class in .NET 4:
Queue is initialized with a parameter (say 100) to indicate number of worker tasks. Task.Factory.StartNew() is called to create a bunch of tasks.
Then new work item is added to the queue, the consumer takes this task and executes it.
Now based on the above, there is seems to be a limit of how many tasks you can execute at the same time, while using Thread.QueueUserWorkItem, CLR will use thread pool with default pool size.
Basically what I'm trying to do is figure out is using Tasks with BlockingCollection is appropriate in a scenario where I want to create a Windows service that polls a database for jobs that are ready to be run. If job is ready to be executed, the timer in Windows service (my only producer) will add a new work item to the queue where the work will then be picked up and executed by a worker task.
Does it make sense to use Producer/Consumer queue in this case? And what about number of worker tasks?
I am not sure about whether using the Producer/Consumer queue is the best pattern to use but with respect to the threads issue.
As I believe it. The .NET4 Tasks still run as thread however you do not have to worry about the scheduling of these threads as the .NET4 provides a nice interface to it.
The main advantages of using tasks are:
That you can queue as many of these up as you want with out having the overhead of 1M of memory for each queued workitem that you pass to Thread.QueueUserWorkItem.
It will also manages which threads and processors your tasks will run on to improve data flow and caching.
You can build in a hierarchy of dependancies for your tasks.
It will automatically use as many of the cores avaliable on your machine as possible.
I am building an application where I have inputs from printers over the network (on specific ports) and other files which are created into a folder locally or through the network. The user can create different threads to monitor different folders at the same time, as well as threads to handle the input from threes printers over the network. The application is supposed to process the input data according to its type and output it. On the other end of the application, there would be 4 threads waiting for input data from the input threads (could be 10 or 20 threads) to process and apply 4 different tasks.
As we will have many threads running at the same time, I thought I would use MSMQ to manage these threads. Does using MSMQ fit in this scenario or should I use another technique? Managing these threads in terms of scheduling, prioritizing, etc.
(P.S: I was thinking to build my own ThreadEngine class that will take care of all of these things until I heard about MSMQ, which am still not sure if it’s the right thing to use)
MSMQ would be useful for managing your input/output data not for your threads. .Net already has the ThreadPool, the CCR and the TPL to assist you with concurrency and multithreading so I would suggest reading up on those technologies and choosing the most appropriate one.
MSMQ is a system message queue, not a thread pool manager.
This could be interesting in a case where you don't really mind poor performance and are really going for a system where tasks are persistent and transactional to guarantee execution.
If you are looking for performance then I agree with other folks and highly discourage you from doing this - even with non-durable (ram queues).
I am writing an application server (again, non-related with a question I already posted here) and I am wondering what are the strategies to use when creating worker threads that work on the database. Some preliminary dates: the server receives xml and sends back xml, all the requests query a database - each request could take a few milliseconds to a few seconds.
Say for example that your server services a small to medium number of clients which in turn send a small number of requests per connection. Is it safe to have one worker thread per connection or should it be per request? Also should a thread pool be used to limit the resources used by the server or a worker should be added each time a new connection/request is made?
Should the server limit the number of threads it creates to an upper limit?
Hope I am not too vague ... I can hardly keep my eyes open.
If you don't have extensive experience writing application servers is a daunting task. It can be eased by using frameworks like ACE that allow you to build different configurations of your app serving infrastructure like thread per connection, thread pools, leader follower and then load the appropriate configuration with an extensible service framework.
I would recommend to read these books on ACE to get
C++ Network Programming: Mastering Complexity Using ACE and Patterns
C++ Network Programming: Systematic Reuse with ACE and Frameworks
to get an idea about what the framework can do for you.
The way I write apps like this is to make the number of threads configurable via the command line and/or a configuration file. I then do some load testing with different numbers of threads - there is always an optimal number beyond which performance begins to degrade.
If you follow the model adopted by Java EE app server developers, there's a queue for incoming requests and a pool of worker threads to service them. It's one thread per request. When a worker thread fulfills a request it goes back into the pool. If the incoming requests show up faster than the worker thread pool can service them, the queue allows them to stack up until a worker thread is released. Both the queue size and the thread pool can be tuned to match for your situation.
I'd wonder why anyone would feel the need to write their own server from scratch, especially when the scenario you describe is solved so well by others. If your wish is education, good luck. If you think you're going to improve on what's been done in the past, I'd re-examine that assumption.