I stumbled upon worker threads in NodeJS and I started investigating on lower level abstraction how intercommunication and data sharing works between them, especially postMessage function that is being used to send message data between threads.
Looking at this line of code const { Worker, isMainThread, parentPort } = require('worker_threads'); one would guess that it uses sockets in order to communicate as keyword port is being used, but I found no open port connections when searching them trough command prompt.
I want to understand what communication protocol is worker_thread mechanism using? Is it TCP or its some other mechanism of sharing data and messages in between threads? This is based on a research that I want to commit myself in order to understand efficiency of transmitting large amount of data in between worker_threads versus ICP communication between child processes using memory sharing/TCP.
Workers don't communicate with their parents with any sort of TCP / IP or other interprocess communication protocol. The messages passed between workers, and between workers and parents, pass data back and forth directly. Workers and their parents share a single address space and V8 instance.
.postMessage() looks like this, where transferList is optional.
port.postMessage(value, transferList)
The items in the first parameter are cloned, and copies passed from the sender to the receiver of the message.
The items in the second parameter are passed without cloning them. Only certain array-style data types can be transferred this way. The sender loses access to these items and the receiver gains access. This saves the cloning time and makes it much quicker to pass large data structures like images. This sort of messaging works in both browser and nodejs code.
Child processes can pass data back and forth from the spawned process to the spawning process. That data is copied and pushed through the interprocess communication method set up at spawning time. In many cases the IPC mechanism uses OS-level pipes; those are as efficient as any IPC mechanism, especially on UNIX-derived OSs. Child processes do not share memory with parents, so they can't use a transferList change-of-ownership scheme directly.
Related
Now I am building an application to send big data from client to server via UDP. I have some questions are:
Should I use one thread to send data or multi-threads to send data?
If I should use multi-threads to send data, I will use one socket for all threads or one socket per one thread?
Thanks,
Should I use one thread to send data or multi-threads to send data?
Either way can work, so it's mostly a matter of personal preference. If it was me, I would use a single thread rather than multiple threads, because multiple threads are a lot harder to implement correctly, and in this case they won't buy you any additional performance, since your throughput bottleneck is almost certainly going to be either your hard disk or your network card, not the speed of your CPU core(s).
If I should use multi-threads to send data, I will use one socket for all threads or one socket per one thread?
Again, either way will work (for UDP), but if it was me, I would use one socket per thread, only because then you don't have to worry so much about race conditions during process-setup and process-shutdown (i.e. each thread simply creates and destroys its own separate/private socket, so there's no worrying about who does what to the socket when)
I am trying to write a server program which supports one client till now and over the few days i was trying to develop it, I concluded i needed threads. The reason for such a decision was since I take input from a wifi socket and later process it and finally write to a file, the processing time is slow and hence i needed a input thread -> circular buffer -> output thread pattern with producer consumer model which is quite common in network programming.
Now, The situation becomes complicated, as I need to manage client disconnection and re connection. I thought of using pthread_exit() and cleaning up all the semaphores and then re initializing them each time the single client re connects.
My question is that is this a efficient approach i.e. everytime killing the threads and semaphores and re creating them. Are there any better solutions.
Thanks.
My question is that is this a efficient approach i.e. everytime killing the threads and semaphores and re creating them. Are there any better solutions.
Learn how to use non-blocking sockets and an event loop. Or use a library that provides TCP sessions for you using non-blocking sockets under the hood. Such as boost::asio.
Learn how to use multi-threading without polluting your code with any synchronization primitives by using message passing to communicate between threads, not shared state. The event loop library you use for non-blocking I/O should also provide means for cross-thread message passing.
Some comments and suggestions.
1-In TCP detecting that the other side has silently disconnected it very difficult if not impossible. A client could disconnect sending a RST TCP message to the server or sending a FIN message, this is the good case. Sometimes the client can disconnect without notice (crash, cable disconnection, etc).
One suggestion here is that you consider the way client and server will communicate. For example, you can use function “select” to set a timeout for receiving a message from client and detect a silent client.
Additionally, depending on the programming language and operating system you may need to handle broken pipe (SIGPIPE) signal (in Linux, with C/C++), for a server trying to send a message through a connection closed by the client.
2-Regarding semaphores, you shouldn’t need to clean semaphores in any especial way when a client disconnect. By applying common good practices of locking and unlocking mutexes should be enough. Also with resources like file descriptors, you need to release them before ending the thread either by returning from the thread start function or with pthread_exit. Maybe I didn’t understand this part of the question.
3-Regarding threads: if you work with multiple threads to optimum is to have a pool of pre-created consumer/worker threads that will check the circular buffer to consume the next available connection. Creating and destroying threads is costly for the operating system.
Threads are resource consuming and you may exhaust operating system resources if you need to create 1,000 threads for example.
Another alternative, is to have only one consumer thread that manages all connections (sockets) asynchronously: a) Each connection has its own state. b) The main thread goes through all connections and use function “select” to detect when connection reads or a writes are ready. 3)Use of non-blocking sockets but this is not essential because from select you know which sockets are ready and will not block.
You can use functions select, poll, epoll.
One link about select and non-blocking sockets: Using select() for non-blocking sockets
Other link with an example: http://linux.die.net/man/2/select
I have two processes: a producer which pushes messages via ZMQ to a consumer in a simple PULL-PUSH point-to-point pattern. The producer has several internal threads that send() via zmq. However, 0MQ's docs suggest not to share sockets between threads.
Must I use a single thread to send?
Assuming there is no strict requirement for keeping the sending order between the threads, doesn't the fact that the socket is a one-directional simplex allow multiple threads to use it without introducing locks?
The easiest thing to do is to create a separate PUSH socket on each of producer's threads and connect all these sockets to a single PULL socket in consumer.
It's explicitly stated in the guide that ZeroMQ sockets must be used on a single thread. I'd say that violating this requirement is not a good idea, even if it seems to work: things may break in the next version of the library or on some specific platform or in some specific load scenario. So, it's just too risky.
Can someone explain in detail how the core cluster module works in Node.js?
How the workers are able to listen to a single port?
As far as I know that the master process does the listening, but how it can know which ports to listen since workers are started after the master process? Do they somehow communicate that back to the master by using the child_process.fork communication channel? And if so how the incoming connection to the port is passed from the master to the worker?
Also I'm wondering what logic is used to determine to which worker an incoming connection is passed?
I know this is an old question, but this is now explained at nodejs.org here:
The worker processes are spawned using the child_process.fork method,
so that they can communicate with the parent via IPC and pass server
handles back and forth.
When you call server.listen(...) in a worker, it serializes the
arguments and passes the request to the master process. If the master
process already has a listening server matching the worker's
requirements, then it passes the handle to the worker. If it does not
already have a listening server matching that requirement, then it
will create one, and pass the handle to the worker.
This causes potentially surprising behavior in three edge cases:
server.listen({fd: 7}) -
Because the message is passed to the master,
file descriptor 7 in the parent will be listened on, and the handle
passed to the worker, rather than listening to the worker's idea of
what the number 7 file descriptor references.
server.listen(handle) -
Listening on handles explicitly will cause the
worker to use the supplied handle, rather than talk to the master
process. If the worker already has the handle, then it's presumed that
you know what you are doing.
server.listen(0) -
Normally, this will cause servers to listen on a
random port. However, in a cluster, each worker will receive the same
"random" port each time they do listen(0). In essence, the port is
random the first time, but predictable thereafter. If you want to
listen on a unique port, generate a port number based on the cluster
worker ID.
When multiple processes are all accept()ing on the same underlying
resource, the operating system load-balances across them very
efficiently. There is no routing logic in Node.js, or in your program,
and no shared state between the workers. Therefore, it is important to
design your program such that it does not rely too heavily on
in-memory data objects for things like sessions and login.
Because workers are all separate processes, they can be killed or
re-spawned depending on your program's needs, without affecting other
workers. As long as there are some workers still alive, the server
will continue to accept connections. Node does not automatically
manage the number of workers for you, however. It is your
responsibility to manage the worker pool for your application's needs.
NodeJS uses a round-robin decision to make load balancing between the child processes. It will give the incoming connections to an empty process, based on the RR algorithm.
The children and the parent do not actually share anything, the whole script is executed from the beginning to end, that is the main difference between the normal C fork. Traditional C forked child would continue executing from the instruction where it was left, not the beginning like NodeJS. So If you want to share anything, you need to connect to a cache like MemCache or Redis.
So the code below produces 6 6 6 (no evil means) on the console.
var cluster = require("cluster");
var a = 5;
a++;
console.log(a);
if ( cluster.isMaster){
worker = cluster.fork();
worker = cluster.fork();
}
Here is a blog post that explains this
As an update to #OpenUserX03's answer, nodejs has no longer use system load-balances but use a built in one. from this post:
To fix that Node v0.12 got a new implementation using a round-robin algorithm to distribute the load between workers in a better way. This is the default approach Node uses since then including Node v6.0.0
I'm looking using to transfer an accept()ed socket between processes using sendmsg(). In short, I'm trying to build a simple load balancer that can deal with a large number of connections without having to buffer the stream data.
Is this a good idea when dealing with a large number (let's say hundreds) of concurrent TCP connections? If it matters, my system is Gentoo Linux
You can share the file descriptor as per the previous answer here.
Personally, I've always implemented servers using pre-fork. The parent sets up the listening socket, spawns (pre-forks) children, and each child does a blocking accept. I used pipes for parent <-> child communication.
Until someone does a benchmark and establishes how "hard" it is to send a file descriptor, this remains speculation (someone might pop up: "Hey, sending the descriptor like that is dirt-cheap"). But here goes.
You will (likely, read above) be better off if you just use threads. You can have the following workflow:
Start a pool of threads that just wait around for work. Alternatively you can just spawn a new thread when a request arrives (it's cheaper than you think)
Use epoll(7) to wait for traffic (wait for connections + interesting traffic)
When interesting traffic arrives you can just dispatch a "job" to one of the threads.
Now, this does circumvent the whole descriptor sending part. So what's the catch ? The catch is that if one of the threads crashes, the whole process crashes. So it is up to you to benchmark and decide what's best for your server.
Personally I would do it the way I outlined it above. Another point: if the workers are children of the process doing the accept, sending the descriptor is unnecessary.