Now, suppose we are designing an application, consists of 2 Erlang Nodes. On Node A, will be very many processes, in the orders of thousands. These processes access resources on Node B by sending a message to a registered process on Node B. At Node B, lets say you have a process started by executing the following function:
start_server()->
register(zeemq_server,spawn(?MODULE,server,[])),ok.<br>
server()->
receive
{{CallerPid, Ref}, {Module, Func, Args}} ->
Result = (catch erlang:apply(Module, Func, Args)),
CallerPid ! {Ref, Result},
server();
_ -> server()
end.
On Node A, any process that wants to execute any function in a given module on Node B, uses the following piece of code:
call(Node, Module, Func, Args)->
Ref = make_ref(),
Me = self(),
{zeemq_server,Node} ! {{Me, Ref}, {Module, Func, Args}},
receive
{Ref, Result} -> Result
after timer:minutes(3) ->
error_logger:error_report(["Call to server took so long"]),
{error,remote_call_failed}
end.
So assuming that Process zeemq_server on Node B, will never be down, and that the network connection between Node A and B is always up, please answer the following questions:
Qn 1: Since there is only one receiving process on Node B, its mail box is most likely to be full , all the time. This is because, the processes are many on Node A and at a given interval, say, 2 seconds, every process at least ,makes a single call to the Node B server. In which ways, can the reception be made redundant on the Node B ? , e.g. Process Groups e.t.c. and explain (the concept) how this would replace the server side code above. Show what changes would happen on the Client side.
Qn 2: In a situation where there is only one receiver on Node B, is there a maximum number of messages allowable in the process mail box ? how would erlang respond , if a single process mail ox is flooded with too many messages ?
Qn 3: In what ways, using the very concept showed above, can i guarantee that every process which sends a request , gets back an answer as soon as possible before the timeout occurs ? Could converting the reception part on the Node B to a parallel operation help ? like this:
start_server()->
register(zeemq_server,spawn(?MODULE,server,[])),ok.<br>
server()->
receive
{{CallerPid, Ref}, {Module, Func, Args}} ->
<b>spawn(?MODULE,child,[Ref,CallerPid,{Module, Func, Args}]),</b>
server();
_ -> server()
end.
child(Ref,CallerPid,{Module, Func, Args})->
Result = (catch erlang:apply(Module, Func, Args)),
CallerPid ! {Ref, Result},
ok.
The method showed above, may increase the instantaneous number of processes running on the Node B, and this may affect the service greatly due to memory. However, it looks good and makes the server() loop to return immediately to handle the next request. What is your take on this modification ?
Lastly : Illustrate how you would implement a Pool of receiver Threads on Node B, yet appearing to be under one Name as regards Node A. Such that, incoming messages are multiplexed amongst the receiver threads and the load shared within this group of processes. Keep the meaning of the problem the same.
The maximum number of messages in a process mailbox is unbounded, except by the amount of memory.
Also, if you need to inspect the mailbox size, use
erlang:process_info(self(),[message_queue_len,messages]).
This will return something like:
[{message_queue_len,0},{messages,[]}]
What I suggest is that you first convert your server above into a gen_server. This your worker.
Next, I suggest using poolboy ( https://github.com/devinus/poolboy ) to create a pool of instances of your server as poolboy workers (there are examples in their github Readme.md). Lastly, I suggest creating a module for callers with a helper method that creates a poolboy transaction and applies a Worker arg from the pool to a function. Example below cribbed from their github:
squery(PoolName, Sql) ->
poolboy:transaction(PoolName, fun(Worker) ->
gen_server:call(Worker, {squery, Sql})
end).
That said, would Erlang RPC suit your needs better? Details on Erlang RPC at http://www.erlang.org/doc/man/rpc.html. A good treatment of Erlang RPC is found at http://learnyousomeerlang.com/distribunomicon#rpc.
IMO spawning a new process to handle each request may be overkill, but it's hard to say without knowing what has to be done with each request.
You can have a pool of process to handle each msg, using a round robin method to distribute the requests or based on type of request ether handle it, send it to a child process or spawn a process. You can also monitor the load of the pooled processes by looking at their msg queues and starting new children if they are overloaded. Using a supervisor.. just use a send_after in the init to monitor the load every few seconds and act accordingly. Use OTP if you can, there's overhead but it is worth it.
I wouldn't use http for a dedicated line communication, I believe it's too much overhead. You can control the load using a pool of processes to handle it.
Related
I have 3 classes that represent nearly isolated processes that can be run concurrently (meant to be persistent, like 3 main() loops).
class DataProcess:
...
def runOnce(self):
...
class ComputeProcess:
...
def runOnce(self):
...
class OtherProcess:
...
def runOnce(self):
...
Here's the pattern I'm trying to achieve:
start various streams
start each process
allow each process to publish to any stream
allow each process to listen to any stream (at various points in it's loop) and behave accordingly (allow for interruption of it's current task or not, etc.)
For example one 'process' Listens for external data. Another process does computation on some of that data. The computation process might be busy for a while, so by the time it comes back to start and checks the stream, there may be many values that piled up. I don't want to just use a queue because, actually I don't want to be forced to process each one in order, I'd rather be able to implement logic like, "if there is one or multiple things waiting, just run your process one more time, otherwise go do this interruptible task while you wait for something to show up."
That's like a lot, right? So I was thinking of using an actor model until I discovered RxPy. I saw that a stream is like a subject
from reactivex.subject import BehaviorSubject
newData = BehaviorSubject()
newModel = BehaviorSubject()
then I thought I'd start 3 threads for each of my high level processes:
thread = threading.Thread(target=data)
threads = {'data': thread}
thread = threading.Thread(target=compute)
threads = {'compute': thread}
thread = threading.Thread(target=other)
threads = {'other': thread}
for thread in threads.values():
thread.start()
and I thought the functions of those threads should listen to the streams:
def data():
while True:
DataProcess().runOnce() # publishes to stream inside process
def compute():
def run():
ComuteProcess().runOnce()
newData.events.subscribe(run())
newModel.events.subscribe(run())
def other():
''' not done '''
ComuteProcess().runOnce()
Ok, so that's what I have so far. Is this pattern going to give me what I'm looking for?
Should I use threading in conjunction with rxpy or just use rxpy scheduler stuff to achieve concurrency? If so how?
I hope this question isn't too vague, I suppose I'm looking for the simplest framework where I can have a small number of computational-memory units (like objects because they have internal state) that communicate with each other and work in parallel (or concurrently). At the highest level I want to be able to treat these computational-memory units (which I've called processes above) as like individuals who mostly work on their own stuff but occasionally broadcast or send a message to a specific other individual, requesting information or providing information.
Am I perhaps actually looking for an actor model framework? or is this RxPy setup versatile enough to achieve that without extreme complexity?
Thanks so much!
i remember the default thread pool size for node is 4 (or based on cpu count). This brings my question like this.
For the very basic simplified case, i'm writing a service1 in node, which sends requests to service2, wait till it finishes the computation and then continue. Now service2 in another server can handle 1000 requests at the same time, it takes time, also it's a blocking call (which is out of my control).
If i do the java way, i can create 1000 threads from glassfish, so the 1st 1000 blast requests can be processed at the same time. The 1001th may need to wait a little bit.
1000 incoming req -> java server1 -> 1000 threads -> 1000 outgoing req -> server2
But in node, if the thread pool size is 4 given it's a 4 core CPU machine, that means node app will be slower than java in this case ? What happens if i increase the pool size to 1000 ? Can i increase to 1000 ?
1000 incoming req -> node server1 -> ~4 threads -> 1000 outgoing req -> server2
I don't see an easy for node, or i can let node handle most stuff, for the above blocking call, add a small java server and dispatch outing req to that ? Any suggestion ?
UPDATE: found this, We use setTimeout( function(){} , 0 ); to create asynchronous functions in JavaScript!
https://medium.com/from-the-scratch/javascript-writing-your-own-non-blocking-asynchronous-functions-60091ceacc79
Guess if i convert the block call into async function, it can solve my issue, i hope, praying !!!
Node hands it's I/O tasks off to the operating system to handle, which are generally multi-threaded. It takes the approach of not having to wait for requests to finish (by blocking a thread), because it wastes time sitting. So, Node hands these tasks off and just tells it to poke Node when it's done. There is a very good related question.
How, in general, does Node.js handle 10,000 concurrent requests?
Currently I am working on a database that is updated by another java application, but need a NodeJS application to provide Restful API for website use. To maximize the performance of NodeJS application, it is clustered and running in a multi-core processor.
However, from my understanding, a clustered NodeJS application has a their own event loop on each CPU core, if so, does that mean, with cluster architect, NodeJS will have to face traditional concurrency issues like in other multi-threading architect, for example, writing to same object which is not writing protected? Or even worse, since it is multi-process running at same time, not threads within a process blocked by another...
I have been searching Internet, but seems nobody cares that at all. Can anyone explain the cluster architect of NodeJS? Thanks very much
Add on:
Just to clarify, I am using express, it is not like running multiple instances on different ports, it is actually listening on the same port, but has one process on each CPUs competing to handle requests...
the typical problem I am wondering now is: a request to update Object A base on given Object B(not finish), another request to update Object A again with given Object C (finish before first request)...then the result would base on Object B rather than C, because first request actually finishes after the second one.
This will not be problem in real single-threaded application, because second one will always be executed after first request...
The core of your question is:
NodeJS will have to face traditional concurrency issues like in other multi-threading architect, for example, writing to same object which is not writing protected?
The answer is that that scenario is usually not possible because node.js processes don't share memory. ObjectA, ObjectB and ObjectC in process A are different from ObjectA, ObjectB and ObjectC in process B. And since each process are single-threaded contention cannot happen. This is the main reason you find that there are no semaphore or mutex modules shipped with node.js. Also, there are no threading modules shipped with node.js
This also explains why "nobody cares". Because they assume it can't happen.
The problem with node.js clusters is one of caching. Because ObjectA in process A and ObjectA in process B are completely different objects, they will have completely different data. The traditional solution to this is of course not to store dynamic state in your application but to store them in the database instead (or memcache). It's also possible to implement your own cache/data synchronization scheme in your code if you want. That's how database clusters work after all.
Of course node, being a program written in C, can be easily extended in C and there are modules on npm that implement threads, mutex and shared memory. If you deliberately choose to go against node.js/javascript design philosophy then it is your responsibility to ensure nothing goes wrong.
Additional answer:
a request to update Object A base on given Object B(not finish), another request to update Object A again with given Object C (finish before first request)...then the result would base on Object B rather than C, because first request actually finishes after the second one.
This will not be problem in real single-threaded application, because second one will always be executed after first request...
First of all, let me clear up a misconception you're having. That this is not a problem for a real single-threaded application. Here's a single-threaded application in pseudocode:
function main () {
timeout = FOREVER
readFd = []
writeFd = []
databaseSock1 = socket(DATABASE_IP,DATABASE_PORT)
send(databaseSock1,UPDATE_OBJECT_B)
databaseSock2 = socket(DATABASE_IP,DATABASE_PORT)
send(databaseSock2,UPDATE_OPJECT_C)
push(readFd,databaseSock1)
push(readFd,databaseSock2)
while(1) {
event = select(readFD,writeFD,timeout)
if (event) {
for (i=0; i<length(readFD); i++) {
if (readable(readFD[i]) {
data = read(readFD[i])
if (data == OBJECT_B_UPDATED) {
update(objectA,objectB)
}
if (data == OBJECT_C_UPDATED) {
update(objectA,objectC)
}
}
}
}
}
}
As you can see, there's no threads in the program above, just asynchronous I/O using the select system call. The program above can easily be translated directly into single-threaded C or Java etc. (indeed, something similar to it is at the core of the javascript event loop).
However, if the response to UPDATE_OBJECT_C arrives before the response to UPDATE_OBJECT_B the final state would be that objectA is updated based on the value of objectB instead of objectC.
No asynchronous single-threaded program is immune to this in any language and node.js is no exception.
Note however that you don't end up in a corrupted state (though you do end up in an unexpected state). Multithreaded programs are worse off because without locks/semaphores/mutexes the call to update(objectA,objectB) can be interrupted by the call to update(objectA,objectC) and objectA will be corrupted. This is what you don't have to worry about in single-threaded apps and you won't have to worry about it in node.js.
If you need strict temporally sequential updates you still need to either wait for the first update to finish, flag the first update as invalid or generate error for the second update. Typically for web apps (like stackoverflow) an error would be returned (for example if you try to submit a comment while someone else have already updated the comments).
We have run into a high CPU usage situation when one of our EventHandlers broke.
Let's say we have several consumers (EventHanlders), that are configured to run sequentially over the buffer. If the first EventHandler throws an exception, is there a way to halt (and awake them later) all the other EventHandlers.
What we are doing is putting the failing thread to sleep and after we try to consume the same event again. But we have notice that the other threads continue running and trying to read from the RingBuffer even where there are not events to read, raising the CPU behind acceptable levels.
For the moment I'm discarding that this is happening because WaitStrategy of the disruptor, because under normal conditions is working as expected. We are using a BlockingWaitStrategy there.
Some more explanations for the sake of the example
INPUT -> [A*] -> [B] -> [C] -> [D]
Where INPUT is the event polled from the RingBuffer and A, B, C and D are the different EventHandlers that are executing sequentially. A* is the consumer throwing an exception.
What we want to achieve is that when consumer A cannot consume an event (eg. after an exception happens), the OnEvent(...) method of that consumer does not exit but will stay in a loop with regular sleeps trying to consume again the same event when it wakes up. In the meanwhile all the other consumers should be parked or sleeping until A succeeds.
We are using disruptor version 3.3.0.
I have been googling but haven't found a working solution.
Thanks in advance.
Salva.
A college has founded out that this issue could be related with a while loop in the waitFor method in BlockingWaitStrategy.
long availableSequence;
while((availableSequence = dependentSequence.get()) < sequence) {
barrier.checkAlert();
}
After several test we have came across this possible solution:
var availableSequence: Long = dependentSequence.get()
while(availableSequence < sequence) {
this.lock.lock()
this.lock.unlock()
availableSequence = dependentSequence.get()
}
availableSequence
Basically it makes that one thread locks the resource and with that we park momentary all the other consumers avoiding the high usage of CPU.
The second point here is the while condition. This is happening just when the available sequence (that is the sequence of the dependent threads) is below the current sequence number. And that only happens when one thread is holding the lock, for example when A throws the exception.
We still investigating if this is a valid solution, or if it can have some undesired side effects.
Any though about it is welcome.
Imagine you are going to have a lot of long processor intensive tasks of translating some strings into something else. You are going to want to have a pool of actual threads to keep the main node thread going and to make use of your cores.
The main way to do this is to either implement Threads-a-gogo or Webworker-Threads, and start a pool of 16 threads (e.g. on a Intel with 8 cores you usually have 16 threads concurrently).
Doing a request to a thread is called an event or a message. Getting a response is also catching an event or getting a message. But how does this work with a threadPool?
If you skip the Webworker API, TAGG and Webworkers for node have the same underlying API. You can load your translation function in all workers using threadPool.load and que a task to one of them using threadPool.any.
But imagine I now have 50 tasks (strings to translate) to be queued. The threadPool will eventually emit 50 events (responses with a translated string) without telling me what task the response belongs to?
I think I am fundamentally misunderstanding one thing about the threadPool.
Is there a way I can just add a task to the threadPool queue and receive a callback when that particular task is done?
Why emit events from the thread pool when you can just return the translated string? The value returned by the code is received by the callback you passed to threadpool.any.eval(). Example:
threadPool.any.eval('return "hello world"', function(err, data) {
// data === 'hello world'
});