Spray routing is based on the Akka actor system. In all the sample code I remember, routing is done "fast" and actual work is spawned to other actors, unless it needs to be done synchronically to gain a response.
I would need to validate a POST input that may take some time (100's of milliseconds). Is the HTTP server going to be busy during this time, with regard to other incoming requests (s.a. normal GETs)?
In other words, what's the Spray routing multithreading model, really?
I can spawn the validation to another actor, but in such a case the REST API response will no longer be able to report if there is an error with the incoming contents. What's the optimum way to handle this?
1) The listener parameter of Http.Bind.apply() can be an actor pool. In this case you will have several identical actors to run your route with several HTTP requests simultaneously.
2) Usually you should not do any blocking calls and/or heavy tasks inside an actor, including a Spray routing actor. In general will be better to use another pool of actors for doing such tasks, using ask pattern from your spray route, or create a temporary per-request actor (don't forget to set setReceiveTimeout and handle timeouts in it) whom can send a message to another actor and wait for the answer (and die after answering to the HTTP request) or create a simple Future, include the Spray's request context (ctx) to this Actor or Future and allow them to do such a work in their separate threads, completing the request context with proper HTTP status and entity when all work will be done (but you should avoid passing the Spray request context to any actors as a message or a message part, because it has heavy context including not serializable parts in it).
Related
We have our HTTP layer served by Play Framework in Scala. One of our APIs is something of the form:
POST /customer/:id
Requests are sent by our UI team which calls these APIs through a React Framework.
The issue is that, sometimes, the requests are issued in batches, successively one after the other for the same customer ID. When this happens, different threads process these requests and so our persistent layer (MySQL) reaches an inconsistent state due to the difference in the timestamp of the handling of these requests.
Is it possible to configure some sort of thread affinity in Play Scala? What I mean by that is, can I configure Play to ensure that requests of a particular customer ID are handled by the same thread throughout the life-cycle of the application?
Batch is
put several API calls into a single HTTP request.
A batch request is a set of command in one HTTP request, like here https://developers.facebook.com/docs/graph-api/making-multiple-requests/
You describe it as
The issue is that, sometimes, the requests are issued in batches, successively one after the other for the same customer ID. When this happens, different threads process these requests and so our persistent layer (MySQL) reaches an inconsistent state due to the difference in the timestamp of the handling of these requests.
This is a set of concurrent requests. Play framework usually works as a stateless server. I assume you also organize it as stateless. There is nothing that binds one request to another, you can't control order. Well, you can, if you create a special protocol, like "opening batch request", request #1, #2, ... "closing batch request". You need to check if exactly all request was correct. You also need to run some stateful threads and some queues ... Thought akka can help with this but I am pretty sure you wan't do it.
This issue is not a "play-framework" depended. You will reproduce it in any server. For example, the general case: Is it possible to receive out-of-order responses with HTTP?
You can go in either way:
1. "Batch" the command in one request
You need to change the client so it jams "batch" requests into one. You also need to change server so it processes all the commands from the batch one after another.
Example of the requests: https://developers.facebook.com/docs/graph-api/making-multiple-requests/
2. "Pipeline" requests
You need to change the client so it sends the next request after receive the response from the previous.
Example: Is it possible to receive out-of-order responses with HTTP?
The solution to this is to pipeline Ajax requests, transmitting them serially. ... . The next request sent only after the previous one has returned successfully."
I'm investigating what reactive means and because it is kind of low level difference, compared to the common non-reactive approach, I'd like to understand what is going on. Let's take Tomcat as a server(I guess it will be different for netty)
Non-reactive
Connection from the browser is created.
For each request thread from thread pool is taken, which will process it.
After the thread finished processing, it returns the result through the connection back to other side.
Reactive???
How is it done for Tomcat or Netty. I cannot find any decent article about how Tomcat supports reactive apps and how Netty does that differently(Connection, Thread, request level explanation)
What bothers me is how reactive is making the webserver unblocking, when you still need to wait for the response. You can get first part of the response quicker maybe with reactive, but is it all? I guess the main point of reactivness is effective thread utilization and this is what I am asking about.
The last point by you : " I guess the main point of reactiveness is effective thread utilization and this is what I am asking about.", is exactly what reactive approach was designed for.
So how does effective utilization achieved?
Well, as an example, lets say you are requesting data from a server multiple times.
In a typical non-reactive way, you will be creating/using multiple threads(may be from a thread-pool) for each of your requests. And job of one particular thread is only to serve that particular request. The thread will take the request, give it to the server and waits for its response till the data is fetched from the server, and then bring that data back to the client.
Now, in a Reactive way, once there is a request, a thread will be allocated for it. Now if another request comes up, there won't be creation of another thread, rather it will be served by the same thread. How?
The thread when takes a request to the server, it won't wait for any immediate response from the server, rather it will come back and serve other request.
Now, when server searches for the data and it is available with the server, an event will be raised, and then the thread will go to fetch that data. This is called Event-loop mechanism as all the work of calling the thread when data is available is achieved by invoking an event.
Now, there is complexity assigned with it to map exact response to requests.
And all these complexity is abstracted by Spring-Webflux(in Java).
So the whole process becomes non-blocking. And as only one thread is enough to serve all the requests, there will be no thread switching we can have one thread per CPU core. Thus achieving effective utilization of threads.
Few images over the net to help you understand: ->
I'm wondering what is an "official" name for the design pattern where you have a single thread that actually handles some resource (database, file, communication interface, network connection, log, ...) and other threads that wish to do something with that resource have to pass a message to this thread and - optionally - wait for a notification about completion?
I've found some articles that refer to this method as "Central Controller", but googling doesn't give much about that particular phrase.
One the other hand this is not exactly a "message pump" or "event queue", because it's not related to GUI or the operating system passing some messages to the application.
It's also not "work queue" or "thread pool", as this single thread is dedicated only to this single activity (managing single resource), not meant to be used to do just about anything that is thrown at it.
For example assume that there's a special communication interface managed by one thread (for example let that be Modbus, but this really doesn't matter). This interface is completely hidden inside an object with it's thread and a message queue. This object has member functions that allow to "read" or "write" data using that communication interface, and these functions can be used by multiple threads without any special synchronization. That's because internally the code of these function converts the arguments to a message/request and passes that via the queue to the handler thread, which just serves these requests one at a time.
This design pattern may be used instead of explicit synchronization with a mutex protecting the shared resource, which would have to be locked/unlocked by each thread that wishes to interact with that resource.
The best pattern that fits here may be the Broker pattern:
The Broker architectural pattern can be used to structure distributed
software systems with decoupled components that interact by remote
service invocations. A broker component Is responsible for
coordinating communication, such as forwarding requests. as well as
for transmitting results and exceptions.
I would simply call it asynchronous IO, or more catchy: non-blocking IO.
As: does it really matter what that "single thread side" is doing in detail? Does it make a difference if you deal "async" with a data base; or some other remote server?
They key attribute is: your code is not waiting for answers; but expecting information to flow in "later".
I’m trying to get my head around why use Akka/Actor model. Suppose a machine has allocated to my stateless application only 2 threads and I have two Actors (Actor1 & Actor2). When a requests comes to Actor1 it pushes it onto Actor2, freeing up Actor1 to serve more requests. In this case at least 1 thread is always occupied, either by Actor1 or Actor2.
Why shouldn’t I just serve requests coming into the Actor1 onto the Actor2 in a single thread (blocking way) which will also use 1 thread and the benefit of using this code would be
easier to debug the code this way for a stateless application because it's running in a single thread.
The second thread can serve other requests.
the processing time of the full request might be quicker because to invoke say DAO from Service is just a function call whereas in Akka's case I will be delegating from Actor1 to Actor2 and Akka will be doing some thread management magic under the hood which means more code to execute and slower execution.
I will be writing less code to return responses for HTTP requests because it's blocked and in a single thread and having a return statement will cascade the response back to the user. Whereas in Akka I will passing the request context around (In Sprays-can case) and will be invoking complete {} way deep in the application layer when the request is actually complete. Or I will be using some special Akka pattern to serve Response for every request. Sounds like I will be writing to be too much code here
Backpressure will only be required at the Client request end point.
Can Akka/Actor model help me gain more out of a machine which only offers 1 or 2 threads to my application ?
Suppose I have an enterprise Java application that basically does the following:
gather user input, query the backend databases (maybe multiple), run some algorithm (say do some in-memory calculation of the queried data sets to produce some statistics etc.), then return the data in some html pages.
My question is: If the bottleneck of the application is on the db query, how can NodeJS helps me in this scenarios since I still need to do all those post-db algorithm before I render the page? How an application architecture looks like?
Of course node can't speed up your storage layer and make that single request that's incurring so much backend processing satisfy that request any faster to the end user. But what it can do is not tie up a thread in the application server thread pool. The single thread can continue on it's loop while that work is going on and accept another request.
That other request might be a cheaper request that will return when it's work is done. That can also happen in an application server with a thread pool model ... that is unless all the threads in the thread pool model are tied up blocked on I/O requests - along with the overhead of each thread. The cheaper request will get queued waiting on a thread out of the thread pool because they are all blocking. Nodes single thread would loop and server the cheap request.
This works because node mandates that all I/O is async and the only work that blocks the loop is your code. That's why the saying "everything in node runs in parallel except your code". While it's possible to write async code in other application servers and achieve similar results, many offer non-async thread pool models where the coding is easier but sometimes less scalable.
For example, this hanselman post illustrates how asp.net is capable of doing async requests but it's not the common model that most have used.