I use GF 4 as JavaEE server.
This is how I understand servlet processing: There is a pool of threads and when request comes one thread from this pool is taken to process the request. After that the thread is put back to pool.
Based on the information above I suppose (I am not sure) that websockets (server end points) are processed this way: There is pool of threads, when
Client creates new websocket a thread is taken from pool to create new instance of ServerEndpoint and to execute #OnOpen method. After that thread is put back to pool.
Client sends message over websocket to server. Thread is taken from pool to execute #OnMessage method. After that thread is put back to pool.
Client closes the websocket - thread is taken from pool to execute #OnClose method. After that thread is put back to pool.
It all means that every method of ServerEndpoint can be executed by different threads. Is my understanding right?
Yes.
The ServerEndpoint instance lives as long as the associated WebSocket session is available as Session argument during #OnOpen. During that WebSocket session, many HTTP and WebSocket requests may be fired. Each such request accounts as an individual thread.
In other words, if your ServerEndpoint class needs to deal with instance variables in multiple methods for some reason, it must be implemented in a thread safe manner. Depending on the concrete functional requirement, you'd probably better use Session#getUserProperties() instead to carry around state associated with the WS session (think of it as session attributes).
Noted should be that this all is regardless of the container and WS implementation used.
Related
Say I'm using gRPC server-side streaming. If I send multiple client requests in a for loop, in this case on the server side, multiple threads will run the same service instance, and they will share the same StreamObserver object. If in one thread, the .onCompleted is called, will it prevent other threads from calling .onNext?
StreamObserver is not thread-safe, so should never be called from multiple threads simultaneously. The service instance is called multiple times, one for each server-streaming RPC. (For server-streaming RPCs, that is the same as once per client request.) But each time it is called it receives a different StreamObserver instance. You can call the different instances on different threads.
Since each RPC has its own StreamObserver instance, calling onCompleted() on one RPC has no impact to being able to call onNext() for a different RPC. The important part is to not call onNext() after onCompleted() for a single RPC/StreamObserver instance.
I have a spring controller. The request thread from the controller is passed to the #Service annotated Service class. Now I want to do some background work and the request thread must some how trigger the background thread and continue with it's own work and should not wait for the background thread to complete.
My first question : is this safe to do this.?
Second question : how to do this.?
Is this safe
Not really. If you have many concurrent users, you'll spawn a thread for everyone of them, and the high number of threads could bring your server to its knees. The app server uses a pool of threads, precisely to avoid this problem.
How to do this
I would do this by using the asynchronous capabilities of Spring. Call a service method annotated with #Async, and the service method will be executed by another thread, from a configurable pool.
I'm code reviewing a WCF service.
In the header of each message we inject data that the service is going to use later to build a connection string to a DB.
That's because the service is going to be used by a number of different sites, each with its own DB that the service has to query.
We use wcf extensibility. We have a custom MessageInspector that, after receiving the request, extracts the data from the message header, creates a context (that implements IExtension) and adds it to OperationContext.Current.Extensions.
Before sending the reply the custom context is removed from the Extencions collection.
This is a fairly common pattern, as discussed here:
Where to store data for current WCF call? Is ThreadStatic safe?
and here:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/319cac66-66e8-4dfe-9a82-dfd289c9df1f/wcf-doesnt-have-session-storage-so-where-should-one-store-call-specific-data?forum=wcf
This all works fine as long as the service receives a request, processes it, sends the reply and receives the next request.
But what if the service receives a request and before being able to reply it gets a second request? I built a small console application to test it. I send 2 messages from 2 different threads, I made the wcf service wait for 2 seconds, to ensure the second request comes in before the first one is completed and this is what I get:
Site Id : test1450 ; Session: uuid:2caf47cf-7d46-4d72-9275-d9c037fa0e70;id=2 : Thread Id: 6
Site Id : test1450 ; Session: uuid:2caf47cf-7d46-4d72-9275-d9c037fa0e70;id=3 : Thread Id: 22
It looks like wcf creates 2 sessions executing on 2 different threads, but Site Id is the same. It shouldn't. Judging from this it looks like OperationContext.Current.Extensions is a collection shared between threads.
Right now I'm inclined to think my test is wrong and I missed something.
Has anyone tried something similar and found out that OperationContext.Current is not thread safe?
OperationContext.Current like other similar properties such as HttpContext.Current have thread affine (or thread static) values. So they are thread safe in the sense that multiple threads can read them, but different threads will get different instances. They can be thought of as dictionaries between specific threads and instances.
So in this context they are not thread safe.
Requests are served by a thread pool so concurrent requests will have different thread ids. (up to a point where the thread pool is full, then requests will be put on hold)
I'm using multithreaded wcf maxConcurrentCalls = 10. By logging calls to my service I see that 10 different threads are executing in my service class and that they are reused in the following calls.
Can I tell WCF to destroy/delete a thread so it will create a new one on the next call?
This is because I have thread-static state that I sometimes want to be cleared (on unexpected exceptions). I am using the thread-static scope to gain performance.
WCF doesn't create new threads. It uses threads from a thread pool to service requests. So when a request begins it draws a thread from this pool to execute the request and after it finishes it returns the thread to the pool. The way that WCF uses threads underneath is an implementation detail that you should not rely on. You should never use Thread Static in ASP.NET/WCF to store state.
In ASP.NET you should use HttpContext.Items and in WCF OperationContext to store some state that would be available through the entire request.
Here's a good blog post you may take a look at which illustrates a nice way to abstract this.
Situation:
I'm am using named pipes on Windows for IPC, using C++.
The server creates a named pipe instance via CreateNamedPipe, and waits for clients to connect via ConnectNamedPipe.
Everytime a client calls CreateFile to access the named pipe, the server creates a thread using CreateThread to service that client. After that, the server reiterates the loop, creating a pipe instance via CreateNamedPipe and listening for the next client via ConnectNamedPipe, etc ...
Problem:
Every client request triggers a CreateThread on the server. If clients come fast and furious, there would be many calls to CreateThread.
Questions:
Q1: Is it possible to reuse already created threads to service future client requests?
If this is possible, how should I do this?
Q2: Would Thread Pool help in this situation?
I wrote a named pipe server today using IOCompletion ports just to see how.
The basic logic flow was:
I created the first named pipe via CreateNamedPipe
I created the main Io Completion Port object using that handle: CreateIoCompletionPort
I create a pool of worker threads - as a thumb suck, CPUs x2. Each worker thread calls GetQueuedCompletionStatus in a loop.
Then called ConnectNamedPipe passing in an overlapped structure. When this pipe connects, one of the GetQueuedCompletionStatus calls will return.
My main thread then joins the pool of workers by also calling GetQueuedCompletionStatus.
Thats about it really.
Each time a thread returns from GetQueuedCompletionStatus its because the associated pipe has been connected, has read data, or has been closed.
Each time a pipe is connected, I immediately create a unconnected pipe to accept the next client (should probably have more than one waiting at a time) and call ReadFile on the current pipe, passing an overlapped structure - ensuring that as data arrives GetQueuedCompletionStatus will tell me about it.
There are a couple of irritating edge cases where functions return a fail code, but GetLastError() is a success. Because the function "failed" you have to handle the success immediately as no queued completion status was posted. Conversely, (and I belive Vista adds an API to "fix" this) if data is available immediately, the overlapped functions can return success, but a queued completion status is ALSO posted so be careful not to double handle data in that case.
On Windows, the most efficient way to build a concurrent server is to use an asynch model with completion ports. But yes you can use a thread pool and use blocking i/o too, as that is a simpler programming abstraction.
Vista/Windows 2008 provide a thread pool abstraction.