I'm code reviewing a WCF service.
In the header of each message we inject data that the service is going to use later to build a connection string to a DB.
That's because the service is going to be used by a number of different sites, each with its own DB that the service has to query.
We use wcf extensibility. We have a custom MessageInspector that, after receiving the request, extracts the data from the message header, creates a context (that implements IExtension) and adds it to OperationContext.Current.Extensions.
Before sending the reply the custom context is removed from the Extencions collection.
This is a fairly common pattern, as discussed here:
Where to store data for current WCF call? Is ThreadStatic safe?
and here:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/319cac66-66e8-4dfe-9a82-dfd289c9df1f/wcf-doesnt-have-session-storage-so-where-should-one-store-call-specific-data?forum=wcf
This all works fine as long as the service receives a request, processes it, sends the reply and receives the next request.
But what if the service receives a request and before being able to reply it gets a second request? I built a small console application to test it. I send 2 messages from 2 different threads, I made the wcf service wait for 2 seconds, to ensure the second request comes in before the first one is completed and this is what I get:
Site Id : test1450 ; Session: uuid:2caf47cf-7d46-4d72-9275-d9c037fa0e70;id=2 : Thread Id: 6
Site Id : test1450 ; Session: uuid:2caf47cf-7d46-4d72-9275-d9c037fa0e70;id=3 : Thread Id: 22
It looks like wcf creates 2 sessions executing on 2 different threads, but Site Id is the same. It shouldn't. Judging from this it looks like OperationContext.Current.Extensions is a collection shared between threads.
Right now I'm inclined to think my test is wrong and I missed something.
Has anyone tried something similar and found out that OperationContext.Current is not thread safe?
OperationContext.Current like other similar properties such as HttpContext.Current have thread affine (or thread static) values. So they are thread safe in the sense that multiple threads can read them, but different threads will get different instances. They can be thought of as dictionaries between specific threads and instances.
So in this context they are not thread safe.
Requests are served by a thread pool so concurrent requests will have different thread ids. (up to a point where the thread pool is full, then requests will be put on hold)
Related
I use GF 4 as JavaEE server.
This is how I understand servlet processing: There is a pool of threads and when request comes one thread from this pool is taken to process the request. After that the thread is put back to pool.
Based on the information above I suppose (I am not sure) that websockets (server end points) are processed this way: There is pool of threads, when
Client creates new websocket a thread is taken from pool to create new instance of ServerEndpoint and to execute #OnOpen method. After that thread is put back to pool.
Client sends message over websocket to server. Thread is taken from pool to execute #OnMessage method. After that thread is put back to pool.
Client closes the websocket - thread is taken from pool to execute #OnClose method. After that thread is put back to pool.
It all means that every method of ServerEndpoint can be executed by different threads. Is my understanding right?
Yes.
The ServerEndpoint instance lives as long as the associated WebSocket session is available as Session argument during #OnOpen. During that WebSocket session, many HTTP and WebSocket requests may be fired. Each such request accounts as an individual thread.
In other words, if your ServerEndpoint class needs to deal with instance variables in multiple methods for some reason, it must be implemented in a thread safe manner. Depending on the concrete functional requirement, you'd probably better use Session#getUserProperties() instead to carry around state associated with the WS session (think of it as session attributes).
Noted should be that this all is regardless of the container and WS implementation used.
I have a webservice using Hibernate as DAL - using MySql with InnoDB.
Since I want to make webservice calls really short (for better user experience in client side) I am using 2 threads with msg queue to do some work.
1 thread gets userId in the message, loads the user from the DB and gets it email address and send email to.
Second thread is used like this:
Webservice call.... doing some actions.
Adding ActivityLog into the DB.
Doing session.save( log ); session.commit();
Now we send message to the thread with the logId.
Message received - insert new entries into timeline table (userid, logId). Session is different session object than the main logic session object.
Should I have problems in this? in lazy loading? in the threads, since message is sent to the thread after commit()?
A Webservice -> Message queue architecture is pretty standard in case you don't need synchronous reply to the web service.
In the web service, store whatever you want into the database and the message queue will pick that up later.
A different session is not a problem, but if you have a single static Session for each of these threads, they may be subject to Session bloat: objects piling up in the Session cache.
More on this: http://suryagaddipati.wordpress.com/2008/02/15/hibernate-rich-clients-and-long-running-sessions/
When a Web Role places a message onto a Storage Queue, how can it poll for a specific, correlated response? I would like the back-end Worker Role to place a message onto a response queue, with the intent being that the caller would pick the response up and go from there.
Our intent is to leverage the Queue in order to offload some heavy processing onto the back-end Worker Roles in order to ensure high performance on the Web Roles. However, we do not wish to respond to the HTTP requests until the back-end Workers are finished and have responded.
I am actually in the middle of making a similar decision. In my case i have a WCF service running in a web role which should off-load calculations to worker-roles. When the result has been computed, the web role will return the answer to the client.
My basic data structure knowledge tells me that i should avoid using something that is designed as a queue in a non-queue way. That means a queue should always be serviced in a FIFO like manner. So basically if using queues for both requests and response, the threads awaiting to return data to the client will have to wait untill the calculation message is at the "top" of the response queue, which is not optimal. If storing the responses by using Azure tables, the threads poll for messages creating unnecessary overhead
What i belive is a possible solution to this problem is using a queue for the requests. This enables use of the competeing consumers pattern and thereby load-balancing. On messages sent into this queue you set the correlationId property on the message. For reply the pub/sub part ("topics") part of Azure service bus is used togehter with a correlation filter. When your back-end has processed the request, it published a result to a "responseSubject" with the correlationId given in the original request. Now this response ca be retrieved by your client by calling CreateSubscribtion (Sorry, i can't post more than two links apparently, google it) using that correlation filter, and it should get notified when the answer is published. Notice that the CreateSubscribtion part should just be done one time in the OnStart method. Then you can do an async BeginRecieve on that subscribtion and the role will be notified in the given callback when a response for one of it's request is available. The correlationId will tell you which request the response is for. So your last challenge is giving this response back to the thread holding the client connection.
This could be achieved by creating Dictionary with the correlationId's (probably GUID's) as key and responses as value. When your web role gets a request it creates the guid, set it as correlationId, add it the hashset, fire the message to the queue and then call Monitor.Wait() on the Guid object. Then have the recieve method invoked by the topic subscribition add the response to the dictionary and then call Monitor.Notify() on that same guid object. This awakens your original request-thread and you can now return the answer to your client (Or something. Basically you just want your thread to sleep and not consume any ressources while waiting)
The queues on the Azure Service Bus have a lot more capabilities and paradigms including pub / sub capabilities which can address issues dealing with queue servicing across multiple instance.
One approach with pub / sub, is to have one queue for requests and one for the responses. Each requesting instance would also subscribe to the response queue with a filter on the header such that it would only receive the responses targeted for it. The request message would, of course contain the value to the placed in the response header to drive the filter.
For the Service Bus based solution there are samples available for implementing Request/Response pattern with Queues and Topics (pub-sub)
Let worker role keep polling and processing the message. As soon as the message is processed add an entry in Table storage with the required corelationId(RowKey) and the processing result, before deleting the processed message from the queue.
Then WebRoles just need to do a look up of the Table with the desired correlationId(RowKey) & PartitionKey
Have a look at using SignalR between the worker role and the browser client. So your web role puts a message on the queue and returns a result to the browser (something simple like 'waiting...') and hook it up to the worker role with SignalR. That way your web role carries on doing other stuff and doesn't have to wait for a result from asynchronous processing, only the browser needs to.
There is nothing intrinsic to Windows Azure queues that does what you are asking. However, you could build this yourself fairly easily. Include a message ID (GUID) in your push to the queue and when processing is complete, have the worker push a new message with that message ID into a response channel queue. Your web app can poll this queue to determine when processing is completed for a given command.
We have done something similar and are looking to use something like SignalR to help reply back to the client when commands are completed.
I'm building a multi-threaded service application in Delphi XE2. Each thread serves its own purpose apart from the other ones. The main service thread is only responsible for keeping the other threads going and saving a log file, etc. Each of these threads reports back to the main service thread through synchronized event triggers. These threads are created when the service starts and destroyed when the service ends.
I'd like to introduce a separate thread as a centralized database connection to avoid having to create many instances of TADOConnection. My service code can call standard functions such as UserListDataSet := DBThread.GetUserList(SomeUserListDataSet); or it would also be nice if I could send direct SQL statements like SomeDataSet := DBThread.Get(MySqlText);. I'd also like to avoid too many occasions of CoInitialize() etc.
The job threads will need to use this db thread. I need to figure out how to "ask" it for certain data, "wait" for a response, and "acquire" that response back in the thread which requested it. I'm sure there are many approaches to this, but I need to know which one is best suited for my scenario. Windows messages? Events? Should I have some sort of queue? Should it send data sets or something else? Is there already something that can do this? I need to figure out how to structure this DB thread in a way that it can be re-used from other threads.
The structure looks like this:
+ SvcThread
+ DBThread
+ TADOConnection
+ Thread1
+ Thread2
+ Thread3
I need threads 1 2 and 3 to send requests to the DBThread. When a thread sends any request to it, it needs to wait until it gets a response. Once there's a response, the DB Thread needs to notify the asking thread. Each of the threads might send a request to this DB Thread at the same time too.
A good tutorial on how to accomplish this would be perfect - it just needs to be a suitable fit for my scenario. I don't need to know just "how to make two threads talk together" but rather "how to make many threads talk to a centralized database thread". These job threads are created as children of the main service thread, and are not owned by the db thread. The db thread has no knowledge of the job threads.
Normally, you'd have a request queue where all the requests are stored. Your database thread reads a request from the queue, handles it, then invokes a callback routine specified by the requester to handle the result. Not sure how this maps to Delphi paradigms, but the basics should be the same.
Do any of the "requesting" threads have anything profitable that they could be doing while they are waiting for a response to be obtained from the database? If the answer is "no," as I suspect that it is quite likely to be, then perhaps you can simplify your situation quite a bit by eliminating the need for "a DB thread" completely. Perhaps all of the threads can simply share a single database-connection in turn, employing a mutual-exclusion object to cause them to "wait their turn."
Under this scenario, there would be one database-connection, and any thread which needed to use it would do so. But they would be obliged to obtain a mutex object first, hold on to the mutex during the time they were doing database queries, and then release the mutex so that the next thread could have its turn.
If you decide that it is somehow advantageous (or a necessity...) to dedicate a thread to managing the connection, then perhaps you could achieve the result using (a) a mutex to serialize the requests, as before; and (b) one event-object to signal the DB-thread that a new request has been posted, and (c) another event-object to signal the requester that the request has been completed.
In either case, if you have indeed determined that the requester threads have nothing useful that they could be doing in the meantime, you have the threads "simply sleeping" until their turn comes up. Then, they do their business, either directly or indirectly. There are no "queues," no complicated shared data-structures, simply because you have (say...) determined that there is no need for them.
I think using a DB connection pool would be a better fit for your problem. This would also allow you to scale your application later on without having to then create additional DB thread and then having to manage "load balancing" for those DB threads.
Since you are mentioning using TADOConnection please have a look at this implementation made by Cary Jensen http://cc.embarcadero.com/item/19975.
I am successfully using this DB connection pool in several applications. I have modified it in several ways, including using an ini file to control: maximum number of connections, cleanup time, timeout times etc.
Cary has written several articles that serves as documentation for it. One is here http://edn.embarcadero.com/article/30027.
I'm using multithreaded wcf maxConcurrentCalls = 10. By logging calls to my service I see that 10 different threads are executing in my service class and that they are reused in the following calls.
Can I tell WCF to destroy/delete a thread so it will create a new one on the next call?
This is because I have thread-static state that I sometimes want to be cleared (on unexpected exceptions). I am using the thread-static scope to gain performance.
WCF doesn't create new threads. It uses threads from a thread pool to service requests. So when a request begins it draws a thread from this pool to execute the request and after it finishes it returns the thread to the pool. The way that WCF uses threads underneath is an implementation detail that you should not rely on. You should never use Thread Static in ASP.NET/WCF to store state.
In ASP.NET you should use HttpContext.Items and in WCF OperationContext to store some state that would be available through the entire request.
Here's a good blog post you may take a look at which illustrates a nice way to abstract this.