I have a WCF web service hosted in IIS- This service has a method - lets call it DoSomething(). DoSomething() is called from a client-side application.
DoSomething performs some work and returns the answer to the user. Now I need to log how often DoSomething is being called. I can add it to the DoSomething function so that it will for every call write to an sql database and update a counter, but this will slow down the DoSomething method as the user needs to wait for this extra database call.
Is it a good option to let the DoSomething method spawn a new thread which will update the counter in the database, and then just return the answer from the DoSomething method to the user without waiting for the thread to finnish? Then I will not know if the database update fails, but that is not critical.
Any problems with spawning a new background thread and not wait for it to finnish in WCF? Or is there a better way to solve this?
Update: To ask the question in a little different way. Is it a bad idea to spawn new threads insde a wcf web service method?
The main issue is one of reliability. Is this a call you care about? If the IIS process crashes after you returned the response, but before your thread completes, does it matter? If no, then you can use client side C# tools. If it does matter, then you must use a reliable queuing technology.
If you use the client side then spawning a new thread just to block on a DB call is never the correct answer. What you want is to make the call async, and for that you use SqlCommand.BeginExecute after you ensure that AsyncronousProcessing is enabled on the connection.
If you need reliable processing then you can use a pattern like Asynchronous procedure execution which relies on persisted queues.
As a side note things like logging, or hit counts, and the like are a huge performance bottleneck if done in the naive approach of writing to the database on every single HTTP request. You must batch and flush.
If you want to only track a single method like DoSomething() in service then you can create an custom operation behavior and apply it over the method.
The operation behavior will contain the code that logs the info to database. In that operation behavior you can use the .NET 4.0's new TPL library to create a task that will take care of database logging. If you use TPL you don't need to worry about directly creating threads.
The advantage of using operation behvaior tomorrow you need to track another method then at that time instead of duplicating the code there you are just going to mark the method with the custom operation behavior. If you want to track all the methods then you should go for service behavior.
To know more about operation behaviors check http://msdn.microsoft.com/en-us/library/system.servicemodel.operationbehaviorattribute.aspx
To know more about TPL(Task Parallel Library) check http://msdn.microsoft.com/en-us/library/dd460717.aspx
Related
I am trying to turn my app multithreading. What I want to achieve is:
- Receive command via TidHTTPServer
- Execute local action (might involve using tidHTTP to send/receive data to other services)
- return execution result to the original caller
since I am pretty new to multi-threading I would like to know if my design-idea is correct
TMsgHandler=Class(TThread)
in TidHTTPServer.OnCommandGet I create a new instance of TMsgHandler and pass ARequestInfo and AResponseInfo
TMsgHandler.Excecute interprest the data
Can TMsgHandler.Execeute use Objects (descendants of TidHTTP) in my Main to communicate with other services?
TMsgHandler sends answer through AResponseInfo and terminates.
will this work?
This is not the correct design.
THTTPServer is a multi-threaded component. Its OnCommand... events are fired in the context of worker threads that Indy creates for you.
As such, you do not need to derive your TMsgHandler from TThread. Do your TIdHTTP directly in the context of the OnCommand... thread instead. A response will not be sent back to the client until your event handler exits (unless you send one manually). However, you should not share a single TIdHTTP from the main thread (unless you absolute need to, in which case you would need to synchronize access to it). You should create a new TIdHTTP dynamically directly in your OnCommand.../TMsgHandler code as needed.
I have an asp.net core Web Api application.
In my application I have Web Api method which I want to prevent multi request from the same user to enter simultaneously. I don't mind request from different users to perform simultaneously.
I am not sure how to create the lock and where to put it. I thought about creating some kind of a dictionary which will contains the user id and perform the lock on the item but I don't think i'm getting it right. Also, what will happen if there is more than one server and there is a load balancer?
Example:
Let assume each registered user can do 10 long task each month. I need to check for each user if he exceeded his monthly limit. If the user will send many simultaneously requests to the server, he might be allowed to perform more than 10 operations. I understand that I need to put a lock on the method but I do want to allow other users to perform this action simultaneously.
What you're asking for is fundamentally not how the Internet works. The HTTP and underlying IP protocols are stateless, meaning each request is supposed to run independent of any knowledge of what has occurred previously (or concurrently, as the case may be). If you're worried about excessive load, your best bet is to implement rate limiting/throttling tied to authentication. That way, once a user burns through their allotted requests, they're cut off. This will then have a natural side-effect of making the developers programming against your API more cautious about sending excessive requests.
Just to be a bit more thorough, here, the chief problem with the approach you're suggesting is that I know of no way it can be practically implemented. You can use something like SemaphoreSlim to create a lock, but that needs to be static so that the same instance is used for each request. Being static is going to limit your ability to use a dictionary of them, which is what you'll need for this. It can technically be done, I suppose, but you'd have to use a ConcurrentDictionary and even then, there's no guarantee of single-thread additions. So, concurrent requests for the same user could load concurrent semphaphores into it, which defeats the entire point. I suppose you could front-load the dictionary with a semphaphore for each user from the start, but that could become a huge waste of resources, depending on your user-base. Long and short, it's one of those things where when you're finding a solution this darn difficult, it's a good sign you're likely trying to do something you shouldn't be doing.
EDIT
After reading your example, I think this really just boils down to an issue of trying to handle the work within the request pipeline. When there's some long-running task to be completed or just some heavy work to be done, the first step should always be to pass it off to a background service. This allows you to return a response quickly. Web servers have a limited amount of threads to handle requests with, and you want to service the request and return a response as quickly as possible to keep from exhausting your threadpool.
You can use a library like Hangfire to handle your background work or you can implement an IHostedService as described here to queue work on. Once you have your background service ready, you would then just immediately hand off to that any time your get a request to this endpoint, and return a 202 Accepted response with a URL the client can hit to check the status. That solves your immediate issue of not wanting to allow a ton of requests to this long-running job to bring your API down. It's now essentially doing nothing more that just telling something else to do it and then returning immediately.
For the actual background work you'd be queuing, there, you can check the user's allowance and if they have exceeded 10 requests (your rate limit), you fail the job immediately, without doing anything. If not, then you can actually start the work.
If you like, you can also enable webhook support to notify the client when the job completes. You simply allow the client to set a callback URL that you should notify on completion, and then when you've finish the work in the background task, you hit that callback. It's on the client to handle things on their end to decide what happens when the callback is it. They might for instance decide to use SignalR to send out a message to their own users/clients.
EDIT #2
I actually got a little intrigued by this. While I still think it's better for your to offload the work to a background process, I was able to create a solution using SemaphoreSlim. Essentially you just gate every request through the semaphore, where you'll check the current user's remaining requests. This does mean that other users must wait for this check to complete, but then your can release the semaphore and actually do the work. That way, at least, you're not blocking other users during the actual long-running job.
First, add a field to whatever class you're doing this in:
private static readonly SemaphoreSlim _semaphore = new SemaphoreSlim(1, 1);
Then, in the method that's actually being called:
await _semaphore.WaitAsync();
// get remaining requests for user
if (remaining > 0)
{
// decrement remaining requests for user (this must be done before this next line)
_semaphore.Release();
// now do the work
}
else
{
_semaphore.Release();
// handle user out of requests (return error, etc.)
}
This is essentially a bottle-neck. To do the appropriate check and decrementing, only one thread can go through the semaphore at a time. That means if your API gets slammed, requests will queue up and may take a while to complete. However, since this is probably just going to be something like a SELECT query followed by an UPDATE query, it shouldn't take that long for the semaphore to release. You should definitely do some load testing and watch it, though, if you're going to go this route.
I know that blocking code is discouraged in node.js because it is single-threaded. My question is asking whether or not blocking code is acceptable in certain circumstances.
For example, if I was running an Express webserver that requires a MongoDB connection, would it be acceptable to block the event loop until the database connection was established? This is assuming that all pages served by Express require a database query (which would fail if MongoDB was not initialized).
Another example would be an application that requires the contents of a configuration file before being initializing. Is there any benefit in using fs.readFile over fs.readFileSync in this case?
Is there a way to work around this? Is wrapping all the code in a callback or promise the best way to go? How would that be different from using blocking code in the above examples?
It is really up to you to decide what is acceptable. And you would do that by determining what the consequences of blocking would be ... on a case-by-case basis. That analysis would take into account:
how often it occurs,
how long the event loop is likely to be blocked, and
the impact that blocking in that context will have on usability1.
Obviously, there are ways to avoid blocking, but these tend to add complexity to your application. Really, you need to decide ... on a case-by-case basis ... whether that added complexity is warranted.
Bottom line: >>you<< need to decide what is acceptable based on your understanding of your application and your users.
1 - For example, in a game it would be more acceptable to block the UI while switching "levels" than during active play. Or for a general web service, "once off" blocking while a config file is loaded or a DB connection is established during webserver startup is more acceptable that if this happened on every request.
From my experience most tasks should be handled in a callback or by returning a promise. You DO NOT want to block code in a Node application. That's what makes it so nice! Mostly with MongoDB it will crash before it has a chance to connect if there is no connection. It won't' really have an effect on an API call because your server will be dead!
Source: I'm a developer at a bootcamp that teaches MEAN stack.
Your two examples are completely different. The distinction actually answers the question in and of itself.
Grabbing data from a database is dependent on being connected to that database. Any code that is dependent upon that data is then dependent upon that connection. These things have to happen serially for the app to function and be meaningful.
On the other hand, readFileSync will block ALL code, not just code that is reliant on it. You could start reading a csv file while simultaneously establishing a database connection. Once both are done, you could add that csv data to the database.
In our ColdFusion application we have stateless model objects.
All the data I want I can get with one method call (it calls other internally without saving the state).
Methods usually ask the database for the data. All methods are read only, so I don't have to worry about thread safety (please correct me if I'm wrong).
So there is no need to instantiate objects at all. I could call them statically, but ColdFusion doesn't have static methods - calling the method would mean instantiating the object first.
To improve performance I have created singletons for every Model object.
So far it works great - each object is created once and then accessed as needed.
Now my worry is that all requests for data would go through only 1 model object.
Should I? I mean if on my object I have a method getOfferData() and it's time-consuming.
What if a couple of clients want to access it?
Will second one wait for the first request to finish or is it executed in a separate thread?
It's the same object after all.
Should I implement some kind of object pool for this?
The singleton pattern you are using won't cause the problem you are describing. If getOfferData() is still running when another call to that function gets called on a different request then this will not cause it to queue unless you do one of the following:-
Use cflock to grant an exclusive lock
Get queueing connecting to your database because of locking / transactions
You have too many things running and you use all the available concurrent threads available to ColdFusion
So the way you are going about it is fine.
Hope that helps.
I'm building a multi-threaded service application in Delphi XE2. Each thread serves its own purpose apart from the other ones. The main service thread is only responsible for keeping the other threads going and saving a log file, etc. Each of these threads reports back to the main service thread through synchronized event triggers. These threads are created when the service starts and destroyed when the service ends.
I'd like to introduce a separate thread as a centralized database connection to avoid having to create many instances of TADOConnection. My service code can call standard functions such as UserListDataSet := DBThread.GetUserList(SomeUserListDataSet); or it would also be nice if I could send direct SQL statements like SomeDataSet := DBThread.Get(MySqlText);. I'd also like to avoid too many occasions of CoInitialize() etc.
The job threads will need to use this db thread. I need to figure out how to "ask" it for certain data, "wait" for a response, and "acquire" that response back in the thread which requested it. I'm sure there are many approaches to this, but I need to know which one is best suited for my scenario. Windows messages? Events? Should I have some sort of queue? Should it send data sets or something else? Is there already something that can do this? I need to figure out how to structure this DB thread in a way that it can be re-used from other threads.
The structure looks like this:
+ SvcThread
+ DBThread
+ TADOConnection
+ Thread1
+ Thread2
+ Thread3
I need threads 1 2 and 3 to send requests to the DBThread. When a thread sends any request to it, it needs to wait until it gets a response. Once there's a response, the DB Thread needs to notify the asking thread. Each of the threads might send a request to this DB Thread at the same time too.
A good tutorial on how to accomplish this would be perfect - it just needs to be a suitable fit for my scenario. I don't need to know just "how to make two threads talk together" but rather "how to make many threads talk to a centralized database thread". These job threads are created as children of the main service thread, and are not owned by the db thread. The db thread has no knowledge of the job threads.
Normally, you'd have a request queue where all the requests are stored. Your database thread reads a request from the queue, handles it, then invokes a callback routine specified by the requester to handle the result. Not sure how this maps to Delphi paradigms, but the basics should be the same.
Do any of the "requesting" threads have anything profitable that they could be doing while they are waiting for a response to be obtained from the database? If the answer is "no," as I suspect that it is quite likely to be, then perhaps you can simplify your situation quite a bit by eliminating the need for "a DB thread" completely. Perhaps all of the threads can simply share a single database-connection in turn, employing a mutual-exclusion object to cause them to "wait their turn."
Under this scenario, there would be one database-connection, and any thread which needed to use it would do so. But they would be obliged to obtain a mutex object first, hold on to the mutex during the time they were doing database queries, and then release the mutex so that the next thread could have its turn.
If you decide that it is somehow advantageous (or a necessity...) to dedicate a thread to managing the connection, then perhaps you could achieve the result using (a) a mutex to serialize the requests, as before; and (b) one event-object to signal the DB-thread that a new request has been posted, and (c) another event-object to signal the requester that the request has been completed.
In either case, if you have indeed determined that the requester threads have nothing useful that they could be doing in the meantime, you have the threads "simply sleeping" until their turn comes up. Then, they do their business, either directly or indirectly. There are no "queues," no complicated shared data-structures, simply because you have (say...) determined that there is no need for them.
I think using a DB connection pool would be a better fit for your problem. This would also allow you to scale your application later on without having to then create additional DB thread and then having to manage "load balancing" for those DB threads.
Since you are mentioning using TADOConnection please have a look at this implementation made by Cary Jensen http://cc.embarcadero.com/item/19975.
I am successfully using this DB connection pool in several applications. I have modified it in several ways, including using an ini file to control: maximum number of connections, cleanup time, timeout times etc.
Cary has written several articles that serves as documentation for it. One is here http://edn.embarcadero.com/article/30027.