I am trying to correctly trace transactions in a spark-kotlin server with newrelic. This particular server has intensive coroutine usage since it makes a lot of requests to other servers.
Newrelic has custom instrumentation that allows to track work being done in other threads from the one the transaction was originated at. The key instruments for this are:
Token
Trace with "async = true"
So to track the work on another thread, we must:
Issue a Token in the original thread
Link the token in the "worker thread"
Expire the token when no more threads will be asociated with the transaction
(See doc)
We also know, that one coroutine can be execute across multiple threads, so to correctly track the the work done in one coroutine we must link each of the threads. For this I figured a continuation interceptor might be used as explained in this question. The problem araises when we want to expire the token. Lets say we issue a token before calling a coroutine builder and set it in the coroutine context. We can then link the token in the continuation interceptor, but how can I know when a coroutine is finishing in order to expire the token? I tried using "ContinuationInterceptor::releaseInterceptedContinuation()" but it turns out it may be called several times in a single coroutine, as it is called after each continuation is relased.
I found that Job have a "invokeOnCompletion" method which is called by the framework once a coroutine finishes. I expired the token there!
Related
I have an asp.net core Web Api application.
In my application I have Web Api method which I want to prevent multi request from the same user to enter simultaneously. I don't mind request from different users to perform simultaneously.
I am not sure how to create the lock and where to put it. I thought about creating some kind of a dictionary which will contains the user id and perform the lock on the item but I don't think i'm getting it right. Also, what will happen if there is more than one server and there is a load balancer?
Example:
Let assume each registered user can do 10 long task each month. I need to check for each user if he exceeded his monthly limit. If the user will send many simultaneously requests to the server, he might be allowed to perform more than 10 operations. I understand that I need to put a lock on the method but I do want to allow other users to perform this action simultaneously.
What you're asking for is fundamentally not how the Internet works. The HTTP and underlying IP protocols are stateless, meaning each request is supposed to run independent of any knowledge of what has occurred previously (or concurrently, as the case may be). If you're worried about excessive load, your best bet is to implement rate limiting/throttling tied to authentication. That way, once a user burns through their allotted requests, they're cut off. This will then have a natural side-effect of making the developers programming against your API more cautious about sending excessive requests.
Just to be a bit more thorough, here, the chief problem with the approach you're suggesting is that I know of no way it can be practically implemented. You can use something like SemaphoreSlim to create a lock, but that needs to be static so that the same instance is used for each request. Being static is going to limit your ability to use a dictionary of them, which is what you'll need for this. It can technically be done, I suppose, but you'd have to use a ConcurrentDictionary and even then, there's no guarantee of single-thread additions. So, concurrent requests for the same user could load concurrent semphaphores into it, which defeats the entire point. I suppose you could front-load the dictionary with a semphaphore for each user from the start, but that could become a huge waste of resources, depending on your user-base. Long and short, it's one of those things where when you're finding a solution this darn difficult, it's a good sign you're likely trying to do something you shouldn't be doing.
EDIT
After reading your example, I think this really just boils down to an issue of trying to handle the work within the request pipeline. When there's some long-running task to be completed or just some heavy work to be done, the first step should always be to pass it off to a background service. This allows you to return a response quickly. Web servers have a limited amount of threads to handle requests with, and you want to service the request and return a response as quickly as possible to keep from exhausting your threadpool.
You can use a library like Hangfire to handle your background work or you can implement an IHostedService as described here to queue work on. Once you have your background service ready, you would then just immediately hand off to that any time your get a request to this endpoint, and return a 202 Accepted response with a URL the client can hit to check the status. That solves your immediate issue of not wanting to allow a ton of requests to this long-running job to bring your API down. It's now essentially doing nothing more that just telling something else to do it and then returning immediately.
For the actual background work you'd be queuing, there, you can check the user's allowance and if they have exceeded 10 requests (your rate limit), you fail the job immediately, without doing anything. If not, then you can actually start the work.
If you like, you can also enable webhook support to notify the client when the job completes. You simply allow the client to set a callback URL that you should notify on completion, and then when you've finish the work in the background task, you hit that callback. It's on the client to handle things on their end to decide what happens when the callback is it. They might for instance decide to use SignalR to send out a message to their own users/clients.
EDIT #2
I actually got a little intrigued by this. While I still think it's better for your to offload the work to a background process, I was able to create a solution using SemaphoreSlim. Essentially you just gate every request through the semaphore, where you'll check the current user's remaining requests. This does mean that other users must wait for this check to complete, but then your can release the semaphore and actually do the work. That way, at least, you're not blocking other users during the actual long-running job.
First, add a field to whatever class you're doing this in:
private static readonly SemaphoreSlim _semaphore = new SemaphoreSlim(1, 1);
Then, in the method that's actually being called:
await _semaphore.WaitAsync();
// get remaining requests for user
if (remaining > 0)
{
// decrement remaining requests for user (this must be done before this next line)
_semaphore.Release();
// now do the work
}
else
{
_semaphore.Release();
// handle user out of requests (return error, etc.)
}
This is essentially a bottle-neck. To do the appropriate check and decrementing, only one thread can go through the semaphore at a time. That means if your API gets slammed, requests will queue up and may take a while to complete. However, since this is probably just going to be something like a SELECT query followed by an UPDATE query, it shouldn't take that long for the semaphore to release. You should definitely do some load testing and watch it, though, if you're going to go this route.
I want to have a global variable for inter thread communication.
Current setup:
Thread Group 1: It is a multi-user thread, that will be accessing certain API's in a flow.
Thread group 2: Now, as per our current system, the auth-token refreshes after every 10 min. So in this thread group, I am calling refreshToken API.
Approach1: Tried using jmeter property. But since all threads are in parallel, so if one thread updates the token, all other threads fail.
Approach2: Using InterThreadCommunication plugin. Since I don't know when and where in the system, the token will expire, so don't know where to call in the inter thread communication pre-processor.
Use thread-specific properties like:
In "Refresh Token" Thread Group specify individual properties to hold tokens for each virtual user like:
${__setProperty(token_${__threadNum},YOUR_TOKEN_VALUE_HERE,)}
In "PA APIs" Thread Group access individual token values like:
${__P(token_${__threadNum},)}
References:
__setProperty() function
__threadNum() function
Apache JMeter Functions - An Introduction
In approach 1 with jmeter property you can continue with adding a Critical Section Controller with same key as a parent for Samplers who is trying to get or refresh your token.
That way Threads will wait until previous (get/)update token will be over and then get the updated token.
Notice that it won't work in Distributed testing:
Critical Section Controller takes locks only within one JVM, so if using Distributed testing ensure your use case does not rely on all threads of all JVMs blocking.
Situation: A high-scale Azure IIS7 application, which must do this:
Receive request
Place request payload onto a queue for decoupled asynchronous processing
Maintain connection to client
Wait for a notification that the asynchronous process has completed
Respond to client
Note that these will be long-running processes (30 seconds to 5 minutes).
If we employ Monitor.Wait(...) here, waiting for a callback to the same web application, from the asynchronous process, to invoke Monitor.Pulse(...) on the object we invoked Monitor.Wait() on, will this effectively create thread starvation in a hurry?
If so, how can this be mitigated? Is there a better pattern to employ here for awaiting the callback? For example, could we place the Response object into a thread-safe dictionary, and then somehow yield, and let the callback code lock the Response and proceed to respond to the client? If so, how?
Also, what if the asynchronous process dies, and never invokes the callback, thus never causing Monitor.Pulse() to fire? Is our thread hung now?
Given the requirement you have, I would suggest to have a look at AsyncPage/AsyncController (depends on whether you use ASP.NET WebForms or ASP.NET MVC). These give you the possibility to execute long running tasks in IIS without blocking I/O threads.
I have a WCF web service hosted in IIS- This service has a method - lets call it DoSomething(). DoSomething() is called from a client-side application.
DoSomething performs some work and returns the answer to the user. Now I need to log how often DoSomething is being called. I can add it to the DoSomething function so that it will for every call write to an sql database and update a counter, but this will slow down the DoSomething method as the user needs to wait for this extra database call.
Is it a good option to let the DoSomething method spawn a new thread which will update the counter in the database, and then just return the answer from the DoSomething method to the user without waiting for the thread to finnish? Then I will not know if the database update fails, but that is not critical.
Any problems with spawning a new background thread and not wait for it to finnish in WCF? Or is there a better way to solve this?
Update: To ask the question in a little different way. Is it a bad idea to spawn new threads insde a wcf web service method?
The main issue is one of reliability. Is this a call you care about? If the IIS process crashes after you returned the response, but before your thread completes, does it matter? If no, then you can use client side C# tools. If it does matter, then you must use a reliable queuing technology.
If you use the client side then spawning a new thread just to block on a DB call is never the correct answer. What you want is to make the call async, and for that you use SqlCommand.BeginExecute after you ensure that AsyncronousProcessing is enabled on the connection.
If you need reliable processing then you can use a pattern like Asynchronous procedure execution which relies on persisted queues.
As a side note things like logging, or hit counts, and the like are a huge performance bottleneck if done in the naive approach of writing to the database on every single HTTP request. You must batch and flush.
If you want to only track a single method like DoSomething() in service then you can create an custom operation behavior and apply it over the method.
The operation behavior will contain the code that logs the info to database. In that operation behavior you can use the .NET 4.0's new TPL library to create a task that will take care of database logging. If you use TPL you don't need to worry about directly creating threads.
The advantage of using operation behvaior tomorrow you need to track another method then at that time instead of duplicating the code there you are just going to mark the method with the custom operation behavior. If you want to track all the methods then you should go for service behavior.
To know more about operation behaviors check http://msdn.microsoft.com/en-us/library/system.servicemodel.operationbehaviorattribute.aspx
To know more about TPL(Task Parallel Library) check http://msdn.microsoft.com/en-us/library/dd460717.aspx
I'm building a multi-threaded service application in Delphi XE2. Each thread serves its own purpose apart from the other ones. The main service thread is only responsible for keeping the other threads going and saving a log file, etc. Each of these threads reports back to the main service thread through synchronized event triggers. These threads are created when the service starts and destroyed when the service ends.
I'd like to introduce a separate thread as a centralized database connection to avoid having to create many instances of TADOConnection. My service code can call standard functions such as UserListDataSet := DBThread.GetUserList(SomeUserListDataSet); or it would also be nice if I could send direct SQL statements like SomeDataSet := DBThread.Get(MySqlText);. I'd also like to avoid too many occasions of CoInitialize() etc.
The job threads will need to use this db thread. I need to figure out how to "ask" it for certain data, "wait" for a response, and "acquire" that response back in the thread which requested it. I'm sure there are many approaches to this, but I need to know which one is best suited for my scenario. Windows messages? Events? Should I have some sort of queue? Should it send data sets or something else? Is there already something that can do this? I need to figure out how to structure this DB thread in a way that it can be re-used from other threads.
The structure looks like this:
+ SvcThread
+ DBThread
+ TADOConnection
+ Thread1
+ Thread2
+ Thread3
I need threads 1 2 and 3 to send requests to the DBThread. When a thread sends any request to it, it needs to wait until it gets a response. Once there's a response, the DB Thread needs to notify the asking thread. Each of the threads might send a request to this DB Thread at the same time too.
A good tutorial on how to accomplish this would be perfect - it just needs to be a suitable fit for my scenario. I don't need to know just "how to make two threads talk together" but rather "how to make many threads talk to a centralized database thread". These job threads are created as children of the main service thread, and are not owned by the db thread. The db thread has no knowledge of the job threads.
Normally, you'd have a request queue where all the requests are stored. Your database thread reads a request from the queue, handles it, then invokes a callback routine specified by the requester to handle the result. Not sure how this maps to Delphi paradigms, but the basics should be the same.
Do any of the "requesting" threads have anything profitable that they could be doing while they are waiting for a response to be obtained from the database? If the answer is "no," as I suspect that it is quite likely to be, then perhaps you can simplify your situation quite a bit by eliminating the need for "a DB thread" completely. Perhaps all of the threads can simply share a single database-connection in turn, employing a mutual-exclusion object to cause them to "wait their turn."
Under this scenario, there would be one database-connection, and any thread which needed to use it would do so. But they would be obliged to obtain a mutex object first, hold on to the mutex during the time they were doing database queries, and then release the mutex so that the next thread could have its turn.
If you decide that it is somehow advantageous (or a necessity...) to dedicate a thread to managing the connection, then perhaps you could achieve the result using (a) a mutex to serialize the requests, as before; and (b) one event-object to signal the DB-thread that a new request has been posted, and (c) another event-object to signal the requester that the request has been completed.
In either case, if you have indeed determined that the requester threads have nothing useful that they could be doing in the meantime, you have the threads "simply sleeping" until their turn comes up. Then, they do their business, either directly or indirectly. There are no "queues," no complicated shared data-structures, simply because you have (say...) determined that there is no need for them.
I think using a DB connection pool would be a better fit for your problem. This would also allow you to scale your application later on without having to then create additional DB thread and then having to manage "load balancing" for those DB threads.
Since you are mentioning using TADOConnection please have a look at this implementation made by Cary Jensen http://cc.embarcadero.com/item/19975.
I am successfully using this DB connection pool in several applications. I have modified it in several ways, including using an ini file to control: maximum number of connections, cleanup time, timeout times etc.
Cary has written several articles that serves as documentation for it. One is here http://edn.embarcadero.com/article/30027.