Okay here is the scenario:
Clients send a request at 10:00:00 (H:M:S). That request is stored in the IIS app pool QUEUE until there is an available thread for that request. Thread is released and the app pool now received the request that it has to process. The time is 10:00:15.
When did the client start waiting for his response - at 10:00:00 or 10:00:15?
Client timeout period started at 10:00:00. The client has no idea what's going on with the internals of the server, nor even network latency. All it knows is when the request was sent and when a response was received (if at all).
While there may be more granular timeouts at the platform-specific message handler level (SendTimeout, ReceiveHeadersTimeout, ReceiveDataTimeout), the Timeout defined on .NET Standard-compliant implementations of HttpClient is end-to-end. Per Microsoft:
The HttpClient.Timeout property is intended to be exactly what you are
referring to as the 99% case: an end-to-end timeout after which the
request expires. The WinHttpHandler API is intended to provide a
deeper control to developers for more advanced scenarios. In keeping
with this intention, we have more granular timeouts on that type since
we have gotten developer requests in the past who asked for control
over a specific stage of the request.
Related
We have a service which pings our EP1 Premium service and yesterday we received 3 client side timeout errors after 2 minutes of waiting. When opening the trace in App insights, these requests which time out are not even logged and have no trace of ever being received Azure side, and therefore stay unanswered. By looking at the metrics provided in the Azure Functions app, I found out that 1-2 minutes after the request has been sent, the app loses all its ability to work as its Total App Domains falls to 0 as well as all connections, threads and so on and this state lasts until the next request is received, therefore "skipping" the request that happened beforehand. This is a big issue as I need to make sure requests get answered in a timely manner.
The client service sent HTTP requests to the Azure Functions app expecting an answer, only to time out while the Azure-side doesn't have any record of ever receiving the request.
I believe this issues is related to Consumption Plan of Azure Functions called Cold Start behaviour. The "skipping" mechanism is explained below:
Apps may scale to zero when idle, meaning some requests may have additional latency at startup. The consumption plan does have some optimizations to help decrease cold start time, including pulling from pre-warmed placeholder functions that already have the function host and language processes running.https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale#cold-start-behavior
Please also consider of having look on this article, which explains the behaviour. https://azure.microsoft.com/en-us/blog/understanding-serverless-cold-start/
I have Web application on IIS server.
I have POST method that take a long time to run (Around 30-40min).
After period time the application stop running (Without any exception).
I set Idle timeout to be 0 and It is not help for me.
What can I do to solve it?
Instead of doing all the work initiated by the request before responding at all:
Receive the request
Put the information in the request in a queue (which you could manage with a database table, ZeroMQ, or whatever else you like)
Respond with a "Request recieved" message.
That way you respond within seconds, which is acceptable for HTTP.
Then have a separate process monitor the queue and process the data on it (doing the 30-40 minute long job). When the job is complete, notify the user.
You could do this through the browser with a Notification or through a WebSocket or use a completely different mechanism (such as by sending an email to the user who made the request).
We have our HTTP layer served by Play Framework in Scala. One of our APIs is something of the form:
POST /customer/:id
Requests are sent by our UI team which calls these APIs through a React Framework.
The issue is that, sometimes, the requests are issued in batches, successively one after the other for the same customer ID. When this happens, different threads process these requests and so our persistent layer (MySQL) reaches an inconsistent state due to the difference in the timestamp of the handling of these requests.
Is it possible to configure some sort of thread affinity in Play Scala? What I mean by that is, can I configure Play to ensure that requests of a particular customer ID are handled by the same thread throughout the life-cycle of the application?
Batch is
put several API calls into a single HTTP request.
A batch request is a set of command in one HTTP request, like here https://developers.facebook.com/docs/graph-api/making-multiple-requests/
You describe it as
The issue is that, sometimes, the requests are issued in batches, successively one after the other for the same customer ID. When this happens, different threads process these requests and so our persistent layer (MySQL) reaches an inconsistent state due to the difference in the timestamp of the handling of these requests.
This is a set of concurrent requests. Play framework usually works as a stateless server. I assume you also organize it as stateless. There is nothing that binds one request to another, you can't control order. Well, you can, if you create a special protocol, like "opening batch request", request #1, #2, ... "closing batch request". You need to check if exactly all request was correct. You also need to run some stateful threads and some queues ... Thought akka can help with this but I am pretty sure you wan't do it.
This issue is not a "play-framework" depended. You will reproduce it in any server. For example, the general case: Is it possible to receive out-of-order responses with HTTP?
You can go in either way:
1. "Batch" the command in one request
You need to change the client so it jams "batch" requests into one. You also need to change server so it processes all the commands from the batch one after another.
Example of the requests: https://developers.facebook.com/docs/graph-api/making-multiple-requests/
2. "Pipeline" requests
You need to change the client so it sends the next request after receive the response from the previous.
Example: Is it possible to receive out-of-order responses with HTTP?
The solution to this is to pipeline Ajax requests, transmitting them serially. ... . The next request sent only after the previous one has returned successfully."
I have a namespace in ASB with a single topic and subscription. I can see the message count from when I was testing the consumer (ruby client), the first spike, but after I stopped testing and nothing in the client was running there were 10 incoming requests from 'somewhere'. The second spike in the graph. The machine was off at this point so it must have come from Azure.
Within half an hour of turning on my machine there were 6 incoming requests, the third spike but nothing is running as it's a commandline client so I assume it's Azure again.
I don't have anything else in Azure (functions apps etc). Literally nothing is running/polling/pulling/peeking etc.
Is it possible to identify where these requests are coming from? Graph is below.
Incoming requests are not incoming messages. When there are no messaging and you receive nothing, there are still operations taking place. This is due to the fact that the client (assuming you're using client with a message handler rather than trying to retrieve messages yourself), the client is performing long-polling. I.e. the client will poll for new messages rather than the broker to push those. Unlike RabbitMQ, which will push to a client when messages become available.
Incoming Requests is a 'Request Metric' that shows number of requests made to the "Service Bus service" over a specified period. Even without any message flow within the ServiceBus Namespace there can be incoming requests.
Probably it is not possible to find the origin of the incoming request.
I'm trying to make sure I understand the meaning of the poolExhaustedAction values for a threading profile. I'm not seeing a lot of examples out there.
Assume I have a thread pool on an HTTP endpoint that has maxThreadsActive set to "16". I receive 20 inbound requests in a short period (faster than I can process any of them).
If poolExhaustedAction is set to "WAIT" then the last 4 requests will wait for threadWaitTimeout. Is this correct?
If poolExhaustedAction is set to "RUN" then the last 4 requests will ????...use the thread that carried the request to the endpoint to run the flow???? I'm a bit confused on this one. Specifically, if set to "RUN", will the service ever reject a request (assuming Mule has threads to deliver messages to it)?
Have you read http://www.mulesoft.org/documentation/display/current/Tuning+Performance? Especially this part?
Answers to your questions are:
Yes.
Indeed the thread that received the request will be used to process it in the flow. The service will start rejecting requests when inbound socket connections will time-out because the thread in charge of routing them in Mule is too busy to accept them.