I have an Azure python HTTP trigger function that needs to execute dynamic code. If 100 users executing dynamic code simultaneously, even if one user has bad code (infinite loop), other valid requests were failing. Is there a way in Azure to invoke HTTP function as it's own instance so other API requests were not impacted or programmatically terminate invalid request?
I tried functionTimeout in host.json but this is terminating invalid and other valid requests too that were processing simultaneously.
Thanks
This behavior could be due to the single threaded architecture of Python. This is an expected behavior.
It is documented in Python Functions Developer reference on how to handle such scenario’s: https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python#scaling-and-concurrency
Here are the two methods to handle this:
Use Async calls
Add more Language worker processes per host, this can be done by using application setting : FUNCTIONS_WORKER_PROCESS_COUNT up to a maximum value of 10. ( So basically, for the CPU-bound workload you are simulating with any loops, we do recommend setting FUNCTIONS_WORKER_PROCESS_COUNT to a higher number to parallelize the work given to a single instance.
[Please note that each new language worker is spawned every 10 seconds until they are warmed up.]
Here is a GitHub issue which talks about this issue in detail : https://github.com/Azure/azure-functions-python-worker/issues/236
Related
Using Ktor and Kotlin 1.5 to implement a REST service backed by Netty. A couple of things about this service:
"Work" takes non-trivial amount of time to complete.
A unique client endpoint sends multiple requests in parallel to this service.
There are only a handful of unique client endpoints.
The service is not scaling as expected. We ran a load test with parallel requests coming from a single client and we noticed that we only have two threads on the server actually processing the requests. It's not a resource starvation problem - there is plenty of network, memory, CPU, etc. and it doesn't matter how many requests we fire up in parallel - it's always two threads keeping busy, while the others are sitting idle.
Is there a parameter we can configure to increase the number of threads available to process requests for specific endpoints?
Netty use what is called Non-blocking IO model (http://tutorials.jenkov.com/java-concurrency/single-threaded-concurrency.html).
In this case you have only a single thread and it can handle a lot of sub-processes in parallel, as long as you follow best practices (not blocking the main thread event loop).
You might need to check the following configuration options for Netty https://ktor.io/docs/engines.html#configure-engine
connectionGroupSize = x
workerGroupSize = y
callGroupSize = z
Default values usually are set rather low and tweaking them could be useful for the time-consuming 'work'. The exact values might vary depending on the available resources.
I'm using the Azure Fluent API, Azure Management Libraries for .NET, to scale the DTU's within an Azure Elastic Pool and would like to know if it's possible to trigger an update without having to wait for the processing to complete.
Currently the following block of code will wait until the Elastic Pool has finished scaling before it continues execution. With a large premium Elastic Pool this could mean that the this line will take up to 90 minutes to complete.
ElasticPool
.Update()
.WithDtu(1000)
.Apply();
There's also a ApplyAsync() method which i could deliberately not await to allow the program to continue execution, if i take this approach the program will end execution shortly after calling this line and i am unsure if this library has been designed to work in this fashion.
Does anyone know of a better solution to trigger an update without having to wait on a response? Or if it is safe to fire the async method without waiting for a response?
There is currently no way to make a fire and forget calls in the Fluent SDK for update scenarios but we are looking to the ways of enabling a manual status polling in the future. One option would be to create a thread that will wait on the completion. The other one is to use the Inner getter and make a low level BeginCreateOrUpdateAsync/BeginUpdateAsync method calls and then do manual polls.
On the side note if you need to make multiple calls and then wait for completion of all of them you can use Task.WaitAll(...) and provide the list of the ApplyAsync tasks.
Please log an issue in the repo if you will hit any errors because that way you will be able to track the progress of the fix.
edit: FYI the call is blocking not because SDK is waiting for the response from Azure but that SDK waits until the call is completed, operation of update is finished and the resource is ready to be used for further operations. Just firing an update and then trying to use resource will cause error responses if in your case Elastic Pool is still in the middle of the update.
I'm working on an AWS Lambda function that currently makes hundreds of API calls but when going into production it will make hundreds of thousands. The problem is that I can't test at that scale.
I'm using the async module to execute my api calls with async.eachLimit so that I can limit the concurrency (I currently set it a 300).
The thing that I don't understand is the limits on AWS Lambda. Here's what the docs say:
AWS Lambda Resource Limits per Invocation
Number of file descriptors: 1,024
Number of processes and threads (combined total): 1,024
As I understand it, Node.js is single threaded so I don't think I would exceed that limit. I'm not using child processes and the async library doesn't either so OK on that front too.
Now about those file descriptors, my function strictly calls the rest of AWS's API and I'm never writing to disk so I don't think I'm using them.
The other important AWS Lambda limits are execution time and memory consumed. Those are very clearly reported on each execution and I am perfectly aware when I'm close to reaching them or not, so let's ignore these for now.
A little bit of context:
The exact nature of my function is that every time a sports match starts I need to subscribe all mobile devices to the appropriate SNS topics, so basically I'm calling our own MySQL database and then the AWS SNS endpoint repeatedly.
So the question is...
How far can I push async's concurrency in AWS Lambda in this context? Are there any practical limits or something else that might come into play that I'm not considering?
As I understand it, Node.js is single threaded so I don't think I
would exceed that limit. I'm not using child processes and the async
library doesn't either so OK on that front too.
Node.js is event driven, not single threaded.
The Javascript engine runs on a single thread (the event loop) and delegates I/O operation to an internal library (libuv) which handles its thread pool and asynchronous operations.
async doesn't open a child process on its own, but behind the scenes, whether you're making an HTTP request or interacting with the file system, you're delegating these operations to libuv.
In other words, you've answered your own question well with the resources limits:
How far can I push async's concurrency in AWS Lambda in this context? Are there any practical limits or something else that might come into play that I'm not considering?
AWS Lambda Resource Limits per Invocation
Number of file descriptors: 1,024
Number of processes and threads (combined total): 1,024
It's hard to say whether libuv would open a new thread for each I/O operation, so you might get away with a little more than the numbers listed above. But you will probably run out or memory way before reaching those limits anyway.
The bottom line is no, you won't be able to make hundreds of thousands of calls in a single lambda execution.
Regarding the context of your function, depending on how often your job needs to run, you might want to refactor your lambda to multiple executions (it would also run faster), or have it on an EC2 with auto scaling triggered by lambda.
I have a site that makes the standard data-bound calls, but then also have a few CPU-intensive tasks which are ran a few times per day, mainly by the admin.
These tasks include grabbing data from the db, running a few time-consuming different algorithms, then reuploading the data. What would be the best method for making these calls and having them run without blocking the event loop?
I definitely want to keep the calculations on the server so web workers wouldn't work here. Would a child process be enough here? Or should I have a separate thread running in the background handling all /api/admin calls?
The basic answer to this scenario in Node.js land is to use the core cluster module - https://nodejs.org/docs/latest/api/cluster.html
It is an acceptable API to :
easily launch worker node.js instances on the same machine (each instance will have its own event loop)
keep a live communication channel for short messages between instances
this way, any work done in the child instance will not block your master event loop.
I haven't used threading in my program before. But there is a problem I am having with this 3rd party application.
It is an offsite backup solution and it has a server and many clients. We have an admin console to manage all the clients and that is where there is a problem.
If one of the client side application gets stuck, or is running in a broken condition, the admin console waits forever to get a response and does not display anything.
$for(client= client1; client < last_client; client++){
if (getOServConnection(client, &socHandler)!=NULL) { .. }
}
I want two solutions to this. I want to know if there is anyway, I can set a timeout for the function getOServConnection, so that I get a response within X seconds.
And, I want to know how to call this function in parallel for all clients, so that I get the response from all clients within X seconds.
the getOServConnection contains a WSAConnect call, and I don't want to use any options on the socket, since it is used by other modules and it will affect the application severely.
First.. If you move the call that hangs into a separate thread you can use the main thread for starting a timer an waiting for the timeout. If you are using Visual C++ and if you are in Win32 you can use the (rather old) MFC based timer. Once this timer expires it will launch a function call OnTimer. This timer does not affect your application's main thread as it works in a different system based thread.
Second.. If you need to start any number of threads with that connection you should start thinking of a design pattern to use for that. You could use a fixed number of threads, and in that case you may want to use a object pool. Or if the number of threads is (relatively) limitless you may want to use a factory method