Does the Iron framework support multithreaded mode? How do I enable & configure it?
It is enabled by default.
The primary structure in Iron is Iron, you can start the server using either the http or listen_with methods.
For http:
Defaults to a threadpool of size 8 * num_cpus.
For listen_with: the signature takes a threads usize argument:
Kick off the server process with X threads.
Where I suppose X stands for the number of threads passed as argument.
Related
Using Ktor and Kotlin 1.5 to implement a REST service backed by Netty. A couple of things about this service:
"Work" takes non-trivial amount of time to complete.
A unique client endpoint sends multiple requests in parallel to this service.
There are only a handful of unique client endpoints.
The service is not scaling as expected. We ran a load test with parallel requests coming from a single client and we noticed that we only have two threads on the server actually processing the requests. It's not a resource starvation problem - there is plenty of network, memory, CPU, etc. and it doesn't matter how many requests we fire up in parallel - it's always two threads keeping busy, while the others are sitting idle.
Is there a parameter we can configure to increase the number of threads available to process requests for specific endpoints?
Netty use what is called Non-blocking IO model (http://tutorials.jenkov.com/java-concurrency/single-threaded-concurrency.html).
In this case you have only a single thread and it can handle a lot of sub-processes in parallel, as long as you follow best practices (not blocking the main thread event loop).
You might need to check the following configuration options for Netty https://ktor.io/docs/engines.html#configure-engine
connectionGroupSize = x
workerGroupSize = y
callGroupSize = z
Default values usually are set rather low and tweaking them could be useful for the time-consuming 'work'. The exact values might vary depending on the available resources.
I have an Azure python HTTP trigger function that needs to execute dynamic code. If 100 users executing dynamic code simultaneously, even if one user has bad code (infinite loop), other valid requests were failing. Is there a way in Azure to invoke HTTP function as it's own instance so other API requests were not impacted or programmatically terminate invalid request?
I tried functionTimeout in host.json but this is terminating invalid and other valid requests too that were processing simultaneously.
Thanks
This behavior could be due to the single threaded architecture of Python. This is an expected behavior.
It is documented in Python Functions Developer reference on how to handle such scenario’s: https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python#scaling-and-concurrency
Here are the two methods to handle this:
Use Async calls
Add more Language worker processes per host, this can be done by using application setting : FUNCTIONS_WORKER_PROCESS_COUNT up to a maximum value of 10. ( So basically, for the CPU-bound workload you are simulating with any loops, we do recommend setting FUNCTIONS_WORKER_PROCESS_COUNT to a higher number to parallelize the work given to a single instance.
[Please note that each new language worker is spawned every 10 seconds until they are warmed up.]
Here is a GitHub issue which talks about this issue in detail : https://github.com/Azure/azure-functions-python-worker/issues/236
Let us say, a gRPC client makes two requests R1 and R2 to gRPC server, one after the other (assume without any significant time gap, i.e R2 is made when R1 is still not served). Also, assume that R1 takes much more time than R2.
In this case, should I expect R2's response first as it takes less time or should I expect R1's response first as this request is made prior to R2? What will happen and why?
As far as what I have observed, I think requests are served in FCFS fashion, so, R1's response will be received by the client first and then R2's, but I am not sure.
Theoretically nothing discourages server and client process gRPC requests in parallel. GRPC connection is made over HTTP/2 one that can handle multiple requests at once. So yes - if server doesn't use some specific synchronization or limitation mechanisms then requests would be processes with overlapping. If server resources or policy doesn't allow it then they should be processed one by one. Also I can add than request can have a Timeout after which it would be cancelled. So long wait can lead to cancellation and non-processing at all.
All requests should be processed in parallel. The gRPC architecture for the Java implementation for example, it is divided into 2 "parts":
The event loop runs in a thread work group - It is similar to what we have to reactive implementations. One thread per core to handle the incoming requests.
The request processing is done in a dedicated thread which will be created using the CachedThreadPool system by default.
For single-thread languages like Javascript, I am not sure how they are doing it, but I would guess it is done in the same thread and therefore it would end up queuing the requests.
I am currently using the CassFuture callback future to implement an asynchronous pattern for processing Cassandra queries. It appears that all of the callbacks are coming in on the same thread. Is this the expected behavoir?
Assuming you are using the current version of the driver (v2.9.0) the number of threads is configured via cass_cluster_set_num_threads_io() before creating a session and defaults to 1. If you increase the number of IO threads in the cluster configuration you will notice that CassFuture callbacks will begin to show up on different threads.
NOTE: If your callback is slow, consider running on a separate thread; otherwise the callback might block IO operations for other requests being executed on the callback/IO thread. Another recommendation is not use the complete number of cores/virtual cores available in your hardware configuration as this could starve resources from your client application (and potentially from the OS services).
I haven't used threading in my program before. But there is a problem I am having with this 3rd party application.
It is an offsite backup solution and it has a server and many clients. We have an admin console to manage all the clients and that is where there is a problem.
If one of the client side application gets stuck, or is running in a broken condition, the admin console waits forever to get a response and does not display anything.
$for(client= client1; client < last_client; client++){
if (getOServConnection(client, &socHandler)!=NULL) { .. }
}
I want two solutions to this. I want to know if there is anyway, I can set a timeout for the function getOServConnection, so that I get a response within X seconds.
And, I want to know how to call this function in parallel for all clients, so that I get the response from all clients within X seconds.
the getOServConnection contains a WSAConnect call, and I don't want to use any options on the socket, since it is used by other modules and it will affect the application severely.
First.. If you move the call that hangs into a separate thread you can use the main thread for starting a timer an waiting for the timeout. If you are using Visual C++ and if you are in Win32 you can use the (rather old) MFC based timer. Once this timer expires it will launch a function call OnTimer. This timer does not affect your application's main thread as it works in a different system based thread.
Second.. If you need to start any number of threads with that connection you should start thinking of a design pattern to use for that. You could use a fixed number of threads, and in that case you may want to use a object pool. Or if the number of threads is (relatively) limitless you may want to use a factory method