Im using VisualVM to profile an application that uses akka-streams-kafka.
It shows a lot of Kafka coordinator blocking threads
Who are these coordinators?
I also have three Kafka consumers that are also blocking threads
Do I need to create a separate Execution context for them?
From Java Kafka client API
One Consumer Per Thread A simple option is to give each thread its own consumer instance. Here are the pros and cons of this approach:
PRO: It is the easiest to implement PRO: It is often the fastest as no
inter-thread co-ordination is needed PRO: It makes in-order processing
on a per-partition basis very easy to implement (each thread just
processes messages in the order it receives them). CON: More consumers
means more TCP connections to the cluster (one per thread). In general
Kafka handles connections very efficiently so this is generally a
small cost. CON: Multiple consumers means more requests being sent to
the server and slightly less batching of data which can cause some
drop in I/O throughput. CON: The number of total threads across all
processes will be limited by the total number of partitions.
Related
I've been "scaling out" a single-threaded socket.io app by using the cluster module long before node.js added worker threads.
But as having multiple processes involves (in my case) heavy usage of IPC, copying the same high-bandwidth data to all workers results in significant overhead. So instead -
is it possible for socket.io to make use of worker threads? The goal is to use more than just one core.
I can think of two different ways that this may be possible:
having multiple, completely separate socket.io instances, each running on its own worker thread (e.g. each servicing a different tcp port - that's how my current cluster solution works)
having a single socket.io instance use multiple threads internally (in which case it would perhaps have to be doing something like epoll - so I kind of doubt socket.io has that built-in)
N.B. As worker threads are supposed to be used for performing CPU-intensive JavaScript operations, it remains to be seen how much can be gained by switching from multiple processes to multiple threads. But in general, I would not expect in-process to be slower than inter-process communication. Also, in the case of worker threads, data can be shared (rather than copied).
Which one of these are better to implement in a server? I was wondering about the tradeoffs between spawning a thread per request versus using a worker pool consisting of a fixed number of threads, and how the performance will vary as the number of users increases. These are some questions that came up to my mind what I started to think about the way I want to implement my server.
I'm just wondering if there would be any reason I might want to lock a queue. I am working on an application that has several threads that reads and writes to a database. In order to reduce traffic, I want to reduce the amount of calls to that database at any given point (I know many databases can handle some traffic already). Would it make any sense to make a queue for the read/write requests and only the request at the top executes and then protect the queue's push and pop commands with a lock? Is having a lock on each read/write call enough? Isn't a lock implemented as a "queue" by the OS anyways? Could size of this "queue" be an issue or would there be any other reason I wouldn't use a lock by itself?
Thanks!
You could limit the number of threads that are engaged in database requests or if that's not feasible due to the nature of your app, you could use a more granular approach to limit access to the shared resource. In python, you can use the built-in semaphore objects for inter-thread synchronization. For inter-process synchronization (or inter-thread), you'd use posix_ipc. It depends what your service's execution model is.
Most database clients wouldn't require any application-level throttling. In a typical system, the database connections would be pooled and the connection manager would be responsible for acquiring an available connection. Internally this usually involves a queue of some sort with timeouts to prevent waiting indefinitely. The database itself would then handle the scheduling of individual operations made by each connection.
However, a semaphore is a signalling primitive that can be used to limit the number of concurrent operations: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Semaphore.html
Tasks can also be modeled as a producer-consumer problem which involves a shared queue, however you'll have to deal with the added complexity of managing the consumer threads in addition to the producers.
I see from the server side, the benefit of NIO is the capability to manage multiple network connections with fewer thread comparing to the comparing to one thread per connection blocking IO.
However, if I have a IO client which connects to thousand of servers at the same time, can I just have similar approach to manage these connections IO using fewer threads. I tried the approach in Netty 4 multiple client and found it spawn a "Reader" thread for each channel it created.
So, my questions are:
1) what are the benefits using netty/NIO in the client side?
2) is it possible to manage multiple connections with fewer threads in the client side?
Thanks!
I have uploaded the code samples in github: https://github.com/hippoz/ogop-lseb
The sample server/client class is moc.ogop.ahsp.demo.nio.MultipleConnectionNioMain and moc.ogop.ahsp.demo.nio.NettyNioServerMain
Having lots of threads creates a context-switch problem in the kernel where lots more memory is being loaded and unloaded from each core as the kernel tries to reschedule the threads across the cores.
The benefit of NIO anywhere is performance. Thats pretty much the only reason we use it. Using Blocking IO is MUCH more simple. Using the worker model and NIO you can limit the number of threads (and potential computational time) the process uses. So if you have two workers and they go bonkers using 100% cpu time the whole system won't go to a crawl because you have 2-4 more cores available.
Have fun!
https://en.wikipedia.org/wiki/Context_switch
Why should I use non-blocking or blocking sockets?
I know that Vibe.D implementation is based on Fibers.
But I don't know how high load scenarios are handled by Vibe.D. Is it the scheduler in Vibe.D allocating fibers on multiple threads or just one thread for all fibers?
This consideration is very important because even with the high efficiency of Fibers a lot of CPU time is wasted is no more than one thread is used to attend all incoming requests.
Their front page says yes:
http://vibed.org/
this page has the details
http://vibed.org/features#multi-threading
Distributed processing of incoming connections
The HTTP server (as well as any other TCP based server) can be instructed to process incoming connections across the worker threads of the thread pool instead of in the main thread. For applications that don't need to share state across different connections in the process, this can increase the maximum number of requests per second linearly with the number of cores in the system. This feature is enabled using the HTTPServerOption.distribute or TCPListenOptions.distribute settings.