I am creating a web service that creates a huge amount of small java timer threads over (10k). I can only seem to create 2k timer threads before I get the OutOfMemoryError: unable to create new native thread. How do i solve this? I am using a macbook pro to run my Tomcat server on. I'v configured the ulimit (-u) max user processes to double what it used to be but I still get the same problem. What are my options, if any, to make this doable?
It's often a bad idea for web applications to start their own (few) threads, let alone 10K threads - and then "as timers"? Seriously? Don't go there.
What can you do?
Don't rely on the ability to create those threads.
Change your architecture! Use a scheduler library that has solved this problem already (e.g. Quartz or others).
If you don't want to use an external library (why wouldn't you?): Implement a single timer thread that executes the scheduled operations when they're due. Do not use a new thread for each scheduled operation
If you wanted to boil 100 eggs, would you buy 100 timers?
Related
I have a Spring Boot application in which everytime API call is made, I am creating an ExecutorService with fixedThreadPool size of 5 threads and passing around 500 tasks to CompletableFuture to run Async. I am using this for a migration of lakhs of data.
As I started the migration, initially API was working fine and each API Call ( Basically code logic + ThreadPool Creation + Jobs Assignment to threads ) was taking around just 200 ms or so. But as API calls increased and new threadpools kept on creating, I can see gradual increase in time being taken to Create the thread Pool and assign the jobs, as a result API response time went till 4 secs.
Note : After the jobs are done, i am shutting down the executor service in finally block.
Question :
Can multiple creation create overhead to the application and do those pools keep on piling up?
Wont there be any automatic garbage collection to this ?
Will there be any limit to how many pools get created ?
And what could be causing this time delay ..
I can add further clarifications based on specific queries..
Can multiple creation create overhead to the application and do those pools keep on piling up?
Yes absolutely. Unless you shutdown the thread pools, they won't be destroyed automatically and consume resources. See next question for more details.
Wont there be any automatic garbage collection to this ?
You need to take care that the thread pools are destructed after they are no longer needed. For example, the javadoc of ThreadPoolExecutor provides some hints:
A pool that is no longer referenced in a program AND has no remaining threads will be shutdown automatically. If you would like to ensure that unreferenced pools are reclaimed even if users forget to call shutdown(), then you must arrange that unused threads eventually die, by setting appropriate keep-alive times, using a lower bound of zero core threads and/or setting allowCoreThreadTimeOut(boolean).
Will there be any limit to how many pools get created ?
There is no hard limit on how many threads are supported by Java, however there may be restrictions depending on your operating system and available resources such as memory. This is quite a complex question, more details can be found in the answers to this question: How many threads can a Java VM support?
And what could be causing this time delay?
I assume that you don't have a proper cleanup / shutdown mechanism in place for the thread pools. Every thread allocates at least 1 MB of memory for the thread stack. For example, the more threads you create, the more memory your application consumes. Depending on the system / jvm configuration, the application may utilize swap which dramatically slows down the performance.
There may be other things that cause a drop in performance, so this is just what came to my mind right now.
Profilers will help you to identify performance issues or resource leaks. This article by Baeldung shows a few profilers you could use.
I have a web application that simply acts as a Front Controller using Spring Boot to call other remote REST services where I am combining Spring's DeferredResult with Observables subscribed on Scheduler.computation().
We are also using JMeter to stress out the web application, and we have noticed that requests start to fail with a 500 status, no response data and no logs anywhere when the number of concurrent threads scheduled in JMeter increases from 25, which obviously is a very "manageable" number for Tomcat.
Digging into the issue with the use of VisualVM to analyze how the threads were being created and used, we realized that the use of rx.Schedulers was somehow impacting the number of threads created by Tomcat NIO. Let me summarize our tests based on the rx.Scheduler used and a test in JMeter with 100 users (threads):
SCHEDULERS.COMPUTATION()
As we're using the Schedulers.computation() and my local machine has 4 available processors, then 4 EventLoop thread pools are created by RxJava (named RxComputationThreadPool-XXX) and ONLY 10 of Tomcat (named http-nio-8080-exec-XXX), as per VisualVM:
http://screencast.com/t/7C9La6K4Kt6
SCHEDULERS.IO() / SCHEDULERS.NEWTHREAD()
This scheduler seems to basically act as the Scheduler.newThread(), so a new thread is always created when required. Again, we can see lots of threads created by RxJava (named RxNewThreadScheduler-XXX), but ONLY 10 for Tomcat (named http-nio-8080-exec-XXX), as per VisualVM:
http://screencast.com/t/K7VWhkxci09o
SCHEDULERS.IMMEDIATE() / NO SCHEDULER
If we disable the creation of new threads in RxJava, either by setting the Schedulers.immediate() or removing it from the Observable, then we see the expected behaviour from Tomcat's threads, i.e. 100 http-nio-8080-exec corresponding to the number of users defined for the JMeter test:
http://screencast.com/t/n9TLVZGJ
Therefore, based on our testing, it's clear to us that the combination of RxJava with Schedulers and Tomcat 8 is somehow constraining the number of threads created by Tomcat... And we have no idea why or how this is happening.
Any help would be much appreciated as this is blocking our development so far.
Thanks in advance.
I am in the process of moving a newsletter service from a Windows server running Microsoft.NET 4.5 to a Linux server running Mono 3.0.3. The service uses Amazon's "Simple Email Service" (SES) to deliver the emails, via the official .NET SDK (wrapping a REST interface).
While sending emails via SES sequentially from Mono turns out to be slightly faster than Microsoft.NET using similar hardware, I am running into serious performance trouble when attempting to deliver multiple mails in parallel. Below is a chart showing the time required to send 128 emails on both platforms using a varying number of threads. As you can see, performance on Mono degrades rapidly after 8 threads, and with 128 threads I get only HTTP timeouts – not a single email is delivered.
Profiling via console output, it turns out that the first "batch" of mails is the source of the slowdown. With two threads, sending one email in each, both threads finish in around 2200 ms. With four threads, sending one email in each, they all finish in around 4400 ms. Eight threads, around 8800 ms, etc. It seems as if the web service, while spawned simultaneously, are run sequentially and are required to wait for one another before returning.
Any ideas what might be triggering this behavior? The source code for the Amazon SDK is available on GitHub, but I have not been able to pinpoint anything suspiciously. Maybe the use of the async methods on HttpWebRequest?
Yes, stop using async HttpWebRequest* for now because there is a bug being discussed in the Mono list. A patch has been provided, but apparently is not good enough and has been reverted from master.
If you're good with low level code, it would be nice that you contribute a patch.
* The fastest way to stop using the async infrastructure is calling mono witht eh environment variable MONO_DISABLE_AIO=1. By the way, if you're using more than one thread anyway, maybe a Parallel.For would be enough but keeping the code non-asynchronous? The best use-case of async is actually to avoid threading and still manage to achieve parallelization (or rather, avoid blocking waits).
I've been reading bunch of articles regarding new TPL in .NET 4. Most of them recommend using Tasks as a replacement for Thread.QueueUserWorkItem. But from what I understand, tasks are not threads. So what happens in the following scenario where I want to use Producer/Consumer queue using new BlockingCollection class in .NET 4:
Queue is initialized with a parameter (say 100) to indicate number of worker tasks. Task.Factory.StartNew() is called to create a bunch of tasks.
Then new work item is added to the queue, the consumer takes this task and executes it.
Now based on the above, there is seems to be a limit of how many tasks you can execute at the same time, while using Thread.QueueUserWorkItem, CLR will use thread pool with default pool size.
Basically what I'm trying to do is figure out is using Tasks with BlockingCollection is appropriate in a scenario where I want to create a Windows service that polls a database for jobs that are ready to be run. If job is ready to be executed, the timer in Windows service (my only producer) will add a new work item to the queue where the work will then be picked up and executed by a worker task.
Does it make sense to use Producer/Consumer queue in this case? And what about number of worker tasks?
I am not sure about whether using the Producer/Consumer queue is the best pattern to use but with respect to the threads issue.
As I believe it. The .NET4 Tasks still run as thread however you do not have to worry about the scheduling of these threads as the .NET4 provides a nice interface to it.
The main advantages of using tasks are:
That you can queue as many of these up as you want with out having the overhead of 1M of memory for each queued workitem that you pass to Thread.QueueUserWorkItem.
It will also manages which threads and processors your tasks will run on to improve data flow and caching.
You can build in a hierarchy of dependancies for your tasks.
It will automatically use as many of the cores avaliable on your machine as possible.
How to use multithreading in c# for sending the SMS to multiple person at a times? Use must of multithread. means must execute sms sending code/process independently at a time. (synchronisely) how can i do this ? please guide.
Start reading the documentation - or a book like "c# in 21 days".
System.Threading is your namespace for threads. Opening a thread is trivial, but I would not go that way.
Look into ThreadPool and queue a WorkItem for every SMS. The ThreadPool will automatically start threads. This is more memory efficient than using static threads, especially if you use that in multiple places of your application (as threads may get shared).
There are ample of samples for using WorkItems.
http://msdn.microsoft.com/en-us/library/system.threading.threadpool.queueuserworkitem%28VS.71%29.aspx
is a decent start documentation wise.
Well, In my project in WCF service I use for instance ThreadPool class for sending emails. In that case emails will be quequed and this will ensure that service will not "hang". Creating lots of different threads may lead to clogging of the system