i am trying to do an application,like if there are 10 separate tasks and 4 threads are running.My application has to maintain two queue one for tasks and another for threads.If any task needs to execute it should find which thread is free and assign the task to that thread.i dont know how to produce this.Anyone knows what are the concepts i have to look,please help me.
It sounds like you just need a thread pool of the kind returned by Executors.newFixedThreadPool. Just submit the tasks to the thread pool when you need to, and it will be executed accordingly.
The ThreadPoolExecutor does pretty well exactly that for you.
I suppose you could make your tasks Runnable and use a ThreadPoolExecutor to run them.
Yes Executor would be best approach for this problem. BUt u need to keep various things in mind while using the executor and also INs and OUTs of executor as Executor is a service which if used properly can be a blessing and if not it can be a big mistake.
Related
I have a process Scheduled using Timer and TimerTask that runs nightly. Currently ti takes about an hour to finish. Considering there are only 6000 records to loop through the process and the upper management feels like it is very inefficient Job. So I wanted to know if I could span multiple threads of the same job with different datasets. Probaby each thread processes only 500 records at a time.
If i am hitting the same table for read/insert and update using
multiple threads would that be ok to do it?
if so how do i run multiple threads within a timer task? I suppose I could
just create threads and run but how do i ensure they run simultaneously but not sequentially?
I am using java 1.4 and this runs on a jboss 2.4 and i make use EJB 1.1 session beans in the process to read/update/add data.
There isn't enough info in your post for a surefire answer, but I'll share some thoughts:
It depends. Generally you can do reads in parallel, but not writes. If you're doing much more reading than writing, you're probably ok, but you may find yourself dealing with frustrating race conditions.
It depends. You are never guaranteed to have threads run in parallel. That's up to the cpu/kernel/jvm to decide. You just make threads to tell the machine that it's allowed to execute them in parallel.
I am using the tpl to process thousands of files in a multithreaded fashion.All good.
However there is some part of the application that I must process those files single thread.
Setting maxdegreeParallelism=1 means 1 thread x core is this correct?
When you dont you parallelism and you have 4 cores does it still use 1 thread x core?
The problem is that tpl does lot of hard work for you and also not been very familiar with threading does not help.
Bottom line I need to make sure that maxdegreeParallelism=1 is single threaded
Sorry for silly question but could not find a straight answer by googling.
See here.
No. It is not the case that necessarily one thread is run per core when you set the `MaxDegreePrallelism". It has a different meaning. It limits the number of parallel tasks done in the entire parallel operation. If you set that to one, it basically renders your parallel approach useless.
The TPL schedules a task on the threadpool. Once the task is scheduled, the threadpool decides how all the tasks to be done are to be distributed among threads, cores and processors. This is based on certain heuristics like the virtual address space, number of threads currently in blocked state, etc.
Now, if you mean that there is a part of your application in which the tasks should be done in a sequential form, there are ways to achieve that. Take a look at ContinueWith.
Documentation doesnt say anything about CPU-Cores but concurrent operations.
So this means, setting to 1 equals 1 Thread in total. Though it is a different thread than the callingthread.
Fiddle to poorly prove assumption.
Problem: Read file of size > 10 MB and load it in staging table using Spring Batch. How can we maintain state while reading a file, in order to restart the job if it fails?
As per the documentation the FileItemReader is not thread safe and if we try to make it thread safe, we end up loosing restartability. So basic questions are:
Is there a way to read the file in blocks and each thread knows which block it needs to read?
If we make the read synchronous, what changes are required to make the job restartable in this scenario?
If anyone has faced similar issues or have any analysis of how it performs would help us take decision.
Also, any pointers or sample codes are appreciated.
Multithreading is only useful if your threads are doing different things at the same time. For example, you can have two threads running on separate CPUs. Or one thread can be waiting for a network message while the other is painting the screen.
But in your case, both threads would be waiting for the same IO from the same device, so there's no point using more than one.
See also this question Reading a file by multiple threads
Could someone clear up to me how these things correlate:
Task
Thread
ThreadPool's thread
Paraller.For/ForEach/Invoke
I.e. when I create a Task and run it, where does it get a thread to execute on? And when I call Parallel.* what is really going on under the covers?
Any links to articles, blogposts, etc are also very welcomed!
The ideal state of a system is to have 1 actively running thread per CPU core. By defining work in more general terms of "tasks", the TPL can dynamically decide how many threads to use and which tasks to do on each one in order to come closest to achieving that ideal state. These are decisions that are almost always best made dynamically at runtime because when writing the code you can't know for sure how many CPU cores will be available to your application, how busy they are with other work, etc.
Thread: is a real OS thread, has handle and ID.
ThreadPool: is a collection of already-created OS Threads. These threads are owned/maintained by the runtime, and your code is only allowed to "borrow" them for a while, you can only do work short-termed work in these threads, and you can't modify any thread state, nor delete these threads.
Best guesses on these two:
Task: might get run on a pre-created thread in the thread pool, or might get run as part of user-mode scheduling, this is all depending on what the runtime thinks is best. Another guess: with TPL, the user-mode scheduling is NOT based on OS Fibers, but is its own complete (and working) implementation).
Parallel.For: actually, no clue how this is implemented. The runtime might create new threads to do the parallel bits, or much more likely use the thread pool's threads for the parallelism.
I have seen several comments to the effect that Executors are better than Threads, but if you have a number of Threads communicating via bounded buffers (as in Flow-Based Programming) why would you use Executors when you have to use Threads anyway (with newCachedThreadPool (?)). Also, I use methods like isAlive(), interrupt() - how do I get hold of the Thread handle?
Does anyone have sample code that I can plagiarize? ;-)
Executors are basically an abstraction over Threads. They make you isolate your potentially parallel logic in Runnable/Callable instances while liberating you from the duties of manually creating and starting a thread or managing a pool. You still need to handle dependencies as part of your application logic.
If you want to interact / are comfortable with Threads for your application logic, you may skip using Executors. Regarding getting hold of the thread, you can always execute Thread.currentThread() to get hold of the current thread from any executing context.