I am using an bpmn process which is already running using thread and also using spring ftp where the Task scheduler thread is running but I found the application is cannot switch from the threads. Is there any way to invoke the task-scheduler process without any interrupt and I am using InboundchannelAdapter to copy files from FTP. Please suggest any feasible way to resolve the issue.
I don't see any issues in your question. And to be honest it fully isn't clear.
Please, be more specific and sharing some code/config/logs/stack-trace sometime is really useful. More info, more chance to get quick and proper answer.
I guess your problem that you download files from FTP and in the same thread run a BPM process which might block eventually waiting for some actor action.
Fro this purpose you should shift Spring Integration flow on the <poller> to different thread and don't steal task-scheduler resources. They are really so expensive for the whole system. Consider to use enough big ThreadPoolTaskExecutor for the task-executor reference on the <poller>. Also there is an ExecutorChannel for you with similar thread shifting capabilities.
Related
I have a scenario where i am spawning multiple threads (each of them working on a browser instance). But all threads share 1 common operation which i need to synchronize. I was thinking of creating locks using file created on disk, but am afraid will that really work ?
Basically i need some locking mechanism on windows platform. I am spawning multiple threads, each thread is an automated script (javascript/batch) working over browser, but i need to synchronise any operation which triggers opening of save as/upload dialog in browser.
Regards
Depends strongly on the language, platform, OS, etc you want to develop on.
For POSIX you could achieve that e.g. with mutexes:
http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_lock.html
If you develop with C++17:
http://en.cppreference.com/w/cpp/thread/mutex
Regarding your question: you could also do that with file locks. In that case you would have to wait for the file to be locked again, which can be achieved with select()
In Grails I have a very time consuming task that simply reads about a 1000 csv files each containing one word per line. Then it reads each file and moves the word per line to a database.
What would be the difference in memory consumed if I perform this task in controller (main thread) or in another thread spawned in controller?
I have already faced out of memory errors due to which jvm totally quit the live website and I had to restart Tomcat.
Could it be due to time consuming task in controller thread(hence more memory consumed)?
P.S. I am more interested in finding the difference in memory consumed in controller thread and another thread created in controller.
It depends on your overall requirements, load (how often you do this), architecture, available software and hardware.
First of all, this is usually a task for ETL (Extract Transform Load) systems. If you can't afford one (or your boss is strictly against it) you can use several other approaches to offload the controller.
dedicated server - install this DB load onto another box. Define communication channels. Asynchronously get notified when the load finishes and pass this info onto the user.
In process processing. This is when you just spawn another thread (or threads) to do the heavy lifting. GPars could come to the rescue here.
You could also use the Grails built-in async features as specified here
Back to your original question. I wouldn't even think about loading 100s of CSV in a controller. If I was to implement this, I'd certainly go for an ETL as it's most suitable for this kind of tasks.
I have a site which sometimes takes particularly long to process a request (and that's not a defect). 99% of the time it's pretty quick because it almost doesn't do any processing.
I want to show a message that says "Loading" when the site takes long to process the request. My site uses mod_wsgi and Apache. The way I see it, I would respond saying 'Loading' before completing the processing and do one of two things right before:
-spawn a (daemon) thread to take care of the processing.
-communicate through socket with other process and tell it to take care of the processing (most likely send request to http://localhost:8080/do_processing).
What are the pros and cons of one approach vs the other?
Using a separate process is better. It does not have to be hard at all as suggested in another answer as you can use an existing system for doing exactly that such as Celery (http://celeryproject.org/). Relying on in process threads is not necessarily a good idea unless you are going to implement an internal job queueing system of your own to prevent blowing out of number of threads. Also, in a multiprocess server configuration you cant be guaranteed a request comes back to the same process and so not easy to get status of a running operation. Finally, the web server processes could get killed off and thus your background task could also be killed before it finishes. You would need to have a mechanism for holding state which can survive such an event if that was important. Far easier to use something like Celery.
The process route requires quite a bit of a system processing. Creation of a separate process is relatively expensive and slow. However if your process crashes it doesn't affect your main governing process (you will receive the exit status code and will have an opportunity to respawn a new working process). You will also need some sort of InterProcessCommunication layer (can be a socket, pipe, shared memory, etc...) which is adds to complexity if your project.
Threads are lightweight and cheap. All you need to do is to manage concurrent access to shared resources. So it really depends on the task you have in mind. Threads probably will be more likely the appropriate way to implement your task.
How to use multithreading in c# for sending the SMS to multiple person at a times? Use must of multithread. means must execute sms sending code/process independently at a time. (synchronisely) how can i do this ? please guide.
Start reading the documentation - or a book like "c# in 21 days".
System.Threading is your namespace for threads. Opening a thread is trivial, but I would not go that way.
Look into ThreadPool and queue a WorkItem for every SMS. The ThreadPool will automatically start threads. This is more memory efficient than using static threads, especially if you use that in multiple places of your application (as threads may get shared).
There are ample of samples for using WorkItems.
http://msdn.microsoft.com/en-us/library/system.threading.threadpool.queueuserworkitem%28VS.71%29.aspx
is a decent start documentation wise.
Well, In my project in WCF service I use for instance ThreadPool class for sending emails. In that case emails will be quequed and this will ensure that service will not "hang". Creating lots of different threads may lead to clogging of the system
I am building an application where I have inputs from printers over the network (on specific ports) and other files which are created into a folder locally or through the network. The user can create different threads to monitor different folders at the same time, as well as threads to handle the input from threes printers over the network. The application is supposed to process the input data according to its type and output it. On the other end of the application, there would be 4 threads waiting for input data from the input threads (could be 10 or 20 threads) to process and apply 4 different tasks.
As we will have many threads running at the same time, I thought I would use MSMQ to manage these threads. Does using MSMQ fit in this scenario or should I use another technique? Managing these threads in terms of scheduling, prioritizing, etc.
(P.S: I was thinking to build my own ThreadEngine class that will take care of all of these things until I heard about MSMQ, which am still not sure if it’s the right thing to use)
MSMQ would be useful for managing your input/output data not for your threads. .Net already has the ThreadPool, the CCR and the TPL to assist you with concurrency and multithreading so I would suggest reading up on those technologies and choosing the most appropriate one.
MSMQ is a system message queue, not a thread pool manager.
This could be interesting in a case where you don't really mind poor performance and are really going for a system where tasks are persistent and transactional to guarantee execution.
If you are looking for performance then I agree with other folks and highly discourage you from doing this - even with non-durable (ram queues).