Spring boot multithreaded async not working - multithreading

The task is to call a database, retrieve certain records update and save them.
As the amount of records if fairly large we want to do this Async, however, this doesn't seem to be implemented correctly.
The main class:
#SpringBootApplication
#EnableAsync
MainApplication() {
#Bean("threadPoolExecutor")
public TaskExecutor getAsyncExecutor(){
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(DataSourceConfig.getTHREAD_POOL_SIZE());
executor.setMaxPoolSize(DataSourceConfig.getTHREAD_POOL_SIZE());
executor.setWaitForTasksToCompleteOnShutdown(true);
executor.setThreadNamePrefix("RetryEnhancement-");
return executor;
}
}
Method in the first service:
#Service
public class FirstService() {
#Transactional
public void fullProcess() {
for(int counter = 0; counter < ConfigFile.getTHREADS(); counter++){
secondaryService.threads();
}
}
}
Method in the second service:
#Service
public class SecondService () {
#Async("threadPoolExecutor")
public void threads() {
while(thirdService.threadMethod()) {
//doNothing
}
}
}
Method in the third service:
#Service
public class ThirdService() {
#Transactional
public boolean threads() {
Record record = repository.fetchRecord();
if(record!=null) {
updateRecord(record);
saveRecord(record);
return true;
} else {
return false;
}
}
}
Repository:
public interface repository extends CrudRepository<Record, long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
Record fetchRecord();
}
The issue I'm finding is that, while the code executes perfectly fine, it seems to have a Synchronous execution (found by adding a .sleep and watching the execution in the logger).
The seperate threads seem to be waiting until the other is executed.
I'm probably doing something wrong and if another thread already explains the issue, than please refer it, though I have not been able to find this issue in a different thread.

Your solution is way to complex. Ditch all of that and just inject the TaskExecutor and do the updateRecord in a separate thread (you might need to retrieve it again as you are now using a different thread and thus connection.
Something like this should do the trick
private final TaskExecutor executor; // injected through constructor
public void process() {
Stream<Record> records = repository.fetchRecords(); // Using a stream gives you a lazy cursor!
records.forEach(this::processRecord);
}
private void processRecord(Record record) {
executor.submit({
updateRecord(record);
saveRecord(record);
});
}
You might want to put the processRecord into another object and make it #Transactional or wrap it in a TransactionTemplate to get that behavior.

Related

how to do something when liferay module stop

i am making cron job like loop to do something using new thread.
when module stop, this thread keeps running, so when i deployed updated module, i'm afraid it will make duplicate thread doing similar task
#Component(immediate = true, service = ExportImportLifecycleListener.class)
public class StaticUtils extends Utils{
private StaticUtils() {}
private static class SingletonHelper{
private static final StaticUtils INSTANCE = new StaticUtils();
}
public static StaticUtils getInstance() {
return SingletonHelper.INSTANCE;
}
}
public class Utils extends BaseExportImportLifecycleListener{
public Utils() {
startTask();
}
protected Boolean CRON_START = true;
private void startTask() {
new Thread(new Runnable() {
public void run() {
while (CRON_START) {
System.out.println("test naon bae lah ");
}
}
}).start();
}
#Deactivate
protected void deactivate() {
CRON_START = false;
System.out.println(
"cron stop lah woooooooooooooooooy");
}
}
i'm using liferay 7
I have populated task that i store from db, so this thread is checking is there a task that it must do, then if it exist execute it.
I'm quite new in osgi and liferay. i've try to use scheduler and failed and also exportimportlifecycle listener but dont really get it yet
think again: Do you really need something to run all the time in the background, or do you just need some asynchronous processing in the background, when triggered? It might be better to start a background task as a one-off, that automatically terminates
Liferay provides an internal MessageBus, that you can utilize to listen to events and implement background processing, without the need for a custom thread
You're in the OSGi world, so you can utilize #Activate, #Modified, #Deactivate (from org.osgi.service.component.annotations) or use a org.osgi.framework.BundleActivator.
But, in general, it's preferable if you don't start your own thread

How to execute a callable task asynchronously on a separate thread using without using Threadpool Executors?

I need to execute a callable task on a separate thread and perform a processing action without using Threadpool Executors. Below is the solution that worked for me using RxJava. Hope this helps someone out there. If you have any suggestion or a different solution, please feel free to answer it below.
#Component
public class TaskProcessor<T> {
#FunctionalInterface
public interface ProcessResult<V> {
void process(V v);
}
public void runTaskOnNewThread(Callable<T> task,ProcessResult<T> action) {
Assert.notNull(task, "task cannot be null");
Observable.fromCallable(task)
.subscribeOn(Schedulers.newThread())
.subscribe(new Action1<T>() {
#Override
public void call(T result) {
action.process(result);
}
});
}
}
(new Thread(()-> {action.process(task.call())}).start();

Java: Running transaction in multithreaded environment

We are launching a website that will have a very heavy volume for a short period of time. It is basically giving tickets. The code is written in Java, Spring & Hibernate. I want to mimic the high volume by spawning multiple threads and trying to get the ticket using JUnit test case. The problem is that in my DAO class the code just simply dies after I begin transaction. I mean there is no error trace in the log file or anything like that. Let me give some idea about the way my code is.
DAO code:
#Repository("customerTicketDAO")
public class CustomerTicketDAO extends BaseDAOImpl {// BaseDAOImpl extends HibernateDaoSupport
public void saveCustomerTicketUsingJDBC(String customerId) {
try{
getSession().getTransaction().begin(); //NOTHING HAPPENS AFTER THIS LINE OF CODE
// A select query
Query query1 = getSession().createSQLQuery("my query omitted on purpose");
.
.
// An update query
Query query2 = getSession().createSQLQuery("my query omitted on purpose");
getSession().getTransaction().commite();
} catch (Exception e) {
}
}
Runnable code:
public class InsertCustomerTicketRunnable implements Runnable {
#Autowired
private CustomerTicketDAO customerTicketDAO;
public InsertCustomerTicketRunnable(String customerId) {
this.customerId = customerId;
}
#Override
public void run() {
if (customerTicketDAO != null) {
customerTicketDAO.saveCustomerTicketUsingJDBC(customerId);
}
}
}
JUnit method:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations={"file:src/test/resources/applicationContext-test.xml"})
public class DatabaseTest {
#Before
public void init() {
sessionFactory = (SessionFactory)applicationContext.getBean("sessionFactory");
Session session = SessionFactoryUtils.getSession(sessionFactory, true);
TransactionSynchronizationManager.bindResource(sessionFactory, new SessionHolder(session));
customerTicketDAO = (CustomerTicketDAO)applicationContext.getBean("customerTicketDAO");
}
#After
public void end() throws Exception {
SessionHolder sessionHolder = (SessionHolder) TransactionSynchronizationManager.unbindResource(sessionFactory);
SessionFactoryUtils.closeSession(session);
}
#Test
public void saveCustomerTicketInMultipleThreads () throws Exception {
ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
for (int i=0; i<1000; i++) {
executor.submit(new InsertCustomerTicketRunnable(i));
}
// This will make the executor accept no new threads
// and finish all existing threads in the queue
executor.shutdown();
// Wait until all threads are finish
executor.awaitTermination(1, TimeUnit.SECONDS);
}
I see no data being inserted into the database. Can someone please point me as to where I am going wrong?
Thanks
Raj
SessionFactory is thread safe but Session is not. So my guess is that you need to call SessionFactoryUtils.getSession() from within each thread, so that each thread gets its own instance. You are currently calling it from the main thread, so all children threads try to share the same instance.
Naughty, naughty!
public void saveCustomerTicketUsingJDBC(String customerId) {
try {
getSession().getTransaction().begin(); //NOTHING HAPPENS AFTER THIS LINE OF CODE
.
.
} catch (Exception e) {
}
}
You should never (well, hardly ever) have an empty catch block, if there is a problem you will find that your code 'just simply dies' with no log messages. Oh look, that's what's happening ;)
At the very minimum you should log the exception, that will go a long way towards you helping you find what the problem is (and from there, the solution).

Running windows service in separate thread and use autofac for DI

I'm trying to create a long running windows service, so I need to run the actual worker class on a separate thread, to avoid the "service did not respond in a timely fashion" error when I right click and select start in Windows Service Manager.
The worker class ("NotificationProcess") has a whole raft of dependencies and I'm using Autofac to satisfy these.
I'm really not sure how to set up Autofac for the worker class. At the moment I'm getting errors telling me that the DbContext has been disposed when I go to use it in the "Execute" method of the worker class.
I guess I'm looking for how to write a windows service and use a new thread for the worker class with dependencies satisfied by autofac.
I've googled and can't find any examples of this.
Any suggestions would be awesome.
Here's what I've got so far...
Program.cs:
static class Program
{
static void Main()
{
using (var container = ServiceStarter.CreateAutoFacContainer())
{
var service = container.Resolve<NotificationService>();
if (Environment.UserInteractive)
{
service.Debug();
}
else
{
ServiceBase.Run(container.Resolve<NotificationService>());
}
}
The Service class:
public partial class NotificationService : ServiceBase
{
private NotificationProcess _app;
readonly ILifetimeScope _lifetimeScope;
public NotificationService(ILifetimeScope lifetimeScope)
{
_lifetimeScope = lifetimeScope;
InitializeComponent();
}
protected override void OnStart(string[] args)
{
_app = _lifetimeScope.Resolve<NotificationProcess>();
_app.Start();
}
The worker class:
public class NotificationProcess
{
private Thread _thread;
private readonly IBankService _bankService;
private readonly IRateService _rateService;
private readonly IEmailService _emailService;
private readonly IRateChangeSubscriberService _rateChangeSubscriberService;
private readonly IRateChangeNotificationService _rateChangeNotificationService;
private readonly ILogManager _logManager;
public NotificationProcess(IBankService bankService, ILogManager logManager, IRateService rateService, IEmailService emailService,
IRateChangeSubscriberService rateChangeSubscriberService, IRateChangeNotificationService rateChangeNotificationService)
{
_bankService = bankService;
_rateService = rateService;
_emailService = emailService;
_rateChangeSubscriberService = rateChangeSubscriberService;
_rateChangeNotificationService = rateChangeNotificationService;
_logManager = logManager;
}
public void Start()
{
_thread = new Thread(new ThreadStart(Execute));
_thread.Start();
}
public void Execute()
{
try
{
var rateChangeToNotify = _rateService.GetRateChangesForNotification();
foreach (var rateChange in rateChangeToNotify)
{
//do whatever business logic.....
}
}
}
The answer is actually simple: use scoping! You should do the following:
Register all services (such as DbContext) that should live for the duration of a request or action with the LifetimeScope lifestyle. You'll usually have a timer in your windows service. Each 'pulse' can be considered a request.
On the beginning of each request begin a lifetime scope.
Within that scope, resolve the root object from the object graph and call its method.
Dispose the scope.
In your case that means you need to change your design, since NotificationService is resolved once and its dependencies are reused on another thread. This is a no-no in dependency injection land.
Here's an alternative design:
// This method is called on a background thread
// (possibly in a timely manner)
public void Run()
{
try
{
using (var scope = container.BeginLifetimeScope())
{
var service = scope.Resolve<NotificationService>();
service.Execute();
}
}
catch (Exception ex)
{
// IMPORTANT: log exception.
// Not logging an exception will leave us in the dark.
// Not catching the exception will kill our service
// because we run in a background thread.
}
}
Using a lifetime scope allows you to get a fresh DbContext for every request and it would even allow you to run requests in parallel (with each request its own DbContext).

How to implement cancellable worker thread

I'm trying to implement a cancellable worker thread using the new threading constructs in System.Threading.Tasks namespace.
So far I have have come up with this implementation:
public sealed class Scheduler
{
private CancellationTokenSource _cancellationTokenSource;
public System.Threading.Tasks.Task Worker { get; private set; }
public void Start()
{
_cancellationTokenSource = new CancellationTokenSource();
Worker = System.Threading.Tasks.Task.Factory.StartNew(
() => RunTasks(_cancellationTokenSource.Token),
_cancellationTokenSource.Token
);
}
private static void RunTasks(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
Thread.Sleep(1000); // simulate work
}
}
public void Stop()
{
try
{
_cancellationTokenSource.Cancel();
Worker.Wait(_cancellationTokenSource.Token);
}
catch (OperationCanceledException)
{
// OperationCanceledException is expected when a Task is cancelled.
}
}
}
When Stop() returns I expect Worker.Status to be TaskStatus.Canceled.
My unit tests have shown that under certain conditions Worker.Status remains set to TaskStatus.Running.
Is this a correct way to implement a cancellable worker thread?
I believe that the problem is in your call to
Worker.Wait(_cancellationTokenSource.Token);
That's waiting for the token to be signalled - which it already is, because you've just called Cancel(). If you change that to just
Worker.Wait();
then I believe you'll see a state of RanToCompletion. You won't see Canceled, because your task isn't throwing OperationCanceledException. If you change your RunTasks method to call
cancellationToken.ThrowIfCancellationRequested()
at the end, then you'll need to catch an AggregateException in Stop - but then you'll see a state of Canceled at the end.
At least, that's what my experimentation shows :)

Resources