After a job failed finally no matter if it's out of retries or it's a ProblematicExceptionAndMustNotRetry, I want to react on it.
Is there any hook I haven't seen so far?
Currently I'm using a CustomRetryFilter as follows:
public class CustomRetryFilter extends RetryFilter {
#Override
public void onStateElection(Job job, JobState newState) {
if (this.isFailed(newState)) {
if (!this.isProblematicExceptionAndMustNotRetry(newState)
&& !this.maxAmountOfRetriesReached(job)) {
job.scheduleAt(Instant.now().plusSeconds(this.getSecondsToAdd(job)),
String.format("Retry %d of %d", this.getFailureCount(job), this.getMaxNumberOfRetries(job)));
} else {
// Publish event
}
}
}
}
What is best practice for such a scenario?
No, this is currently the way to do it.
I'll add a feature request for it.
Related
The task is to call a database, retrieve certain records update and save them.
As the amount of records if fairly large we want to do this Async, however, this doesn't seem to be implemented correctly.
The main class:
#SpringBootApplication
#EnableAsync
MainApplication() {
#Bean("threadPoolExecutor")
public TaskExecutor getAsyncExecutor(){
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(DataSourceConfig.getTHREAD_POOL_SIZE());
executor.setMaxPoolSize(DataSourceConfig.getTHREAD_POOL_SIZE());
executor.setWaitForTasksToCompleteOnShutdown(true);
executor.setThreadNamePrefix("RetryEnhancement-");
return executor;
}
}
Method in the first service:
#Service
public class FirstService() {
#Transactional
public void fullProcess() {
for(int counter = 0; counter < ConfigFile.getTHREADS(); counter++){
secondaryService.threads();
}
}
}
Method in the second service:
#Service
public class SecondService () {
#Async("threadPoolExecutor")
public void threads() {
while(thirdService.threadMethod()) {
//doNothing
}
}
}
Method in the third service:
#Service
public class ThirdService() {
#Transactional
public boolean threads() {
Record record = repository.fetchRecord();
if(record!=null) {
updateRecord(record);
saveRecord(record);
return true;
} else {
return false;
}
}
}
Repository:
public interface repository extends CrudRepository<Record, long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
Record fetchRecord();
}
The issue I'm finding is that, while the code executes perfectly fine, it seems to have a Synchronous execution (found by adding a .sleep and watching the execution in the logger).
The seperate threads seem to be waiting until the other is executed.
I'm probably doing something wrong and if another thread already explains the issue, than please refer it, though I have not been able to find this issue in a different thread.
Your solution is way to complex. Ditch all of that and just inject the TaskExecutor and do the updateRecord in a separate thread (you might need to retrieve it again as you are now using a different thread and thus connection.
Something like this should do the trick
private final TaskExecutor executor; // injected through constructor
public void process() {
Stream<Record> records = repository.fetchRecords(); // Using a stream gives you a lazy cursor!
records.forEach(this::processRecord);
}
private void processRecord(Record record) {
executor.submit({
updateRecord(record);
saveRecord(record);
});
}
You might want to put the processRecord into another object and make it #Transactional or wrap it in a TransactionTemplate to get that behavior.
I have a test project with Room database. Using Asynctask I can successfully insert an object with some test data into the database. I'm trying to learn RxJava and replace Asynctask with RxJava's observer, but it doesn't work. I have read alot of documentation and watched tutorials, but I don't think I quite get it. Here's the relevant code:
Here I set my Room object with the data from my List:
for(ObjectForArray item: listToDatabase) {
myRoomEntity.setName( item.getName() );
Log.d( "TAG", myRoomEntity.getName() );
}
Then I try to use RxJava Observable to insert data into the database. This was originally and successfully done using Asynctask:
Observable<MyRoomEntity> myRX = Observable
.just(myRoomEntity)
.subscribeOn( Schedulers.io() )
.observeOn( AndroidSchedulers.mainThread() );
myRX.subscribe( new Observer<MyRoomEntity>() {
#Override
public void onSubscribe(Disposable d) {
Log.d("TAG ONSUBSCRIBE", d.toString());
try {
myViewModel.insertDatabase( myRoomEntity );
Log.d( "TAG", "Populating database Success" );
}catch(Error error) {
Log.d( "TAG", error.toString() );
}
}
The OnNext, OnError and OnComplete are empty.
When I run the project it crashes with the error:
Cannot access database on the main thread since it may potentially lock the UI for a long period of time.
I'm obviously using RxJava wrong since the point is to do asynchronous tasks away from the main thread.
i have use RX java in replace of Asyntask as it has been deprecated in android 9
there are multiple replacements that android provides like Executors, threads, Listenable Futures , Coroutines 🔥, so you are looking how to implement this with rxjava and how RX Java java helps your to migrate just add these dependencies first in gradle
implementation "io.reactivex.rxjava2:rxjava:2.2.20"
implementation "io.reactivex.rxjava2:rxandroid:2.1.1"
once you import lets start working with RX java i will let you know where you can put background task, pre execute, on post execute like asynctask
lets start codding with Rx java first , i have comment in the method that will help you to put the code
Observable.fromCallable(new Callable<Boolean>() {
#Override
public Boolean call() throws Exception {
/// here is your background task
return true;
}
}).subscribeOn(Schedulers.io()).observeOn(AndroidSchedulers.mainThread())
.subscribe(new Observer<Boolean>() {
#Override
public void onSubscribe(Disposable d) {
//// pre execute here is my progress dialog
showProgressDialog(getString(R.string.scanning));
}
#Override
public void onNext(Boolean aBoolean) {
//// here is on sucess you can do anystuff here like
if (aBoolean){
/// if its value true you can go ahead with this
}
}
#Override
public void onError(Throwable e) {
/// this helps you to go if there is any error show dialog whatever you wants here
Log.e("error of kind",e.getMessage() );
}
#Override
public void onComplete() {
/// when your task done means post execute
}
});
once its done lets start working with implementation
Observable.fromCallable(new Callable<Boolean>() {
#Override
public Boolean call() throws Exception {
/// here is your background task
uribitmap = getScannedBitmap(original, points);
uri = Utils.getUri(getActivity(), uribitmap);
scanner.onScanFinish(uri);
return true;
}
}).subscribeOn(Schedulers.io()).observeOn(AndroidSchedulers.mainThread())
.subscribe(new Observer<Boolean>() {
#Override
public void onSubscribe(Disposable d) {
//// pre execute here is my progress dialog
showProgressDialog(getString(R.string.scanning));
}
#Override
public void onNext(Boolean aBoolean) {
//// here is on sucess you can do anystuff here like
if (aBoolean){
/// if its value true you can go ahead with this
}
}
#Override
public void onError(Throwable e) {
/// this helps you to go if there is any error show dialog whatever you wants here
Log.e("error of kind",e.getMessage() );
}
#Override
public void onComplete() {
/// when your task done means post execute
uribitmap.recycle();
dismissDialog();
}
});
now i will do this with executors :
/// pre execute you can trigger to progress dialog
showProgressDialog(getString(R.string.scanning));
ExecutorService executors = Executors.newSingleThreadExecutor();
executors.execute(new Runnable() {
#Override
public void run() {
//// do background heavy task here
final Bitmap uribitmap = getScannedBitmap(original, points);
uri = Utils.getUri(getActivity(), uribitmap);
scanner.onScanFinish(uri);
new Handler(Looper.getMainLooper()).post(new Runnable() {
#Override
public void run() {
//// Ui thread work like
uribitmap.recycle();
dismissDialog();
}
});
}
});
You are getting this error because you are trying to insert an Object on the main (UI) thread.
You should do something like this:
Observable.fromCallable(() -> myViewModel.insertDatabase( myRoomEntity ))
.subscribeOn( Schedulers.io() )
.observeOn( AndroidSchedulers.mainThread() );
And then use an Observer to subscribe to the Observable.
Please try restructuring your code like this:
Completable.fromAction(() -> myViewModel.insertDatabase(myRoomEntity))
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(() -> Log.d("TAG", "Populating database Success"),
throwable -> Log.d("TAG", throwable.toString()))
Considerations:
If your myRoomEntity is not available before this whole construct gets subscribed, make sure you use defer http://reactivex.io/documentation/operators/defer.html
Your subscribe section handlers are operating on "main", that's why you were receiving a crash.
If possible, avoid unnecessary just calls
I need to execute a callable task on a separate thread and perform a processing action without using Threadpool Executors. Below is the solution that worked for me using RxJava. Hope this helps someone out there. If you have any suggestion or a different solution, please feel free to answer it below.
#Component
public class TaskProcessor<T> {
#FunctionalInterface
public interface ProcessResult<V> {
void process(V v);
}
public void runTaskOnNewThread(Callable<T> task,ProcessResult<T> action) {
Assert.notNull(task, "task cannot be null");
Observable.fromCallable(task)
.subscribeOn(Schedulers.newThread())
.subscribe(new Action1<T>() {
#Override
public void call(T result) {
action.process(result);
}
});
}
}
(new Thread(()-> {action.process(task.call())}).start();
I'm using Ninject 2.0 with an MVC 2/EF 4 project in order to inject my repositories into my controllers. I've read that when doing something like that, one should bind using InRequestScope(). When I do that, I get a new repository per request, but the old repositories aren't being disposed. Since the old repositories are remaining in memory, I get conflicts with multiple ObjectContexts existing at the same time.
My concrete repositories implement IDisposable:
public class HGGameRepository : IGameRepository, IDisposable
{
// ...
public void Dispose()
{
if (this._siteDB != null)
{
this._siteDB.Dispose();
}
}
}
And my Ninject code:
public class NinjectControllerFactory : DefaultControllerFactory
{
private IKernel kernel = new StandardKernel(new HandiGamerServices());
protected override IController GetControllerInstance(System.Web.Routing.RequestContext requestContext, Type controllerType)
{
try
{
if (controllerType == null)
{
return base.GetControllerInstance(requestContext, controllerType);
// return null;
}
}
catch (HttpException ex)
{
if (ex.GetHttpCode() == 404)
{
IController errorController = kernel.Get<ErrorController>();
((ErrorController)errorController).InvokeHttp404(requestContext.HttpContext);
return errorController;
}
else
{
throw ex;
}
}
return (IController)kernel.Get(controllerType);
}
private class HandiGamerServices : NinjectModule
{
public override void Load()
{
Bind<HGEntities>().ToSelf().InRequestScope();
Bind<IArticleRepository>().To<HGArticleRepository>().InRequestScope();
Bind<IGameRepository>().To<HGGameRepository>().InRequestScope();
Bind<INewsRepository>().To<HGNewsRepository>().InRequestScope();
Bind<ErrorController>().ToSelf().InRequestScope();
}
}
}
What am I doing wrong?
I'm quite sure that you are wrong about the guess that your objects are not disposed. This just does not happen when you think it will happen. But the fact that this does happen later should not give you any problems with ObjectContexts unless you are doing something wrong. With a high load you will have a lot of ObjectContexts at the same time anyway.
What can become a problem though is that the memory usage increases. That's why the request scope needs to be released actively. The Ninject MVC extensions will take care of that. Otherwise have a look at the OnePerRequestModule to see how it is done:
https://github.com/ninject/Ninject.Web.Common/blob/master/src/Ninject.Web.Common/OnePerRequestHttpModule.cs
I've installed excel module in order to generate reports from datas recorded by my application into database.
It works fine : i can create report simply by clicking on a link into my main page and render into excel template.
But i'd rather generate excel file periodically (using a job) and save it into a shared folder, and that without any human action (so not by clicking on a link).
It's like I want to trigger the associated controller to render into my template automatically.
Does anyone got any tips on it for me?
So the problem is you can't pass some parameters into the job, or...?
Using something like this just doesn't work?
#On("0 45 4-23 ? * MON-FRI")
public class ExcelJob extends Job {
public void doJob() {
// generate excel
}
}
I wrote my own Excel generator using JExcel, and I use it for scheduled generation without a problem. It also doesn't require a template, because the report structure is derived from annotations. This is roughly 20 lines of code - you may want to try it for yourself.
This is really rough and lacks good user feedback, but gives you the idea...
Excel generator - not Play-specific in any way
public class ExcelGenerator
{
public void generateReport(Function successCallback,
Function failureCallback)
{
try
{
byte[] report = // generate your report somehow
successCallback.execute(report);
}
catch (Exception e)
{
failureCallback.execute(e.getMessage());
}
}
}
A function interface for callbacks (very basic)
public interface Function
{
public void execute(Object... args);
}
Your Play controller
public class MyController extends Controller
{
public static void index()
{
render();
}
public static void createReport()
{
Function failureCallback = new Function()
{
public void execute(Object... args)
{
flash.error(args[0]);
indxe();
}
};
Function successCallback = new Function()
{
public void execute(Object... args)
{
renderBinary((byte[])args[0]);
}
};
ExcelGenerator excelGenerator = new ExcelGenerator();
excelGenerator.generateReport(successCallback,
failureCallback);
}
}
Finally, re-use the ExcelGenerator from your job
public class MyJob extends Job
{
public void doJob()
{
Function failureCallback = new Function()
{
public void execute(Object... args)
{
Logger.error(args[0]);
}
}
Function successCallback = new Function()
{
public void execute(Object... args)
{
byte[] report = (byte[])args[0];
// write report to disk
}
}
ExcelGenerator excelGenerator = new ExcelGenerator();
excelGenerator.generateReport(successCallback,
failureCallback);
}
}
You'll still need to write your own report generator, or refactor the existing excel module to provide what you need.
So if you want to run and manage several jobs you can do something like this
for (int i = 0; i < 10; i++) {
SendingMessageJob sendingMessageJob = new SendingMessageJob();
promises.add(sendingMessageJob.now());
}
boolean allDone = false;
while (!allDone) {
allDone = true;
for (F.Promise promise : promises) {
if (!promise.isDone()) {
allDone = false;
break;
}
}
}
// when arrive here all jobs have finished their process
You can check the Play documentation, specifically the section on jobs, where you'll see examples on how to create automatically triggered methods. This should solve your issue.
EDIT (update on comment):
You can manually trigger a job, do this:
new MyExcelGeneratorJob().doJob();
Thing is, Play is stateless, so the job should use data from the database. Instead of trying to pass parameters from your request into the Job (won't work) try to store that data in a staging area in the database that the job loads and processes to generate the excel.