I am using this signature of delay:
public final <U> Observable<T> delay(Func1<? super T,? extends Observable<U>> itemDelay)
javadoc
I am using Func1 to return an Observable which acts as a sort of "trigger". My goal is to delay items until an outside async operation completes. Once that operation completes I want to emit all items that have been delayed and all future items in order.
Here is some sample code that shows what I'm trying to do:
import java.util.concurrent.atomic.AtomicBoolean;
import io.reactivex.Observable;
import io.reactivex.ObservableTransformer;
import io.reactivex.functions.Consumer;
import io.reactivex.schedulers.Schedulers;
import io.reactivex.subjects.ReplaySubject;
public class Example {
private ReplaySubject<Object> delayTrigger = ReplaySubject.create(); // (1)
public void main() {
System.out.println("============ MAIN ============");
SourceThread sourceThread = new SourceThread();
sourceThread.start();
sourceThread.stream
.compose(doOnFirst(integer -> startAsyncOperation())) // (2)
.delay(integer -> delayTrigger) // (3)
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe((Integer integer)
-> System.out.println("onNext: " + integer));
}
private void startAsyncOperation() {
System.out.println(">>>>>>> long async operation started");
SomeOtherThread someOtherThread = new SomeOtherThread();
someOtherThread.start();
}
private void onAsyncOperationComplete() {
System.out.println("<<<<<<< long async operation completed");
delayTrigger.onNext(new Object()); // (4)
}
/**
* From https://stackoverflow.com/a/32366794
*/
private <T> ObservableTransformer<T, T> doOnFirst(Consumer<? super T> consumer) {
return observableTransformer -> Observable.defer(() -> {
final AtomicBoolean first = new AtomicBoolean(true);
return observableTransformer.doOnNext(t -> {
if (first.compareAndSet(true, false)) {
consumer.accept(t);
}
});
});
}
/**
* Some thread to simulate a some time delayed source.
* This is not really part of the problem,
* we just need a time delayed source on another thread
*/
private final class SourceThread extends Thread {
private ReplaySubject<Integer> stream = ReplaySubject.create();
#Override
public void run() {
super.run();
for (int i = 0; i < 100; i++) {
stream.onNext(i);
System.out.println("Source emits item: " + i);
try {
Thread.sleep(20);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
private final class SomeOtherThread extends Thread {
#Override
public void run() {
super.run();
try {
Thread.sleep(1000);
onAsyncOperationComplete();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
At (1) I create a ReplaySubject that will act as my trigger, at (2) I start the async operation, at (3) I delay until the trigger emits something; finally at (4) I put something into the trigger stream when async operation completes.
This works fine for the most part, except when the async operation completes, the stream returned from delay is out of order.
I/System.out: Source emits item: 46
I/System.out: Source emits item: 47
I/System.out: <<<<<<< long async operation completed
I/System.out: Source emits item: 48
I/System.out: onNext: 0
I/System.out: onNext: 48 <---- problem here!!!
I/System.out: onNext: 1
I/System.out: onNext: 2
I/System.out: onNext: 3
Item 48 is emitted from delay before items 1 - 47. Item 49 will get emitted out of order as well. This will continue until items 1-47 are emitted, then the stream cleans up. But there is a big section of un-ordered items. Is there a way I can fix this? Am I using delay correctly? Is this a bug in delay?
For reference this is just a sample. In my "real" problem I have no way to re-order the emitted items once they get out of order (i.e. they aren't nicely numbered).
That delay operator has no ordering guarantees because the inner source for item #1 may signal later than another inner source for item #2 in general. Any async signal may throw off the ordering, even if coming from a source such as a terminated ReplaySubject.
I assume you want to prefetch the main source but not let it through before the external signal, right? In this case, you can use concatArrayEager where the first source's completion triggers the emission of the prefetched second source:
PublishSubject<Integer> delayer = PublishSubject.create();
Observable.concatArrayEager(
delayer,
sourceThread.stream
)
// somewhere async
delayer.onComplete();
Related
I'm trying to define a flow for a single-threaded handler. Messages come in great number and the handler is slow (it's inefficient to process them one by one). So I want to make the handler consume all messages available in the channel at once (or wait until a few messages accumulate) with Java DSL. If there are no messages in the channel and the handler has processed a previous group it should wait for a certain period of time (timeout "a") for a few messages to accumulate in the channel. But if messages keep coming, the handler MUST consume them after a certain period of time from the previous execution (timeout "b"). Therefore time intervals between handler executions should be no more than "b" (unless no messages arrive in the channel).
There is no reason to make multiple instances of that sort of handler: it generates data for interfaces. The code below describes some basic configuration. My problem is that I'm not able to come up with debouncing (the timeout "b") and releasing the group once the handler execution is completed.
#Configuration
public class SomeConfig {
private AtomicBoolean someHandlerBusy = new AtomicBoolean(false);
#Bean
StandardIntegrationFlow someFlow() {
return IntegrationFlows
.from("someChannel")
.aggregate(aggregatorSpec -> aggregatorSpec
//The only rule to release a group:
//wait 500ms after last message and have a free someHandler
.groupTimeout(500)
.sendPartialResultOnExpiry(true) //if 500ms expired - send group
.expireGroupsUponCompletion(true) //group should be filled again
.correlationStrategy(message -> true) //one group key, all messages in oe group
.releaseStrategy(message -> false) //never release messages, only with timeout
//Send messages one by one. This is not part of this task.
//I just want to know how to do that. Like splitter.
//.outputProcessor(MessageGroup::getMessages)
)
.handle("someHandler")
.get();
}
}
I have the solution with plain Java (kotlin) code: https://pastebin.com/mti3Y5tD
UPDATE
The configuration below does not erase group. The group is growing and growing and it falls wit error at the end.
Error:
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at JPLISAgent.c line: 844
Configuration:
#Configuration
public class InterfaceHandlerConfigJava {
#Bean
MessageChannel interfaceAggregatorFlowChannel() {
return MessageChannels.publishSubscribe("interfaceAggregatorFlowChannel").get();
}
#EventListener(ApplicationReadyEvent.class)
public void initTriggerPacket(ApplicationReadyEvent event) {
MessageChannel channel = event.getApplicationContext().getBean("interfaceAggregatorFlowChannel", MessageChannel.class);
channel.send(MessageBuilder.withPayload(new InterfaceHandler.HandlerReadyMessage()).build());
}
#Bean
StandardIntegrationFlow someFlow(
InterfaceHandler interfaceHandler
) {
long lastMessageTimeout = 10L;
return IntegrationFlows
.from("interfaceAggregatorFlowChannel")
.aggregate(aggregatorSpec -> aggregatorSpec
.groupTimeout(messageGroup -> {
if (haveInstance(messageGroup, InterfaceHandler.HandlerReadyMessage.class)) {
System.out.println("case HandlerReadyMessage");
if (haveInstance(messageGroup, DbChangeStreamConfiguration.InitFromDbMessage.class)) {
System.out.println("case InitFromDbMessage");
return 0L;
} else if (messageGroup.size() > 1) {
long groupCreationTimeout =
messageGroup.getTimestamp() + 500L - System.currentTimeMillis();
long timeout = Math.min(groupCreationTimeout, lastMessageTimeout);
System.out.println("case messageGroup.size() > 1, timeout: " + timeout);
return timeout;
}
}
System.out.println("case Handler NOT ReadyMessage");
return null;
})
.sendPartialResultOnExpiry(true)
.expireGroupsUponCompletion(true)
.expireGroupsUponTimeout(true)
.correlationStrategy(message -> true)
.releaseStrategy(message -> false)
)
.handle(interfaceHandler, "handle")
.channel("interfaceAggregatorFlowChannel")
.get();
}
private boolean haveInstance(MessageGroup messageGroup, Class clazz) {
for (Message<?> message : messageGroup.getMessages()) {
if (clazz.isInstance(message.getPayload())) {
return true;
}
}
return false;
}
}
I want to highlight: this flow is in the cycle. There is IN and no OUT. Messages go to the IN but handler emits HandlerReadyMessage at the end.
Maybe there should be some thread breaker channel?
FINAL VARIANT
As aggregator and handler should not blocks each other and should not try to make a stackoverflow exception they should run in different threads. In the configuration above this achieved with queue channels. Looks that publish-subscribe channels are not running subscribers in different threads (at least for one subscriber).
#Configuration
public class InterfaceHandlerConfigJava {
// acts as thread breaker too
#Bean
MessageChannel interfaceAggregatorFlowChannel() {
return MessageChannels.queue("interfaceAggregatorFlowChannel").get();
}
#Bean
MessageChannel threadBreaker() {
return MessageChannels.queue("threadBreaker").get();
}
#EventListener(ApplicationReadyEvent.class)
public void initTriggerPacket(ApplicationReadyEvent event) {
MessageChannel channel = event.getApplicationContext().getBean("interfaceAggregatorFlowChannel", MessageChannel.class);
channel.send(MessageBuilder.withPayload(new InterfaceHandler.HandlerReadyMessage()).build());
}
#Bean
StandardIntegrationFlow someFlow(
InterfaceHandler interfaceHandler
) {
long lastMessageTimeout = 10L;
return IntegrationFlows
.from("interfaceAggregatorFlowChannel")
.aggregate(aggregatorSpec -> aggregatorSpec
.groupTimeout(messageGroup -> {
if (haveInstance(messageGroup, InterfaceHandler.HandlerReadyMessage.class)) {
System.out.println("case HandlerReadyMessage");
if (haveInstance(messageGroup, DbChangeStreamConfiguration.InitFromDbMessage.class)) {
System.out.println("case InitFromDbMessage");
return 0L;
} else if (messageGroup.size() > 1) {
long groupCreationTimeout =
messageGroup.getTimestamp() + 500L - System.currentTimeMillis();
long timeout = Math.min(groupCreationTimeout, lastMessageTimeout);
System.out.println("case messageGroup.size() > 1, timeout: " + timeout);
return timeout;
}
}
System.out.println("case Handler NOT ReadyMessage");
return null;
})
.sendPartialResultOnExpiry(true)
.expireGroupsUponCompletion(true)
.expireGroupsUponTimeout(true)
.correlationStrategy(message -> true)
.releaseStrategy(message -> false)
.poller(pollerFactory -> pollerFactory.fixedRate(1))
)
.channel("threadBreaker")
.handle(interfaceHandler, "handle", spec -> spec.poller(meta -> meta.fixedRate(1)))
.channel("interfaceAggregatorFlowChannel")
.get();
}
private boolean haveInstance(MessageGroup messageGroup, Class clazz) {
for (Message<?> message : messageGroup.getMessages()) {
if (clazz.isInstance(message.getPayload())) {
return true;
}
}
return false;
}
}
It's not clear what you mean by timer b, but you can use a .groupTimeoutExpression(...) to dynamically determine the group timeout.
You don't need to worry about sending messages one by one; when the output processor returns a collection of Message<?> they are sent one-at-a-time.
Say I have a bunch of transformations on an Observable:
operation()
.flatMap(toSomething())
.map(toSomethingElse())
.flatMap(toYetSomethingElse())
.subscribeOn(Schedulers.newThread())
.observeOn(AdroidSchedulers.mainThread())
.subscribe(observer);
Are all of these operations synchronous except for the last call to flatMap()? Or are all of the operations run on the thread that I told it to subscribe on?
I figured this out, with a test. The following test passes (which means the emissions on the Observable are all on the same background thread):
volatile long observableThreadId;
#Test
public void transformedObservables_shouldRunInSameThread() {
Observable.from(new String[]{"a", "b", "c"}) //
.flatMap(new Func1<String, Observable<Object>>() {
#Override public Observable<Object> call(String s) {
observableThreadId = Thread.currentThread().getId();
return Observable.from((Object) s);
}
}) //
.map(new Func1<Object, String>() {
#Override public String call(Object o) {
long id = Thread.currentThread().getId();
if (id != observableThreadId) {
throw new RuntimeException("Thread ID mismatch");
}
return (String) o;
}
}) //
.flatMap(new Func1<String, Observable<String>>() {
#Override public Observable<String> call(String s) {
long id = Thread.currentThread().getId();
if (id != observableThreadId) {
throw new RuntimeException("Thread ID mismatch");
}
return Observable.from(s);
}
}) //
.subscribeOn(Schedulers.newThread()) //
.observeOn(Schedulers.currentThread()) //
.subscribe(new Observer<String>() {
#Override public void onCompleted() {
assertThat(Thread.currentThread().getId()).isNotEqualTo(observableThreadId);
}
#Override public void onError(Throwable throwable) {
}
#Override public void onNext(String s) {
}
});
System.out.println("blah");
}
===============================
UPDATE:
A better answer can actually be found in the ReactiveX documentation on Scheduler:
By default, an Observable and the chain of operators that you apply to
it will do its work, and will notify its observers, on the same thread
on which its Subscribe method is called. The SubscribeOn operator
changes this behavior by specifying a different Scheduler on which the
Observable should operate. The ObserveOn operator specifies a
different Scheduler that the Observable will use to send notifications
to its observers.
... the SubscribeOn operator designates which thread the Observable will
begin operating on, no matter at what point in the chain of operators
that operator is called. ObserveOn, on the other hand, affects the
thread that the Observable will use below where that operator appears.
For this reason, you may call ObserveOn multiple times at various
points during the chain of Observable operators in order to change on
which threads certain of those operators operate.
I am using threads in blackberry to perform web service calls. I want to get notified as soon as the call gets a response back. I was using
Handlers
in android. I didnt find anything similar in blackberry.
Here is the code I am using to run the thread
class PrimeRun implements Runnable {
long minPrime;
PrimeRun(long minPrime) {
this.minPrime = minPrime;
}
public void run() {
// compute primes larger than minPrime
. . .
}
}
How can I get a notification after the thread finished running?
How can I do this in blackberry?
Thanks
Added more Information : Thanks for your reply. Its really
informative. Let me explain a bit more on my issue. I have a
webservice call which is running on a thread. As soon as I get the
reply back from server I want to execute the next function(next call
to server) which is based on the response from the previous call.So I need to wait until I get a response back. Also
at them same time I need to show a activity indicator on screen. I was
using handler for this in android. I am looking for something similar
on blackberry.
So your question essentially is this
One thread does the job while the other thread waits for completion
The first thread completes the job and "notifies" the second thread.
This is a simple producer consumer problem. Here is the code how you can solve this.
class JobResult
{
boolean done = false;
}
JobResult result = new JobResult();
class Worker extends Thread
{
JobResult _result;
public Worker( JobResult result )
{
_result = result
}
public void run()
{
// Do some very long job
synchronized( _result )
{
// modify result
_result.done = true;
_result.notify();
}
}
}
public class Waiter extends Thread
{
JobResult _result;
public Waiter( JobResult result )
{
_result = result;
}
public void run()
{
synchroinzed( _result ){
while(! _result.done)
{
this.wait();
}
}
// Wait is over. You can do something now.
}
}
As I got the Zach's question - he asks how to execute some code that involves UI changes (something like showing an info popup or closing the progress popup) upon a background thread completion. On Android a Handler created on the UI thread is often used for that purpose.
In BB you can use another way which is similar to Swing on desktop Java. When you need some code to be executed on the UI thread you wrap it in a Runnable and pass to one of the following methods:
// Puts runnable object into this application's event queue,
// and waits until it is processed.
Application.invokeAndWait(Runnable runnable)
// Puts runnable object into this application's event queue.
Application.invokeLater(Runnable runnable)
// Puts runnable object into this application's event queue
// for repeated execution.
Application.invokeLater(Runnable runnable, long time, boolean repeat)
So the behaviour of the above calls is similar to what Handler.post(Runnable r) (and the like) does.
Note, you can always get a handle to your Application instance by a static call Application.getApplication().
So in the end of a background thread it is safe to do something like this:
Application.getApplication().invokeLater(new Runnable() {
public void run() {
progressScreen.close();
Dialog.alert("I am finished!");
}
});
It is similar to Android's:
handler.post(new Runnable() {
public void run() {
progressScreen.dismiss();
showDialog(DIALOG_TASK_FINISHED_ID);
}
});
Android has a much rich multi threading primitives. But you can achieve the same even in Blackberry with equal elegance. The solution I provide below is essentially the same as previous, but with a minor change. Waiter thread can be replaced with built-in utility to perform painting on UI thread using UiApplicaiton's invokeLater method. You don't actually need to "notify" anyone but just update the UI once a particular task is completed. Check the docs for more info.
Anyway, you can model your code along the lines:
class ProgressScreen extends FullScreen
{
LabelField _label;
public void start()
{
}
public void setMessage( final String message )
{
UiApplication.getApplication(
UiApplication.invokeLater(
new Runnable() {
_label.setText( message );
}
)
);
}
public void dismiss()
{
this.close();
}
}
interface WebserviceTask
{
int STATUS_CONDITIONS_NOT_SATISFIED = -3;
int STATUS_NET_ERR = -2;
int STATUS_FAILURE = -1;
int STATUS_SUCCESS = 0;
public int invoke();
}
public class Updater extends Thread
{
final int NUM_TASKS = 10;
WebServiceTask tasks[] = new WebServiceTask[ NUM_TASKS ];
WebServiceTask tasks[0] = new WebServiceTask(){
public int invoke()
{
int retCode = 0;
// invoke a particular web service
return STATUS_SUCCESS;
}
}
public void run()
{
ProgressScreen progress = new ProgressScreen();
progress.start();
for( int i=0; i < NUM_TASKS; i++ )
{
int retcode;
WebServiceTask t = tasks[i];
retcode = t.invoke();
String mesg;
switch( retcode )
{
case STATUS_SUCCESS: { mesg ="Task successfully completed!";} break;
case STATUS_NET_ERR: { mesg ="Could not connect to network";} break;
}
progress.setMessage(message);
}
progress.dismiss();
}
}
Note that I have provided only the stubs to give you an idea how you may accomplish. Let us know how it goes.
I have a problem with this code
if(Handlers.Count==0)
{
GetHandlers();
while (_handlers.Count == 0)
{
Thread.Sleep(100);
}
}
return _showroomLogs;
This method executes:
private void GetHandlers()
{
WebSerive.GetHandlersCompleted += new EventHandler<GetHandlersCompletedEventArgs>(OnGetHandlersCompleted);
WebSerive.GetHandlersAsync(_app.HandlerId);
}
but to this method:
private void OnGetHandlersCompleted(object sender, GetHandlersCompletedEventArgs e)
{
WebSerive.GetHandlersCompleted -= new EventHandler<GetHandlersCompletedEventArgs>(OnGetHandlersCompleted);
_handlers = e.Result;
}
I fall into afterd execution of
return _showroomLogs;
of cource if I remove this piece with While
What I must to do, to have executed OnGetHandlersAsync before
return _showroomLogs;
?
You need to recognise that as soon as you introduce an asynchronous operation to a sequence the whole sequence becomes asynchronous. Using blocking techniques like Sleep is 99.99999% the wrong choice.
Restructure as:-
private void GetHandlers(int handlerID, Action<IList<Handler>> returnResult)
{
EventHandler<GetHandlersCompletedEventArgs> eh = null;
eh = (s, args) =>
{
WebSerive.GetHandlersCompleted -= eh;
returnResult(args.Result);
};
WebSerive.GetHandlersCompleted += eh;
WebSerive.GetHandlersAsync(handerlID);
}
you then call with:-
GetHandlers(_app.HandlerId, handlers =>
{
_handlers = handlers;
// Do other stuff
});
Edit
Let me outline at a conceptual level what the fundemental problem is here. Lets say I have button click event, which calls FuncA. FuncA calls FuncB, FuncB calls FuncC.
click -> FuncA -> FuncB -> FuncC
This whole sequence is synchronous it might look like:-
void Button_Click(object sender, EventArgs e)
{
FuncA();
//Do other stuff
}
void FuncA()
{
var result = FuncB();
//Do stuff with result;
}
string FuncB()
{
return FuncC() + " World";
}
string FuncC()
{
return "Hello";
}
But now lets change FuncC into something that operates asynchronously. It returns immediately but its return value is not available until later, it calls a callback method when done that carries the result as parameter. The problem for FuncB is that it wants to return a value but can't until the async operation of FuncC has completed. Rather than have FuncB block the thread, we need to turn FuncB into an asynchronous operation in the same manner as FuncC. This whole process needs to bubble all the way to the event. It becomes:-
void Button_Click(object sender, EventArgs e)
{
FuncA(() =>
{
//Do other stuff after FuncA has completed
});
}
void FuncA(Action callback)
{
FuncB(result =>
{
//Do stuff with result
// then finally
callback();
});
}
void FuncB(Action<string> returnResult)
{
FuncC(result => returnResult(result + " World"));
}
void FuncC(Action<string> returnResult)
{
Dispatcher.BeginInvoke(() => returnResult("Hello"));
}
This pattern will do when there is only one actual asynchronous operation involved. Things start to get really funky when you have a series of actual async calls to make within the same operation. To avoid excessive callback nesting a little framework help is needed. I call mine the AsyncOperationService which you can read about here.
Part of my Silverlight application requires data from three service requests. Up until now I've been chaining the requests so as one completes the other starts... until the end of the chain where I do what I need to do with the data.
Now, I know thats not the best method(!). I've been looking at AutoResetEvent (link to MSDN example) to thread and then synchronize the results but cannot seem to get this to work with async service calls.
Does anyone have any reason to doubt this method or should this work? Code samples gratefully received!
Take a look at this example:
Will fire Completed event and print 'done' to Debug Output once both services returned.
Key thing is that waiting for AutoResetEvents happens in background thread.
public partial class MainPage : UserControl
{
public MainPage()
{
InitializeComponent();
Completed += (s, a) => { Debug.WriteLine("done"); };
wrk.DoWork += (s, a) =>
{
Start();
};
wrk.RunWorkerAsync();
}
public event EventHandler Completed;
private void Start()
{
auto1.WaitOne();
auto2.WaitOne();
Completed(this, EventArgs.Empty);
}
public AutoResetEvent auto1 = new AutoResetEvent(false);
public AutoResetEvent auto2 = new AutoResetEvent(false);
BackgroundWorker wrk = new BackgroundWorker();
private void Button_Click(object sender, RoutedEventArgs e)
{
ServiceReference1.Service1Client clien = new SilverlightAsyncTest.ServiceReference1.Service1Client();
clien.DoWorkCompleted += new EventHandler<SilverlightAsyncTest.ServiceReference1.DoWorkCompletedEventArgs>(clien_DoWorkCompleted);
clien.DoWork2Completed += new EventHandler<SilverlightAsyncTest.ServiceReference1.DoWork2CompletedEventArgs>(clien_DoWork2Completed);
clien.DoWorkAsync();
clien.DoWork2Async();
}
void clien_DoWork2Completed(object sender, SilverlightAsyncTest.ServiceReference1.DoWork2CompletedEventArgs e)
{
Debug.WriteLine("2");
auto1.Set();
}
void clien_DoWorkCompleted(object sender, SilverlightAsyncTest.ServiceReference1.DoWorkCompletedEventArgs e)
{
Debug.WriteLine("1");
auto2.Set();
}
}
It could be done using the WaitHandle in the IAsyncResult returned by each async method.
The code is simple. In Silverlight I just do 10 service calls that will add an item to a ListBox. I'll wait until all the service calls end to add another message to the list (this has to run in a different thread to avoid blocking the UI). Also note that adding items to the list have to be done through the Dispatcher since they will modify the UI. There're a bunch of lamdas, but it's easy to follow.
public MainPage()
{
InitializeComponent();
var results = new ObservableCollection<string>();
var asyncResults = new List<IAsyncResult>();
resultsList.ItemsSource = results;
var service = new Service1Client() as Service1;
1.To(10).Do(i=>
asyncResults.Add(service.BeginDoWork(ar =>
Dispatcher.BeginInvoke(() => results.Add(String.Format("Call {0} finished: {1}", i, service.EndDoWork(ar)))),
null))
);
new Thread(()=>
{
asyncResults.ForEach(a => a.AsyncWaitHandle.WaitOne());
Dispatcher.BeginInvoke(() => results.Add("Everything finished"));
}).Start();
}
Just to help with the testing, this is the service
public class Service1
{
private const int maxMilliSecs = 500;
private const int minMillisSecs = 100;
[OperationContract]
public int DoWork()
{
int millisSecsToWait = new Random().Next(maxMilliSecs - minMillisSecs) + minMillisSecs;
Thread.Sleep(millisSecsToWait);
return millisSecsToWait;
}
}