I am attempting to close a stream coming from an http request using Retrofit and rxjava, either because it timedOut, or because I need to change details that went into the request. Both appear to work perfectly, as when I cancel subscription I get the doOnCancel debug message and when doOnNext is completed I get the doOnTerminate message. I also do not receive inputLines from multiple threads. However, my thread count rises every single time either of the above actions happen. It appears that responsebody.close is not releasing their resources and therefore the thread is not dying (I also have gotten error messages along the lines of "OKHTTP leaked. did you close youre responseBody?")
Does anyone have any suggestions?
public boolean closeSubscription() {
flowableAlive = false;
subscription.cancel();
return true;
}
public void subscribeToFlowable() {
streamFlowable.observeOn(Schedulers.newThread()).subscribeOn(Schedulers.newThread())
.doOnTerminate(() -> log.debug("TERMINATED")).doOnCancel(() -> log.debug("FLOWABLE CANCELED"))
.subscribe(new Subscriber<ResponseBody>() {
#Override
public void onSubscribe(Subscription s) {
subscription = s;
subscription.request(Long.MAX_VALUE);
}
#Override
public void onNext(ResponseBody responseBody) {
log.debug("onNext called");
String inputLine;
try (InputStream inputStream = responseBody.byteStream()) {
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
while (flowableAlive && ((inputLine = br.readLine()) != null)) {
log.debug("stream receive input line for thread " + name);
log.debug(inputLine);
}
} catch (IOException e) {
log.debug("error occurred");
log.debug(e.getMessage());
}
}
#Override
public void onError(Throwable t) {
log.debug("error");
flowableAlive = false;
}
#Override
public void onComplete() {
log.debug("completed");
closeSubscription();
flowableAlive = false;
}
});
}
The result of subscribe() is Disposable object. You should store it as a filed and call Disposable.dispose() on it later as shown here:
https://proandroiddev.com/disposing-on-android-the-right-way-97bd55cbf970
Tour OkHttp call will be interrupted properly because dispose() interrupts thread on which the call runs and OkHttp checks regularly if Thread was interrupted to stop transfer when that happened - it's called cooperative cancelling/interruption.
I'm trying to use hazelcast ScheduledExecutorService to execute some periodic tasks. I'm using hazelcast 3.8.1.
I start one node and then the other, and the tasks are distributed between both nodes and properly executed.
If I shutdown the first node, then the second one will start to execute the periodic tasks that were previously on the first node.
The problem is that, if I stop the second node instead of the first, then its tasks are not rescheduled to the first one. This happens even if I have more nodes. If I shutdown the last node to receive tasks, those tasks are lost.
The shutdown is always done with ctrl+c
I've created a test application, with some sample code from hazelcast examples and with some pieces of code I've found on the web. I start two instances of this app.
public class MasterMember {
/**
* The constant LOG.
*/
final static Logger logger = LoggerFactory.getLogger(MasterMember.class);
public static void main(String[] args) throws Exception {
Config config = new Config();
config.setProperty("hazelcast.logging.type", "slf4j");
config.getScheduledExecutorConfig("scheduler").
setPoolSize(16).setCapacity(100).setDurability(1);
final HazelcastInstance instance = Hazelcast.newHazelcastInstance(config);
Runtime.getRuntime().addShutdownHook(new Thread() {
HazelcastInstance threadInstance = instance;
#Override
public void run() {
logger.info("Application shutdown");
for (int i = 0; i < 12; i++) {
logger.info("Verifying whether it is safe to close this instance");
boolean isSafe = getResultsForAllInstances(hzi -> {
if (hzi.getLifecycleService().isRunning()) {
return hzi.getPartitionService().forceLocalMemberToBeSafe(10, TimeUnit.SECONDS);
}
return true;
});
if (isSafe) {
logger.info("Verifying whether cluster is safe.");
isSafe = getResultsForAllInstances(hzi -> {
if (hzi.getLifecycleService().isRunning()) {
return hzi.getPartitionService().isClusterSafe();
}
return true;
});
if (isSafe) {
System.out.println("is safe.");
break;
}
}
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
threadInstance.shutdown();
}
private boolean getResultsForAllInstances(
Function<HazelcastInstance, Boolean> hazelcastInstanceBooleanFunction) {
return Hazelcast.getAllHazelcastInstances().stream().map(hazelcastInstanceBooleanFunction).reduce(true,
(old, next) -> old && next);
}
});
new Thread(() -> {
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
IScheduledExecutorService scheduler = instance.getScheduledExecutorService("scheduler");
scheduler.scheduleAtFixedRate(named("1", new EchoTask("1")), 5, 10, TimeUnit.SECONDS);
scheduler.scheduleAtFixedRate(named("2", new EchoTask("2")), 5, 10, TimeUnit.SECONDS);
scheduler.scheduleAtFixedRate(named("3", new EchoTask("3")), 5, 10, TimeUnit.SECONDS);
scheduler.scheduleAtFixedRate(named("4", new EchoTask("4")), 5, 10, TimeUnit.SECONDS);
scheduler.scheduleAtFixedRate(named("5", new EchoTask("5")), 5, 10, TimeUnit.SECONDS);
scheduler.scheduleAtFixedRate(named("6", new EchoTask("6")), 5, 10, TimeUnit.SECONDS);
}).start();
new Thread(() -> {
try {
// delays init
Thread.sleep(20000);
while (true) {
IScheduledExecutorService scheduler = instance.getScheduledExecutorService("scheduler");
final Map<Member, List<IScheduledFuture<Object>>> allScheduledFutures =
scheduler.getAllScheduledFutures();
// check if the subscription already exists as a task, if so, stop it
for (final List<IScheduledFuture<Object>> entry : allScheduledFutures.values()) {
for (final IScheduledFuture<Object> objectIScheduledFuture : entry) {
logger.info(
"TaskStats: name {} isDone() {} isCanceled() {} total runs {} delay (sec) {} other statistics {} ",
objectIScheduledFuture.getHandler().getTaskName(), objectIScheduledFuture.isDone(),
objectIScheduledFuture.isCancelled(),
objectIScheduledFuture.getStats().getTotalRuns(),
objectIScheduledFuture.getDelay(TimeUnit.SECONDS),
objectIScheduledFuture.getStats());
}
}
Thread.sleep(15000);
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}).start();
while (true) {
Thread.sleep(1000);
}
// Hazelcast.shutdownAll();
}
}
And the task
public class EchoTask implements Runnable, Serializable {
/**
* serialVersionUID
*/
private static final long serialVersionUID = 5505122140975508363L;
final Logger logger = LoggerFactory.getLogger(EchoTask.class);
private final String msg;
public EchoTask(String msg) {
this.msg = msg;
}
#Override
public void run() {
logger.info("--> " + msg);
}
}
I'm I doing something wrong?
Thanks in advance
-- EDIT --
Modified (and updated above) the code to use log instead of system.out. Added logging of task statistics and fixed usage of the Config object.
The logs:
Node1_log
Node2_log
Forgot to mention that I wait until all the task are running in the first node before starting the second one.
Bruno, thanks for reporting this, and it really is a bug. Unfortunately it was not so obvious with multiple nodes as it is with just two. As you figured by your answer its not losing the task, but rather keep it cancelled after a migration. Your fix, however is not safe because a Task can be cancelled and have null Future at the same time, eg. when you cancel the master replica, the backup which never had a future, just gets the result. The fix is very close to what you did, so in the prepareForReplication() when in migrationMode we avoid setting the result. I will push a fix for that shortly, just running a few more tests. This will be available in master and later versions.
I logged an issue with your finding, if you don't mind, https://github.com/hazelcast/hazelcast/issues/10603 you can keep track of its status there.
I was able to do a quick fix for this issue by changing the ScheduledExecutorContainer class of the hazelcast project (used 3.8.1 source code), namely the promoteStash() method. Basically I've added a condition for the case were we task was cancelled on a previous migration of data.
I don't now the possible side effects of this change, or if this is the best way to do it!
void promoteStash() {
for (ScheduledTaskDescriptor descriptor : tasks.values()) {
try {
if (logger.isFinestEnabled()) {
logger.finest("[Partition: " + partitionId + "] " + "Attempt to promote stashed " + descriptor);
}
if (descriptor.shouldSchedule()) {
doSchedule(descriptor);
} else if (descriptor.getTaskResult() != null && descriptor.getTaskResult().isCancelled()
&& descriptor.getScheduledFuture() == null) {
// tasks that were already present in this node, once they get sent back to this node, since they
// have been cancelled when migrating the task to other node, are not rescheduled...
logger.fine("[Partition: " + partitionId + "] " + "Attempt to promote stashed canceled task "
+ descriptor);
descriptor.setTaskResult(null);
doSchedule(descriptor);
}
descriptor.setTaskOwner(true);
} catch (Exception e) {
throw rethrow(e);
}
}
}
I have got a REST-API in my Java Web Application. This has a method to take orders from a customer's Android app (client) and send (after a bunch of tasks, like price calculating etc.) a response back to the client.
#POST
#Path("order")
#Produces({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
public OrderResponse takeOrder(OrderRequest request
) throws IOException {
OrderResponse response = new OrderResponse();
String token = request.getTokenString();
CustomerSession session = sessionPool.getSession(token);
if (session != null) {
OrderHeader order = new OrderHeader();
order.setFkOrderHeaderCustomerID(session.getFkCustomerID());
order.setOrderCreationDate(new Date());
Tasks as getting the session for authentication etc. have to be done synchronously, sure. 'Cause the response for the clients depends on it's success or failure.. So far so good.
At the end of this method the client gets an email about the state of his order request.
Email email = EmailGenerator.createOrderEmail(order);
try {
emailService.send(email);
} catch (MessagingException ex) {
Logger.getLogger(CustomerREST.class.getName()).log(Level.SEVERE, null, ex);
}
response.setStatus(OrderStatusEnum.SUCCESS);
} else {
response.setStatus(OrderStatusEnum.TOKEN_INVALID);
}
return response;
}
This sometimes takes up to a few seconds for which the client has to wait for the response. That hurts.
Is there any way to send the response and do that email stuff in the background?
Thread mailingThread = new Thread() {
#Override
public void run() {
try {
Email email = EmailGenerator.createOrderEmail(order);
emailService.send(email);
} catch (MessagingException | IOException ex) {
Logger.getLogger(CustomerREST.class.getName()).log(Level.SEVERE, null, ex);
}
}
};
mailingThread.start();
Thaks Kyle! This seems to do what I attempted!
We are developing normal bar code scanner app for windows surface tablets. In onNavigatedTo method of page we are invoking camera..
var devices = await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture);
await App.camera.InitializeAsync(new MediaCaptureInitializationSettings
{
VideoDeviceId = devices[1].Id
});
cap.Source = App.camera;
cap.Visibility = Visibility.Visible;
await App.camera.StartPreviewAsync();
It is working fine.
Now App goes to background and comes back to foreground. But we are unable to invoke camera
again on resume using same code
await App.camera.StartPreviewAsync();
How to do it?
You need to use events such as Activated and VisibilityChanged :
private MediaCapture m_mediaCaptureMgr;
public MainPage()
{
this.InitializeComponent();
Window.Current.Activated+=Current_Activated;
Window.Current.VisibilityChanged+= Current_VisibilityChanged;
}
And then Initialize and StartPreview your camera on Current_Activated :
private async void Current_Activated(object sender, WindowActivatedEventArgs e)
{
try
{
m_mediaCaptureMgr = new Windows.Media.Capture.MediaCapture();
await m_mediaCaptureMgr.InitializeAsync();
cap.Source = m_mediaCaptureMgr;
await m_mediaCaptureMgr.StartPreviewAsync();
}
catch (Exception exception)
{
new MessageDialog("Unable to start the video capture.").ShowAsync();
}
}
And Stop Camera on Current_VisibilityChanged :
private async void Current_VisibilityChanged(object sender, VisibilityChangedEventArgs e)
{
try
{
await m_mediaCaptureMgr.StopPreviewAsync();
}
catch (Exception exception)
{
}
}
I have a panel with a JTabbedpane and in every tab you can set parameters to execute a query. When one query is busy retrieving his data from the database, you can already open a new tab to set the new parameters. To avoid overload on the database only one query may be executed at once. But when you click execute the program must remember which queries to execute in the right order. During the execution a loader icon is shown and the GUI may not be frozen, because there is a stop button you can click to stop the execution.
I used a swingworker to avoid the GUI from blocking while executing the query and that works fine. But now I want to prevent the next query to start before the previous has finished. In a model, common for the whole panel, I initialized a semaphore: private final Semaphore semaphore = new Semaphore(1, true);
This is the code which starts the swingworker (I've added println commands to see which is started, stopped or finished)
private void doStoredQuery() {
try {
semaphore.acquire();
System.out.println(queryName + "started");
worker.execute();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
And this is my swingworker (initializeWorker() is called from the constructor of the main class):
private SwingWorker<StoredQueryDataModel, Integer> initializeWorker() {
worker = new SwingWorker<StoredQueryDataModel, Integer>() {
#Override
protected StoredQueryDataModel doInBackground() throws Exception {
try {
StoredQueryDataModel dataModel = null;
publish(0);
try {
dataModel = new StoredQueryDataModel(queryRunner, ldbName, queryName, params);
} catch (S9SQLException e) {
//
} catch (Throwable e) {
showErrorMessage(e);
}
return dataModel;
}
finally {
semaphore.release();
System.out.println(queryName + "finished");
}
}
#Override
protected void process(List<Integer> chunks) {
//ignore chunks, just reload loader icon
panel.repaint();
}
#Override
protected void done() {
String error;
try {
result = get();
error = null;
} catch (Exception e) {
error = e.getMessage();
}
if(result == null) {
semaphore.release();
System.out.println(queryName + " stopped");
}
if(error == null) {
// process result
}
else {
showErrorMessage(new Throwable(error));
}
}
};
return worker;
}
I've tried putting the acquire and release on other positions in the code, but nothing seems to work. I am bot in Swingworker and sempahores quite new... Can someone help?
I have found the problem: the semaphore had to be a static variable. In my code there were as many semaphores as there are tabs, which caused them to run at the same time instead of sequentially.