I'm developing an android app that receives and processes mail messages. The app must be connected to an IMAP server and keep the connection alive, so it can see and process new mail messages instantly (mails contains json data from a mail api server). The app have two modes, manual and live connection. Here is some of my code:
class Idler {
Thread th;
volatile Boolean isIdling=false;
boolean shouldsync=false;//we need to see if we have unseen mails
Object idleLock;
Handler handler=new Handler();
IMAPFolder inbox;
public boolean keppAliveConnection;//keep alive connection, or manual mode
//This thread should keep the idle connection alive, or in case it's set to manual mode (keppAliveConnection=false) get new mail.
Thread refreshThread;
synchronized void refresh()
{
if(isIdling)//if already idling, just keep connection alive
{
refreshThread =new Thread(new Runnable() {
#Override
public void run() {
try {
inbox.doCommand(new IMAPFolder.ProtocolCommand() {
#Override
public Object doCommand(IMAPProtocol protocol) throws ProtocolException {
//Why not noop?
//any call to IMAPFolder.doCommand() will trigger waitIfIdle, this
//issues a "DONE" command and waits for idle to return(ideally with a DONE server response).
// So... I think NOOP is unnecessary
//protocol.simpleCommand("NOOP",null); I'm not issuing noop due to what I said ^
//PD: if connection was broken, then server response will never arrive, and idle will keep running forever
//without triggering messagesAdded event any more :'( I see any other explanation to this phenomenon
return null;
}
});
} catch (MessagingException e) {
e.printStackTrace();
}
}
},"SyncThread");
refreshThread.start();
}
else
{
getNewMail();//If manual mode keppAliveConnection=false) get the new mail
}
}
public Idler()
{
th=new Thread(new Runnable() {
#SuppressWarnings("InfiniteLoopStatement")
#Override
public void run() {
while (true)
{
try {
if(refreshThread !=null && refreshThread.isAlive())
refreshThread.interrupt();//if the refresher thread is active: interrupt. I thing this is not necessary at this point, but not shure
initIMAP();//initializes imap store
try {
shouldsync=connectIMAP()||shouldsync;//if was disconnected or ordered to sync: needs to sync
}
catch (Exception e)
{
Thread.sleep(5000);//if can't connect: wait some time and throw
throw e;
}
shouldsync=initInbox()||shouldsync;//if inbox was null or closed: needs to sync
if(shouldsync)//if needs to sync
{
getNewMail();//gets new unseen mail
shouldsync=false;//already refreshed, clear sync "flag"
}
while (keppAliveConnection) {//if sould keep idling "forever"
synchronized (idleLock){}//MessageCountListener may be doing some work... wait for it
isIdling = true; //set isIdling "flag"
handler.removeCallbacksAndMessages(null);//clears refresh scheduled tasks
handler.postDelayed(new Runnable() {
#Override
public void run() {
refresh();
}
},1200000);//Schedule a refresh in 20 minutes
inbox.idle();//start idling
if(refreshThread !=null && refreshThread.isAlive())
refreshThread.interrupt();//if the refresher thread is active: interrupt. I thing this is not necessary at this point, but not shure
handler.removeCallbacksAndMessages(null);//clears refresh scheduled tasks
isIdling=false;//clear isIdling "flag"
if(shouldsync)
break;//if ordered to sync... break. The loop will handle it upstairs.
synchronized (idleLock){}//MessageCountListener may be doing some work... wait for it
}
}
catch (Exception e) {
//if the refresher thread is active: interrupt
//Why interrupt? refresher thread may be waiting for idle to return after "DONE" command, but if folder was closed and throws
//a FolderClosedException, then it could wait forever...., so... interrupt.
if (refreshThread != null && refreshThread.isAlive())
refreshThread.interrupt();
handler.removeCallbacksAndMessages(null);//clears refresh scheduled tasks
}
}
}
},"IdlerThread");
th.start();
}
private synchronized void getNewMail()
{
shouldsync=false;
long uid=getLastSeen();//get last unprocessed mail
SearchTerm searchTerm=new UidTerm(uid,Long.MAX_VALUE);//search from las processed message to the las one.
IMAPSearchOperation so=new IMAPSearchOperation(searchTerm);
try {
so.run();//search new messages
final long[] is=so.uids();//get unprocessed messages count
if (is.length > 0) {//if some...
try {
//there are new messages
IMAPFetchMessagesOperation fop=new IMAPFetchMessagesOperation(is);
fop.run();//fetch new messages
if(fop.messages.length>0)
{
//process fetched messages (internally sets the last seen uid value & delete some...)
processMessages(fop.messages);
}
inbox.expunge();//expunge deleted messages if any
}
catch (Exception e)
{
//Do something
}
}
else
{
//Do something
}
}
catch (Exception e)
{
//Do something
}
}
private synchronized void initIMAP()
{
if(store==null)
{
store=new IMAPStore(mailSession,new URLName("imap",p.IMAPServer,p.IMAPPort,null,p.IMAPUser,p.IMAPPassword));
}
}
private boolean connectIMAP() throws MessagingException {
try {
store.connect(p.IMAPServer, p.IMAPPort, p.IMAPUser, p.IMAPPassword);
return true;
}
catch (IllegalStateException e)
{
return false;
}
}
//returns true if the folder was closed or null
private synchronized boolean initInbox() throws MessagingException {
boolean retVal=false;
if(inbox==null)
{//if null, create. This is called after initializing store
inbox = (IMAPFolder) store.getFolder("INBOX");
inbox.addMessageCountListener(countListener);
retVal=true;//was created
}
if(!inbox.isOpen())
{
inbox.open(Folder.READ_WRITE);
retVal=true;//was oppened
}
return retVal;
}
private MessageCountListener countListener= new MessageCountAdapter() {
#Override
public void messagesAdded(MessageCountEvent ev) {
synchronized (idleLock)
{
try {
processMessages(ev.getMessages());//process the new messages, (internally sets the last seen uid value & delete some...)
inbox.expunge();//expunge deleted messajes if any
} catch (MessagingException e) {
//Do something
}
}
}
};
}
The problem is: Sometimes when the user is refreshing or the app auto-refreshes, in the Alive Connection mode, one or both of this conditions keeps my app from getting new messages. This is from the javamail source code.
1: The IdlerThread enters monitor state in:
//I don't know why sometimes it enters monitor state here.
private synchronized void throwClosedException(ConnectionException cex)
throws FolderClosedException, StoreClosedException {
// If it's the folder's protocol object, throw a FolderClosedException;
// otherwise, throw a StoreClosedException.
// If a command has failed because the connection is closed,
// the folder will have already been forced closed by the
// time we get here and our protocol object will have been
// released, so if we no longer have a protocol object we base
// this decision on whether we *think* the folder is open.
if ((protocol != null && cex.getProtocol() == protocol) ||
(protocol == null && !reallyClosed))
throw new FolderClosedException(this, cex.getMessage());
else
throw new StoreClosedException(store, cex.getMessage());
}
2: The "refresherThread" enters wait state in:
void waitIfIdle() throws ProtocolException {
assert Thread.holdsLock(messageCacheLock);
while (idleState != RUNNING) {
if (idleState == IDLE) {
protocol.idleAbort();
idleState = ABORTING;
}
try {
// give up lock and wait to be not idle
messageCacheLock.wait();//<-----This is the line is driving me crazy.
} catch (InterruptedException ex) { }
}
}
As one of both of this threads "stops" running (wait & monitor state) my app is useless when reach this condition. In my country the mobile data network is very unstable, slow & expensive(GSM) So it must be failure resilient and take care about every transferred bit.
I guess the problem arises when the connection silently fails and the refresherThread starts to do its job. It issues a DONE command if idle is active, but, as the connection is gone, when idle tries to throw a FolderClosedException, one or both threads gets locked indefinitely.
So, my question is: Why is this situation arising and how to prevent it? How can I keep the idle loop securely running without getting locked?
I've tried a lot of things till exhaustion with no results.
Here are some threads I've read without getting a solution to my problem. In my country internet is EXTREMELY expensive too, so I can't research as much as I want, nor list all the urls I've visited looking for information.
JavaMail: Keeping IMAPFolder.idle() alive
JavaMail: Keeping IMAPFolder.idle() alive
Javamail : Proper way to issue idle() for IMAPFolder
Please, excuse my english. Any suggestion will be greatly appreciated. I've heard about this site strictness, so please be gentle, I'm new over here.
Be sure to set the timeout properties to make sure you don't hang waiting for a dead connection or server.
Instead of issuing a nop command directly, you should call Folder.isOpen or Folder.getMessageCount; they'll issue the nop command if needed.
If the folder is closed asynchronously (FolderClosedException), you'll need to restart the idle loop.
Related
I am attempting to close a stream coming from an http request using Retrofit and rxjava, either because it timedOut, or because I need to change details that went into the request. Both appear to work perfectly, as when I cancel subscription I get the doOnCancel debug message and when doOnNext is completed I get the doOnTerminate message. I also do not receive inputLines from multiple threads. However, my thread count rises every single time either of the above actions happen. It appears that responsebody.close is not releasing their resources and therefore the thread is not dying (I also have gotten error messages along the lines of "OKHTTP leaked. did you close youre responseBody?")
Does anyone have any suggestions?
public boolean closeSubscription() {
flowableAlive = false;
subscription.cancel();
return true;
}
public void subscribeToFlowable() {
streamFlowable.observeOn(Schedulers.newThread()).subscribeOn(Schedulers.newThread())
.doOnTerminate(() -> log.debug("TERMINATED")).doOnCancel(() -> log.debug("FLOWABLE CANCELED"))
.subscribe(new Subscriber<ResponseBody>() {
#Override
public void onSubscribe(Subscription s) {
subscription = s;
subscription.request(Long.MAX_VALUE);
}
#Override
public void onNext(ResponseBody responseBody) {
log.debug("onNext called");
String inputLine;
try (InputStream inputStream = responseBody.byteStream()) {
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
while (flowableAlive && ((inputLine = br.readLine()) != null)) {
log.debug("stream receive input line for thread " + name);
log.debug(inputLine);
}
} catch (IOException e) {
log.debug("error occurred");
log.debug(e.getMessage());
}
}
#Override
public void onError(Throwable t) {
log.debug("error");
flowableAlive = false;
}
#Override
public void onComplete() {
log.debug("completed");
closeSubscription();
flowableAlive = false;
}
});
}
The result of subscribe() is Disposable object. You should store it as a filed and call Disposable.dispose() on it later as shown here:
https://proandroiddev.com/disposing-on-android-the-right-way-97bd55cbf970
Tour OkHttp call will be interrupted properly because dispose() interrupts thread on which the call runs and OkHttp checks regularly if Thread was interrupted to stop transfer when that happened - it's called cooperative cancelling/interruption.
I noticed that in a couple implementations of the ble scanner that the scanning is stopped and started again after a given time period, like every 20 seconds.
Here for example here is a scanner class firing up the scanner in a separate thread. You can see in the start() method that the thread is put to sleep for a period of time, and the scanner is then stopped and restarted:
public class BleScanner extends Thread {
private final BluetoothAdapter bluetoothAdapter;
private final BluetoothAdapter.LeScanCallback mLeScanCallback;
private volatile boolean isScanning = false;
public BleScanner(BluetoothAdapter adapter, BluetoothAdapter.LeScanCallback callback) {
bluetoothAdapter = adapter;
mLeScanCallback = callback;
}
public boolean isScanning() {
return isScanning;
}
public void startScanning() {
synchronized (this) {
isScanning = true;
start();
}
}
public void stopScanning() {
synchronized (this) {
isScanning = false;
bluetoothAdapter.stopLeScan(mLeScanCallback);
}
}
#Override
public void run() {
try {
// Thread goes into an infinite loop
while (true) {
synchronized (this) {
// If there is not currently a scan in progress, start one
if (!isScanning) break;
bluetoothAdapter.startLeScan(mLeScanCallback);
}
sleep(Constants.SCAN_PERIOD); // Thread sleeps before stopping the scan
// stop scan
synchronized (this) {
bluetoothAdapter.stopLeScan(mLeScanCallback);
}
// restart scan on next iteration of infinite while loop
}
} catch (InterruptedException ignore) {
} finally { // Just in case there is an error, the scan will be stopped
bluetoothAdapter.stopLeScan(mLeScanCallback);
// The finally block always executes when the try block exits. This ensures that the
// finally block is executed even if an unexpected exception occurs.
}
}
}
Are there any benefits to stopping and restarting the scanner? Why not just let the scan continue perpetually?
There are advantages. On some devices, you would only see an advertisement from a device once per scan. On some you would see all advertisements. Also, restarting the scan cleans up some low-level stuff and is generally better than keeping the scanner active all the time.
I am confused with Async feature introduced in Servlet 3.0 spec
From Oracle site (http://docs.oracle.com/javaee/7/tutorial/doc/servlets012.htm):
To create scalable web applications, you must ensure that no threads
associated with a request are sitting idle, so the container can use
them to process new requests.
There are two common scenarios in which a thread associated with a
request can be sitting idle.
1- The thread needs to wait for a resource to become available or process data before building the response. For example, an application
may need to query a database or access data from a remote web service
before generating the response.
2- The thread needs to wait for an event before generating the response. For example, an application may have to wait for a JMS
message, new information from another client, or new data available in
a queue before generating the response.
The first item happens a lot (nearly always, we always query db or call a remote webservice to get some data). And calling an external resource will always consume some time.
Does it mean that we should ALWAYS use servelt async feature for ALL our servelts and filter ?!
I can ask this way too, if I write all my servelts and filters async, will I lose anything (performance)?!
If above is correct the skeleton of ALL our servlets will be:
public class Work implements ServletContextListener {
private static final BlockingQueue queue = new LinkedBlockingQueue();
private volatile Thread thread;
#Override
public void contextInitialized(ServletContextEvent servletContextEvent) {
thread = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
ServiceFecade.doBusiness();
AsyncContext context;
while ((context = queue.poll()) != null) {
try {
ServletResponse response = context.getResponse();
PrintWriter out = response.getWriter();
out.printf("Bussiness done");
out.flush();
} catch (Exception e) {
throw new RuntimeException(e.getMessage(), e);
} finally {
context.complete();
}
}
} catch (InterruptedException e) {
return;
}
}
}
});
thread.start();
}
public static void add(AsyncContext c) {
queue.add(c);
}
#Override
public void contextDestroyed(ServletContextEvent servletContextEvent) {
thread.interrupt();
}
}
I have a panel with a JTabbedpane and in every tab you can set parameters to execute a query. When one query is busy retrieving his data from the database, you can already open a new tab to set the new parameters. To avoid overload on the database only one query may be executed at once. But when you click execute the program must remember which queries to execute in the right order. During the execution a loader icon is shown and the GUI may not be frozen, because there is a stop button you can click to stop the execution.
I used a swingworker to avoid the GUI from blocking while executing the query and that works fine. But now I want to prevent the next query to start before the previous has finished. In a model, common for the whole panel, I initialized a semaphore: private final Semaphore semaphore = new Semaphore(1, true);
This is the code which starts the swingworker (I've added println commands to see which is started, stopped or finished)
private void doStoredQuery() {
try {
semaphore.acquire();
System.out.println(queryName + "started");
worker.execute();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
And this is my swingworker (initializeWorker() is called from the constructor of the main class):
private SwingWorker<StoredQueryDataModel, Integer> initializeWorker() {
worker = new SwingWorker<StoredQueryDataModel, Integer>() {
#Override
protected StoredQueryDataModel doInBackground() throws Exception {
try {
StoredQueryDataModel dataModel = null;
publish(0);
try {
dataModel = new StoredQueryDataModel(queryRunner, ldbName, queryName, params);
} catch (S9SQLException e) {
//
} catch (Throwable e) {
showErrorMessage(e);
}
return dataModel;
}
finally {
semaphore.release();
System.out.println(queryName + "finished");
}
}
#Override
protected void process(List<Integer> chunks) {
//ignore chunks, just reload loader icon
panel.repaint();
}
#Override
protected void done() {
String error;
try {
result = get();
error = null;
} catch (Exception e) {
error = e.getMessage();
}
if(result == null) {
semaphore.release();
System.out.println(queryName + " stopped");
}
if(error == null) {
// process result
}
else {
showErrorMessage(new Throwable(error));
}
}
};
return worker;
}
I've tried putting the acquire and release on other positions in the code, but nothing seems to work. I am bot in Swingworker and sempahores quite new... Can someone help?
I have found the problem: the semaphore had to be a static variable. In my code there were as many semaphores as there are tabs, which caused them to run at the same time instead of sequentially.
I have a windows service developed in C#. On it's Start method I have a initialization such as:
Task _backgroundTask = null;
CancellationTokenSource _backgroundCancellationSource = null;
protected override void OnStart(string[] args)
{
......
_backgroundCancellationSource = new CancellationTokenSource();
CancellationToken token = backgroundCancellationSource.Token;
_backgroundTask = new Task(() => BackgroundFoldersProcessing(token), token, TaskCreationOptions.LongRunning);
.......
}
Now the method BackgroundFoldersProcessing looks like this:
void BackgroundFoldersProcessing(CancellationToken token)
{
while (true)
{
try
{
if (token.IsCancellationRequested)
{
return;
}
DoSomeWork()
}
catch (Exception ex)
{
.........
}
}
}
Now, the Stop method is as follows:
protected override void OnStop()
{
.................
_backgroundCancellationSource.Cancel();
_backgroundTask.Wait();
_backgroundCancellationSource.Dispose();
_backgroundTask.Dispose();
_backgroundTask = null;
_backgroundCancellationSource = null;
.................
}
Now the problem is when I try to stop the service in a middle of processing, the Wait method of _backgroundTask would not stop the service until and unless the DoSomeWork() method inside the BackgroundFoldersProcessing gets completed, the Windows Service would not stop.
Is there any way, though which I can stop the service and the execution of _backgroundTask would be terminated, even though the DoSomeWork() method gets completed/executed or not? I have also tried token.ThrowIfCancellationRequested() in BackgroundFoldersProcessing method, but that also did not worked. I want that whenever I try to Stop the service from Service Control Manager (SCM), the service should be stopped immediately and the __backgroundTask should stop executing the BackgroundFoldersProcessing method and be terminated as well. How can I achieve this?
You can try use ThreadAbortException:
defining the thread:
ThreadStart threadDelegate = new ThreadStart(BackgroundFoldersProcessing);
Thread thread_ = new Thread(threadDelegate);
thread_.Start();
Add catch to BackgroundFoldersProcessing
catch (ThreadAbortException e)
{
return;
}
and when you want to shut it down use:
thread_.Abort();
thread_.Join();
Then when Abort() will be called ThreadAbortException will be thrown.