apache curator: lock revocation not working - multithreading

short: I want to create a lock with one client and release it with another; so I am trying to use revocation for that end. It isn't working. more details (and my code) are below.
any help is appreciated!
long: have a system where one thread (with its own client) sets the first lock (update), then a second thread (with a client that may or may not be the same as the original client) will set a second lock; then do some work, then release that lock, and then release the first lock this code simulates two threads by having two different clients get locks.
the second client is unable to revoke the lock from the first client, however. the 'revocation listener' is never triggered. have scoured the web and not found examples.
this code assumes that you have a zookeeper server running on your local host at port 2181
ideally, I'd also like to look up from somewhere else real quick to see if the lock is in place, but perhaps an acquire with a very short timeout (5 milliseconds) would accomplish that.
also, is it a good idea to reap away locks after releasing them? (to not clog up the system?)
thanks
-Jill
ps: also posted on the apache curator user mailing list
I'll cross-post answers if I have them.
Got one answer from the mailing list: Jordan Zimmerman
8:46 PM (13 hours ago) There are a
number of problems:
The docs are not clear on this, but makeRevocable() must be called BEFORE the lock is acquired. Please submit a Jira/PR to fix the doc.
In your test, Revoker.attemptRevoke was using the incorrect path. It must be the path of the lock, so: "Revoker.attemptRevoke(client2,
ipMutex.getLockPath());”
InterProcessMutex keeps track of the thread that owns the lock. So, the lock.release(); in your revoker won’t work. I suggest using
InterProcessSemaphoreMutex instead.
import java.util.Collection;
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.apache.curator.RetryPolicy;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.framework.recipes.locks.InterProcessMutex;
import org.apache.curator.framework.recipes.locks.RevocationListener;
import org.apache.curator.framework.recipes.locks.Revoker;
import org.apache.curator.retry.ExponentialBackoffRetry;
public class MultipleClientExample {
/*entry point
*/
public static void main(String[] args){
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
String zookeeperConnectionString = "127.0.0.1:2181";
CuratorFramework client = CuratorFrameworkFactory.newClient(zookeeperConnectionString, retryPolicy);
client.start();
try {
System.out.println("testing locks....");
InterProcessMutex ipMutex = new InterProcessMutex(client, "/mpx-updates/guid123/update");
boolean acquired = ipMutex.acquire(3, TimeUnit.SECONDS);
System.out.println("got the lock(update)?" + acquired);
RevocationListener<InterProcessMutex> rl = new MyRevocationListener();
ipMutex.makeRevocable(rl);
InterProcessMutex ipMutex2 = new InterProcessMutex(client, "/mpx-updates/guid123/swap");
boolean a2 = ipMutex2.acquire(3, TimeUnit.SECONDS);
System.out.println("got the lock(swap)?" + a2);
System.out.println("got the first lock in this process? " + ipMutex.isAcquiredInThisProcess());
// make a new client; see if it can get the lock!
CuratorFramework client2 = CuratorFrameworkFactory.newClient(zookeeperConnectionString, retryPolicy);
client2.start();
InterProcessMutex ipMutex1Retry = new InterProcessMutex(client2, "/mpx-updates/guid123/update");
boolean a3 = ipMutex1Retry.acquire(3, TimeUnit.SECONDS);
System.out.println("got the lock(retry/update) ?" + a3);
System.out.println("got the first lock in this process? " + ipMutex1Retry.isAcquiredInThisProcess());
Revoker.attemptRevoke(client2, "/mpx-updates/guid123/update");
a3 = ipMutex1Retry.acquire(3, TimeUnit.SECONDS);
System.out.println("AGAIN: got the lock(retry/update) ?" + a3);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public class MyRevocationListener implements RevocationListener<InterProcessMutex> {
/*
* (non-Javadoc)
*
* #see org.apache.curator.framework.recipes.locks.RevocationListener#revocationRequested(java.lang.Object)
*/
#Override
public void revocationRequested(InterProcessMutex lock) {
//this seems to never be called
Collection<String> participantNodes = null;
try {
System.out.println("revocation was requested....");
System.out.println("ick ick revocation requested....");
participantNodes = lock.getParticipantNodes();
lock.release();
System.out.println("revoked lock at path: " + participantNodes);
} catch (Exception e) {
System.out.println("problem revoking lock with path: " + participantNodes + "; it was not revoked");
}
}
}
}

Related

How does RecursiveAction work?

I downloaded some existing code from internet. I ran it with few modifications. In one scenario, I did not get what I was looking for. Here is the code -
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveAction;
public class MyRecursiveAction extends RecursiveAction{
private long workload = 0;
public MyRecursiveAction(long workload) {
this.workload = workload;
}
#Override
protected void compute() {
if(this.workload > 16) {
System.out.println("Splitting workload :: " + this.workload);
List<MyRecursiveAction> subtasks = new ArrayList<MyRecursiveAction>();
subtasks.addAll(createSubtasks());
for(RecursiveAction subtask : subtasks) {
subtask.fork();
}
}else {
System.out.println("Doing work myself1 " + this.workload);
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("Done it ya " + this.workload);
}
}
private List<MyRecursiveAction> createSubtasks() {
List<MyRecursiveAction> subTasks = new ArrayList<>();
MyRecursiveAction subtask1 = new MyRecursiveAction(this.workload / 2);
MyRecursiveAction subtask2 = new MyRecursiveAction(this.workload / 2);
subTasks.add(subtask1);
subTasks.add(subtask2);
return subTasks;
}
public static void main(String[] args) {
MyRecursiveAction myRecursiveAction = new MyRecursiveAction(24);
ForkJoinPool forkJoinPool = new ForkJoinPool(4);
forkJoinPool.invoke(myRecursiveAction);
}
}
Check the following excerpt -
System.out.println("Doing work myself1 " + this.workload);
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("Done it ya " + this.workload);
I added a sleep of 1 second and then I printed another statement. However if I run the code, I don't see that statement getting printed. I don't understand why. Why will that not get printed ? In fact the result of the execution is -
Splitting workload :: 24
Doing work myself1 12
Doing work myself1 12
I was expecting the following line as well - "Done it ya"..
Make workload static and volatile:
private static volatile long workload = 0;
Loose the this.workload for just workload.
Alter if statement to:
if(workload > 0) {
Then you will get to "Done it ya".
I have found the reason as to why the last line was not getting printed.This is because fork works in asynchronous way. So its altogether a different thread which sleeps for some time. In asynchronous programming, there is no need for the main thread to wait for the response to come back unless we via code add some constructs. In this case by the time thread wakes up after 1 second, the main thread is already over.
To force the main thread to wait for execution of other threads, we need to use JOIN.
ForkJoinTask.join(): This method blocks until the result of the computation is done.
So if I add the following block
for(RecursiveAction subtask : subtasks) {
subtask.join();
}
the main thread waits and we get all the expected lines printed on the console.

My app crashes when I try to send a request to the server

Well, it actually works pretty well on my android studio simulator but when I try to run it on my phone it just crashes.
I just want to send a number to the server and get a response with the data that I need to that number. so this is my code which do that:
thread = new Thread() {
#Override
public void run() {
//server stuff
try {
//Connecting
if(!userClass.equals("")) {
Log.i(debugString, "Attempting to connect to server");
socket = new Socket(hostname, portnumber);
Log.i(debugString, "Connection established!");
BufferedWriter bw = new BufferedWriter((new OutputStreamWriter(socket.getOutputStream())));
bw.write("" + userClass);
bw.newLine();
bw.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
input = br.readLine();
}
} catch (IOException e) {
Log.e(debugString, e.getMessage());
} finally {
threadComplete = true;
}
}
};
thread.start();
while(!threadComplete)
continue;
then I just use this thread whenever I want to get the updated info for my request like that:
String getUserClass = userClass;
if(!getUserClass.equals(""))
{
threadComplete = false;
userClass = getUserClass;
thread.start();
while (!threadComplete)
continue;
changes.setText(input);
}
else Toast.makeText(this, "Error, choose your class", Toast.LENGTH_SHORT).show();
BTW, in the end of every thread (on the emulator because on my phone it crashes) I get a message:
Skipped 91 frames! The application may be doing too much work on its main thread.
and I have another problem, I also use IntentService to run my app service on the background, and obviously I don't want it to run constantly forever, so I made a loop which contains at the end of each loop a wait() command, but the problem is that when I set the time to wait for longer than 3000 milliseconds or so, the service crashes.
my code for the background service:
synchronized (this) {
int count = 0;
while (count<4) {
try {
wait(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
if (notifications && !userClass.equals("")) {
new Thread() {
#Override
public void run() {
//server stuff
try {
//Connecting
if (!userClass.equals("")) {
Log.i("debug", "Attempting to connect to server");
socket = new Socket(hostname, portnumber);
Log.i("debug", "Connection established!");
BufferedWriter bw = new BufferedWriter((new OutputStreamWriter(socket.getOutputStream())));
bw.write("" + userClass);
bw.newLine();
bw.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
input = br.readLine();
}
} catch (IOException e) {
Log.e("debug", e.getMessage());
} finally {
complete = true;
}
}
}.start();
while (!complete)
continue;
Toast.makeText(this, "" + input, Toast.LENGTH_SHORT).show();
NotificationManager mNotifyMgr = (NotificationManager) getSystemService(NOTIFICATION_SERVICE);
NotificationCompat.Builder mBuilder =
new NotificationCompat.Builder(this)
.setSmallIcon(R.drawable.chanka)
.setContentTitle("ביטול שיעורים: ")
.setContentText(input);
mNotifyMgr.notify(mNotificationId, mBuilder.build());
mNotificationId++;
Toast.makeText(this, "" + input, Toast.LENGTH_SHORT).show();
count++;
}
}
}
This following piece of code is the culprit -
while (!threadComplete)
continue;
You are kind of putting the main thread on a long loop. Android does not allow that. The general construct in these kind of use cases is this -
Step 1 - Show a progress dialog to the user indicating that you are
doing something important and user needs to wait till that is
complete. Show some meaningful text in the progress dialog which makes
sense to the user.
Step 2 - Start a async connection to the server. There are lot of
options in Android to do this. But for your purpose AsyncTask might
be useful. Connect to your server, fetch and parse data in the
doInBackground method of AsyncTask and once the task is complete,
let onPostExecute publish the same to the Main thread.
Step 3 - Once you get back the result from the Async task, you may
dismiss the progress dialog and continue with whatever you were doing.
Please note that the main thread should not be blocked at any time. This is the event handling thread of the app and handles all events (User initiated or system initiated). If the thread is blocked, you get the kind of error you are seeing now. Specifically in your case, Android system is not able to do some draw operations because of the while loop.
Create a new Asynctask and run the socket establisment codes inside it :)
socket = new Socket(hostname, portnumber);

Issue with Mutex getting lock before it was released and how to distribute locks equally?

The method GetItemSearchResponse will be called by multiple console application. But I wanted to call this method one by one. So I applied mutex. So that this method will be locked for other threads.
public class AWSItemSearchClient : IDisposable
{
// the name of the global mutex;
private const string MutexName = "FAA9569-7DFE-4D6D-874D-19123FB16CBC-8739827-[SystemSpecicString]";
private Mutex _globalMutex;
private bool _owned = false;
//This method call should be synchronized
public ItemSearchResponse GetItemSearchResponse(ItemSearch itemSearch)
{
ItemSearchResponse response = null;
RequestAgain:
try
{
_globalMutex = new Mutex(true, MutexName, out _owned);
while (!_owned)
{
// did not get the mutex, wait for it.
_owned = _globalMutex.WaitOne(2000);
}
AWSECommerceServicePortTypeClient amazonClient = new AWSECommerceServicePortTypeClient();
response = amazonClient.ItemSearch(itemSearch);
Thread.Sleep(2000);
AppLogger.ExamineThreadAcquisitionLog("Lock acquired by Thread " + Process.GetCurrentProcess().Id + " Exe: " + AppDomain.CurrentDomain);
}
catch (Exception ex)
{
AppLogger.ExamineThreadAcquisitionLog("Error in Item Search request : " + ex.Message);
goto RequestAgain;
}
return response;
}
public void Dispose()
{
if (_owned)
{
_globalMutex.ReleaseMutex();
Thread.Sleep(1500);
AppLogger.ExamineThreadAcquisitionLog("Lock released by Thread " + Process.GetCurrentProcess().Id + " Exe: " + AppDomain.CurrentDomain);
}
_globalMutex = null;
}
}
But the log which I am getting is not convincing the proper execution of code.
----
Lock acquired by Thread 9916 Exe: Name:FB.ABC.vshost.exe
There are no context policies.
12/28/2016 5:27:06 PM
----
Lock Released by Thread 17396 Exe: Name:FB.XYZ.vshost.exe
There are no context policies.
12/28/2016 5:27:06 PM
----
Lock Released by Thread 9916 Exe: Name:FB.ABC.vshost.exe
There are no context policies.
12/28/2016 5:27:09 PM
----
Lock acquired by Thread 17396 Exe: Name:FB.XYZ.vshost.exe
There are no context policies.
12/28/2016 5:27:10 PM
----
Lock acquired by Thread 9916 Exe: Name:FB.ABC.vshost.exe
There are no context policies.
It should be like one thread got lock and it should be released by that thread first then the lock should be acquired by other thread.
I am calling this method in console apps as following :
using (var client = new AWSItemSearchClient())
{
response = client.GetItemSearchResponse(itemSearch);
}
Also the above method is called by 3 console apps but the lock is acquired always by two threads, 1 and 9. How can I distribute locks equally?

Winforms updates with high performance

Let me setup this question with some background information, we have a long running process which will be generating data in a Windows Form. So, obviously some form of multi-threading is going to be needed to keep the form responsive. But, we also have the requirement that the form updates as many times per second while still remaining responsive.
Here is a simple test example using background worker thread:
void bw_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
int reportValue = (int)e.UserState;
label1.Text = reportValue;
//We can put this.Refresh() here to force repaint which gives us high repaints but we lose
//all other responsiveness with the control
}
void bw_DoWork(object sender, DoWorkEventArgs e)
{
for (int x = 0; x < 100000; x++)
{
//We could put Thread.Sleep here but we won't get highest performance updates
bw.ReportProgress(0, x);
}
}
Please see the comments in the code. Also, please don't question why I want this. The question is simple, how do we achieve the highest fidelity (most repaints) in updating the form while maintaining responsiveness? Forcing the repaint does give us updates but we don't process windows messages.
I have also try placing DoEvents but that produces stack overflow. What I need is some way to say, "process any windows messages if you haven't lately". I can see also that maybe a slightly different pattern is needed to achieve this.
It seems we need to handle a few issues:
Updating the Form through the non UI thread. There are quite a few solution to this problem such as invoke, synchronization context, background worker pattern.
The second problem is flooding the Form with too many updates which blocks the message processing and this is the issue around which my question really concerns. In most examples, this is handles trivially by slowing down the requests with an arbitrary wait or only updating every X%. Neither of these solutions are approriate for real-world applications nor do they meet the maximum update while responsive criteria.
Some of my initial ideas on how to handle this:
Queue the items in the background worker and then dispatch them in a UI thread. This will ensure every item is painted but will result in lag which we don't want.
Perhaps use TPL
Perhaps use a timer in the UI thread to specify a refresh value. In this way, we can grab the data at the fastest rate that we can process. It will require accessing/sharing data across threads.
Update, I've updated to use a Timer to read a shared variable with the Background worker thread updates. Now for some reason, this method produces a good form response and also allows the background worker to update about 1,000x as fast. But, interestingly it only 1 millisecond accurate.
So we should be able to change the pattern to read the current time and call the updates from the bw thread without the need for the timer.
Here is the new pattern:
//Timer setup
{
RefreshTimer.SynchronizingObject = this;
RefreshTimer.Elapsed += RefreshTimer_Elapsed;
RefreshTimer.AutoReset = true;
RefreshTimer.Start();
}
void bw_DoWork(object sender, DoWorkEventArgs e)
{
for (int x = 0; x < 1000000000; x++)
{
//bw.ReportProgress(0, x);
//mUiContext.Post(UpdateLabel, x);
SharedX = x;
}
}
void RefreshTimer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
label1.Text = SharedX.ToString();
}
Update And here we have the new solution that doesn't require the timer and doesn't block the thread! We achieve a high performance in calculations and fidelity on the updates with this pattern. Unfortunately, ticks TickCount is only 1 MS accurate, however we can run a batch of X updates per MS to get faster then 1 MS timing.
void bw_DoWork(object sender, DoWorkEventArgs e)
{
long lastTickCount = Environment.TickCount;
for (int x = 0; x < 1000000000; x++)
{
if (Environment.TickCount - lastTickCount > 1)
{
bw.ReportProgress(0, x);
lastTickCount = Environment.TickCount;
}
}
}
There is little point in trying to report progress any faster than the user can keep track of it.
If your background thread is posting messages faster than the GUI can process them, (and you have all the symtoms of this - poor GUI resonse to user input, DoEvents runaway recursion), you have to throttle the progress updates somehow.
A common approach is to update the GUI using a main-thread form timer at a rate sufficiently small that the user sees an acceptable progress readout. You may need a mutex or critical section to protect shared data, though that amy not be necessary if the progress value to be monitored is an int/uint.
An alternative is to strangle the thread by forcing it to block on an event or semaphore until the GUI is idle.
The UI thread should not be held for more than 50ms by a CPU-bound operation taking place on it ("The 50ms Rule"). Usually, the UI work items are executed upon events, triggered by user input, completion of an IO-bound operation or a CPU-bound operation offloaded to a background thread.
However, there are some rare cases when the work needs to be done on the UI thread. For example, you may need to poll a UI control for changes, because the control doesn't expose proper onchange-style event. Particularly, this applies to WebBrowser control (DOM Mutation Observers are only being introduced, and IHTMLChangeSink doesn't always work reliably, in my experience).
Here is how it can be done efficiently, without blocking the UI thread message queue. A few key things was used here to make this happen:
The UI work tasks yields (via Application.Idle) to process any pending messages
GetQueueStatus is used to decide on whether to yield or not
Task.Delay is used to throttle the loop, similar to a timer event. This step is optional, if the polling needs to be as precise as possible.
async/await provide pseudo-synchronous linear code flow.
using System;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace WinForms_21643584
{
public partial class MainForm : Form
{
EventHandler ContentChanged = delegate { };
public MainForm()
{
InitializeComponent();
this.Load += MainForm_Load;
}
// Update UI Task
async Task DoUiWorkAsync(CancellationToken token)
{
try
{
var startTick = Environment.TickCount;
var editorText = this.webBrowser.Document.Body.InnerText;
while (true)
{
// observe cancellation
token.ThrowIfCancellationRequested();
// throttle (optional)
await Task.Delay(50);
// yield to keep the UI responsive
await ApplicationExt.IdleYield();
// poll the content for changes
var newEditorText = this.webBrowser.Document.Body.InnerText;
if (newEditorText != editorText)
{
editorText = newEditorText;
this.status.Text = "Changed on " + (Environment.TickCount - startTick) + "ms";
this.ContentChanged(this, EventArgs.Empty);
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
async void MainForm_Load(object sender, EventArgs e)
{
// navigate the WebBrowser
var documentTcs = new TaskCompletionSource<bool>();
this.webBrowser.DocumentCompleted += (sIgnore, eIgnore) => documentTcs.TrySetResult(true);
this.webBrowser.DocumentText = "<div style='width: 100%; height: 100%' contentEditable='true'></div>";
await documentTcs.Task;
// cancel updates in 10 s
var cts = new CancellationTokenSource(20000);
// start the UI update
var task = DoUiWorkAsync(cts.Token);
}
}
// Yield via Application.Idle
public static class ApplicationExt
{
public static Task<bool> IdleYield()
{
var idleTcs = new TaskCompletionSource<bool>();
if (IsMessagePending())
{
// register for Application.Idle
EventHandler handler = null;
handler = (s, e) =>
{
Application.Idle -= handler;
idleTcs.SetResult(true);
};
Application.Idle += handler;
}
else
idleTcs.SetResult(false);
return idleTcs.Task;
}
public static bool IsMessagePending()
{
// The high-order word of the return value indicates the types of messages currently in the queue.
return 0 != (GetQueueStatus(QS_MASK) >> 16 & QS_MASK);
}
const uint QS_MASK = 0x1FF;
[System.Runtime.InteropServices.DllImport("user32.dll")]
static extern uint GetQueueStatus(uint flags);
}
}
This code is specific to WinForms. Here is a similar approach for WPF.

Qt thread call issues

Please help me. I am struck-up with thread concept. Actually my problem : I want to display the cities List in the combobox. I am getting cities list from the webservice. I am using thread for update the combo box value after webserice call finished.
Here I can call the webservice. But I couldn't get the Reply.
I am using the following code.
MainWindow.cpp:
MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
CGNetwork *cgNetwork = new CGNetwork();
ui->setupUi(this);
renderThread = new RenderThread(cgNetwork);
renderThread->start();
connect(renderThread,SIGNAL(finished()),this,SLOT(initControls()));
}
void MainWindow::initControls()
{
CGMainWindowUtility *pointer = CGMainWindowUtility::instance();
QStringList cityitems;
cityitems <<tr("All");
cityitems.append(pointer->getCityList());
QStringListModel *cityModel = new QStringListModel(cityitems, this);
ui->cityComboBox->setModel(cityModel);
}
RenderThread.cpp:
RenderThread::RenderThread(CGNetwork *cgnetwork)
{
cityUrl = "http://112.138.3.181/City/Cities";
categoryUrl = "http://112.138.3.181/City/Categories";
}
void RenderThread::run()
{
qDebug()<< "THREAD Started";
CGNetwork *cgnetworks = new CGNetwork();
cgnetworks->getCityList(cityUrl);
}
CGNetwork.cpp:
void CGNetwork ::getCityList(const QUrl url)
{
cityGuideNetworkAccessManager = new QNetworkAccessManager(this);
qDebug()<<"connection";
connect(cityGuideNetworkAccessManager, SIGNAL(finished(QNetworkReply*)),
this, SLOT(parseCityList()));
const QNetworkRequest cityRequest(url);
cityReply= cityGuideNetworkAccessManager->get(cityRequest);
connect(cityReply, SIGNAL(error(QNetworkReply::NetworkError)),
this, SLOT(slotError()));
}
void CGNetwork::parseCityList()
{
qDebug()<<"Parsing";
cgParser = new CGJsonParser();
cgParser->CityRead(cityReply);
}
Since QNetworkAccessManager works asynchronously, there's no need for a separate thread. You can call getCityList directly from your main thread and it won't block.
I think your slots weren't called because your QThread::run returned before any of the work its been doing had a chance to complete, since getCityList just initiated an http request without waiting for it (because QNetworkAccessManager::get doesn't block like I said above).
Also as a side note, your slots aren't getting the same parameters as their corresponding signals, I don't remember if Qt supports this.

Resources