What's the best way to atomically modify an object in GG?
GridCache<String, Message> cache = GridGain.grid().cache("message");
Message message = cache.get(messageId);
if(message != null) {
message.setReadDate(new Date());
cache.putxAsync(messageId, message);
}
I'm hoping to replace it with atomic operation that is executed on the data node
cache.execute(messageId, new Task() {
public void execute(V value) {
value.setReadDate();
return value;
}
});
GridGain has cache.transform(...) and cache.transformAndCompute(...) methods which work exactly like you need.
Please take a look at Delta (a.k.a. Partial) Updates section in documentation.
Related
I have implemented the Azure - Offline Sync based on the documentation / Sample provided by Microsoft Sample in my Xamarin Forms Application.
In the sample / documentation provided, they are using the default Service Handler.
// Simple error/conflict handling. A real application would handle the various errors like network conditions,server conflicts and others via the IMobileServiceSyncHandler.
Since I need to implement a retry logic for 3 times if the Pull / Push fails. As per the documentation I have created a custom Service Handler(IMobileServiceSyncHandler).
Please find my code logic here.
public class CustomSyncHandler : IMobileServiceSyncHandler
{
public async Task<JObject> ExecuteTableOperationAsync(IMobileServiceTableOperation operation)
{
MobileServiceInvalidOperationException error = null;
Func<Task<JObject>> tryExecuteAsync = operation.ExecuteAsync;
int retryCount = 3;
for (int i = 0; i < retryCount; i++)
{
try
{
error = null;
var result = await tryExecuteAsync();
return result;
}
catch (MobileServiceConflictException e)
{
error = e;
}
catch (MobileServicePreconditionFailedException e)
{
error = e;
}
catch (MobileServiceInvalidOperationException e)
{
error = e;
}
catch (Exception e)
{
throw e;
}
if (error != null)
{
if(retryCount <=3) continue;
else
{
//Need to implement
//Update failed, reverting to server's copy.
}
}
}
return null;
}
public Task OnPushCompleteAsync(MobileServicePushCompletionResult result)
{
return Task.FromResult(0);
}
}
But I am not sure how to handle / revert server copy in case all the 3 retry failed.
In the TODO sample they where reverting it based on the
MobileServicePushFailedException. But which is available when we implement IMobileServiceSyncHandler.
More over if we include custom IMobileServiceSyncHandler it wont execute the code after PushAsync / PullAsync. Even the try catch wont fire in case any exception.
try
{
await this.client.SyncContext.PushAsync();
await this.todoTable.PullAsync(
//The first parameter is a query name that is used internally by the client SDK to implement incremental sync.
//Use a different query name for each unique query in your program
"allTodoItems",
this.todoTable.CreateQuery());
}
catch (MobileServicePushFailedException exc)
{
if (exc.PushResult != null)
{
syncErrors = exc.PushResult.Errors;
}
}
// Simple error/conflict handling. A real application would handle the various errors like network conditions,
// server conflicts and others via the IMobileServiceSyncHandler.
if (syncErrors != null)
{
foreach (var error in syncErrors)
{
if (error.OperationKind == MobileServiceTableOperationKind.Update && error.Result != null)
{
//Update failed, reverting to server's copy.
await error.CancelAndUpdateItemAsync(error.Result);
}
else
{
// Discard local change.
await error.CancelAndDiscardItemAsync();
}
Debug.WriteLine(#"Error executing sync operation. Item: {0} ({1}). Operation discarded.", error.TableName, error.Item["id"]);
}
}
}
Note
In my application I am only trying to achieve retry for 3 time in case any server error. I am not looking for to resolve conflicts. Thant is the reason I haven't added the code for the same.
If someone came across similar issues and resolved it please help.
Stez.
You say you aren't trying to resolve conflicts, but you need to resolve them one way or another (without telling the user what's going on, perhaps) by accepting the server version of the object or updating the client operation. Otherwise it will just keep telling you about the same conflict each time it retries the operation.
You need to have a subclass of the Microsoft.WindowsAzure.MobileServices.Sync.MobileServiceSyncHandler class, which overrides OnPushCompleteAsync() in order to handle conflicts and other errors. Let's call the class SyncHandler:
public class SyncHandler : MobileServiceSyncHandler
{
public override async Task OnPushCompleteAsync(MobileServicePushCompletionResult result)
{
foreach (var error in result.Errors)
{
await ResolveConflictAsync(error);
}
await base.OnPushCompleteAsync(result);
}
private static async Task ResolveConflictAsync(MobileServiceTableOperationError error)
{
Debug.WriteLine($"Resolve Conflict for Item: {error.Item} vs serverItem: {error.Result}");
var serverItem = error.Result;
var localItem = error.Item;
if (Equals(serverItem, localItem))
{
// Items are the same, so ignore the conflict
await error.CancelAndUpdateItemAsync(serverItem);
}
else // check server item and local item or the error for criteria you care about
{
// Cancels the table operation and discards the local instance of the item.
await error.CancelAndDiscardItemAsync();
}
}
}
Include an instance of this SyncHandler() when you initialize your MobileServiceClient:
await MobileServiceClient.SyncContext.InitializeAsync(store, new SyncHandler()).ConfigureAwait(false);
Read up on the MobileServiceTableOperationError to see other conflicts you can handle as well as its methods to allow resolving them.
The exception carries with it a copy of the server version. In my implementation of IMobileServiceSyncHandler I therefore just return error.Value and this seems to work.
A more extensive example of this kind of logic can be found in this MSDN blog.
The same author has another example where he shows how you can resolve the conflict in favour of the server copy or the client copy, here.
I am using Android camera2 to create a custom camera. The cameraDevice.close() method is slow and it makes UI freeze for 1 sec. I put it in another thread and it seems to work just fine. I want to know if this will cause some serious problem and whether there is another way to achieve this. Here is my closeCamera method:
private void closeCamera() {
boolean release = false;
try {
mCameraOpenCloseLock.acquire();
release = true;
} catch (InterruptedException e) {
release = false;
}
try {
preparing = true;
if (mCaptureSession != null) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M && mCaptureSession.isReprocessable()
|| validCameraSession) {
mCaptureSession.close();
}
mCaptureSession = null;
validCameraSession = false;
}
} catch (IllegalStateException e) {
mCaptureSession = null;
} catch (Exception e) {
mCaptureSession = null;
}
try {
new Thread(new Runnable() {
#Override
public void run() {
if (mCameraDevice != null) {
if (openCamera) {
mCameraDevice.close();
mCameraDevice = null;
}
}
}
}).start();
} catch (IllegalStateException e) {
Log.e(TAG, "closeCamera: mCaptureSession - ", e);
} catch (Exception e) {
Log.e(TAG, "closeCamera: mCaptureSession - ", e);
}
if (release) {
if (mCameraOpenCloseLock != null) {
int lock = mCameraOpenCloseLock.availablePermits();
if (lock > 1) mCameraOpenCloseLock.release(lock - 1);
else if (lock == 0) mCameraOpenCloseLock.release();
}
}
}
I think it may cause crash when mCameraDevice has not been closed but user open camera again. But it is rare case, and I am thinking of putting another check before open camera again. I don't want my UI to freeze 1 sec for it to close, is there any other way I can achieve that except putting it in seperate thread?
As Alex Cohn mentions, the recommended practice is to do all camera-related work on a separate thread from the UI.
It also takes a long time to open the camera, or create a capture session, relatively speaking, so doing those operations not on the UI thread is also a good idea.
That said, as long as you're not losing track of your own app state (so that you don't try to use a camera device you've already closed by accident, for example), there's no reason you can't mix calls to the camera device or capture session from multiple threads. The classes themselves are thread-safe.
As far as I know, such freeze with cameraDevice.close() happens on some unfortunate devices, and sometimes is cured by performing a normal system upgrade.
But this is a little consolation if this happens to you, on your device. Actually, you are kind of lucky that you can prepare a fix for that. The end-users of your app will benefit from your misfortune.
Your code looks OK, if it delivers desired improvements for you. As I explained, it may be hard to reproduce this problem on another device.
I would rather put all closeCamera() logic on the same background thread. If you provided a Handler to openCamera(), as in the official example,
manager.openCamera(mCameraId, mStateCallback, mBackgroundHandler);
then I would suggest posting all closeCamera() sequence to this mBackgroundHandler.
this is the structure of my firebase db:
/UserData
/DeviceMgmt
/Counters
/NumberOfAll:
/NumberOfSelected
/TotalDownloaded
...
/Devices
/pushId1
/uid
/toSelect=true (optional)
/downloaded
/lastDownload
/pushId2
/pushId2
...
My code:
exports.countNumberOfAllDevices = functions.database.ref('/UserData/DeviceMgmt/Devices/{pushId}').onWrite(
(change) => {
const collectionRef = change.after.ref;// /UserData/DeviceMgmt/Devices/{pushId}
const countRef = collectionRef.parent.parent.child('Counters/NumberOfAll');
let increment;
if (change.after.exists() && !change.before.exists()) {
increment = 1;
} else if (!change.after.exists() && change.before.exists()) {
increment = -1;
} else {
return null;
}
return countRef.transaction((current) => {
return (current || 0) + increment;
}).then(() => {
return console.log('counter /UserData/Counters/NumberOfAll updated.');
});
});
is based on functions-samples/child-count/ where pushed messages are replaced by my devices, but my device have children and messages in the example are childless.
My problem is:
1.
When my device is created in java one writing is object creation, another one is update of its child, so the NumberOfAll counter increments by 2.
At every changes within {pushId} device the function is run useless because that does not change the number of devices.
Should I replace countNumberOfAllDevices with
two functions:
incrementNumberOfAllDevices with onCreate (after creating device object no child is created later)
decrementNumberOfAllDevices with onDelete (before deleting device object no child is deleted earlier)
OR
go deeper with snapshot reference from
/Devices/{pushId}
to
/Devices/{pushId}/uid
when uid is never changed between creation and deletion ?
My Java code that triggers the Cloud Function:
testAddNewDeviceToDevices(String token) {
Device device = new Device( "Test", 0, token);
String deviceKey = dbRefDevices.push().getKey();
dbRefDevices.child(deviceKey).setValue(device)
.addOnFailureListener(new OnFailureListener() {
#Override public void onFailure(#NonNull Exception e) {
Log.e(TAG, "failed. Exception: ", e);
}
});
}
You should use onCreate and onDelete. They were introduced in version 1.0 of the SDK for exactly the the case you're describing. It will simplify your code drastically.
IMO, onWrite is not that useful. It's kind of a legacy trigger from pre-1.0 days.
It is easy enough in D to create a Queue type using the std.container.dlist.
I would like to have multiple threads but have them communicate with a queue, not with message passing (https://tour.dlang.org/tour/en/multithreading/message-passing). As I understand it the messages are designed to always receive data at particular points in the code; the receiving thread will block until the expected data is received.
(EDIT: I was informed about receiveTimeout but having a no timeout and just a check is really more appropriate in this case (maybe a timeout of 0?). Also I am not sure what the message API will do if multiple messages are sent before any any are received. I will have to play with that.)
void main() {
spawn(&worker, thisTid);
// This line will block until the expected message is received.
receive (
(string message) {
writeln("Received the message: ", text);
},
)
}
What I am needing is to merely receive data if there is some. Something like this:
void main() {
Queue!string queue// custom `Queue` type based on DList
spawn(&worker, queue);
while (true) {
// Go through any messages (while consuming `queue`)
for (string message; queue) {
writeln("Received a message: ", text);
}
// Do other stuff
}
}
I have tried using shared variables (https://tour.dlang.org/tour/en/multithreading/synchronization-sharing) but DMD is complaining that "Aliases to mutable thread-local data not allowed." or some other errors, depending.
How would this be done in D? Or, is there a way to use messages to do this kind of communication?
This doesn't answer the specific question but ti does clear up what I think is a misunderstanding of the message passing api...
just call receiveTimeout instead of plain receive
http://dpldocs.info/experimental-docs/std.concurrency.receiveTimeout.html
I use this:
shared class Queue(T) {
private T[] queue;
synchronized void opOpAssign(string op)(T object) if(op == "~") {
queue ~= object;
}
synchronized size_t length(){
return queue.length;
}
synchronized T pop(){
assert(queue.length, "Please check queue length, is 0");
auto first = queue[0];
queue = queue[1..$];
return first;
}
synchronized shared(T[]) consume(){
auto copy = queue;
queue = [];
return copy;
}
}
I have gotten the answer I need.
Simply put, use core.thread rather than std.concurrency. std.concurrency manages messages for you and does not allow you to manage it yourself. core.thread is what std.concurrency uses internally.
The longer answer, here is how I fully implemented it.
I have created a Queue type that is based on an Singly Linked List but maintains a pointer of the last element. The Queue also uses standard component inputRange and outputRange (or at least I think it does) per Walter Brights vision (https://www.youtube.com/watch?v=cQkBOCo8UrE).
The Queue is also built to allow one thread to write and another to read with very little mutexing internally so it should be fast.
The Queue I shared here https://pastebin.com/ddyPpLrp
A simple implementation to have a second thread read input:
Queue!string inputQueue = new Queue!string;
ThreadInput threadInput = new ThreadInput(inputQueue);
threadInput.start;
while (true) {
foreach (string value; inputQueue) {
writeln(value);
}
}
ThreadInput being defined as thus:
class ThreadInput : Thread {
private Queue!string queue;
this(Queue!string queue) {
super(&run);
this.queue = queue;
}
private void run() {
while (true) {
queue.put(readln);
}
}
}
The code https://pastebin.com/w5jwRVrL
The Queue again https://pastebin.com/ddyPpLrp
Hi i am trying to grab a value from my threading but it seem work not so find to me course i found that my code structure are unstable enough..here is my code i name my thread class as "clsThreadCount" and below is my implementation
public volatile bool Grab = false;
public volatile int count = 0;
public void Initialization(int i)
{
count = i;
}
public void Play()
{
Grab = false;
_shouldStop = false;
ThreadTest();
}
public void Stop()
{
_shouldStop = true;
workerThread.Join(1);
workerThread.Abort();
}
private void ThreadTest()
{
workerThread = new Thread(DoWork);
workerThread.Start();
while (!workerThread.IsAlive) ;
}
private void DoWork()
{
try
{
while (!_shouldStop)
{
if (Grab)
{
count++;
Grab = false;
}
}
}
catch (Exception)
{
Play();
}
finally
{
}
}
when my program(main menu) are starting to run i will trigger the initialize function at pass the parameter as 7
ObjThreadCount.Initialization(7); // count = 7
ObjThreadCount.Play(); // the thread are running
ObjThreadCount.Grab = true; // the grab equal to true, count++ are trigger
Thread.Sleep(100); // wait awhile
lblResult.Text = ObjThreadCount.count.ToString(); // sometime i can get count++ result (e.g. 8)
ObjThreadCount.Stop(); // thread stop
sometime my program can able to get a right counting from the thread but sometime are not.
i realize at my while loop implementation there are something are missing..
something like waitone or waitautoevent..can i ignore Thread.Sleep(100) ?? what are the suitable code should i add in the while loop ?
Please help me~ :S
** sorry in the first upload i forgot to write down "volatile" into the variable
thank you..
If C# (and C and java, and probably C++), you need to declare _shouldStop and Grab as volatile.