Azure function return unique sequence number - multithreading

I am new to Azure. I would like to create a function that returns the sequence number. I have created a function using Thread mutex to lock the sequence number. I tested the below code with around 10k parallel requests. The problem is I am getting duplicates sequence number when doing the testing, mutex is not working. I am not sure what to do to avoid duplication instead generate running number for each request
Public class MySharedMutexCounter {
public static long count = 0;
public static Mutex ObjMutex = new Mutex(false,"SeqGenerator");
}
public long GetSequnceNo(){
long seqId = -1;
try{
MySharedMutexCounter.ObjMutex.waitOne();
seqId = ++MySharedMutexCounter.count;
if(seqId > 100){
MySharedMutexCounter.count = 0;
seqId = ++MySharedMutexCounter.count;
}
return seqId;
}finally{
MySharedMutexCounter.ObjMutex.RelaseMutex();
}
return -1;
}

Thing is, an azure function can scale to multiple instances running on different machines so you need a distributed lock of some kind or another way to guarantee there won't be concurrent access to the state.
How about using a Durable Entity? It is basically a piece of state that can be accessed by a Durable Function and operations against the state are performed in a safe way:
To prevent conflicts, all operations on a single entity are guaranteed to execute serially, that is, one after another.
(source)
A durable entity is like a distributed object, so other instances of the function will use the same entity.
The Developer Guide demonstrates a nice example using a counter. Kind of fits your scenario.

Hi #Peter Bons I tried the below code but taking lot of time. May be something wrong in my code. Is it possible to get the value in a fraction of second bcos I shd return the value less than a second.
[FunctionName("FunctionOrchestrator")]
public static async Task<int> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
int currentValue = -1;
var input = context.GetInput<CounterParameter>();
if (input != null && !string.IsNullOrWhiteSpace(input.OperationName))
{
var entityId = new EntityId("Counter", "myCounter");
// Perform the requested operation on the entity
currentValue = await context.CallEntityAsync<int>(entityId, input.OperationName);
}
return currentValue;
}
[FunctionName("Counter")]
public static int Counter([EntityTrigger] IDurableEntityContext ctx, ILogger log)
{
log.LogInformation($"Request for operation {ctx.OperationName} on entity.");
switch (ctx.OperationName.Trim().ToLowerInvariant())
{
case "increment":
ctx.SetState(ctx.GetState<int>() + 1);
break;
}
// Return the latest value
return ctx.GetState<int>();
}
[FunctionName("AutoIncrement")]
public static async Task<HttpResponseMessage> HttpAutoIncrement(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestMessage req,
[DurableClient] IDurableOrchestrationClient starter,
[DurableClient] IDurableEntityClient client,
ILogger log)
{
// Function input comes from the request content.
var input = new CounterParameter { OperationName = "Increment" };
string instanceId = await starter.StartNewAsync("FunctionOrchestrator", input);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
await starter.WaitForCompletionOrCreateCheckStatusResponseAsync(req, instanceId);
var entityId = new EntityId("Counter", "myCounter");
try
{
// An error will be thrown if the counter is not initialised.
var stateResponse = await client.ReadEntityStateAsync<int>(entityId);
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(stateResponse.EntityState.ToString())
};
}
catch (System.NullReferenceException)
{
return new HttpResponseMessage(HttpStatusCode.NotFound)
{
Content = new StringContent("Counter is not yet initialised. " +
"Initialise it by calling increment or decrement HTTP Function.")
};
}
}

Related

How can I turn a synchronous structure to asynchronous on ASP.NET CORE

I have tree layer of classes to get data from database and serve it within an Action. I get data from database old way, using SqlParameter classes.
I need to/want to reconstruct this methods to asynchronous methods. What truly I want is to learn how to make these synchronous methods to Asynchronous. And make them work different threads and use them across entire app.
What I don't want to do is to Use Ado.Net or using ready methods like HttpClient.GetAsync
I read this question: Calling async methods from non-async code but couldn't apply to my structure and couldn't be sure if it is work properly, avoids deadlocks.
My old structure is like that:
My Action in My BlockController:
public ActionResult Index()
{
return View(new BlockDb().Select());
}
The Select Method in BlockDb class:
public List<Block> Select()
{
SqlDataReader dr = DbJobs.ExecuteReader("BlockSelect");
List<Block> list = new List<Block>();
Block cs;
while (dr.Read())
{
cs = new Block();
cs.ID = Convert.ToInt32(dr["ID"]);
cs.Name= Convert.ToString(dr["Name"]);
cs.Dt = Convert.ToDateTime(dr["Dt"]);
cs.Ok = Convert.ToBoolean(dr["Ok"]);
list.Add(cs);
}
dr.Close();
dr.Dispose();
return list;
}
And the last one ExecuteReader Method in DbJobs static class:
public static SqlDataReader ExecuteReader(string ProcName, params SqlParameter[] prmtr)
{
SqlDataReader Result = null;
SqlConnection connect = new SqlConnection(cnn);
try
{
SqlCommand command = new SqlCommand(ProcName, connect);
command.CommandType = CommandType.StoredProcedure;
foreach (SqlParameter item in prmtr)
command.Parameters.Add(item);
if (connect.State == ConnectionState.Closed)
connect.Open();
Result = command.ExecuteReader(CommandBehavior.CloseConnection);
//connect.Close();
//connect.Dispose();
}
catch { }
return Result;
}
I can't be sure bu the result of an Action could be like that:
public async Task<ActionResult> Index()
{
return View(await new BlockDb().SelectAsync());
}
How can I achive that?
To convert code to async, it's best to start at the lowest level and work your way up. So, starting with DbJobs, change every method to its asynchronous equivalent and await the result:
public static SqlDataReader ExecuteReader(string ProcName, params SqlParameter[] prmtr)
{
SqlDataReader Result = null;
SqlConnection connect = new SqlConnection(cnn);
try
{
SqlCommand command = new SqlCommand(ProcName, connect);
command.CommandType = CommandType.StoredProcedure;
foreach (SqlParameter item in prmtr)
command.Parameters.Add(item);
if (connect.State == ConnectionState.Closed)
await connect.OpenAsync(CancellationToken.None);
Result = await command.ExecuteReaderAsync(CommandBehavior.CloseConnection);
}
catch { }
return Result;
}
This will give you a compiler error telling you to change your DbJobs.ExecuteReader signature. So, if you do what the compiler error tells you to do, you'll end up with:
public static async Task<SqlDataReader> ExecuteReaderAsync(string ProcName, params SqlParameter[] prmtr)
Now you'll get compiler errors for all the old code that called DbJobs.ExecuteReader, e.g., BlockDb.Select. So, change those to use asynchronous methods, too:
public List<Block> Select()
{
SqlDataReader dr = await DbJobs.ExecuteReaderAsync("BlockSelect");
List<Block> list = new List<Block>();
Block cs;
while (await dr.ReadAsync(CancellationToken.None))
{
cs = new Block();
cs.ID = Convert.ToInt32(dr["ID"]);
cs.Name= Convert.ToString(dr["Name"]);
cs.Dt = Convert.ToDateTime(dr["Dt"]);
cs.Ok = Convert.ToBoolean(dr["Ok"]);
list.Add(cs);
}
dr.Close();
dr.Dispose();
return list;
}
Again, you'll get a compiler error telling you how to change BlockDb.Select:
public async Task<List<Block>> SelectAsync()
And finally, you'll do the same for Index:
public async Task<ActionResult> Index()
{
return View(await new BlockDb().SelectAsync());
}

do we need to synchronise the DB calls if multithreading is involved?

I have a scenario where two threads invoke a method and this method generated a sequence using postgres nextval(test_sequence).
test_sequence is initailly assigned to 1.
public String createNotification() {
logger.info("createNotification ENTRY");
Future<String> futRes = this.threadPool.submit(new Callable<String>() {
#Override
public String call() {
String notificationID = getNotificationId();//DB CALL TO GENERATE THE NEXT SEQUENCE.
boolean isInsertSuccess = notificationDaoService.insertNotificationIntoDB(notificationID);
if (isInsertSuccess == true) {
return notificationID;
} else {
return null;
}
}
});
try {
return futRes.get(5, TimeUnit.SECONDS);
} catch (Exception e) {
logger.error("Issue while getting value from future with exception :", e);
return null;
}
}
So in the above snippet, getNotificationId() will generate the sequence and insertNotificationIntoDB() wil insert the generated notification id to the table.
I some times observing the primary key voilation exception when multiple threads try to invoke createNotification().
So i am thinking to synchronise the db calls as mentioned below,
synchronised(object)
{
String notificationID = getNotificationId();
boolean isInsertSuccess = notificationDaoService.insertNotificationIntoDB(notificationID);
}
is this solution ok?
and also i want to ask if i can generalise that if multiple threads are accessing a function and if that function has DB calls that does basic CRUD, then all the DB calls needs to be synchronised. Is this right inference?

C# Parse how to wait until Query returns a value

I'm trying to retrieve user data from Parse (xamarin.ios using c#). I'm using an async method with await. My challenge is,each time I navigate to the tableView in the app, which should populate the user data in question,the table is always empty.
I would like to wait until the results have been returned before proceeding with the other portion of code.I have tried to use the ContinueWith() function but constantly ran into a build error -
Cannot implicitly convert type 'void' to System.Collections.Generic.IEnumerable<Parse.ParseObject>
My Questions:
Is this the best way to wait for the result?
How do I solve the build error?
Here is my current implementation:
public async void retrieveData(string username)
{
try
{
this.requests.ClearRequests();
refreshed = false;
var query = ParseObject.GetQuery("Requests").WhereEqualTo("username", username);
IEnumerable<ParseObject> results = await query.FindAsync().ContinueWith(t =>{
if(results != null)
{
foreach(ParseObject parseObject in results)
{
UserRequest request = new UserRequest();
request.objectId = parseObject.ObjectId;
request.make = parseObject.Get<string> ("item1");
request.model = parseObject.Get<string> ("item2");
request.year = parseObject.Get<string> ("item3");
request.userName = parseObject.Get<string> ("username");
this.requests.addRequest (request);
}
refreshed = true;
}
});
}
catch(ParseException e) {
Console.WriteLine (e.Message + e.StackTrace);
}
}
You shouldn't need a ContinueWith...that's what the await should handle.
await waits on a Task and then brings back the result with the proper return type. ContinueWith returns a Task, so you would have to grab the Result from the task to make it usable.
For more on this type of thing, you may want to check out Difference between await and ContinueWith
You can try something like this.
public async void retrieveData(string username, )
{
try
{
this.requests.ClearRequests();
refreshed = false;
var query = ParseObject.GetQuery("Requests").WhereEqualTo("username", username);
IEnumerable<ParseObject> results = await query.FindAsync();
if(results != null)
{
foreach(ParseObject parseObject in results)
{
UserRequest request = new UserRequest();
request.objectId = parseObject.ObjectId;
request.make = parseObject.Get<string> ("item1");
request.model = parseObject.Get<string> ("item2");
request.year = parseObject.Get<string> ("item3");
request.userName = parseObject.Get<string> ("username");
this.requests.addRequest (request);
}
refreshed = true;
}
//This is your refresh method for your TableView
this.RefreshTableView();
//or, if in iOS
NSNotificationCenter.DefaultCenter.PostNotificationName("resultsRetrieved", null);
}
catch(ParseException e) {
Console.WriteLine (e.Message + e.StackTrace);
}
}
To show the results in the tableView, I would recommend moving the refreshing of the tableView to a separate method that gets triggered synchronously after the results have been retrieved and parsed. This is shown with the this.RefreshTableView() call above.
If in iOS on Xamarin, another option is to post a notification to the NSNotificationCenter (the Xamarin documentation for which is here). Use the PostNotificationName part seen above instead and then add an observer in the ViewControllers that you want to be dependent on the data. This is done as follows:
Make a notificationToken object:
NSObject notificationToken;
Then in your setup method (you could put this inside of your ViewDidLoad):
void Setup ()
{
notificationToken = NSNotificationCenter.DefaultCenter.AddObserver ("resultsRetrieved", RefreshData);
}
Make your RefeshData method:
void RefreshData (NSString notifString)
{
this.tableView.ReloadData();
}
And then, make sure you dispose of the notification observer when you tear down the class
void Teardown ()
{
NSNotificationCenter.DefaultCenter.RemoveObserver (notificationToken);
}
I had a similar issue so started using callbacks. I'm using them in Xamarin.Android, pretty sure they're available in Xamarin.iOS.
Method that starts the task method - Note I am passing in a method of this class as a parameter
private async void updatedData()
{
await Utils.DataTasks.getNewLiveTips(populateTipsList);
}
Method that calls for data from server
public class DataTasks
{
public static async Task getAllData(Action<IEnumerable<ParseObjects>> callback) {
var query = new ParseQuery<ParseObjects>().OrderByDescending("updatedAt").Limit(5);
IEnumerable<ParseObjects> parseTips = await query.FindAsync();
foreach (var tip in parseTips)
{
// Save data to DB if needed
}
callback(parseTips);
}
Method I passed as parameter in the first instance is now called
private void populateTipsList(IEnumerable<ParseObjects> results)
{
mAdapter = new TipAdapter(this.Activity, results);
mRecyclerView.SetAdapter(mAdapter);
refresher.Refreshing = false;
}

node.js and edge.js - continuously running task (thread)

I would like to make a running task (ListenResponse()), which continuously listens and sends data to serial port. At the same time, I would like to add another task (WriteParameter) through which I would set some parameters, which are then sent to serial port. The problem is, that now I'm using two async tasks, but the data in thread is not getting updated fast enough. It updates after few iterations of ListenResponse() task.
Pseudocode is below:
string data;
void ListenResponseSerial() {
byte b;
while (true) {
b = getSerialData(&b);
parse(b);
if (parsed) {
Respond(data);
}
}
}
public async Task<object> ListenResponse(dynamic input) {
(new Task(ListenResponseSerial)).Start();
}
public async Task<object> Write(dynamic input) {
IDictionary<string, object> payload = (IDictionary<string, object>)input;
data = (string) payload["data"];
return null;
}
Thank you very much for your help.

Silverlight - limit application to one WCF call at a time

Silverlight can only send a certain number of simultaneous WCF requests at a time. I am trying to serialize the requests that a particular section of my application is performing because I don't need them to run concurrently.
The problem is as follows (summary below):
"WCF proxies in Silverlight applications use the SynchronizationContext of the thread from which the web service call is initiated to schedule the invocation of the async event handler when the response is received. When the web service call is initiated from the UI thread of a Silverlight application, the async event handler code will also execute on the UI thread."
http://tomasz.janczuk.org/2009/08/improving-performance-of-concurrent-wcf.html
summary: basically, if you block the thread that is calling the async method, it will never get called.
I can't figure out the right model of threading this such which would give me what I want in a reasonable way.
My only other requirement is that I don't want the UI thread to block.
As far as I can see, what should work is if the UI thread has a worker thread which queues up the calls as Action delegates, then uses an AutoResetEvent to execute a task one at a time in yet another worker thread. There are two problems:
1) The thread that calls async can't block, because then async will never get called. In fact, if you put that thread into a wait loop, I've noticed it doesn't get called either
2) You need a way to signal from the completed method of the async call that it is done.
Sorry that was so long, thanks for reading. Any ideas?
I have used a class that i build on my own to execute load operations synchronous. With the class you can register multiple load operations of diffrent domaincontexts and then execute them one by one. You can provide an Action to the constructor of the class that gets called, when all operations are finished (successful or failed).
Here´s the code of the class. I think it´s not complete and you have to change it to match your expectations. Maybe it can help you in your situation.
public class DomainContextQueryLoader {
private List<LoadOperation> _failedOperations;
private Action<DomainContextQueryLoader> _completeAction;
private List<QueuedQuery> _pendingQueries = new List<QueuedQuery>();
public DomainContextQueryLoader(Action<DomainContextQueryLoader> completeAction) {
if (completeAction == null) {
throw new ArgumentNullException("completeAction", "completeAction is null.");
}
this._completeAction = completeAction;
}
/// <summary>
/// Expose the count of failed operations
/// </summary>
public int FailedOperationCount {
get {
if (_failedOperations == null) {
return 0;
}
return _failedOperations.Count;
}
}
/// <summary>
/// Expose an enumerator for all of the failed operations
/// </summary>
public IList<LoadOperation> FailedOperations {
get {
if (_failedOperations == null) {
_failedOperations = new List<LoadOperation>();
}
return _failedOperations;
}
}
public IEnumerable<QueuedQuery> QueuedQueries {
get {
return _pendingQueries;
}
}
public bool IsExecuting {
get;
private set;
}
public void EnqueueQuery<T>(DomainContext context, EntityQuery<T> query) where T : Entity {
if (IsExecuting) {
throw new InvalidOperationException("Query cannot be queued, cause execution of queries is in progress");
}
var loadBatch = new QueuedQuery() {
Callback = null,
Context = context,
Query = query,
LoadOption = LoadBehavior.KeepCurrent,
UserState = null
};
_pendingQueries.Add(loadBatch);
}
public void ExecuteQueries() {
if (IsExecuting) {
throw new InvalidOperationException("Executing of queries is in progress");
}
if (_pendingQueries.Count == 0) {
throw new InvalidOperationException("No queries are queued to execute");
}
IsExecuting = true;
var query = DequeueQuery();
ExecuteQuery(query);
}
private void ExecuteQuery(QueuedQuery query) {
System.Diagnostics.Debug.WriteLine("Load data {0}", query.Query.EntityType);
var loadOperation = query.Load();
loadOperation.Completed += new EventHandler(OnOperationCompleted);
}
private QueuedQuery DequeueQuery() {
var query = _pendingQueries[0];
_pendingQueries.RemoveAt(0);
return query;
}
private void OnOperationCompleted(object sender, EventArgs e) {
LoadOperation loadOperation = sender as LoadOperation;
loadOperation.Completed -= new EventHandler(OnOperationCompleted);
if (loadOperation.HasError) {
FailedOperations.Add(loadOperation);
}
if (_pendingQueries.Count > 0) {
var query = DequeueQuery();
ExecuteQuery(query);
}
else {
IsExecuting = false;
System.Diagnostics.Debug.WriteLine("All data loaded");
if (_completeAction != null) {
_completeAction(this);
_completeAction = null;
}
}
}
}
Update:
I´ve just noticed that you are not using WCF RIA Services, so maybe this class will not help your.
There are some options:
- You can take a look at the Agatha-rrsl either by inspecting the implementation of it or by just using it instead of pure wcf. The framework allows you to queue requests. You can read more here.
- Another option is to use the Reactive extension. There is a SO example here and more info here and here.
- You can try the Power Thread library from Jeffrey Richter. He describes it on his book CLR via C#. You can find the library here. This webcast gives you some info about it.
- You can always roll your own implementation. The yield statement is a good help here. Error handling makes it very difficult to get the solution right.

Resources