I'm running into some tough to explain oddities when trying to retrieve messages from my local storage queues. I'm fairly sure this isn't happening in production using actual Azure queues.
The line in particular causing this issue is:
msgs = await priorityQueue.GetMessagesAsync(Settings.NumberOfMessagesToGet, visibilityTimeSpan, null, null);
Which will just do nothing and doesn't seem to ever return. However, replacing it with:
msgs = priorityQueue.GetMessages(Settings.NumberOfMessagesToGet, visibilityTimeSpan, null, null);
Returns back once it's done and seems fine.
Am I using the await here right? Any ideas why this isn't working?
I'm using the Windows Azure SDK 2.8, with the Windows Azure Storage Emulator 4.2.0.0, in case it gives any clues.
The Azure Storage Queues sample on GitHub demonstrates how to use async patterns with the .NET Client library from a console application:
https://github.com/Azure-Samples/storage-queue-dotnet-getting-started
Notice that at the top level of the program, the "Wait()" method is used:
ProcessBatchOfMessagesAsync(queue).Wait();
Related
I'm calling this code from Windows IoT Core on RPi3 and getting this error. I'm trying to send a message to a blob in Azure. However, it only does it once and silently fails.
The Code:
s_deviceClient = DeviceClient.Create(s_iotHubUri, new
DeviceAuthenticationWithRegistrySymmetricKey(s_myDeviceId, s_deviceKey),
TransportType.Mqtt);
await s_deviceClient.SendEventAsync(message);
The Error:
microsoft azure devices client "I/O Error Occurred".
I was told that a using SetPolicy/ExponentialBackoff might work but I haven't been successful in implementing it. I'm calling it from a static class if that means anything.
I found a solution with a dynamic class, but I'd have to change the architecture of my app to use it.
https://azureiot.wordpress.com/2018/05/03/azure-iot-hub-device-sdk-retry-policy/
I have a very long running process which is hosted using Azure Function App (though it's not recommended for long running processes) targeting v2.0. Earlier it was targeting v1.0 runtime so I didn't face any function timeout issue.
But now after updating the runtime to target v2.0, I am not able to find any way to set the function timeout to Infinite as it was in case of v1.0.
Can someone please help me out on this ?
From your comments it looks like breaking up into smaller functions or using something other than functions isn't an option for you currently. In such case, AFAIK you can still do it with v2.0 as long as you're ready to use "App Service Plan".
The max limit of 10 minutes only applies to "Consumption Plan".
In fact, documentation explicitly suggests that if you have functions that run continuously or near continuously then App Service Plan can be more cost-effective as well.
You can use the "Always On" setting. Read about it on Microsoft Docs here.
Azure Functions scale and hosting
Also, documentation clearly states that default value for timeout with App Service plan is 30 minutes, but it can be set to unlimited manually.
Changes in features and functionality
UPDATE
From our discussion in comments, as null value isn't working for you like it did in version 1.x, please try taking out the "functionTimeout" setting completely.
I came across 2 different SO posts mentioning something similar and the Microsoft documentation text also says there is no real limit. Here are the links to SO posts I came across:
SO Post 1
SO Post 2
One way of doing it is to implement Eternal orchestrations from Durable Functions. It allows you to implement an infinite loop with dynamic intervals. Of course, you need to slightly modify your code by adding support for the stop/start function at any time (you must pass the state between calls).
[FunctionName("Long_Running_Process")]
public static async Task Run(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
var initialState = context.GetInput<object>();
var state = await context.CallActivityAsync("Run_Long_Running_Process", initialState);
if (state == ???) // stop execution when long running process is completed
{
return;
}
context.ContinueAsNew(state);
}
You cannot set an Azure Function App timeout to infinite. I believe the longest any azure function app will consistently run is 10 minuets. As you stated Azure functions are not meant for long running processes. You may need find a new solution for your app, especially if you will need to scale up the app at all in the future.
I have a webjob running in azure that is processing data sent to an event hub.
In the eventprocessor I want to save information to a SQL server. To make sure that everything is inserted correctly I want to use transactions.
When I run the code locally everything works perfect. But when running in Azure nothing happens, no error is thrown.
What I have read it should be possible to use TransactionScope. This example code below is not working.
using (TransactionScope scope = new TransactionScope())
{
dataImportDao.StartProcessingMessage(mappedMessage);
scope.Complete();
}
Any suggestions how to solve it or if I should go with a different approach is very appreciated.
I am running an azure webjobs SDK console application (continuous) with the recommended setup:
public static void ProcessQueueMessage([QueueTrigger("logqueue")] string logMessage, TextWriter logger)
The azure queue I am running against has ~6000 messages in it and I am running the web-job locally, as a console application.
The problem I'm having is that the processing randomly stops after processing between zero and ~30 messages. The console stays open, but no more console messages are displayed.
For example, it might just process 2 messages:
Executing: 'Functions.ProcessQueueMessage' - Reason: 'New queue message detected on 'QueueName'.'
Executed: 'Functions.ProcessQueueMessage' (Succeeded)
Executing: 'Functions.ProcessQueueMessage' - Reason: 'New queue message detected on 'QueueName'.'
Executed: 'Functions.ProcessQueueMessage' (Succeeded)
And then, nothing. There doesn't seem to be anything wrong with my internet connection and I can't trace the issues down to any particular messages.
Has anyone else had issues with this SDK?
Update:
I made sure that I was using the right versions of all of the dependencies by removing the nuget packages and then re-running install-package Microsoft.Axure.Webjobs. I am now using webjobs version 1.1.0 which has pulled in version 4.3 of azure storage.
As recommended by Matthew, I have pulled down the source code for azure webjobs to determine where the process is freezing up. Once the freez-up occurs, I pause execution and checked the running threads for what I believe is the culprit within Microsoft.Azure.WebJobs.Host.CompositeTraceWriter
protected virtual void InvokeTextWriter(TraceEvent traceEvent)
{
if (_innerTextWriter != null)
{
string message = traceEvent.Message;
if (!string.IsNullOrEmpty(message) &&
message.EndsWith("\r\n", StringComparison.OrdinalIgnoreCase))
{
// remove any terminating return+line feed, since we're
// calling WriteLine below
message = message.Substring(0, message.Length - 2);
}
_innerTextWriter.WriteLine(message);
if (traceEvent.Exception != null)
{
_innerTextWriter.WriteLine(traceEvent.Exception.ToDetails());
}
}
}
The line it freezes on is line 66 : _innerTextWriter.WriteLine(message);
_innerTextWriter is an instance of System.IO.TextWriter.SyncTextWriter
Is it possible there is some deadlock issue with this class or the way it is being used?
Some notes:
I am running in the debugger, so in this case I believe the textwriter is forwarding to the console internally
I have my batchsize set to 1 via config.Queues.BatchSize = 1;, not sure if that could matter
I'm currently working on setting up an environment on another computer so that I can see if it is reproducible somewhere other than this machine (surface book).
Update
The issue was me not understanding how the new windows 10 command prompt works. Any time you click on the command window, it goes into "select" mode which completely pauses execution of the process.
Basically: https://superuser.com/questions/419717/windows-command-prompt-freezing-randomly?newreg=ece53f5584254346be68f85d1fd2f18d
You can tell it is in this state because it will prefix the window title with the word "Select":
You have to press enter or click again to get it going once again.
So, two final comments:
1) What an incredibly confusing and un-intuitive behavior for a command window!
2) I hope some admin will come take pity on the shame I have brought upon myself and my family by deleting this question.
To get rid of this strange behavior, you can disable QuickEdit mode:
Strange. When it is in this stuck state, can you try adding a new queue message to the queue and see if that triggers? Are you sure your function isn't hanging internally? What version of the SDK are you using? You might also try upgrading to v1.1.0 which we just released last week. If there are really a bunch of messages in the queue waiting to be processed, I can't think of anything that would cause this. The queue listener in the SDK should chug along, reading batches of messages in parallel and dispatching them to your function. Have you changed any of the JobHostConfiguration.Queues configuration knobs? You haven't force updated the version of the Azure SDK have you to something higher than the WebJobs SDK supports?
Another option if you can't figure this out might be to clone the SDK, build it and debug it locally. The repo is here. The main queue processing loop is here.
I have a console based application as WebJob. Now internally i am trying to map a CloudDrive using the storageconnectionstring UseDevelopmentStorage=true
It is throwing exception ERROR_AZURE_DRIVE_DEV_PATH_NOT_SET. I searched for this error and found that WebJobs do not run locally in Azure emulator. Is this information still valid?
Is there any plan to provide emulator (storage) support for webjobs in near future say in a week or so?
thanks
The information is still valid - we don't support the Azure emulator.
We have that work item on our backlog but I cannot give you any ETA.
Boo hoo Microsoft... This seems rather stupid given that you want us to start adopting the use of Azure Web Jobs!
There are new few lines of code in current version, which I believe solves this issue
static void Main()
{
var config = new JobHostConfiguration();
if (config.IsDevelopment)
{
config.UseDevelopmentSettings();
}
var host = new JobHost();
// The following code ensures that the WebJob will be running continuously
host.RunAndBlock();
}