QueueTrigger is not picking messages- Azure WebJobs SDK 3.0 - azure

I'm trying to develop WebJob using SDK 3.0.x, and testing it locally. I've followed the sample in github without any success.
When running it locally everything is going ok, it also see the ProcessQueueMessage function but it doesn't pick the messages from the queue.
Program.cs
static void Main(string[] args)
{
var builder = new HostBuilder();
//builder.UseEnvironment(EnvironmentName.Development);
builder.ConfigureWebJobs(b =>
{
b.AddAzureStorageCoreServices();
b.AddAzureStorage();
});
builder.ConfigureAppConfiguration((context, config) =>
{
config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);
});
builder.ConfigureLogging((context, b) =>
{
b.AddConsole();
// If the key exists in settings, use it to enable Application Insights.
string instrumentationKey = context.Configuration["APPINSIGHTS_INSTRUMENTATIONKEY"];
if (!string.IsNullOrEmpty(instrumentationKey))
{
b.AddApplicationInsights(o => o.InstrumentationKey = instrumentationKey);
}
});
builder.ConfigureServices((context, services) =>
{
//services.AddSingleton<IJobActivator, MyJobActivator>();
services.AddScoped<Functions, Functions>();
services.AddSingleton<IHostService, HostService>();
})
.UseConsoleLifetime();
var host = builder.Build();
using (host)
{
host.Run();
}
}
Functions.cs
public class Functions
{
private readonly IHostService _hostService;
public Functions(IHostService hostService)
{
_hostService = hostService;
}
// This function will get triggered/executed when a new message is written
// on an Azure Queue called queue.
public void ProcessQueueMessage([QueueTrigger("newrequests")] string dd,
//DateTimeOffset expirationTime,
//DateTimeOffset insertionTime,
//DateTimeOffset nextVisibleTime,
//string queueTrigger,
//string id,
//string popReceipt,
//int dequeueCount,
ILogger logger)
{
var newRequestItem = new RequestQueueItem();
logger.LogTrace($"New queue item received...");
//logger.LogInformation($" QueueRef = {id} - DequeueCount = {dequeueCount} - Message Content [Id = {newRequestItem.Id}, RequestDate = {newRequestItem.RequestDate}, Mobile = {newRequestItem.Mobile}, ProviderCode = {newRequestItem.ProviderCode}, ItemIDClass = {newRequestItem.MappingIDClass}]");
// TODO: Read the DatabaseConnectionString from App.config
logger.LogTrace($" Getting DB ConnectionString...");
var connectionString = ConfigurationManager.ConnectionStrings["DatabaseConnection"].ConnectionString;
// TODO: Initiation of provider service instance
logger.LogTrace($" Init IalbayanmtnclientserviceClient service instance...");
var bayanService = new AlbayanMtnWCFService.IalbayanmtnclientserviceClient();
// TODO: sending request to provider service endpoint and wait for response
logger.LogTrace($" Sending request to Service Endpoint...");
var response= bayanService.requestpaymenttransactionAsync("agentcode", "agentpassword", "accountno", int.Parse(newRequestItem.TransactionType), newRequestItem.MappingIDClass, newRequestItem.Mobile, (int)newRequestItem.Id).Result;
logger.LogTrace($"Done processing queue item");
}
}
Here is the screen shot for the output
Appreciate your help
Screen shot for queue messages 'newrequests'
enter image description here

From your snapshot, your webjob runs well on local. It didn't pick message because you don't add message in the newrequests queue.
The function only be triggered after you add the message. Or I will get the same result just like yours.
About the tutorial , your could refer to the official doc:Get started with the Azure WebJobs SDK. And make sure you set the right storage account. The below is my appsettings.json. Make sure the "Copy to output directory" property of the appSettings.json file is set to either Copy if newer or Copy always. Or it will run into exception:Storage account 'Storage' is not configured.
{
"ConnectionStrings": {
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=key;..."
}
}
Hope this could help you, if you still have other questions, please let me know.

Related

Azure Event Hub to ELK Event Replay

I have configured function apps in Azure to persist log entries from the function app insights logs to an event hub which then also persists these log entries to ELK. I recently noticed that some log entries were missing and I followed this example to create a console app that would replay the log events. The console app I created based on the examples works and no exceptions are thrown, however I never find the log entries when querying ELK, they don't appear to be persisted there the way I would expect. Below is some example code of what my console app is doing.
private const string connectionString = "CONNECTION_STRING";
private const string eventHubName = "EVENT_HUB_NAME";
static EventHubBufferedProducerClient producerClient;
static async Task Main()
{
string eventDataString = "{"message": "Here's a log entry"}";*/
var eventData = JsonConvert.DeserializeObject<EventData>(eventDataString);
eventData.MessageId = "MessageIDLog1";
eventData.ContentType = "application/json";
eventData.EventBody = new BinaryData(Encoding.UTF8.GetBytes("{message": "Here's a log entry}"));
producerClient = new EventHubBufferedProducerClient(connectionString, eventHubName);
producerClient.SendEventBatchFailedAsync += args =>
{
Console.WriteLine($"Publishing failed for { args.EventBatch.Count } events. Error: { args.Exception.Message }");
return Task.CompletedTask;
};
producerClient.SendEventBatchSucceededAsync += args =>
{
Console.WriteLine($"{ args.EventBatch.Count } events were published to partition: { args.PartitionId }.");
return Task.CompletedTask;
};
try
{
await producerClient.EnqueueEventAsync(eventData);
}
finally
{
await producerClient.DisposeAsync();
}
}
I'm hoping someone out there has some experience with this and could point me in the right direction as I've been on this for a while and I'm not sure what I'm doing incorrectly.

Azure Functions Dependency Tracking for SQL Server and Service Bus Into Application Insights

Previously I have Azure Web App (.net core) and It successfully track the SQL Server and Service Bus dependency into Application Insights. It is not working some how with Azure Functions.
Environment
dotnet 6
dotnet-isolated mode
log level default set to "Information".
Azure Environment using Consumption plan for Azure Functions.
Application Insights key is configured.
I have Azure API management at front and backend is Azure Function and that call SQL Server and Service Bus.
Api Management Service to Azure function dependency successfully resolved but Azure Function to other component is not working.
I know I am posting my own answer. Also there are chance that in future there may be some good solution or it get integrated the way it is in in-process mode.
By then follow steps.
Add Package
Microsoft.ApplicationInsights.WorkerService
In program.cs in configuring host.
services.AddApplicationInsightsTelemetryWorkerService();
More info at
https://learn.microsoft.com/en-us/azure/azure-monitor/app/worker-service
The only way I've managed to solve this issue so far was by setting up custom Middleware
.ConfigureFunctionsWorkerDefaults(config =>
{
config.UseMiddleware<AiContextMiddleware>();
})
In the IServiceCollection you need to setup simply
.AddApplicationInsightsTelemetryWorkerService()
public class AiContextMiddleware : IFunctionsWorkerMiddleware
{
private readonly TelemetryClient _client;
private readonly string _hostname;
public AiContextMiddleware(TelemetryClient client)
{
_client = client;
_hostname = Environment.GetEnvironmentVariable("AI_CLOUD_ROLE_NAME");
}
public async Task Invoke(FunctionContext context, FunctionExecutionDelegate next)
{
var operationId = ExtractOperationId(context.TraceContext.TraceParent);
// Let's create and start RequestTelemetry.
var requestTelemetry = new RequestTelemetry
{
Name = context.FunctionDefinition.Name,
Id = context.InvocationId,
Properties =
{
{ "ai.cloud.role", _hostname},
{ "AzureFunctions_FunctionName", context.FunctionDefinition.Name },
{ "AzureFunctions_InvocationId", context.InvocationId },
{ "AzureFunctions_OperationId", operationId }
},
Context =
{
Operation =
{
Id = operationId,
ParentId = context.InvocationId,
Name = context.FunctionDefinition.Name
},
GlobalProperties =
{
{ "ai.cloud.role", _hostname},
{ "AzureFunctions_FunctionName", context.FunctionDefinition.Name },
{ "AzureFunctions_InvocationId", context.InvocationId },
{ "AzureFunctions_OperationId", operationId }
}
}
};
var operation = _client.StartOperation(requestTelemetry);
try
{
await next(context);
}
catch (Exception e)
{
requestTelemetry.Success = false;
_client.TrackException(e);
throw;
}
finally
{
_client.StopOperation(operation);
}
}
private static string ExtractOperationId(string traceParent)
=> string.IsNullOrEmpty(traceParent) ? string.Empty : traceParent.Split("-")[1];
}
It's definitely not a perfect solution as you then get two starting logs, but as end result, you get all logs traces + dependencies correlated to an operation.
I've solved this issue in the first place like that, now I'm revisiting whether there are any better ways to solve this.
Let me know too whether you managed to solve this issue on your side.

Azure Functions isolated .Net 6.0 + SignalR

My goal is to:
In scheduled functions - add message to SignalR
In SPA application (vue.js) subscribe to the event and call API to update the view
For now I'm trying to get anything to/from SignalR in my Function app (isolated, .net 6.0).
What I have in a function app:
[Function("negotiate")]
public HttpResponseData Negotiate(
[HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestData req,
[SignalRConnectionInfoInput(HubName = "AdminHub", ConnectionStringSetting = "AzureSignalRConnectionString")] SignalRConnectionInfo connectionInfo)
{
_logger.LogInformation($"SignalR Connection URL = '{connectionInfo.Url}'");
var response = req.CreateResponse(HttpStatusCode.OK);
response.Headers.Add("Content-Type", "text/plain; charset=utf-8");
response.WriteString($"Connection URL = '{connectionInfo.Url}'");
return response;
}
}
[Function("SendMessage")]
[SignalROutput(HubName = "AdminHub", ConnectionStringSetting = "AzureSignalRConnectionString")]
public SignalRMessage SendMessage(
[HttpTrigger(AuthorizationLevel.Anonymous, "get")] Microsoft.Azure.Functions.Worker.Http.HttpRequestData req)
{
return
new SignalRMessage
{
Target = "cancelToHandle",
MethodName = "cancelToHandle",
Arguments = new[] { "hello" }
};
}
[Function("SignalRTest")]
public static async Task SignalRTest([SignalRTrigger("AdminHub", "messages", "cancelToHandle", ConnectionStringSetting = "AzureSignalRConnectionString")] string message, ILogger logger)
{
logger.LogInformation($"Receive {message}.");
}
Negotiate function is not called. When should it be called?
If I call SendMessage, no error, but nothing happens in SignalR service. Should I see connections and messages there? (zero in the Metrics for now).
I've tried to create a test "emulator" client - just a console application:
var url = "http://<azureSignalRUrl>/AdminHub";
var connection = new HubConnectionBuilder()
.WithUrl(url)
.WithAutomaticReconnect()
.Build();
// receive a message from the hub
connection.On<string, string>("cancelToHandle", (user, message) => OnReceiveMessage(user, message));
await connection.StartAsync();
// send a message to the hub
await connection.InvokeAsync("SendMessage", "ConsoleApp", "Message from the console app");
void OnReceiveMessage(string user, string message)
{
Console.WriteLine($"{user}: {message}");
}
and it throws the exception ": 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. (:80)'
I think I'm missing overall understanding of what is supposed to happen:
when should negotiate function be triggered
can I see messages that I send in Azure portal (in SignalR service)?
how can I easily receive them in testing purposes
what do parameters/properties mean (target / method name / category). Example:
SignalRTriggerAttribute has the following constructor
public SignalRTriggerAttribute(string hubName, string category, string #event, params string[] parameterNames);
and Output binding receives any custom model I create?
which settings should be set in SignalR service - for now all I set it Serverless mode + CORS
Here are the few link which will help in using SignalIr service extension in functions
Using SignalIr service extension
https://github.com/Azure/azure-functions-dotnet-worker/blob/main/samples/Extensions/SignalR/SignalRFunction.cs
Below is the sample link for triggering negotiate function
https://github.com/Log234/azure-functions-signalr-dotnet-isolated-demo/blob/main/SignalRDemo/NegotiationFunctions.cs
for complete understanding of SignalIR here is the Github and MS document.

Azure Storage Queue Trigger - Use remote queue name

I am using Azure App Configuration Store to store configuration. I am using the following code in startup.cs to load my config from Azure.
var builder = new ConfigurationBuilder();
builder.AddAzureAppConfiguration(options =>
{
options.Connect(this.Values.AppConfigConnectionString);
options.Select(keyFilter: KeyFilter.Any, labelFilter: this.Values.Env);
});
var config = builder.Build();
Now this config variable contains my queue names. I need this dynamic so to create and handle it in 4 different environments. Dev / Stage / QA / Prod.
public async Task Run(
[QueueTrigger("%QueueName%", Connection = "StorageConnection")]VoiceHubEvent item)
This isn't working as my local.settings.json file doesn't contain QueueName entry.
Is it possible to make use of config variable in Run() to resolve queuename? By reloading queue trigger function or something?
Thanks,
Kiran.
Is it possible to make use of config variable in Run() to resolve queuename? By reloading queue trigger function or something?
Yes, you can.
Create an extensions method for the IWebJobsBuilder interface to set up a connection to AzureAppConfiguration.
public static IWebJobsBuilder AddAzureConfiguration(this IWebJobsBuilder webJobsBuilder)
{
//-- Get current configuration
var configBuilder = new ConfigurationBuilder();
var descriptor = webJobsBuilder.Services.FirstOrDefault(d => d.ServiceType == typeof(IConfiguration));
if (descriptor?.ImplementationInstance is IConfigurationRoot configuration)
configBuilder.AddConfiguration(configuration);
var config = configBuilder.Build();
//-- Add Azure Configuration
configBuilder.AddAzureAppConfiguration(options =>
{
var azureConnectionString = config[TRS.Shared.Constants.CONFIGURATION.KEY_AZURECONFIGURATION_CONNECTIONSTRING];
if (string.IsNullOrWhiteSpace(azureConnectionString)
|| !azureConnectionString.StartsWith("Endpoint=https://"))
throw new InvalidOperationException($"Missing/wrong configuration value for key '{TRS.Shared.Constants.CONFIGURATION.KEY_AZURECONFIGURATION_CONNECTIONSTRING}'.");
options.Connect(azureConnectionString);
});
//build the config again so it has the key vault provider
config = configBuilder.Build();
return webJobsBuilder;
}
Where the azureConnectionString is read from you appsetting.json and should contain the url to the Azure App Configuration.
In startup.cs:
public void Configure(IWebJobsBuilder builder)
{
builder.AddAzureConfiguration();
ConfigureServices(builder.Services)
.BuildServiceProvider(true);
}
For more details, you could refer to this SO thread.

Azure service bus message delivery count is not increasing or is reset when topic subscription disabling/enabling

I have the following workflow:
Service bus receives messages.
Azure function triggers and tries to deliver this messages via HTTP to some service.
If delivery failed - function throws exception (custom) and disables topic subscription via code below:
The other function in parallel pings special health check endpoint of the service, and if it gets 200 - it tries to enable subscription and make the flow work again.
The steps could be reproduced N times, cause health check will return 200, thus the delivery url of point 2 - 4xx code.
After the next attempt to enable subscription and deliver the message, I expect that delivery count will be increased and in the end (after 10 deliveries attempt) it will get to dead-letter.
Actual - it equals 1.
I assume, that it may reset when I call CreateOrUpdate with status changed.
If yes - what is the other way to manage subscription status instead of Microsoft.Azure.Management package so that the messages delivery count will not be reset?
UPDATE: Function code
public static class ESBTESTSubscriptionTrigger
{
private static readonly HttpClient Client = new HttpClient();
private static IDatabase redisCache;
[FunctionName("ESBTESTSubscriptionTrigger")]
[Singleton]
public static async Task Run([ServiceBusTrigger("Notifications", "ESBTEST", AccessRights.Listen, Connection = "NotificationsBusConnectionString")]BrokeredMessage serviceBusMessage, TraceWriter log, [Inject]IKeyVaultSecretsManager keyVaultSecretsManager)
{
var logicAppUrl = await keyVaultSecretsManager.GetSecretAsync("NotificationsLogicAppUrl");
if (redisCache == null)
{
redisCache = RedisCacheConnectionManager.GetRedisCacheConnection(
keyVaultSecretsManager.GetSecretAsync("RedisCacheConnectionString").GetAwaiter().GetResult());
}
if (string.IsNullOrWhiteSpace(logicAppUrl))
{
log.Error("Logic App URL should be provided in Application settings of function App.");
throw new ParameterIsMissingException("Logic App URL should be provided in Application settings of function App.");
}
var applicaitonId = serviceBusMessage.Properties["applicationId"].ToString();
var eventName = serviceBusMessage.Properties.ContainsKey("Event-Name") ? serviceBusMessage.Properties["Event-Name"].ToString() : string.Empty;
if (string.IsNullOrWhiteSpace(applicaitonId))
{
log.Error("ApplicationId should be present in service bus message properties.");
throw new ParameterIsMissingException("Application id is missing in service bus message.");
}
Stream stream = serviceBusMessage.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
string s = reader.ReadToEnd();
var content = new StringContent(s, Encoding.UTF8, "application/json");
content.Headers.Add("ApplicationId", applicaitonId);
HttpResponseMessage response;
try
{
response = await Client.PostAsync(logicAppUrl, content);
}
catch (HttpRequestException e)
{
log.Error($"Logic App responded with {e.Message}");
throw new LogicAppBadRequestException($"Logic App responded with {e.Message}", e);
}
if (!response.IsSuccessStatusCode)
{
log.Error($"Logic App responded with {response.StatusCode}");
var serviceBusSubscriptionsSwitcherUrl = await keyVaultSecretsManager.GetSecretAsync("ServiceBusTopicSubscriptionSwitcherUri");
var sbSubscriptionSwitcherResponse = await Client.SendAsync(
new HttpRequestMessage(HttpMethod.Post, serviceBusSubscriptionsSwitcherUrl)
{
Content =
new
StringContent(
$"{{\"Action\":\"Disable\",\"SubscriptionName\":\"{applicaitonId}\"}}",
Encoding.UTF8,
"application/json")
});
if (sbSubscriptionSwitcherResponse.IsSuccessStatusCode == false)
{
throw new FunctionNotAvailableException($"ServiceBusTopicSubscriptionSwitcher responded with {sbSubscriptionSwitcherResponse.StatusCode}");
}
throw new LogicAppBadRequestException($"Logic App responded with {response.StatusCode}");
}
if (!string.IsNullOrWhiteSpace(eventName))
{
redisCache.KeyDelete($"{applicaitonId}{eventName}DeliveryErrorEmailSent");
}
}
}

Resources