Azure service bus message delivery count is not increasing or is reset when topic subscription disabling/enabling - azure

I have the following workflow:
Service bus receives messages.
Azure function triggers and tries to deliver this messages via HTTP to some service.
If delivery failed - function throws exception (custom) and disables topic subscription via code below:
The other function in parallel pings special health check endpoint of the service, and if it gets 200 - it tries to enable subscription and make the flow work again.
The steps could be reproduced N times, cause health check will return 200, thus the delivery url of point 2 - 4xx code.
After the next attempt to enable subscription and deliver the message, I expect that delivery count will be increased and in the end (after 10 deliveries attempt) it will get to dead-letter.
Actual - it equals 1.
I assume, that it may reset when I call CreateOrUpdate with status changed.
If yes - what is the other way to manage subscription status instead of Microsoft.Azure.Management package so that the messages delivery count will not be reset?
UPDATE: Function code
public static class ESBTESTSubscriptionTrigger
{
private static readonly HttpClient Client = new HttpClient();
private static IDatabase redisCache;
[FunctionName("ESBTESTSubscriptionTrigger")]
[Singleton]
public static async Task Run([ServiceBusTrigger("Notifications", "ESBTEST", AccessRights.Listen, Connection = "NotificationsBusConnectionString")]BrokeredMessage serviceBusMessage, TraceWriter log, [Inject]IKeyVaultSecretsManager keyVaultSecretsManager)
{
var logicAppUrl = await keyVaultSecretsManager.GetSecretAsync("NotificationsLogicAppUrl");
if (redisCache == null)
{
redisCache = RedisCacheConnectionManager.GetRedisCacheConnection(
keyVaultSecretsManager.GetSecretAsync("RedisCacheConnectionString").GetAwaiter().GetResult());
}
if (string.IsNullOrWhiteSpace(logicAppUrl))
{
log.Error("Logic App URL should be provided in Application settings of function App.");
throw new ParameterIsMissingException("Logic App URL should be provided in Application settings of function App.");
}
var applicaitonId = serviceBusMessage.Properties["applicationId"].ToString();
var eventName = serviceBusMessage.Properties.ContainsKey("Event-Name") ? serviceBusMessage.Properties["Event-Name"].ToString() : string.Empty;
if (string.IsNullOrWhiteSpace(applicaitonId))
{
log.Error("ApplicationId should be present in service bus message properties.");
throw new ParameterIsMissingException("Application id is missing in service bus message.");
}
Stream stream = serviceBusMessage.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
string s = reader.ReadToEnd();
var content = new StringContent(s, Encoding.UTF8, "application/json");
content.Headers.Add("ApplicationId", applicaitonId);
HttpResponseMessage response;
try
{
response = await Client.PostAsync(logicAppUrl, content);
}
catch (HttpRequestException e)
{
log.Error($"Logic App responded with {e.Message}");
throw new LogicAppBadRequestException($"Logic App responded with {e.Message}", e);
}
if (!response.IsSuccessStatusCode)
{
log.Error($"Logic App responded with {response.StatusCode}");
var serviceBusSubscriptionsSwitcherUrl = await keyVaultSecretsManager.GetSecretAsync("ServiceBusTopicSubscriptionSwitcherUri");
var sbSubscriptionSwitcherResponse = await Client.SendAsync(
new HttpRequestMessage(HttpMethod.Post, serviceBusSubscriptionsSwitcherUrl)
{
Content =
new
StringContent(
$"{{\"Action\":\"Disable\",\"SubscriptionName\":\"{applicaitonId}\"}}",
Encoding.UTF8,
"application/json")
});
if (sbSubscriptionSwitcherResponse.IsSuccessStatusCode == false)
{
throw new FunctionNotAvailableException($"ServiceBusTopicSubscriptionSwitcher responded with {sbSubscriptionSwitcherResponse.StatusCode}");
}
throw new LogicAppBadRequestException($"Logic App responded with {response.StatusCode}");
}
if (!string.IsNullOrWhiteSpace(eventName))
{
redisCache.KeyDelete($"{applicaitonId}{eventName}DeliveryErrorEmailSent");
}
}
}

Related

Azure Functions isolated .Net 6.0 + SignalR

My goal is to:
In scheduled functions - add message to SignalR
In SPA application (vue.js) subscribe to the event and call API to update the view
For now I'm trying to get anything to/from SignalR in my Function app (isolated, .net 6.0).
What I have in a function app:
[Function("negotiate")]
public HttpResponseData Negotiate(
[HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestData req,
[SignalRConnectionInfoInput(HubName = "AdminHub", ConnectionStringSetting = "AzureSignalRConnectionString")] SignalRConnectionInfo connectionInfo)
{
_logger.LogInformation($"SignalR Connection URL = '{connectionInfo.Url}'");
var response = req.CreateResponse(HttpStatusCode.OK);
response.Headers.Add("Content-Type", "text/plain; charset=utf-8");
response.WriteString($"Connection URL = '{connectionInfo.Url}'");
return response;
}
}
[Function("SendMessage")]
[SignalROutput(HubName = "AdminHub", ConnectionStringSetting = "AzureSignalRConnectionString")]
public SignalRMessage SendMessage(
[HttpTrigger(AuthorizationLevel.Anonymous, "get")] Microsoft.Azure.Functions.Worker.Http.HttpRequestData req)
{
return
new SignalRMessage
{
Target = "cancelToHandle",
MethodName = "cancelToHandle",
Arguments = new[] { "hello" }
};
}
[Function("SignalRTest")]
public static async Task SignalRTest([SignalRTrigger("AdminHub", "messages", "cancelToHandle", ConnectionStringSetting = "AzureSignalRConnectionString")] string message, ILogger logger)
{
logger.LogInformation($"Receive {message}.");
}
Negotiate function is not called. When should it be called?
If I call SendMessage, no error, but nothing happens in SignalR service. Should I see connections and messages there? (zero in the Metrics for now).
I've tried to create a test "emulator" client - just a console application:
var url = "http://<azureSignalRUrl>/AdminHub";
var connection = new HubConnectionBuilder()
.WithUrl(url)
.WithAutomaticReconnect()
.Build();
// receive a message from the hub
connection.On<string, string>("cancelToHandle", (user, message) => OnReceiveMessage(user, message));
await connection.StartAsync();
// send a message to the hub
await connection.InvokeAsync("SendMessage", "ConsoleApp", "Message from the console app");
void OnReceiveMessage(string user, string message)
{
Console.WriteLine($"{user}: {message}");
}
and it throws the exception ": 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. (:80)'
I think I'm missing overall understanding of what is supposed to happen:
when should negotiate function be triggered
can I see messages that I send in Azure portal (in SignalR service)?
how can I easily receive them in testing purposes
what do parameters/properties mean (target / method name / category). Example:
SignalRTriggerAttribute has the following constructor
public SignalRTriggerAttribute(string hubName, string category, string #event, params string[] parameterNames);
and Output binding receives any custom model I create?
which settings should be set in SignalR service - for now all I set it Serverless mode + CORS
Here are the few link which will help in using SignalIr service extension in functions
Using SignalIr service extension
https://github.com/Azure/azure-functions-dotnet-worker/blob/main/samples/Extensions/SignalR/SignalRFunction.cs
Below is the sample link for triggering negotiate function
https://github.com/Log234/azure-functions-signalr-dotnet-isolated-demo/blob/main/SignalRDemo/NegotiationFunctions.cs
for complete understanding of SignalIR here is the Github and MS document.

Error "Exception while executing function" from Azure Service Bus Listener

We use an Azure Service Bus to post all of our requests from our Xamarin mobile app. The Azure Service Bus is bound to an Azure Function which is triggered each time a requests hits the Azure Service Bus.
We have found that we are getting errors from this Azure Function when we send data above a certain size. We can send up to 800 records without a problem but when we send >=850 records we get the following error:
[Error] Exception while executing function:
Functions.ServiceBusQueueTrigger. mscorlib: Exception has been thrown
by the target of an invocation. mscorlib: One or more errors occurred.
A task was canceled.
The service that is being invoked is an ASP.NET Web API RESTful service that saves the data records into a database. This doesn't generate any errors at all.
Here is my Azure Function code.
#r "JWT.dll"
#r "Common.dll"
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using Microsoft.ServiceBus.Messaging;
public static void Run(BrokeredMessage message, TraceWriter log)
{
log.Info($"C# ServiceBus queue trigger function processed message: {message.MessageId}");
if (message != null)
{
Common.Entities.MessageObjectEntity messageObject = message?.GetBody<Common.Entities.MessageObjectEntity>();
string msgType = messageObject?.MessageType;
var msgContent = messageObject?.MessageContent;
log.Info($"Message type: {msgType}");
double timestamp = (DateTime.UtcNow - new DateTime(1970, 1, 1)).TotalSeconds;
string subscriber = "MYSUBSCRIBER";
string privatekey = "MYPRIVATEKEY";
Dictionary<string, object> payload = new Dictionary<string, object>()
{
{"iat", timestamp},
{"subscriber", subscriber}
};
string token = JWT.JsonWebToken.Encode(payload, privatekey, JWT.JwtHashAlgorithm.HS256);
using (var client = new HttpClient())
{
string url = $"http://myexamplewebservices.azurewebsites.net/api/routingtasks?formname={msgType}";
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(subscriber, token);
HttpContent content = new StringContent((string)msgContent, Encoding.UTF8, "application/json");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var response = client.PostAsync(new Uri(url), content);
if (response == null)
{
log.Info("Null response returned from request.");
}
else
{
if (response.Result.IsSuccessStatusCode && response.Result.StatusCode == System.Net.HttpStatusCode.OK)
{
log.Info("Successful response returned from request.");
}
else
{
log.Info($"Unsuccessful response returned from request: {response.Result.StatusCode}.");
}
}
}
log.Info("Completing message.");
}
}
This code has been working for several years and works across all our other apps / web sites.
Any ideas why we're getting errors wehen we post large amounts of data to our Azure Service Bus / Azure Function?
It may caused by "new httpclient", there is a limit to how quickly system can open new sockets so if you exhaust the connection pool, you may get some errors. You can refer to this link: https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/
And could you please share some more error message ?
I can see that you are creating httpclient connection on each request which possibly be causing this issue. Httpclient creates a socket connection underneath it and has hard limit on it. Even when you dispose it it remains there for couple of mins that can't be used. A good practice is to create single static httpclient connection and reuse it. I am attaching some documents for you to go through.
AzFunction Static HttpClient , Http Client Working , Improper instantiation

Configuring Twilio SMS from Azure Functions v2

I have some code where I'm reading messages off of an Azure Event Hub that I want to either send an email or send an SMS.
The email is working through send grid, but I'm not sure how to configure the SMS part though.
I think I'd want to use Twilio and here's a sample of what my code's like. The "messageCollector" works for sending Email since there's some configuration for SendGrid in the local json. How do I configure Twilio?
[FunctionName("SendAlert")]
public static async Task Run(
[EventHubTrigger("v1-email-hub", Connection = "EventHubConnection")] EventData[] events,
[SendGrid] IAsyncCollector<SendGridMessage> messageCollector,
[TwilioSms] IAsyncCollector<CreateMessageOptions> smsCollector,
[Inject] NotificationEventLogic eventLogic,
ILogger log)
{
foreach (EventData eventData in events)
{
string messageBody = Encoding.UTF8.GetString(eventData.Body.Array, eventData.Body.Offset, eventData.Body.Count);
var notificationEvents = JsonConvert.DeserializeObject<List<NotificationEvent>>(messageBody);
foreach (var ev in notificationEvents)
{
if (ev.NotificationEventType == NotificationEventType.Email)
{
var message = new SendGridMessage();
// ... ... make message and add it
await messageCollector.AddAsync(message);
}
else if (ev.NotificationEventType == NotificationEventType.SMS)
{
// Not sure how to get this to work
var mobileMessage = new CreateMessageOptions(new PhoneNumber(ev.Data))
{
Body = $"Notification {ev.NotificationId}"
};
await smsCollector.AddAsync(mobileMessage);
}
// await smsCollector.AddAsync()
await eventLogic.CreateEventAsync(ev);
}
}
}
You will need to configure it in attribute
[TwilioSms(AccountSidSetting = "TwilioAccountSid", AuthTokenSetting = "TwilioAuthToken", From = "+1425XXXXXXX")]
as it mentioned in documentation
TwilioAccountSid This value must be set
to the name of an app setting that holds your Twilio Account Sid e.g.
TwilioAccountSid. If not set, the default app setting name is
"AzureWebJobsTwilioAccountSid".
TwilioAuthToken This value must be set to
the name of an app setting that holds your Twilio authentication token
e.g. TwilioAccountAuthToken. If not set, the default app setting name
is "AzureWebJobsTwilioAuthToken".

How to implement blob storage access timeout and show message

I want to show message to end user when blob is taking so much time for uploading and downloading. I found useful blog here.
Simply linear retry policy
public static RetryPolicy LinearRetry(int retryCount, TimeSpan intervalBetweenRetries)
{
return () =>
{
return (int currentRetryCount, Exception lastException, out TimeSpan retryInterval) =>
{
// Do custom work here
// Set backoff
retryInterval = intervalBetweenRetries;
// Decide if we should retry, return bool
return currentRetryCount < retryCount;
};
};
}
But here I didn't get how to send response to user back while retrying. Is this right way or anything else. Please suggest
OperationContext class in Storage Client Library has an event called Retrying that you can consume and send message back to the client.
For example, I created a simple console application which tries to create a blob container. When I ran this application, I deliberately turned off Internet access so that I can simulate a situation where operation would be retried. Then in this event consumer, I simply write something back to console. You could simply raise another event from there that would send a message back to your client.
var requestOptions = new BlobRequestOptions()
{
RetryPolicy = new ExponentialRetry(),
};
var operationContext = new OperationContext();
operationContext.Retrying += (sender, args) =>
{
Console.WriteLine("I'm retrying ....");
};
var cloudStorageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
var blobClient = cloudStorageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("test");
container.CreateIfNotExists(requestOptions, operationContext);

Azure log showing: "The supplied notification payload is invalid" for official Xamarin.Android sample

So I tried running this Push notification sample for Xamarin.Android http://azure.microsoft.com/en-us/documentation/articles/partner-xamarin-mobile-services-android-get-started-push/ and after following instructions from the docs - I got it up and running. The insertion of items work absolutely fine however push notification refuses to work.
This is the error I get on Azure for push: Error: 400 - The supplied notification payload is invalid.
Anyone else tried running this sample on their device and tried push notifications? The error isn't doing much to help my case.
The sample is using PushSharp.
I'd appreciate any help. Thanks a bunch!
This is how I send push notification to Google Cloud Messaging from the back-end server.
public async Task<bool> SendNotification(int id, int index, string from, string text, string tag)
{
try
{
var payload = new
{
data = new
{
message = new
{
// this part can be anything you want
id,
index,
from,
text,
when = DateTime.UtcNow.ToString("s") + "Z"
}
}
};
var json = JsonConvert.SerializeObject(payload);
await _hubClient.SendGcmNativeNotificationAsync(json, tag);
return true;
}
catch (ArgumentException ex)
{
// This is expected when an APNS registration doesn't exist.
return false;
}
Then in your app Intent Service, you can parse the JSON "message":
protected override void OnMessage(Context context, Intent intent)
{
var message = intent.Extras.GetString("message");
// message is JSON payload
// { "id":"someid", "index":"1", "text":"some text","from"... }
var json = JObject.Parse(message);
var id = json["id"].ToString();
var index = json["index"].ToString();
var text = json["text"].ToString();
var from = json["from"].ToString();
var when = DateTime.Parse(json["when"].ToString());
// do whatever you want with your values here
}

Resources