I have Azure Queue in operation that I write to after each execution on a azure time trigger.The time trigger added message to the queue. But in some times duplicate value added in azure queue. The queue trigger functionality is to send notifications to user in a particular time. But some time user get 3 notification.
In time tigger execute in 15 min interval. When the time trigger execute in 11.45Am, the user get notifications total 3.
At time 11.45Am, 11.50am, 11.55am etc. But actually we need to send only one notification in 11.45 am. I cheked the time trigger log, it execute only in 15 time interval.
public void AddToQueue(string queueData, string name, string connectionString)
{
//"DefaultEndpointsProtocol=https;AccountName=myAccount;AccountKey=c3RyaW5nIGxlbmd0aCB2YWxpZA=="
var storageAccount = CloudStorageAccount.Parse(connectionString);
var queueClient = storageAccount.CreateCloudQueueClient();
var queue = queueClient.GetQueueReference(name);
queue.CreateIfNotExistsAsync();
var message = new CloudQueueMessage(queueData);
queue.AddMessageAsync(message);
}
Is any dulicate added in queue or any issue in my code?
Note: total I have 3 queue trigger functions and one time trigger function.
Related
I have an API that will call Azure Topic to schedule a message. Is there a way to receive that message before the schedule time? For example in my code below, I schedule a message to azure topic and it will be queue after 60mins/1hr. Is there a way to received that message before 1hr?
string queueName = "topic";
var client = new ServiceBusClient("", new ServiceBusClientOptions()
{
TransportType = ServiceBusTransportType.AmqpWebSockets
});
// create the sender
ServiceBusSender sender = client.CreateSender(queueName);
// create a message that we can send. UTF-8 encoding is used when providing a string.
ServiceBusMessage message = new ServiceBusMessage($"Hello world cancel 13 {DateTime.Now}");
// add 5 minutes delay
long seq = await sender.ScheduleMessageAsync(message,
DateTimeOffset.Now.AddMinutes(60)
);
The message sequence number you get back when scheduling is suitable for cancelling but doesn't allow receiving that message earlier. The service doesn't allow early receiving, as anything that gets the messages to get the active messages (not in the future). For this scenario, I would suggest keeping the data in a database and not leveraging the queue as the database.
I am working with an azure service bus queue configured to be FIFO (First input first output). I work on an order application with the following states "Pending", "Received" and "Sent". therefore I have grouped the messages by the "SessionId" service bus option, setting the orderId as sessionId so that it processes the messages in order in case of horizontal scaling.
So far it works perfectly, the problem I have found is when a message in "pending" or "Received" status fails due to a timeout and goes to the dead letter queue. The message in "sent" status is processed correctly and then when the support team re-sends the "Pending" or "Received" status message to the queue it is processed correctly marking the order in a previous status instead of "sent" ".
I can think of several ways to control this, for example that the support team looks at the status of the order before reprocessing the message from the dead letter queue :) but I would like to know if service bus offers the possibility that if there is a message in the dead letter queu all the messages in the session queue that have the same sessionId go to the dead letter queu. Finallly, my question is:
Is there a way to configure azure service bus so that if there are any messages in the dead letter queue it sends all messages with the same sessionId to the dead letter queue?
Thank you very much!!!
I would like to know if service bus offers the possibility that if there is a message in the dead letter queue all the messages in the session queue that have the same sessionId go to the dead letter queue.
No, there is no such offering by Service Bus by default.
Is there a way to configure azure service bus so that if there are any messages in the dead letter queue it sends all messages with the same sessionId to the dead letter queue?
Yes, you can do that. You can first peek the messages in your dead-letter queue to fetch all the session ids. Then you can receive the messages in your main queue whose session id is in the DLQ, and then move those messages to DLQ. Here's one such logic I've implemented in dot net using the latest version of Service Bus SDK.
var queueName = "<queue>";
var connectionString = "<connection-string>";
var client = new ServiceBusClient(connectionString);
var sessionIdInDLQList = new List<string>();
var receiver = client.CreateReceiver(queueName, new ServiceBusReceiverOptions { SubQueue = SubQueue.DeadLetter });
var message = await receiver.PeekMessageAsync();
while (message != null)
{
if (!sessionIdInDLQList.Contains(message.SessionId))
sessionIdInDLQList.Add(message.SessionId);
message = await receiver.PeekMessageAsync();
}
foreach (var sessionId in sessionIdInDLQList)
{
var session = await client.AcceptSessionAsync(queueName, sessionId);
message = await session.ReceiveMessageAsync(TimeSpan.FromSeconds(20));
while (message != null)
{
await session.DeadLetterMessageAsync(message, "Message with this session is to be dead-lettered!");
message = await session.ReceiveMessageAsync(TimeSpan.FromSeconds(20));
}
}
In your case, you need to do this before your consumers start reading the messages, probably you can write this in your consumer application or any trigger application like Azure Function or worker role. That’s upto your method of handling.
You can try this code to read Dead Letter from Queue.
public static async Task GetMessage()
{
string topic = "myqueue1";
string connectionString = "Endpoint = sb://xxx.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxx";
var servicebusclient = new ServiceBusClient(connectionString);
var reciveroptions = new ServiceBusReceiverOptions { SubQueue = SubQueue.DeadLetter };
var reciver = servicebusclient.CreateReceiver(topic, reciveroptions);
// 10 number of message read from Queue
await receiver.PeekMessageAsync(10);
}
after receiving message from Dead Letter you can send to queue.
As per Microsoft official documents
There's no automatic cleanup of the DLQ. Messages remain in the DLQ
until you explicitly retrieve them from the DLQ and call Complete() on
the dead-letter message.
These following document help you.
Thanks Casually Coding for posting post on Read Message from the Dead Letter Queue
Microsoft Documents Using Dead-Letter Queues to Handle Message Transfer Failures , Receive Message from Dead letter queue
I have a azure function that is triggered every 1 minute. The function creates a MessageReceiver like this
var messageReceiver = new MessageReceiver(serviceBusConnectionString, entityPath, ReceiveMode.PeekLock, null, 0);
The subscription has a lock duration of 1 minute.
It then tries to fetch up to 100 messages from the subscription like this:
var allMessages = new List<Message>();
Message message;
do {
message = await messageReceiver.ReceiveAsync(TimeSpan.FromSeconds(2));
if (message != null) allMessages.Add(message);
} while (message != null && allMessages.Count() < 100);
After processing the messages are completed using messageReceiver.CompleteAsync and the messageReceiver is closed using messageReceiver.CloseAsync().
The first time the function runs it fetches up to 100 messages from the subscription, but on the next runs it only fetches 0 or 1 message no matter the number of messages available in the subscription (Service Bus Explorer shows that there > 10 messages in the subscription). So it seems that ReceiveAsync returns null even when there is messages available.
Increasing the timeout for ReceiveAsync doesn't seem to help.
Why does ReceiveAsync return null when there is messages available?
I found a solution or workaround. If I change the the service bus topic to not allow partitioning (it requires deleting the topic and creating it again), ReceiveAsync always returns a message when there is a message available.
The problems seems related to this issue Odd Behavior of Azure Service Bus ReceiveBatch()
Does anyone see a problem with this code? I have a single Azure Queue in operation that I write to after each transaction on a Web Application. The queue is only called once per transaction, but items sometimes appear more than once - 10 duplicates in some cases. This is sporadic and happens rarely, but it is annoying nonetheless.
public async ActionResult DoWork()
{
//Carry out some processing
string jsonMessage = new JavaScriptSerializer().Serialize(MyDataObject);
await storageAccess.AddItemToQueue(new CloudQueueMessage(jsonMessage));
}
public async Task AddItemToQueue(CloudQueueMessage message)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
ConfigurationManager.AppSettings["StorageConnectionString"]);
// Create the blob client.
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
var queue = queueClient.GetQueueReference("outbox");
await queue.CreateIfNotExistsAsync();
await queue.AddMessageAsync(message);
}
If there is a timeout or a warning, could the AddMessageAsync method be trying to send it again until it succeeds?
Inspecting the queue, the items might look like this, where I have 3 items, all containing the same MyObject1 inserted well after the transaction closed.
1. ksasdsad-asdsad-asdsads-s222 12:05 MyObject1
2. sadadsd-weqwe-sdddsd-e21323 12:10 MyObject1
3. 32323-asdsads-2123213-sddwe 12:30 MyObject1
Is there a way to get the message ID after insert it in a queue Azure ?
CloudStorageAccount storageAccount =
CloudStorageAccount.parse(storageConnectionString);
CloudQueueClient queueClient = storageAccount.createCloudQueueClient();
CloudQueue queue = queueClient.getQueueReference("myqueue");
queue.createIfNotExist();
CloudQueueMessage message = new CloudQueueMessage("Hello, World");
queue.addMessage(message);
// Get message ID here ?
I realize it has been 5 years since this was originally asked; however, it is now possible to achieve this.
CloudQueueMessage message = new CloudQueueMessage("Hello, World");
queue.AddMessage(message);
// here's how you get the id
string id = message.Id;
Only way you could get the message id is by getting the message. So you would have to fetch messages from the queue using GetMessage or GetMessages method. However there's no guarantee that you will get the message you just created as GetMessages can only return up to 32 visible messages from the top of the queue.
Since queue lies on the principle "First In First Out" or FIFO, that is why you can't just get the particular message anytime you want but you have to use the GetMessage and iterate on it.