Is there a way to get the message ID after insert it in a queue Azure ?
CloudStorageAccount storageAccount =
CloudStorageAccount.parse(storageConnectionString);
CloudQueueClient queueClient = storageAccount.createCloudQueueClient();
CloudQueue queue = queueClient.getQueueReference("myqueue");
queue.createIfNotExist();
CloudQueueMessage message = new CloudQueueMessage("Hello, World");
queue.addMessage(message);
// Get message ID here ?
I realize it has been 5 years since this was originally asked; however, it is now possible to achieve this.
CloudQueueMessage message = new CloudQueueMessage("Hello, World");
queue.AddMessage(message);
// here's how you get the id
string id = message.Id;
Only way you could get the message id is by getting the message. So you would have to fetch messages from the queue using GetMessage or GetMessages method. However there's no guarantee that you will get the message you just created as GetMessages can only return up to 32 visible messages from the top of the queue.
Since queue lies on the principle "First In First Out" or FIFO, that is why you can't just get the particular message anytime you want but you have to use the GetMessage and iterate on it.
Related
I have an API that will call Azure Topic to schedule a message. Is there a way to receive that message before the schedule time? For example in my code below, I schedule a message to azure topic and it will be queue after 60mins/1hr. Is there a way to received that message before 1hr?
string queueName = "topic";
var client = new ServiceBusClient("", new ServiceBusClientOptions()
{
TransportType = ServiceBusTransportType.AmqpWebSockets
});
// create the sender
ServiceBusSender sender = client.CreateSender(queueName);
// create a message that we can send. UTF-8 encoding is used when providing a string.
ServiceBusMessage message = new ServiceBusMessage($"Hello world cancel 13 {DateTime.Now}");
// add 5 minutes delay
long seq = await sender.ScheduleMessageAsync(message,
DateTimeOffset.Now.AddMinutes(60)
);
The message sequence number you get back when scheduling is suitable for cancelling but doesn't allow receiving that message earlier. The service doesn't allow early receiving, as anything that gets the messages to get the active messages (not in the future). For this scenario, I would suggest keeping the data in a database and not leveraging the queue as the database.
I am working with an azure service bus queue configured to be FIFO (First input first output). I work on an order application with the following states "Pending", "Received" and "Sent". therefore I have grouped the messages by the "SessionId" service bus option, setting the orderId as sessionId so that it processes the messages in order in case of horizontal scaling.
So far it works perfectly, the problem I have found is when a message in "pending" or "Received" status fails due to a timeout and goes to the dead letter queue. The message in "sent" status is processed correctly and then when the support team re-sends the "Pending" or "Received" status message to the queue it is processed correctly marking the order in a previous status instead of "sent" ".
I can think of several ways to control this, for example that the support team looks at the status of the order before reprocessing the message from the dead letter queue :) but I would like to know if service bus offers the possibility that if there is a message in the dead letter queu all the messages in the session queue that have the same sessionId go to the dead letter queu. Finallly, my question is:
Is there a way to configure azure service bus so that if there are any messages in the dead letter queue it sends all messages with the same sessionId to the dead letter queue?
Thank you very much!!!
I would like to know if service bus offers the possibility that if there is a message in the dead letter queue all the messages in the session queue that have the same sessionId go to the dead letter queue.
No, there is no such offering by Service Bus by default.
Is there a way to configure azure service bus so that if there are any messages in the dead letter queue it sends all messages with the same sessionId to the dead letter queue?
Yes, you can do that. You can first peek the messages in your dead-letter queue to fetch all the session ids. Then you can receive the messages in your main queue whose session id is in the DLQ, and then move those messages to DLQ. Here's one such logic I've implemented in dot net using the latest version of Service Bus SDK.
var queueName = "<queue>";
var connectionString = "<connection-string>";
var client = new ServiceBusClient(connectionString);
var sessionIdInDLQList = new List<string>();
var receiver = client.CreateReceiver(queueName, new ServiceBusReceiverOptions { SubQueue = SubQueue.DeadLetter });
var message = await receiver.PeekMessageAsync();
while (message != null)
{
if (!sessionIdInDLQList.Contains(message.SessionId))
sessionIdInDLQList.Add(message.SessionId);
message = await receiver.PeekMessageAsync();
}
foreach (var sessionId in sessionIdInDLQList)
{
var session = await client.AcceptSessionAsync(queueName, sessionId);
message = await session.ReceiveMessageAsync(TimeSpan.FromSeconds(20));
while (message != null)
{
await session.DeadLetterMessageAsync(message, "Message with this session is to be dead-lettered!");
message = await session.ReceiveMessageAsync(TimeSpan.FromSeconds(20));
}
}
In your case, you need to do this before your consumers start reading the messages, probably you can write this in your consumer application or any trigger application like Azure Function or worker role. That’s upto your method of handling.
You can try this code to read Dead Letter from Queue.
public static async Task GetMessage()
{
string topic = "myqueue1";
string connectionString = "Endpoint = sb://xxx.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxx";
var servicebusclient = new ServiceBusClient(connectionString);
var reciveroptions = new ServiceBusReceiverOptions { SubQueue = SubQueue.DeadLetter };
var reciver = servicebusclient.CreateReceiver(topic, reciveroptions);
// 10 number of message read from Queue
await receiver.PeekMessageAsync(10);
}
after receiving message from Dead Letter you can send to queue.
As per Microsoft official documents
There's no automatic cleanup of the DLQ. Messages remain in the DLQ
until you explicitly retrieve them from the DLQ and call Complete() on
the dead-letter message.
These following document help you.
Thanks Casually Coding for posting post on Read Message from the Dead Letter Queue
Microsoft Documents Using Dead-Letter Queues to Handle Message Transfer Failures , Receive Message from Dead letter queue
For purposes of our integration checking, I want to count the number of messages on an Azure queue. The method looks like this:
internal void VerifyMessagesOnQueue(string queueNameKey, int expectedNumberOfMessages)
{
var azureStorageConnectionKey = ConfigurationManager.AppSettings["AzureStorageConnectionKey"];
var storageAccount = CloudStorageAccount.Parse(azureStorageConnectionKey);
var queueClient = storageAccount.CreateCloudQueueClient();
var queue = queueClient.GetQueueReference(ConfigurationManager.AppSettings[queueNameKey]);
var messages = queue.PeekMessages(int.MaxValue);
messages.Count().Should().Be(expectedNumberOfMessages);
}
Right now I'm using var messages = queue.PeekMessages(int.MaxValue); to try to get all the messages on a queue. It returns an HTML repsonse 400. I have tried var messages = queue.PeekMessages(expectedNumberOfMessages);, but when expectedNumberOfMessages is 0, I also get an HTML response 400.
How can I reliably check the number of messages on an Azure queue without disrupting it (this is why I was using .PeekMessage)?
I want to count the number of messages on an Azure queue
I suggest you could try the following code to achieve your goal. I have created a console project to test.
StorageConnectionString in App.config:
<appSettings>
<add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=×××;AccountKey=×××" />
</appSettings>
Code in Program.cs:
static void Main(string[] args)
{
string Queue_Name = "myqueue";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
Microsoft.Azure.CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference(Queue_Name);
queue.FetchAttributes();
var count=queue.ApproximateMessageCount;
Console.WriteLine("message number in queue:"+count);
}
The Result about queue count:
Janley Zhang's answer is correct. To get approximate messages count in a queue, you will need to fetch queue's attribute. Adding a new answer to clarify the following:
Right now I'm using var messages = queue.PeekMessages(int.MaxValue);
to try to get all the messages on a queue. It returns an HTML repsonse
400. I have tried var messages = queue.PeekMessages(expectedNumberOfMessages);, but when
expectedNumberOfMessages is 0, I also get an HTML response 400.
Essentially PeekMessages is used to retrieve the messages from the top of the queue without altering the retrieved messages visibility.
Maximum messages that can be fetched from a queue in a single request is 32 and the minimum is 1. Please check this link (URI Parameters section ) for more details.
In both of scenarios, you're specifying a count that is out of allowed range (1 - 32) and this is why you're getting the 400 error back from the queue.
I have created the code to get the Queue Client data using QueueClient.Receive() with Broken Message
BrokeredMessage deadmessage = client.Receive();
byte[] dataRaw = deadmessage.GetBody<byte[]>();
Due to some corrupted data, I got the exception on second line, while get the body of the broken message. So i was try to get the body of the message on catch block with SteamReader.
Stream stream = deadmessage.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
I experienced with below exception, Could anyone help me with appropriate fixes?
Exception details :
The message body cannot be read multiple times. To reuse it store the value after reading.
To take multiple attempts to read message body you need to read it as a stream first
serviceBusClient.GetBody<Stream>()
Then you can try to interpret it by different ways. For example it can be serialized directly by following way:
var brokeredMessage = new BrokeredMessage(message);
serviceBusClient.Send(brokeredMessage);
but it's better to serialize it to json first.
var brokeredMessage = new BrokeredMessage(JsonConvert.SerializeObject(message));
serviceBusClient.Send(brokeredMessage);
it's more safe in my view because json serialization ignores namespaces of message type, so you will not break your process when you move class of message to another namespace.
Suppose you are starting to send and read messages serialized in json but some old messages can be still binary serialized. In this case you can use the following logic:
public static T DeserializeMessage<T>(BrokeredMessage brokeredMessage)
{
using (var stream = brokeredMessage.GetBody<Stream>())
using (var streamReader = new StreamReader(stream))
{
string bodyText = streamReader.ReadToEnd();
try
{
return JsonConvert.DeserializeObject<T>(bodyText);
}
catch (JsonReaderException)
{
stream.Position = 0;
var reader = XmlDictionaryReader.CreateBinaryReader(stream, XmlDictionaryReaderQuotas.Max);
var serializer = new DataContractSerializer(typeof(T));
var msgBody = (T)serializer.ReadObject(reader);
return msgBody;
}
}
}
If you need to try to deserialize the message as another type, catch System.Runtime.Serialization.SerializationException on serializer.ReadObject(reader).
As Sean Feldman mentioned that if message is corrupted, then it will be handled by dead-letter queue.
Service Bus queues and topic subscriptions provide a secondary sub-queue, called a dead-letter queue (DLQ). The dead-letter queue does not need to be explicitly created and cannot be deleted or otherwise managed independent of the main entity.
The purpose of the dead-letter queue is to hold messages that cannot be delivered to any receiver, or simply messages that could not be processed.
If you need to know how to create and user Service Bus Queue we can refer to get start with Service Bus queues
To reuse it store the value after reading.
If it be can be read correctly then we can store it messageid and vaule for reuse.
The DLQ is mostly similar to any other queue.
If it is corrupted data, we can get it from the dead-letter queue as mormal queue.
string connectionString = CloudConfigurationManager.GetSetting('Microsoft.ServiceBus.ConnectionString');
QueueClient Client = QueueClient.CreateFromConnectionString(connectionString, deletLetterQueueName);
var message = Client.Receive(TimeSpan.FromSeconds(3));
if (message != null)
{
var ret = message.GetBody<stream>();
message.Complete();
}
I found another reason of the exception. The thing is that when I debugged different problem I got this exception. After some experiments I realized that Visual Studio reads the message behind the scene to show it for example in Watch panel, and when my code tried to get the message it was already read by Visual Studio.
To avoid this it needs to wrap the message to a property with backing field, which will store the value. And then I realized that the exception message abstracly says to make this.
So you should consider that it can be read behind the scene
Is there anyway to get a timestamp as to when a message is placed on my Azure queue? What is the best way?
For example, a partner sends a message to my queue and I want to know the time the partner placed a specific message in the queue.
Thanks
If you're using the .NET API, the property InsertionTime in the CloudQueueMessage you get when fetching a message or peeking the queue will contain;
The time that that message was added to the queue.
For example, in Java code the following would retrieve a message (then get the insertion time) and then peek at the next message (and get its insertion time.) This also shows how access to the queue can be controlled by using a revocable stored access policy string.
public class Dequeue {
public static void main(String[] args) throws InvalidKeyException, URISyntaxException, StorageException
{
String sas = "sv=2012-02-12&sig=XUHWUy2ayrcEjNvZUhcgdPbgKZflwSXxtr0BH87XRII%3D&si=heath";
StorageCredentialsSharedAccessSignature credentials = new StorageCredentialsSharedAccessSignature(sas);
URI baseuri = new URI("http://grassy.queue.core.windows.net");
CloudQueueClient queueClient = new CloudQueueClient(baseuri,credentials);
CloudQueue queue = queueClient.getQueueReference("queue1");
CloudQueueMessage retrievedMessage = queue.retrieveMessage();
System.out.println(retrievedMessage.getMessageContentAsString());
System.out.println(retrievedMessage.getInsertionTime());
queue.deleteMessage(retrievedMessage);
CloudQueueMessage peekedMessage = queue.peekMessage();
System.out.println(peekedMessage.getMessageContentAsString());
System.out.println(peekedMessage.getInsertionTime());
} }