Can we Read and Delete one or more message from azure service bus topic using logic app?
what does the Complete Message Action will do on a Logic App.
Read
You could read a message from the service bus. You could use one of the below connectors and triggers :
Get messages from a queue
Get messages from a topic
Or you could use one the following triggers which will get the message content to be read :
When a message is received in a queue (auto-complete)
When a message is received in a queue (peek-lock)
When a message is received in a topic subscription (auto-complete)
When a message is received in a topic subscription (peek-lock)
When one or more messages arrive in a queue (auto-complete) When one
or more messages arrive in a queue (peek-lock)
When one or more messages arrive in a topic (auto-complete)
When one or more messages arrive in a topic (peek-lock)
https://learn.microsoft.com/en-us/connectors/servicebus/#triggers
Complete Message Action (Also - Delete)
The operation completes a message in a queue or topic depending on which conector you use.
What does completes a message mean ?
When the receive operation is initiated, the message is locked by the receiving client. The message in the queue/topic remains undeleted. In case the message is not completed, the message would be available to be consumed again for the receiving clients after the lock expires.
So in short complete a message means that the message should be marked as processed and deleted from the queue or topic.
https://learn.microsoft.com/en-us/connectors/servicebus/#get-messages-from-a-queue-(peek-lock)
Update :
You could do something like below :
Explanation :
The above logic app gets triggered on a http request with the messages count that needs to be read.
Subsequently mentioned count of messages is retrieved from the service bus - topic combination and completed eventually.
Related
Is there a way to have multiple listening clients on one Azure Topic Subscription, and they all recieve ALL messages?
My understanding is that the only implementation of a Subscription is that the Published message is only delivered to ONE client on that subscription, as it is like a queue.
Can these messages be copied to multiple clients using the same Subscription?
EDIT: Potential use case example
A server notifies all of its clients (web clients via browser, or application), that are subscribed to the topic, of an object that has changed its value
More simply, multiple PCs are able to see a data value change
EDIT 2: My setup/what I'm looking for
The issue that I am running into is that a message is marked as consumed by one client, and not delivered to the other client. I have 3 PCs in a test environment:(1 PC publishing messages (we'll call this the Publisher) to the topic, and 2 other PCs subscribed to the topic using the same SubscriptionName (We'll call these Client 1 and Client 2)).
So we have this setup:
Publisher - Publishes to topic
Client 1 - Subscibed using SubscriptionName = Test1
Client 2 - Subscribed using SubscriptionName = Test1
The Publisher publishes 10 messages to the topic.
Client 1 gets Message 0
Client 2 gets Message 1
Client 1 gets Message 2
... And so on (Not all 10 messages are recieved by both Client 1 and Client 2)
I want the Clients to recieve ALL messages, like this:
Client 1 AND Client 2 get Message 0
Client 1 AND Client 2 get Message 1
Client 1 AND Client 2 get Message 2
... And so on.
Service Bus is a one-to-one or end-to-end messaging system.
What you need is Azure Event Hub or Event Grid.
It is not possible for both the client1 and client2 to get the same messsage.
To put it straight, when a message is received by client1 from a subscription and processed successfully, the message is removed from the subscription, so the client2 will not be able to receive the same message again.
Hope this clarifies.
Yes, its a one-to-one implementation, but, if you have real concern about message processing completing in sequential order then it depends on the Receive mode.
You can specify two different modes in which Service Bus receives messages.
Receive and delete.
In this mode, when Service Bus receives the request from the consumer, it marks the message as being consumed and returns it to the consumer application. This mode is the simplest model. It works best for scenarios in which the application can tolerate not processing a message if a failure occurs. To understand this scenario, consider a scenario in which the consumer issues the receive request and then crashes before processing it. As Service Bus marks the message as being consumed, the application begins consuming messages upon restart. It will miss the message that it consumed before the crash.
Peek lock.
In this mode, the receive operation becomes two-stage, which makes it possible to support applications that can't tolerate missing messages.
Finds the next message to be consumed, locks it to prevent other consumers from receiving it, and then, return the message to the application.
After the application finishes processing the message, it requests the Service Bus service to complete the second stage of the receive process. Then, the service marks the message as being consumed.
If the application is unable to process the message for some reason, it can request the Service Bus service to abandon the message. Service Bus unlocks the message and makes it available to be received again, either by the same consumer or by another competing consumer. Secondly, there's a timeout associated with the lock. If the application fails to process the message before the lock timeout expires, Service Bus unlocks the message and makes it available to be received again.
If the application crashes after it processes the message, but before it requests the Service Bus service to complete the message, Service Bus redelivers the message to the application when it restarts. This process is often called at-least once processing. That is, each message is processed at least once. However, in certain situations the same message may be redelivered. If your scenario can't tolerate duplicate processing, add additional logic in your application to detect duplicates. For more information, see Duplicate detection. This feature is known as exactly once processing.
Check this link for more details.
I am using Microsoft Azure Service Bus for .NET Standard.
I'm reading messages from a Service Bus queue using multiple consumers(multiple instances of MessageReceiver) with PeekLock read.
I am not completing the receiving operation for the messages in order for them to be deleted from the queue.
The problem is that the deliveryCount for the second message is basically the deliveryCount for the first message(usually 1) + 1, so 2. Somehow the deliveryCount is passed from one message to another. This problem doesn't occur when I receive both messages with the same receiver(_messageReceiver1 or _messageReceiver2).
msg1 = await _messageReceiver1.ReceiveAsync(80);
Writeline(msg1.SystemProperties.DeliveryCount);
msg2 = await _messageReceiver2.ReceiveAsync(80);
Writeline(msg2.SystemProperties.DeliveryCount);
After exceeding the max delivery count from the portal, because of the delivery count being passed from one message to another, all of my messages are sent to DLQ, even if I was trying to read them or not.
You should go with Topic Subscriptions for multiple consumers. Receiving the messages multiple times will increase the delivery count of the message. Once the count increased more then the Max Delivery Count of the Queue, the messages will be moved to Dead Letter Queue.
I have a simple Storage Queue setup that I'm adding messages too.
These messages were received by an Azure Function but they've failed processing.
Showing 0 of 3 messages in queue
Why can't I see the "failed" messages in the Storage Explorer?
Please note that Azure Storage Explorer is using Peek Messages API to show messages in a queue:
This operation retrieves one or more messages from the front of the queue, but does not alter the visibility of the message.
However, while Azure Function is processing the messages, it's using Get Messages API which makes the messages invisible to other consumers for a while:
The message is not automatically deleted from the queue, but after it has been retrieved, it is not visible to other clients for the time interval specified by the visibilitytimeout parameter.
Typically, when a consumer retrieves a message via Get Messages, that message is usually reserved for deletion until the visibilitytimeout interval expires, but this behavior is not guaranteed. After the visibilitytimeout interval expires, the message again becomes visible to other consumers. If the message is not subsequently retrieved and deleted by another consumer, the original consumer can delete the message using the original pop receipt.
In conclusion, if your messages weren't deleted by Azure Function (I guess it's the case per "Showing 0 of 3 messages in queue" message), you'll be able to see them after visibilitytimeout.
I have been created two logic apps one for send message in queue and another one is for receive messages from queue
Logic App Send Message : This Logic app fired on every 1 minute and insert message in queue
2. Logic App Receive Message : This Logic app Fired after 20 min And i want to fetch all Messages send by send logic in one call
Problem : In receive messages workflow i receive only 1 message per logic app trigger
Setting of Queue :
In Azure Service Bus, if below is the sequence of events, then all's fine -
Create Topic
Create Subscriptions inside Topic
Send Messages to Topic
With above, the subscriptions are triggered when a message is sent. This is expected.
However, if we modify the above sequence like this
Create Topic
Send Messages to Topic
Create Subscriptions inside Topic
In this case, as messages are sent to a topic whilst no subscriptions were in place, when the subscriptions are indeed created, the previously sent messages don't show up in their list. Those messages are essentially 'lost'. Am not able to see those messages in Service Bus Explorer too.
The above sequence flow is relevant because we have detached publishers and subscribers, where the publisher just sends a message and subscribers, when they come online, create the subscriptions and handle them. The order in which the publisher and subscriber come online is not guaranteed.
How can I access/process messages sent to the topic before the subscriptions are created? What happens to such messages in the first place?
Thanks
It turns out that the above behavior is by design - if no subscriptions are there, then the message is lost.
To overcome this, Azure Service Bus provides a property on topic to enable the pre-filtering of messages before they are sent. So, if no filters/subscriptions are available, it'll throw an exception
Set the option on the Topic
namespaceManager.CreateTopicAsync(new TopicDescription(topicName)
{
EnableFilteringMessagesBeforePublishing = true
});
Whilst sending the message, check for exception
try
{
await topicClient.SendAsync(brokeredMessage);
}
catch (NoMatchingSubscriptionException ex)
{
// handle the exception, maybe send it to dead letter queue using DeadLetterAsync
}