In Azure Service Bus, if below is the sequence of events, then all's fine -
Create Topic
Create Subscriptions inside Topic
Send Messages to Topic
With above, the subscriptions are triggered when a message is sent. This is expected.
However, if we modify the above sequence like this
Create Topic
Send Messages to Topic
Create Subscriptions inside Topic
In this case, as messages are sent to a topic whilst no subscriptions were in place, when the subscriptions are indeed created, the previously sent messages don't show up in their list. Those messages are essentially 'lost'. Am not able to see those messages in Service Bus Explorer too.
The above sequence flow is relevant because we have detached publishers and subscribers, where the publisher just sends a message and subscribers, when they come online, create the subscriptions and handle them. The order in which the publisher and subscriber come online is not guaranteed.
How can I access/process messages sent to the topic before the subscriptions are created? What happens to such messages in the first place?
Thanks
It turns out that the above behavior is by design - if no subscriptions are there, then the message is lost.
To overcome this, Azure Service Bus provides a property on topic to enable the pre-filtering of messages before they are sent. So, if no filters/subscriptions are available, it'll throw an exception
Set the option on the Topic
namespaceManager.CreateTopicAsync(new TopicDescription(topicName)
{
EnableFilteringMessagesBeforePublishing = true
});
Whilst sending the message, check for exception
try
{
await topicClient.SendAsync(brokeredMessage);
}
catch (NoMatchingSubscriptionException ex)
{
// handle the exception, maybe send it to dead letter queue using DeadLetterAsync
}
Related
I have an Azure Service Bus topic. Messages send to the topic contain application properties.
I want to set permissions on the topic based on values of one or more application properties, i.e. only when identity A sends a message with property e.g. Destination = 'service' the message is allowed to be added to the topic.
The reason behind this: assume a topic where there are multiple publishers and multiple subscribers to consume the messages from the publishers. But each message shall only go to specific consumers using filters. An actor with bad intentions or just by accident could choose a filter value and send the message therefore to wrong consumers. And I don't want to create multiple topics. I want to limit the publishers to send messages to certain consumers by limiting their permissions by message attributes with certain values on a certain topic.
Is this possible? If yes, how?
You can start a separate app which will peek the message in the queue
check for the properties and then abandon the messages.
To peek create a receiver and use receiver.ReceiveMessageAsync();
to peek the message. After this you can apply your logic to filter
the messages using the properties.
After that use AbandonAsync to abandon the message with wrong properties.
// create a reciever and peek the messages.
await using var client = new ServiceBusClient(connectionString);
//reciever
ServiceBusReceiver receiver = client.CreateReceiver(queueName) ;
// get the messages
ServiceBusReceivedMessage receivedMessage = await receiver.ReceiveMessageAsync();
// write you logic for filtering the message
//now we can abandon the message. Here "reason":" Wrong message" is optional
await receivedMessage.AbandonAsync(new Dictionary<string, object> { { "Reason", "wrong properties"} });
Refernce:
Refer the article by sfeldman
Peek messages
You can't use ServiceBus (or any message broker) the way you describe.
My advice is to consider messages only as signals to do something. But the service is entirely responsible to check if the message is legitimate or not (by calling the source service or another source of truth synchronously for example).
Imagine this scenario:
SourceApp send a DeletedEvent{CustomerId="42"}.
DestApp receives this message, and before deleting data related to this customer, makes an HTTP call to the customer service GET CustomerService/42.
If the customer is deleted => perfect
If not... something went wrong, what you described by an actor with bad intentions or just by accident
Also, if you use the same topic for a lot of applications consider using AAD authentication, with managed identity per app. Or at least forbid sharing SAS keys. Every app should have dedicated connection string, scoped to the subscription or the topic with only the access needed.
I have tried to send a message to the topic of Azure Service Bus but it is not having any subscribers for now and it was showing there is no matching subscription found for it. So I have created a temporary subscriber for that and sent the message successfully.
So my question is it compulsory to having a subscriber to send the message to the topic??
Thanks for the help!
You do not have to have subscriptions under the topic to be able to publish messages. But in order to receive messages, subscriptions have to exist. Subscribers can come online later and fetch those events. Without subscriptions, subscribers will not get anything as the topic doesn't know what to retain and for whom.
Can we Read and Delete one or more message from azure service bus topic using logic app?
what does the Complete Message Action will do on a Logic App.
Read
You could read a message from the service bus. You could use one of the below connectors and triggers :
Get messages from a queue
Get messages from a topic
Or you could use one the following triggers which will get the message content to be read :
When a message is received in a queue (auto-complete)
When a message is received in a queue (peek-lock)
When a message is received in a topic subscription (auto-complete)
When a message is received in a topic subscription (peek-lock)
When one or more messages arrive in a queue (auto-complete) When one
or more messages arrive in a queue (peek-lock)
When one or more messages arrive in a topic (auto-complete)
When one or more messages arrive in a topic (peek-lock)
https://learn.microsoft.com/en-us/connectors/servicebus/#triggers
Complete Message Action (Also - Delete)
The operation completes a message in a queue or topic depending on which conector you use.
What does completes a message mean ?
When the receive operation is initiated, the message is locked by the receiving client. The message in the queue/topic remains undeleted. In case the message is not completed, the message would be available to be consumed again for the receiving clients after the lock expires.
So in short complete a message means that the message should be marked as processed and deleted from the queue or topic.
https://learn.microsoft.com/en-us/connectors/servicebus/#get-messages-from-a-queue-(peek-lock)
Update :
You could do something like below :
Explanation :
The above logic app gets triggered on a http request with the messages count that needs to be read.
Subsequently mentioned count of messages is retrieved from the service bus - topic combination and completed eventually.
I have several subscriptions that listens to different Topics for messages, and some of this messages are dependent on each other. So one message for one subscription "needs" to arrive before another messages in another subscriptions.
I could solve this by storing the messages temporary in a database, but I thought that if I get a message on one subscription and it's correlated messages on a another subscription hasn't arrived yet, I would just wait 1 second and put the first messages back on it's subscription so the correlated messages get's some more time to arrive first.
It's easy if it would have been a Queue, but now it's a subscription and that client don't have any "Send" methods on it.
I don't want to put the messages back on the Topics, since other subscription might not want that messages again.
Since subscriptions basically is a Queue it should be possible, so is there some "base object" that could be used to put messages directly to a subscription queue.
Best Regards
Magnus Gladh
While subscription is a queue behind the scenes, you cannot send messages directly to that queue. Instead, you should target a topic.
If you wish to abort the receive operation, you can when receiving in PeekLock mode.
I've looked everywhere for information on this but can't seem to find what i'm looking for.
I've got an azure topic, with one subscription.
The handler for the subscription failed some messages, they've been put on the deadletter queue.
I can access the messages but i'm at a loss for how to restore them.
I don't want to create a copy of the message and send it to the topic. I specifically want to move it to the subscription queue it came from.
Is there any way to do this?
I don't know if this is possible.
What we mostly do to handle & resume messages, is to use the pattern of defering messages. If a handler fails a message, he can then defer the message and keep track of the messagesequence number (typically in a management / tracking database or component).
Whenever the message needs to be reprocessed (retried), a receiver can then specifically get that message (using the same defered message id) and process it again.