I have this simple application that uses node-amqp10 to read messages from some subscriptions and process/accepts or rejects messages due to some error that might happen.
My question is, is it possible for the consumer to set any additional property into the message when rejecting? (like DeadLetterError)
I've thought the receiverLink.reject would get the error parameter and set into the message automatically, but maybe this is not even possible (consumer setting properties into the message)
I am assuming you are using Azure Service Bus based on the tag in your question. If you use #azure/service-bus package instead, you have the option of updating properties of the message. Also while dead lettering you can provide deadLetterReason and deadLetterErrorDescription like below:
async function receiveMessage() {
// If receiving from a subscription you can use the createReceiver(topicName, subscriptionName) overload
const receiver = sbClient.createReceiver(queueName);
const messages = await receiver.receiveMessages(1);
if (messages.length) {
console.log(
">>>>> Deadletter the one message received from the main queue - ",
messages[0].body
);
// Deadletter the message received
await receiver.deadLetterMessage(messages[0], {
deadLetterReason: "Incorrect Recipe type",
deadLetterErrorDescription: "Recipe type does not match preferences."
});
} else {
console.log(">>>> Error: No messages were received from the main queue.");
}
await receiver.close();
}
For more details, refer Azure Service Bus client library for Javascript.
My question is, is it possible for the consumer to set any additional
property into the message when rejecting? (like DeadLetterError)
No, you can't. I know what you want to do, But the method of the basic module '#azure/service-bus' is not designed as you think.
For NODE.JS, it is deadLetterMessage(ServiceBusReceivedMessage, DeadLetterOptions & [key: string]: any).
JavaScript reference API:
https://learn.microsoft.com/en-us/javascript/api/#azure/service-bus/servicebusreceiver?view=azure-node-latest#deadLetterMessage_ServiceBusReceivedMessage__DeadLetterOptions____key__string___any_
It don't allow you to add custom properties.
But if you using C#, then it is possible:
DeadLetterAsync(String, IDictionary<String,Object>)
C# API reference:
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.servicebus.queueclient.deadletterasync?view=azure-dotnet
In short, for NODE.JS you can only change the description of the dead-letter message, but can't add custom properties.
Related
I have an Azure Message Bus Topic.
I have one "Session enabled" Azure Message Bus Consumer for this Topic.
I have around 3 Worker Services that are using the same Consumer. So the work is shared between these 3 Workers.
The Messages which are sent to the consumer need to be ordered, thats why I am using the "Session Feature" on the Consumer.
I believe that on a first Message, the Session of the Message gets bind to a Worker Service.
For certain Reasons I want to abandon not only a Message but also the session so that it can be picked up by another of the 3 Worker Services.
My questions:
Is this possible?
If yes how can I do this in the code?
Is there something like "Accept Session Or Not" Handler which kicks in when Message received?
See code below:
private void SetupServiceBusSessionProcessors2()
{
var busProcessorOptions = new ServiceBusSessionProcessorOptions();
var busProcessor = _busClient.CreateSessionProcessor("fooTopic", "fooSubscription", busProcessorOptions);
busProcessor.ProcessMessageAsync += args => ProcessSessionMessageHandler2(args);
}
private async Task ProcessSessionMessageHandler2(ProcessSessionMessageEventArgs args)
{
if (false) // Condition here which Abandons Message AND Session
{
// the following line of code seems only to abandon the Message
// but it seems like the session is locked to this service
// i want that other services which are listening via the same consumer can try to handle the session
await args.AbandonMessageAsync(args.Message);
}
}
This is possible in version 7.3.0-beta.1 using the ReleaseSession method on the event args. Note that this is a beta version so the API is subject to change before there is a stable release.
How can I send message to deadletter queue?
serviceBusService.receiveQueueMessage(MESSAGE_QUEUE, {isPeekLock: true}, (error, message) => {
...... // want to put message to deadletter queue if there is exception
serviceBusService.deleteMessage(message, error => {
});
});
Mostly, you'd want to rely on the system to decide when to move a message to DLQ and make it the messaging engine's responsibility as much as possible (and not explicitly put a message on DLQ.) It also appears that the guidance in this scenario was provided via documentation here: How to handle application crashes and unreadable messages
Looks like you are using the older azure-sb package that relies on the HTTP REST apis. If you instead use the newer #azure/service-bus package which uses the faster AMQP implementation, there is a deadletter() method on the message you receive that you can use to send the message to the dead letter queue.
This documentation says, that it is possible to automatically send error messages from global errorChannel to specific destination, the only thing needed is to set the following property:
spring.cloud.stream.bindings.error.destination
I am really confused with this part. Just adding property doesn't work (I use RabbitMQ, Exchange and Queue are configured). So I can see error handled by #ServiceActivator in my application, but it is not sent to "myErrors" exchange in RabbitMQ. Looks like this property does not define outbound binding for errorChannel.
If I create output "error" channel explicitly in my application,
/**
* Error channel name.
*/
String ERROR_ONE = "error";
#Output(MySink.ERROR_ONE)
MessageChannel errorOne();
and explicitly send error message to it from my global handler, for example by declaring outputChannel:
#ServiceActivator(inputChannel = "errorChannel", outputChannel = "error")
public ErrorMessage errorGlobal(ErrorMessage message) {
System.out.println("Handling ERROR GLOBAL SA: " + message);
return message;
}
it will work. But this can be done to binding-specific handlers also, so there is nothig errorChannel-specific here.
The question: Do I miss something? Can publish be done without defining outbound channels explicitly?
That documentation about ...binding.errors... is obsolete and needs to be fixed.
You should use auto-bind-dlq and republish-to-dlq instead and the framwork will publish the failed message to the dlq, with additional headers with information about the failure.
Background
I have several clients sending messages to an azure service bus queue. To match it, I need several machines reading from that queue and consuming the messages as they arrive, using Node.js.
Research
I have read the azure service bus queues tutorial and I am aware I can use receiveQueueMessage to read a message from the queue.
However, the tutorial does not mention how one can listen to a queue and read messages as soon as they arrive.
I know I can simply poll the queue for messages, but this spams the servers with requests for no real benefit.
After searching in SO, I found a discussion where someone had a similar issue:
Listen to Queue (Event Driven no polling) Service-Bus / Storage Queue
And I know they ended up using the C# async method ReceiveAsync, but it is not clear to me if:
That method is available for Node.js
If that method reads messages from the queue as soon as they arrive, like I need.
Problem
The documentation for Node.js is close to non-existant, with that one tutorial being the only major document I found.
Question
How can my workers be notified of an incoming message in azure bus service queues ?
Answer
According to Azure support, it is not possible to be notified when a queue receives a message. This is valid for every language.
Work arounds
There are 2 main work arounds for this issue:
Use Azure topics and subscriptions. This way you can have all clients subscribed to an event new-message and have them check the queue once they receive the notification. This has several problems though: first you have to pay yet another Azure service and second you can have multiple clients trying to read the same message.
Continuous Polling. Have the clients check the queue every X seconds. This solution is horrible, as you end up paying the network traffic you generate and you spam the service with useless requests. To help minimize this there is a concept called long polling which is so poorly documented it might as well not exist. I did find this NPM module though: https://www.npmjs.com/package/azure-awesome-queue
Alternatives
Honestly, at this point, you may be wondering why you should be using this service. I agree...
As an alternative there is RabbitMQ which is free, has a community, good documentation and a ton more features.
The downside here is that maintaining a RabbitMQ fault tolerant cluster is not exactly trivial.
Another alternative is Apache Kafka which is also very reliable.
You can receive messages from the service bus queue via subscribe method which listens to a stream of values. Example from Azure documentation below
const { delay, ServiceBusClient, ServiceBusMessage } = require("#azure/service-bus");
// connection string to your Service Bus namespace
const connectionString = "<CONNECTION STRING TO SERVICE BUS NAMESPACE>"
// name of the queue
const queueName = "<QUEUE NAME>"
async function main() {
// create a Service Bus client using the connection string to the Service Bus namespace
const sbClient = new ServiceBusClient(connectionString);
// createReceiver() can also be used to create a receiver for a subscription.
const receiver = sbClient.createReceiver(queueName);
// function to handle messages
const myMessageHandler = async (messageReceived) => {
console.log(`Received message: ${messageReceived.body}`);
};
// function to handle any errors
const myErrorHandler = async (error) => {
console.log(error);
};
// subscribe and specify the message and error handlers
receiver.subscribe({
processMessage: myMessageHandler,
processError: myErrorHandler
});
// Waiting long enough before closing the sender to send messages
await delay(20000);
await receiver.close();
await sbClient.close();
}
// call the main function
main().catch((err) => {
console.log("Error occurred: ", err);
process.exit(1);
});
source :
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-nodejs-how-to-use-queues
I asked myslef the same question, here is what I found.
Use Google PubSub, it does exactly what you are looking for.
If you want to stay with Azure, the following ist possible:
cloud functions can be triggered from SBS messages
trigger an event-hub event with that cloud function
receive the event and fetch the message from SBS
You can make use of serverless functions which are "ServiceBusQueueTrigger",
they are invoked as soon as message arrives in queue,
Its pretty straight forward doing in nodejs, you need bindings defined in function.json which have type as
"type": "serviceBusTrigger",
This article (https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-service-bus#trigger---javascript-example) probably would help in more detail.
I have a website and and a webjob, where the website is a oneway client and the webjob is worker.
I use the Azure ServiceBus transport for the queue.
I get the following error:
InvalidOperationException: Cannot use ourselves as timeout manager
because we're a one-way client
when I try to send Bus.Defer from the website bus.
Since Azure Servicebus have built in support for timeoutmanager should not this work event from a oneway client?
The documentation on Bus.Defer says: Defers the delivery of the message by attaching a header to it and delivering it to the configured timeout manager endpoint
/// (defaults to be ourselves). When the time is right, the deferred message is returned to the address indicated by the header."
Could I fix this by setting the ReturnAddress like this:
headers.Add(Rebus.Messages.Headers.ReturnAddress, "webjob-worker");
Could I fix this by setting the ReturnAddress like this: headers.Add(Rebus.Messages.Headers.ReturnAddress, "webjob-worker");
Yes :)
The problem is this: When you await bus.Defer a message with Rebus, it defaults to return the message to the input queue of the sender.
When you're a one-way client, you don't have an input queue, and thus there is no way for you to receive the message after the timeout has elapsed.
Setting the return address fixes this, although I admit the solution does not exactly reek of elegance. A nicer API would be if Rebus had a Defer method on its routing API, which could be called like this:
var routingApi = bus.Advanced.Routing;
await routingApi.Defer(recipient, TimeSpan.FromSeconds(10), message);
but unfortunately it does not have that method at the moment.
To sum it up: Yes, setting the return address explicitly on the deferred message makes a one-way client capable of deferring messages.