We have an NServiceBus endpoint that monitors an Azure Service Bus Queue (using Azure as a transport). But not all the clients that send messages to the queue are .NET-based.
Can an NServiceBus endpoint be configured to accept a simple string as input?
I've tried intercepting messages with a class that implements IMutateIncomingMessages, but at this point deserialization from the Azure transport has already failed.
I can inspect the message coming in in a class implementing IMutateIncomingTransportMessages, but I'm not sure if this is the right place.
What is the best way to configure NServiceBus to handle a message being published in the following format (keep in mind this can also come via the Java or Node SDKs, or via an Azure REST endpoint):
var brokered = new BrokeredMessage("This plain string represents the data.");
queueClient.Send(brokered);
Deserialization of this message will fail, because it contains a string, not a byte array as expected by the Azure transport deserializer.
PS: I know it is possible to expose the endpoint as a WCF service, but currently we only have NServiceBus.Host processes that pull from the queue and the WCF solution does not feel like the right solution to me.
As mentioned on twitter earlier, but just including it here for completeness...
If you want to integrate natively, then you have to modify parts of the nsb pipeline to accomodate for your environment.
See https://github.com/yvesgoeleven/NServiceBus.AsbNativeIntegration for an example of such an integration.
Related
I am working on a POC where I have a publisher which is publishing message to Azure Service Bus using Topics. Multiple subscribers have subscribed to the topic.
One of the subscriber wants to send message to Salesforce. What is the best way to send message to Salesforce
I have following options in mind:
Use Azure function to listen to Service bus and then connect with Salesforce to send data. Not sure if a connector already exists.
Read data from Service bus using a .net client and then send message to Azure Logic Apps. From Logic app use Salesforce connector to send message to Salesforce. Not sure if Logic apps can directly subscribe to Service bus.
What are pros can cons of both the options ?
Both approaches are valid but the one to chose depends on the level of comfort you have with the aforementioned technologies.
Integration with Salesforce is done via REST API. The LogicApps connector simplifies it quite a bit. Except, I would not read data from Service Bus message using a .NET client and then send message to Azure Logic Apps, but rather trigger Logic Apps with an incoming message using Service Bus connector and invoke Salesforce connector. This way there will be less moving parts.
In case you’re comfortable with Salesforce REST API, Function is a valid approach as well.
I've got started with Azure Service Bus in Azure. Having gone through some references over the Internet, it seems that people use BrokeredMessage class in Microsoft.ServiceBus.Messaging rather than Message class in Microsoft.Azure.ServiceBus.
BrokeredMessage class: https://learn.microsoft.com/en-us/dotnet/api/microsoft.servicebus.messaging.brokeredmessage?view=azure-dotnet
Message class: https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.servicebus.message?view=azure-dotnet
I can send both of message 'types' to Azure Service Bus and also work with them over Azure Service Bus. Also, both can be used in Asynchronous operation. What are the main differences between the two types?
[Update] This article gives best practices of Azure Service Bus when exchaning brokered message (https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-performance-improvements). I'm unsure if it is also referred to Message in Microsoft.Azure.ServiceBus.
If it's a new project to use Azure Service Bus, I would recommend the following:
Prefer the new .NET Standard client (Microsoft.Azure.ServiceBus) with Message.
Be careful with documentation and various resources. They mostly cater to the old client (hopefully MSFT doco soon to change).
If you need transport transactions spanning different entities, the new client can't provide it yet.
If you need management operations, the new client won't provide it. Ever. Instead you'd have to use Management library or wait till a replacement package for NamespaceManager is out.
If you have old systems emitting messages sent as serialized data and not Stream, implementations using new Client have to know about it and use extension method provided by the client to process those messages. New client only deals with Stream based messages.
As Gaurav Mantiri mentioned that Microsoft.Azure.ServiceBus is the newer version of the library built using .Net Standard.
You could get the detail information from github.
This is the next generation Service Bus .NET client library that focuses on queues & topics. This library is built using .NET Standard 1.3.
We are evaluating "Azure service bus" to use between web server and app server for request response pattern. We are planning to have two queues:
Request Queue
Response Queue
Webserver will push a message to request queue and subscribe to response queue.
By comparing the MessageID and CorrelationId, it can receive the response back, which can be sent back to browser.
But over cloud, using elastic scaling, we can increase/decrease web server (and app server) instances.
We are wondering if this pattern will work here optimally.
To make this work, we will have to have one Request queue and multiple topics (one for each web server instance).
This will have two down sides:
Along with increasing/decreasing web server instance, we will have
to create/delete topic as well.
All the message will be pushed to
all the topics. So, every message will be processed by all the web
servers. And this is not an efficient way.
Please share your thoughts.
Thanks In Advance
When you scale out your endpoint, you don't want to have an instance affinity. You want to rely on the competing consumers and not care which instance of your endpoint processes messages.
For example, if you receive a response and write that to a database, most likely you don't care what instance of an endpoint has written the data. But if you have some in-memory state or anything other info only available to the endpoint that has originated the request and processing reply messages requires that information, then you have instance affinity and need to either remove it or use technology that allows to address that. For example, something like a SignalR with a backplane to communicate a reply message to all your web endpoint instances.
Note that ideally you should avoid instance affinity as much as you can.
I know this is old, but thought I should comment to complete this thread.
I agree with Sean.
In principle, Do not design with instance affinity in mind.
Any design should work irrespective of number of instances and whichever instance runs the code.
Microsoft does recommend the same when designing application architecture for running in the cloud.
In your case, I do not think you should plan to have one topic for each instance.
You should just put the request messages into one topic, with a subscription to allow your receiving app service to process those request messages.
When your receiving app service scales out, that's where your design needs to allow reading messages from the subscription from multiple receivers (multiple instances), which is described in the Competing consumers pattern.
https://learn.microsoft.com/en-us/azure/architecture/patterns/competing-consumers
Please post what you have finally implemented.
I have an asp.net mvc 5 website running in an azure app service. My site allows customers to communicate via email uploading documents if required.
I modeled this as sending email with attachments (max 4mb) using sendgrid with azure webjob.
I cannot use an azure queue since the message size is way too small.
Therefore I have to communicate with a triggered webjob via kudu. I've read the docs and the argument seems to be a simple string which I can either read it off of the arguments or WEBJOBS_COMMAND_ARGUMENTS environment variable.
My poco class to send email has customer properties (mostly string) + the file the user uploads is of type HttpPostedFileBase.
How do I pass this poco class to the triggered webjob via kudu?
Should I json serialize it and pass it as a string?
Any other options?
I need help.
I cannot use an azure queue since the message size is way too small.
That's not a limitation on Azure Queues.
Should I json serialize it and pass it as a string?
That's basically what the azure queues is doing.
My suggestion is if you want to use Azure WebJobs to send an email, you can just send a message to a queue with the appropriate payload. When you listen to the queue you can get the attachment (maybe store it as a blob?) and send it using SendGrid.
When you listen to the message you can either get a string or POCO object. If you're going to get a POCO, I suggest not to send the HttpPostedFileBase property since that can make the message too large (that is a limitation on the message).
Hope this helps
I'm simply trying to work out how best to retrieve messages as quickly as possible from an Azure Service Bus Queue.
I was shocked that there wasn't some way to properly subscribe to the queue for notifications and that I'm going to have to poll. (unless I'm wrong in which case the documentation is terrible).
I got long polling working, but checking a single message every 60 seconds looks like it'll cost around £900 per month (again, unless I've misunderstood that). And if I add a redundant/second service to poll it'll double.
So I'm wondering what the best/most cost efficient way of doing it is.
Essentially I just want to take a message from the queue, perform an API lookup on some internally held data (perhaps using hybrid services?) and then perhaps post a message back to a different queue with some additional information .
I looked at worker roles(?) -- is that something that could do it?
I should mention that I've been looking at doing this with node.js.
Check out these videos from Scott Hanselman and Mark Simms on Azure Queues.
It's C# but you get the idea.
https://channel9.msdn.com/Search?term=azure%20queues%20simms#ch9Search
Touches on:
Storage Queues vs. Service Bus Queues
Grabbing messages in bulk vs. one by one (chunky vs. chatty)
Dealing with poison messages (bad actors)
Misc implementation details
Much more stuff i can't remember now
As for your compute, you can either do a VM, a Worker Role (Cloud Services), App Service Webjobs, or Azure Functions.
The Webjobs SDK and Azure Functions bot have a way to subscribe to Queue events (notify on message).
(Listed from IaaS to PaaS to FaaS - Azure Functions - if such a thing exists).
Azure Functions already has sample code provided as templates to do all that with Node. Just make a new Function and follow the wizard.
If you need to touch data on-prem you either need to look at integrating with a VNET that has site-to-site connectivity back to your prem, or Hybrid Connections (App Service only!). Azure Functions can't do that yet, but every other compute is a go.
https://azure.microsoft.com/en-us/documentation/articles/web-sites-hybrid-connection-get-started/
(That tutorial is Windows only but you can pull data from any OS. The Hybrid Connection Manager has to live on a Windows box, but then it acts as a reverse proxy to any host on your network).
To deal with Azure ServiceBus Queue easily, the best option seems to be Azure Webjob.
There is a ServiceBusTrigger that allows you to get messages from an Azure ServiceBus queue.
For node.js integration, you should have a look at Azure Function. It is built on top of the webjob SDK and have node.js integration :
Azure Functions NodeJS developer reference
Azure Functions Service Bus triggers and bindings for queues and topics
In the second article, there is an example on how get messages from a queue using Azure Function and nodejs :
module.exports = function(context, myQueueItem) {
context.log('Node.js ServiceBus queue trigger function processed message', myQueueItem);
context.done();
};