We need to process Azure Bus queue messages receiving configuration.
We need to implement an application that listens for these messages and processes them when they are received.
We can create .Net core console application to listen the queue.
But we need to host this in Linux. And how to host this console application in linux as always running service ?
Thanks in advance.
Related
I am looking to rewrite a legacy windows based system to make use of Azure serverless features. Issue I have is that the system has to receive messages coming in via TCP due to the upstream system only supporting that approach. The upstream system then receives the ACK of the handshake as confirmation the message has been accepted.
Are there any Azure services that can accept a TCP message on a none standard port that I could then use to place the message on to an Azure queue? I have found some articles using Service Fabric but nothing less than 3 years old so wonder if there is a better way of doing it currently?
So you can use IoT Hub Protocol Gateway which can be the middle-man for accepting TCP\UDP messages which can then forward the message to Event\IoT Hub. You can then use Event Hub trigger in Azure Functions to process your messages.
You can deploy the Protocol Gateway on an Azure VM.
I am writing an application where messages about data changes through an API are published over a RabbitMQ message bus. The messages are consumed by a SignalR hub and pushed to subscribed clients.
Now I find messages in my Eventlog that start with:
Exception: System.Threading.ThreadAbortException: Thread was being aborted.
I found SO questions and answers like:
Why am i getting "Thread was being aborted" in asp.net?
and
What exactly is Appdomain recycling
But that raises the question, if applications running in an AppDomain in the application pool is recycled upon inactivity, how can SignalR maintain a connection to subscribed clients?
Why does it work for SignalR to run in an IIS app pool but not a RabbitMQ consumer?
SignalR client libraries provided by the SignalR might have a default behavior of reconnecting once loosing connection (the JS one definitely does).
So having an Pool Recycle would just make your client reconnect to your server, getting again messages as before interruption.
We are designing a system that is web based but also uses NServiceBus and Azure Service Bus to communicate. We have an on premiss server running IIS for the application and also several cloud services running web roles for communicating with external parties (those parties call on a RESTful interface in the cloud and the message is put on the bus or vice versa).
Both the cloud solutions and the on premiss server need to subscribe and publish messages. The publishing does not seem an issue but what happens to those subscriptions if IIS shuts down the processes, do they get woken again when a message arrives or is the service bus really pull based subscription so requiring an active listener.
I have seen questions on here about hosting publishers but nothing about the safety of subscribers.
Extra Info:
Quite by chance we noticed that sometimes in our development environment the applications would need to be started a couple of times before the messages would start arriving. However it occurred to me that if there were actually messages already in the queue when the application started then they would be processed and otherwise not. So the restart just means that it sees older messages and processes them then once running it gets on just fine. However another colleague noted that nsb related startup log files were only being generated after he visited the website hosted in the same web application for the first time. I have just had a similar problem, messages were in the queue but the breakpoint on the handler was not being hit. When I hit a webapi method on the same iis application instance suddenly messages were being processed. So my conclusion from this is that no, it is not safe to host a subscriber under iis or in this case even a web role.
I am answering my own question based on my experience rather than any deep knowledge of NServiceBus or Azure Service Bus.
It seems that it is not safe to rely on the service bus listener to be active under IIS or a web role. Both can shut down the process and, as the bus listener relies on polling the service rather than messages being pushed to the listener from the service, so no further messages will be received until the process starts again.
The good news is that the process will be restarted when the associated web site or web service is hit. so if you absolutely know that you will receive more traffic on the site than on the bus then you might take the risk that the bus listener will remain active. However for our project we are in the process of splitting the listener into a separate windows service.
I am currently using signalR on Azure Websites with a single instance to push data to clients. No problems.
We're splitting our project into separate web/worker and wcf roles so we can scale them independently.
The site will work like this.
Scenario A
User submits some data to web role and it gets put in a service bus queue ready for worker A, sends a message to worker A that a new item has been added in case it's idle (to save polling). When worker A has processed it, sends a message back to web roles which pushes out to particular clients.
scenario B
receive data in wcf role and it gets put in a different service bus queue ready for worker B, wcf role sends message to worker B that a new item has been added in case it's idle. When worker B has processed it, sends a message to web roles and pushes it out to particular clients.
illustrated badly below:
I am going to enable signalR service bus backplane for the web roles to users. What i'm not sure about is how to get my roles communicating between each other.
I'll need:
web role => worker A
worker A => web role
wcf role => worker B
worker B => web role
Am I creating hubs on web, worker A and worker B all with service bus topics? And then connecting somehow with the signalr .net clients? How do I make sure it goes to all instances of the web role without exposing it publicly?
For some reason it seems simple for hundreds of clients to connect via JavaScript to my web role hub but try and connect some internal ones and I can't quite figure it out.
If anyones interested... What I ended up doing is this:
I created hubs on both the Web and Wcf role. The web role has a connection that allows javascript proxies at /signalr and the web and wcf role had one that didn't at /signalr-internal.
I used the Azure Service Bus as a backplane and let it handle both the web and wcf hubs automatically with no extra tinkering.
In the signalR authentication I probed to see where the connection was coming from (i.e an internal endpoint or the external ssl endpoing and denied / allowed access to particular hubs based on this. This allowed me to use the .net signalr clients on my workers that automatically connect / reconnect etc.
This ended up working nicely with no issues as of yet and it was simple to implement. I'll update if I run into any problems.
EDIT #1:
DO NOT USE THIS METHOD! Everything works splendidly until you actually deploy it into a live environment and then you get a host of issues that made me want to tear my hair out.
What I actually ended up doing (which work perfectly in live) was to use service bus Topics and create subscriptions to them for the listeners. This creates TCP connections and allows your communication to stay 100% internally without any crazy transport or boundary problems.
EDIT #2:
Since this post, Event Hubs were release and we switched over and never looked back. see last comment
Peter, realistically to get this approach to work you would need to switch to Web Roles or IIS hosted on an IaaS VM.
Currently Websites don't support Azure Virtual Networks which is the only way to enable private network inter-connectivity between instances on Azure.
You can add VMs, Web and Worker Roles to a Virual Network which should provide you with the access you're looking for without needing to expose everything via public endpoints.
I have a virtual machine on Azure which will listen to messages over the servicebus of Azure. And another developer needs to connect to this servicebus to send messages to my service. To do so, we need to come up with some protocol for this communication system. And I was thinking about using WSDL to make the server something webservice-like, but instead of listening to the standard HTTP ports it would connect to the service bus and within it a topic with subscription, or whatever. I'm still not sure what would be best.
So, is it possible? Has anyone done something similar before? Are there some examples?
Service Bus provides both Brokered and Relayed messaging models. With using Topics/Queues you are essentially sending and receiving message to/from the broker. So you can use APIs/protocols that the broker supports. As an example you can use the NetMessagingBinding from your service/client to send and receive messages through the queue. There is no contract here other than what you put in the BrokeredMessage body and properties values. The application on either end can decode/decipher that info as needed.
In the case of Relayed messaging however you are making a end-to-end connection between the client and the service with Service Bus serving as the proxy in-between for auth, location-transparency and even load-balancing. So here you can use from a variety of RelayBindings such as NetTcpRelayBinding or webHttpRelayBinding but then both the client and the service need to be available at the same time. Here you need the service and clients to be using the same contract.