I am looking to rewrite a legacy windows based system to make use of Azure serverless features. Issue I have is that the system has to receive messages coming in via TCP due to the upstream system only supporting that approach. The upstream system then receives the ACK of the handshake as confirmation the message has been accepted.
Are there any Azure services that can accept a TCP message on a none standard port that I could then use to place the message on to an Azure queue? I have found some articles using Service Fabric but nothing less than 3 years old so wonder if there is a better way of doing it currently?
So you can use IoT Hub Protocol Gateway which can be the middle-man for accepting TCP\UDP messages which can then forward the message to Event\IoT Hub. You can then use Event Hub trigger in Azure Functions to process your messages.
You can deploy the Protocol Gateway on an Azure VM.
Related
I'm looking into Azure IoT Hub and IoT Edge to understand capabilities and potential use cases roughly. One of the projects I had worked in the past had a strict requirement to disable cloud initiated contact and make things to work based on device initiated polling. For example, when certain settings are changed on the device management portal or new firmware is made available, rather than pushing this down to the devices, each device would become aware of this by polling periodically.
I have no control over this requirement but if that project is to be moved to Azure IoT, I'm certain that the same is going to be requested. Is there an easy way to achieve this with Azure?
I thought about tempering with the network access of the relevant containers but would this lead to the IoT hub going crazy and trying to communicate with 'unresponsive' devices continuously and flag all these failed attempts?
There are two parts to this quest: connections, and messages.
Connections
As far as connections are concerned, one of the security principles of IoT Hub is that the device is responsible for setting up the connection. Azure IoT devices by default do not accept incoming connections. The device sets up an AMQP or MQTT connection with IoT Hub, which will then allow bi-directional communication.
An exception to this is the HTTPS connection, if you use that method, the device will poll IoT Hub for new messages.
Messages
When the bi-directional connection is initiated from the device, this allows the device and IoT Hub to send messages to each other. For a non-edge device, if you don't use the SDK and write your (for instance MQTT) code yourself, you can decide not to listen to any of the incoming messages. You won't subscribe to topics like direct methods and twin updates and only look for them when you want.
For IoT Edge devices, bi-directional communication is handled by the Edge Hub. So your other containers (modules) don't talk to the cloud directly.
That leaves the Edge Agent, which also deals with outgoing connections only, it reports the status of the modules to the cloud.
In the comment of your question, you mention that the requirement is that the cloud can't initiate a connection with the device. In short, that doesn't happen, IoT Hub isn't designed that way.
I am trying to implement the Azure service bus with help of java to receive messages from the queue but I am unable to connect to the service bus because of a proxy, I am getting a connection time exception i.e.
Microsoft.azure.servicebus.primitives.ServiceBusException: Error{condition=proton:io, description='Connection timed out: no further information', info=null}
and when I am trying it from my local network(without proxy) it is connecting to azure bus without any problem. I am using the same sample code which Microsoft has provided on GitHub i.e. QueuesWithProxy.java.
How to configure proxy for azure service bus using java?
I try to connect to Microsoft for this problem so I find out that the previous version of queue client supports ConnectivityMode using TCP, HTTP, HTTPS, and AutoDetect. ServiceBus Explorer is using AutoDetect, trying TCP first and then failing over to HTTPS, regardless of the transport mode we were using (SBMP or AMQP).
With the new client this has changed. TransportMode now combines both options and offers Amqp (AMQP over TCP) or AmqpWebSockets (AMQP over WebSockets). There's no AutoDetect mode. You will have to create your clients and specify TransportType as AmqpWebSockets to bypass blocked.
I am consuming messages from the Azure Event Hub to my application and I am successfully receiving the messages. I want to know whether the messages sent from the Event Hub is encrypted. I searched in learn.microsoft.com but could not find much information. It would be a great help if anyone can help me out to understand.
Short answer: yes.
As you can find in the docs:
Any entity that reads event data from an event hub is an event consumer. All Event Hubs consumers connect via the AMQP 1.0 session and events are delivered through the session as they become available...
Now, there is more documentation around the use of AMQP regarding to the Service Bus and Event Hub, it can be found here:
It is the primary protocol of Azure Service Bus Messaging and Azure Event Hubs. Both services also support HTTPS.
and, more important in this context:
Azure Service Bus ( note: also Event Hub ) requires the use of TLS at all times. It supports connections over TCP port 5671, whereby the TCP connection is first overlaid with TLS before entering the AMQP protocol handshake, and also supports connections over TCP port 5672 whereby the server immediately offers a mandatory upgrade of connection to TLS using the AMQP-prescribed model. The AMQP WebSockets binding creates a tunnel over TCP port 443 that is then equivalent to AMQP 5671 connections.
In addition, Event Hub data is encrypted at rest, see [the docs])(https://learn.microsoft.com/en-us/azure/security/azure-security-encryption-atrest)
I have a backend device with MQTT client connected to opensource MQTT broker (Mosquitto). On the other hand I have many frontend devices (PC, Tablet, Mobile) with GUI application also connected to the broker. So here Mosquitto works as a communication point between backend device and frontend devices and just forwards messages between them. Sometimes amount of data transferred can be quite high (e.g. 1 MB / min).
One backend device + many frontend devices is one installation. I need to prepare infrastructure for thousands of such installations working simultaneously. So my service needs to be very scallable. My company uses Azure cloud solutions, so I've started learning of this solution and I must admit that I am a little bit confused. I've read that I need to use IoT Hub, but it would need MQTT Gateway to be able to speak with MQTT devices. On the other hand, if I understand it well, the Gateway needs to be running on some VM, so here I lose scalability of my solution. Am I right? Now if I will need to support 100k or 500k devices then I will need another VM? One more thing is that I need to integrate all of this with some webservice (for management of backend and frontend devices), so I need some connection between webservice and the MQTT broker...
Before I started to play with Azure, I imagined, that I will simply start an MQTT broker service and magically it will be highly scallable and will be able to provide service for thousands of devices.
Can anybody explain me how to bite that?
Azure IoT Hub now talks MQTT natively. A protocol gateway is no longer required.
https://azure.microsoft.com/en-us/documentation/articles/iot-hub-mqtt-support/
This is going to help you a lot if you've just spent the last hour trying to form the MQTT username and password:
https://github.com/Azure/azure-content/blob/master/articles/iot-hub/iot-hub-devguide.md#example
Example:
Username (DeviceId is case sensitive): iothubname.azure-devices.net/DeviceId
Password (Generate SAS with Device Explorer): SharedAccessSignature sr=iothubname.azure-devices.net%2fdevices%2fDeviceId&sig=kPszxZZZZZZZZZZZZZZZZZAhLT%2bV7o%3d&se=1487709501
Tested with Paho and MQTT.fx on Windows. I could not make it authenticate with mosquitto, and i've put in reasonable effort, even tried using stunnel just in case mosquitto's TLS support wasn't cutting it. Mosquitto probably doesn't handle the long password correctly or something along those lines. It throws an authentication error. Escaping % and & didn't help.
If someone gets Mosquitto to work with Azure IoT Hub, please open my eyes.
...and someone did (Thank you Timothy in the comments)
Mosquitto_pub works, I verified by monitoring with Device Explorer Twin. Example:
mosquitto_pub -h IOTHubACMxxx.azure-devices.net
-p 8883
--cafile "C:\Users\jlaird\Documents\dev\azureca.crt"
-t devices/eACM1/messages/events/
-m "john says hello to azure from mosquitto"
-i eACM1
-u IOTHubACMxxx.azure-devices.net/eACM1/?api-version=2018-06-30
-P "SharedAccessSignature sr=IOTHubACMxxx.azure-devices.net&sig=obfuscate&se=1593013589&skn=device"
Today there isn't an official support for MQTT protocol in Azure but only the public preview of IoT Hub that supports AMQP and HTTP.
To connect MQTT devices to the IoT Hub, Microsoft provides a "framework" named IoT Protocol Gateway (https://github.com/Azure/azure-iot-protocol-gateway) that executes a protocol translation between MQTT and AMQP.
The IoT Protocol Gateway can be installed on premise or in the cloud as an Azure Worker Role. In the second scenario you have the scalability offered by Azure and related to worker role instances.
This solution is absolutely new due to the short life of IoT Hub (still in public preview) and the IoT Protocol Gateway itself.
Your first solution is based on using a third-party MQTT broker (like mosquitto) that you should install in a VM. AFAIK mosquitto doesn't support clustering like HiveMQ broker (see another reply here : Cluster forming with Mosquitto broker).
Last thing about the connection between your web service and the MQTT broker.
In this case the web service should translate calls to him (from front end) to published message on the MQTT broker using an MQTT client that you need to include inside the web service itself.
Even if using AWS, the following link could be useful too :
https://groups.google.com/forum/#!topic/mqtt/19jqofoPLro
Paolo.
I have a virtual machine on Azure which will listen to messages over the servicebus of Azure. And another developer needs to connect to this servicebus to send messages to my service. To do so, we need to come up with some protocol for this communication system. And I was thinking about using WSDL to make the server something webservice-like, but instead of listening to the standard HTTP ports it would connect to the service bus and within it a topic with subscription, or whatever. I'm still not sure what would be best.
So, is it possible? Has anyone done something similar before? Are there some examples?
Service Bus provides both Brokered and Relayed messaging models. With using Topics/Queues you are essentially sending and receiving message to/from the broker. So you can use APIs/protocols that the broker supports. As an example you can use the NetMessagingBinding from your service/client to send and receive messages through the queue. There is no contract here other than what you put in the BrokeredMessage body and properties values. The application on either end can decode/decipher that info as needed.
In the case of Relayed messaging however you are making a end-to-end connection between the client and the service with Service Bus serving as the proxy in-between for auth, location-transparency and even load-balancing. So here you can use from a variety of RelayBindings such as NetTcpRelayBinding or webHttpRelayBinding but then both the client and the service need to be available at the same time. Here you need the service and clients to be using the same contract.