I'm trying to set up an IoT scenario with Azure.
For demo purpose I've used this temperature monitoring scenario:
https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-monitoring-notifications-with-azure-logic-apps
An IoT device is sending temperature data to IoT Hub
IoT Hub is routing messages with a temperature > 30 to a service bus
A logic app is reading that service bus and sends a warning e-mail
Now I have the problem that the logic app sends an e-mail for every message that exceeds the temperature limit (every message that is in my service bus). So my inbox will be spammed as soon as I heat up my IoT device.
In reality I would only want to receive one e-mail if my IoT Device overheats, then wait a certain time. How would I handle that in my logic app?
You can achieve this by simply configuring the concurrency control (you can find this in the Settings of your LA trigger) of your Logic App trigger. By default, the concurrency value will be set to 25 and you have to change it to one.
It allows only one message to be processed in your queue (irrespective to the number of messages in your queue) and you can define the interval in the UI of the Service Bus Queue trigger itself, (the time interval at which you need to get another email)
Related
I have an Azure IoT device connected to an Azure IoT Hub. The device sends 6 - 7 messages per minute. By looking at the D2C message metrics, I found an outlier, that states that at a specific time, the count of the D2C message was zero (see picture). As the messages are routed to a storage, I can check the storage to see if there are messages missing at that specific time, but the data saved in the storage shows that every message was received correctly at that time. Does any one know how that comes or if the metrics are not that reliable generally? If that's the case, what is the best practice to monitor the IoT Hub message transfer?
IoT Hub D2C Message Metrics
EnqueuedTimeUtc in the storage
For precisely monitor the flow of each message through IoT Hub you will need to Trace Azure IoT device-to-cloud messages with distributed tracing (currently in preview)
This trace context includes correlation IDs that allow you to correlate events from one component with events from another component
Automatically log the trace context to Azure Monitor Logs.
Measure and understand message flow and latency from devices to IoT Hub and routing endpoints.
Ref: https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-distributed-tracing
I have an Azure IoT Hub application and a device just started to send messages indicating changes in reported device twin properties every other second. In a matter of few hours, the total messages that day went over 50k. When this number goes of 40k, the IoT Hub becomes VERY slow to respond for ALL customers - not only until the device is shut off but until all those messages have managed to throttle through the system which seems to be after several hours or until next morning.
So if this type of flooding happens, the entire system for all customers grinds to a halt due to slowness.
This is a device bug and needs to be fixed but I was wondering if there is a way - IF this happens - to get the whole IoT hub back to normal where it isn't slow? Something like kicking the offending device or rebooting the hub or something. Or better yet - is there a way to prevent devices from flooding the hub faster than an x amount of messages per minute or something?
You could build some logic to kick the offending device when it starts spamming your hub. One approach might be to route all twinChangeEvents to a separate endpoint and write a Stream Analytics Job to group the messages per deviceId and keep a count of the events in a sliding window of X minutes. After the count reaches a threshold you set, you could call an Azure Function to disable the device and send a notification.
There is one caveat, the docs state:
If the rate of change is too high, or for other reasons such as
internal failures, the IoT Hub might send only one notification that
contains all changes.
I don't know if your device reaches that rate, but I think this would be a suitable approach to kick the offending device.
At some point the IoT hub will start rejecting your messages once the throttling limit is reached as per your tier and units purchased. Now, to handle excess of messages either throttling reached or the IoT hub is slow in processing the message you should auto-scale your IoT hub.
As per the docs in above link provided:-
The sample solution outlined in this article provides the ability to monitor an IoT Hub for the case where the current message count has exceeded a set threshold (for example, 90% of the allowed messages) and, in that case, to automatically scale the IoT Hub up to the next unit of capacity.
At the end of the day you need to also auto-down scale your IoT hub so that at low traffic received the cost is not high for IoT hub. Check the Scaling down section in the article link above.
I have an Azure IoTHub with thousands of devices registered. These devices communicate through a Telco provider who sends messages through an Azure Storage Queue. This Storage Queue triggers an Azure Function which needs to parse the messages and Send an Event to the IoTHub as below.
Currently, we use the Azure IoTHub SDK to create a DeviceClient for each payload and we send the event. Because the DeviceClient represents a device in the IoTHub and is carrying the context of the source of the events, we are having to recreate a device client for each event. This quickly exceeds the threshold of the number of Connections allowed on Azure Functions.
We have tried using the IoTHub Output bindings for Azure Functions, but could not get to work and I do not think it would work because we need to make sure that the events get to the IoTHub with the right context (messages are sent by the right device).
What's the right way to solve this? Can the connections to the IoTHub be reused? Should we abandon Azure Function in favour of something else?
I assume that Telco is some kind of custom device management solution(vendor lock solution), that can also communicate with the device and receive the device telemetry, and eventually forward it to the specified endpoint, correct?
If I may ask and if my assumption is correct, why do you need to deliver the events to IoT Hub, if you are not managing Telco devices through IoT Hub(the arrows on your diagram are only in one direction)?
Using the IoT Hub just as a message broker for essentially cloud-to-cloud communication is not beneficial if that is the only purpose. Also conceptually what you described is cloud-to-cloud communication, and IoT Hub is intended to be used for devices.
Here is what I would do. Setup the API Management(or http triggered Azure Function) as a front door for Telco and pass the messages to the Event Hub.
You can choose here to pass request body for example where your telemetry data is - I assume again.
Keep the IoT Hub, and setup the routing to previously created Event Hub.
Now, in case you have devices that are not vendor locked and that can talk directly to IoT Hub, messages will be re-routed to Event Hub. Also Telco device messages will be routed to exactly the same Event Hub.
Now you can have for example Azure Stream Analytics that can analyze data stream just from the Event Hub, and for both, Telco devices and potentially non-Telco devices.
After trying a few things, I ended up moving away from using the SDK for pushing messages to IoT Hub. This is because the SDK uses AMQP, and creating a DeviceClient for each payload is not viable.
We switched to using HTTPS instead to push the messages to IoT Hub and using HttpClientFactory, we are able to do connection pooling.
I thought I would put this here in case someone has the same issue.
Here is an example of the Http request to send message to IoT Hub
Host: https://<iothubname>.azure-devices.net/devices/<deviceId>/messages/events?api-version=2018-06-30
Authorization: SharedAccessSignature sr=<iothubname>.azure-devices.net&sig=abc123;12344iweoippweruea=iothubowner&se=1570574220
Body: <normal Interval or alarms payloads> // example {"deviceid": "abc", "hello": "world"}
Lastly, thanks #kgalic for the answer but your suggestion would not work. This is not pure B2B integration. Our implementation have to allow for both devices connecting directly to the IoT Hub and devices connecting through the Telco. This is why every device needs to have its own identity and digital twin.
I have a little IoT project with one device. Arduino sends some values to azure where function application processes them and sends instructions for arduino to the endpoint in IoT hub. (/devices/MKR1000/messages/devicebound?api-version=2016-02-03)
I need to get data from this endpoint in real time so I want arduino to read only the last (the newest) message every time, but it starts from the oldest.
It's possible to make the arduino read all the messages from the endpoint and than show the last, but I'm looking for a more efficient way.
Thank you.
You receive old messages because they are still queued in Azure IoT Hub due to the device not "complete" these messages. IoT Hub supports the option to complete/reject/abandon C2D messages over HTTPS and AMQP only at the moment.
Another option is setting ExpiryTimeUtc(in function application?) to release older messages faster(Minimum 1 minute. Default: 1 hour.).
More information you can reference "Send cloud-to-device messages from IoT Hub".
I would like to get some recommendation, for designing a routing of IoT messages in Azure.
I have following scenario:
Senors sending messages to Azure IoT Hub in Google Protobuf format. Depending of the type of a message, I want to route the message to different applications inside a service fabric.
My current approach is to use a service fabric application to receive all messages from the IoT hub, parse the protobuf message, send the message depending on their type (attribute inside the protobuf) to an type-specific Azure event hub. Now the applications fetches the messages from their "own" event hub and process the messages.
I'm not sure if this is the best approach. I don't like the fact, to have one event hub for each type of message. Service Bus Topics are probably not an option, because I have a lot of messages (~30k per second).
Do I realy need a event hub, to decoupling this process, or does it make sense, to send the messages from the "routing application" direct to the different "type applications"?
What do you think?
Regards,
Markus
If you really need high performance you should take a look at IoT Hub and Event Hubs. Azure Event Hubs is a highly scalable data streaming platform and event ingestion service capable of receiving and processing millions of events per second. Event Hubs can process and store events, data, or telemetry produced by distributed software and devices. Data sent to an event hub can be transformed and stored using any real-time analytics provider or batching/storage adapters.
In other hand if you need only 30k messages per second you can go with Premium Messaging.
Comparison of Azure IoT Hub and Azure Event Hubs
Premium Messaging: How fast is it?
What is Event Hubs?