In my case I have 1000+ of devices that stores activity inside. I need to send a http get request to this device to get those data in csv or json format and save it in a storage hosted on azure.
Cab IOT hub require data using get request and can it be scheduled to read daily/weekly?
What other azure services would you suggest to facilitated this scheduled reads?
You have not mentioned which the Azure IoT Hub scale tier is used. Basically there are two price groups such as Basic and Standard with a significant different cost and capabilities. The Basic tier offers only services for one-way communications between the devices and Azure IoT Hub.
Based on that, the following scenarios can be used for your business case:
1. Basic Tier (non event-driven solution)
The device pushs periodicaly a telementry and non-telemetry messages based on the needs to the Azure IoT Hub, where the non-telemetry messages are routed to the Azure Function via the Service Bus Queue/Topic. Responsibility for this non-telemetry pipe is to persist a real device state in the database. Note, that the 6M messages will cost only $50/month. The back-end application can any time to query this database for devices state.
2. Standard Tier (event-driven solution) In this scenario you can use a Device Twin of the Azure IoT Hub to enable storing a real-device state in the cloud-backend (described by #HelenLo). The device can be triggered by C2D message, changing a desired property, invoking a method or based on the device edge trigger to the action for updating a state (reported properties).
The Azure IoT Hub has a capabilities to run your scheduled jobs for multiple devices.
In this solution, the back-end application can call any time a job for ExportDevicesAsync to the blob storage, see more details here. Note, that the 6M messages will cost $250/month.
As you can see the above each scenario needs to build a different device logic model based on the communications capabilities between the devices and Azure IoT Hub and back. Note, there are some limitations for these communications, see more details here.
You can consider using Device Twin of IoT Hub
https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-device-twins
Use device twins to:
Store device-specific metadata in the cloud. For example, the deployment location of a vending machine.
Report current state information such as available capabilities and conditions from your device app. For example, a device is connected to your IoT hub over cellular or WiFi.
Synchronize the state of long-running workflows between device app and back-end app. For example, when the solution back end specifies the new firmware version to install, and the device app reports the various stages of the update process.
Query your device metadata, configuration, or state.
IoT Hub provides you with the ability to connect your devices over various protocols. Preferred protocols are messaging protocols, such as MQTT or AMQP, but HTTPS is also supported. Using IoT hub, you do not request data from the device, though. The device will send the data to the IoT Hub. You have to options to implement that with IoT Hub:
The device connects to the IoT Hub whenever it has some data to be sent, and pushes the data up to IoT Hub
The device does not send any data on its own, but stays always or at least regularly connected to IoT Hub. You then can send a cloud to device message over IoT Hub to the device, requesting the data to be sent. The device then sends the data the same way it would in the first option.
When the data then has been sent to IoT Hub, you need to push it somewhere where it is persistently stored - IoT Hub only keeps messages for 1 day by default. Options for this are:
Create a blob storage account and push to that directly from IoT Hub using a custom endpoint This would probably be the easiest and cheapest. Dependening on how you need to access your data, a blob might not be the best option, though
Create a function app, create a function with an EventHubTrigger, connect it to IoT Hub and let the function process incoming data by outputting it into any kind of data sink, such as SQL, CosmosDB, Table Storage...
Related
I want to implement the following system.
There are users and each user will have an IoT device. The user should be able to do the following:
Login with their email and password.
Control the IoT device in real time (the user will perform some operation on the IoT device and the result will be displayed instantly on the app).
See the state of IoT device in real time. If something changes on the IoT device it should reflect on the app in real time.
I was wondering if this is doable using azure IoT Hub. I came across an architecture where the device is connected to IoT hub and the app is connected to signalR. The messages from IoT device will go to IoT Hub which will update the app using signalR.
But is there any way we can not have additional component like signalR? Can IoT device and mobile app be connected directly to azure IoT hub and exchange data between them without an entity in between other than IoT hub?
Connecting mobile app to IoT hub seems like an option but I did not find any way to implement email and password based authentication to allow users to connect to IoT hub.
Any help in this regard is appreciated.
You could absolutely go without SignalR. You can use the IoT Hub Service SDK to send C2D messages to your device from your phone, and also listen to the device telemetry with the same SDK.
However, it would probably be a better idea to have some restrictions on what you can do with that SDK. If the user first has to log in, I take it that they don't get access to every device in your application? Using some kind of role-based access might be preferable, you would build an API for that, which then uses the Service SDK to give you the data.
Also, even though you could listen to the device data directly from your phone, that means that if you have bad reception/wifi, data might not be received. You might want to consider storing it somewhere? I don't know the details about your application, but if you would then want to listen to events from that storage and send it to your app... SignalR might not be the worst idea.
"See the state of IoT device in real time. If something changes on the IoT device it should reflect on the app in real time."
For this EventGrid integration (https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-event-grid ) would help. For realtime telemetry as Matthijs said SignalR would be a better choice.If you are fine with some delay (say 10 sec) then Azure Time Series Insights can also be used.
How can an IoT device which connects to an IoT Hub using the Azure IoT Hub SDK retrieve larger amounts of sensitive data which is part of the device configuration?
I know i can use Device Twins as configuration documents per Device, but those are limited to 8K in total.
I need larger amounts of data in binary format to be transfered to the device when i a device - for example - gets reset to factory defaults or somehow loses it's local storage as part of a hardware fault.
I was thinking about using a D2C method triggering a request which results in a series of C2D or direct method calls to the requesting device transmitting the information in question (basically mimiking request/response behavior). However this feels like a workaround.
My other thought was about transmitting urls to a storage account as part of the device twin properties allowing the device to download the binary information using the Storage SDK, however given the sensitive nature of the information downloaded i can't directly expose this information to the internet using a public accessible container in Azure Storage.
So my last thought was to access a REST API which controls access to the information. However my concern is, that i break out of the secure communication channel, the IoT Hub SDK provides with it's build in communication methods and i have to secure the communication between device and the REST API using some kind of rotating secret anyway which could expose other risks along with a complicated custom implementation.
What is the suggested way of providing access to this kind of device-specific data for Azure IoT Clients?
I do recommend to use a new feature (still in the preview) of the Azure IoT Hub such as a Device Streams.
You can test this new preview feature using my updated tool Azure IoT Hub Tester (see Appendix A2)
I have an Azure IoTHub with thousands of devices registered. These devices communicate through a Telco provider who sends messages through an Azure Storage Queue. This Storage Queue triggers an Azure Function which needs to parse the messages and Send an Event to the IoTHub as below.
Currently, we use the Azure IoTHub SDK to create a DeviceClient for each payload and we send the event. Because the DeviceClient represents a device in the IoTHub and is carrying the context of the source of the events, we are having to recreate a device client for each event. This quickly exceeds the threshold of the number of Connections allowed on Azure Functions.
We have tried using the IoTHub Output bindings for Azure Functions, but could not get to work and I do not think it would work because we need to make sure that the events get to the IoTHub with the right context (messages are sent by the right device).
What's the right way to solve this? Can the connections to the IoTHub be reused? Should we abandon Azure Function in favour of something else?
I assume that Telco is some kind of custom device management solution(vendor lock solution), that can also communicate with the device and receive the device telemetry, and eventually forward it to the specified endpoint, correct?
If I may ask and if my assumption is correct, why do you need to deliver the events to IoT Hub, if you are not managing Telco devices through IoT Hub(the arrows on your diagram are only in one direction)?
Using the IoT Hub just as a message broker for essentially cloud-to-cloud communication is not beneficial if that is the only purpose. Also conceptually what you described is cloud-to-cloud communication, and IoT Hub is intended to be used for devices.
Here is what I would do. Setup the API Management(or http triggered Azure Function) as a front door for Telco and pass the messages to the Event Hub.
You can choose here to pass request body for example where your telemetry data is - I assume again.
Keep the IoT Hub, and setup the routing to previously created Event Hub.
Now, in case you have devices that are not vendor locked and that can talk directly to IoT Hub, messages will be re-routed to Event Hub. Also Telco device messages will be routed to exactly the same Event Hub.
Now you can have for example Azure Stream Analytics that can analyze data stream just from the Event Hub, and for both, Telco devices and potentially non-Telco devices.
After trying a few things, I ended up moving away from using the SDK for pushing messages to IoT Hub. This is because the SDK uses AMQP, and creating a DeviceClient for each payload is not viable.
We switched to using HTTPS instead to push the messages to IoT Hub and using HttpClientFactory, we are able to do connection pooling.
I thought I would put this here in case someone has the same issue.
Here is an example of the Http request to send message to IoT Hub
Host: https://<iothubname>.azure-devices.net/devices/<deviceId>/messages/events?api-version=2018-06-30
Authorization: SharedAccessSignature sr=<iothubname>.azure-devices.net&sig=abc123;12344iweoippweruea=iothubowner&se=1570574220
Body: <normal Interval or alarms payloads> // example {"deviceid": "abc", "hello": "world"}
Lastly, thanks #kgalic for the answer but your suggestion would not work. This is not pure B2B integration. Our implementation have to allow for both devices connecting directly to the IoT Hub and devices connecting through the Telco. This is why every device needs to have its own identity and digital twin.
I have a node.js server application that communicates with Iot devices using Azure IoT Hub service.
When I examine the sample code sample code from Azure repository about cloud to device message. I see that one client was created. If there are multiple devices and multiple messages to be sent to multiple devices, is it okay to create multiple Iot hub clients, or should I stick to one client solution. What could be the cons and pros of two approach?
Have a look at the Azure IoT Hub feature for your scenario such as a Schedule and broadcast jobs. This feature enables a cloud back-end to schedule and update millions of devices based on your needs.
Is there any way which we can automatically sync data between device MQTT events and Device Twin data ?
For a given device in Azure IOT hub there is a mqtt event endpoint and a device twin endpoint associated to it.
I have tried sending MQTT event data to a particular device through device's event MQTT endpoint : devices//messages/events/ and it was successful. I was able to see the incoming data in the device explorer.
The Problem that I have is that, when we send some data to device through device's event mqtt endpoint, those data will not be automatically sync into the respective device's twin. I think it should happen automatically. In azure we have to manually (programmatically) do a twin data push for each event data that we receive through device's event endpoint.
This can be done through pushing mqtt data to device twin endpoint : $iothub/twin/PATCH/properties/reported/?$rid=ID
Is there anyway that we can do it automatically ? - Means automatically synchronize data from event mqtt stream to Device Twin ?
The primary focus of the Azure IoT Hub is to ingest a device telemetry data into the Azure IoT Stream Pipeline in the real time.
Beside that, each device has a capability to save its state in the Azure storage known as a device-twin such as a json formatted document. The device twin has built-in a support for sync the states between the device and backend cloud in the modern event driven distributed architecture.
The differences between the above two communications such as device-to-cloud and device twin is described in the Reference - IoT Hub quotas and throttling.
Based on this document, you can see, that the device twin is used for non-frequently persistence of the device state, for example: device firmware version, some device configurations, threshold for alert, etc.
In other word, the device twin is not designed for storing a real-time device telemetry data in the cloud backend. In the Internet of things all telemetry data (things) from the devices go to the stream pipeline for their real-time analyzing, etc.
However, the Azure IoT Hub has a capability to help you with your scenario, when a device telemetry data and its device twin changes can be stored in the Azure Blob Storage in the real time, see this feature here. You can use an Azure Function and/or Stream Job for analyzing, query, etc. this blob for your business logic.