I'm developing a basic Azure IoT Remote Monitoring solution with the Azure Solution Accelerator "Remote Monitoring". When I start to actually pay for services and stop using a free account, very soon the cash starts to pile up and there seem to be very many resources created behind the scenes. I'm wondering which resources I really need and which one I could throw away to save money. These are the resources that I have:
App Service plan
App Service
Network interface
Network security group
Public IP address
Virtual network
Storage account
Azure Cosmos DB account
Device Provisioning Service
Event Hubs Namespace
App Service
App Service plan
IoT Hub
Key vault
Logic app
Azure Maps Account
API Connection
Disk
Storage account (2)
Stream Analytics job
Time Series Insights environment
Time Series Insights event source
Virtual machine
CosmosDB is probably one of the more expensive resources in your list so if you can find a way to swap some other datastore for it you can save some money.
Take a look at Remote Monitoring architectural choices. The Azure IoT Remote Monitoring solution accelerator is an open-source, MIT licensed, solution accelerator. To help you speed up your IoT development process, it shows common IoT scenarios such as:
Device connectivity
Device management
Stream processing
The Remote Monitoring solution follows the recommended Azure IoT reference architecture.
This article describes the key architectural and technical choices made in each of the Remote Monitoring subsystems. However, the technical choices Microsoft made in the Remote Monitoring solution aren't the only way to implement a remote monitoring IoT solution. You should regard the technical implementation as a baseline for building a successful application and you should modify it to:
Fit the available skills and experience in your organization.
Meet your vertical application needs.
Related
I am trying to accomplish one task which is below.
What I am doing it.
All my users are on Premises.
Application is hosted on Azure VM IaaS.
Question =>
Azure cloud application talk with Internet and download huge packages and share with client which is on- Primes. So I am trying to understand the Risk and latency matrix between on-Prime users and Azure cloud application.
If any one has done some sort of thing and encounter latency issues and what will be possible fixes for that?
Note=> I can't Migrate user to Azure cloud as of now.
To encounter latency issues, please try the following:
To reduce the latency between on premises client and azure cloud application make use of Azure HPC cache.
Azure HPC Cache reduces latency for applications where data may be tethered to existing infrastructure because of dataset sizes and operational scale.
Azure HPC caches active data automatically that is present in both on-premises and in Azure.
You can make use of Accelerated networking where communication will be done more fast.
Try eliminating network congestion.
Try reducing number of network nodes needed to traverse from one stage to another.
Make use of Azure ExpressRoute and Azure Analysis Services to reduce Network latency.
Azure ExpressRoute creates a private connection between on-premises sources and the Azure.
Azure Analysis Services avoids the need for an on-premises data gateway and generally eliminates network latency.
For more in detail, please refer below links:
https://azure.microsoft.com/en-us/blog/azure-hpc-cache-reducing-latency-between-azure-and-on-premises-storage/
https://blogit.create.pt/gustavobrito/2017/11/27/latency-test-between-azure-and-on-premises-intro/
https://viniciusdeschamps.com.br/3-ways-to-reduce-network-latency-in-azure/#how-can-I-measure-network-latency
I am building an application in Azure, and I really like the application Insights "Application Map". I would like to integrate IoTHub to the map, so I can check the entire stack from device to webinterface in one central place.
What I have tried:
I tried creating a diagnostic setting directly in the IoTHub.
For destination I used the same Log Analytics workspace as my application insights is using.
Results:
I can see logs using the "Logs" tab under the IoTHub, but nothing in the application insights.
The actual question:
How do I connect the IoTHub to the application insights? If this is not possible, what do you normally do to check if everything is as it should be inside the IoTHub?
Application Insights is a feature of Azure Monitor that provides extensible application performance management (APM) and monitoring for live web apps.
What metrics do you plan to track through Application Insights?
When you have critical applications and business processes relying on Azure resources, you want to monitor those resources for their availability, performance, and operation. You can use the features of Azure Monitor to analyze, monitor data generated by Azure IoT Hub and setup alerts.
Azure IoT Hub creates monitoring data using Azure Monitor, which is a full stack monitoring service in Azure that provides a complete set of features to monitor your Azure resources in addition to resources in other clouds and on-premises.
Start with the article Monitoring Azure resources with Azure Monitor, which describes the following concepts:
What is Azure Monitor?
Costs associated with monitoring
Monitoring data collected in Azure
Configuring data collection
Standard tools in Azure for analyzing and alerting on monitoring data
Please see Monitoring Azure IoT Hub and Set up and use metrics and logs with an IoT hub for more details.
I am not understanding when Azure IoT Hub can be preferred over Azure IoT Central. From the readings done so far, IoT central seems better over all the aspects.
Anybody can explain me where are the situations where IoT hub is better than IoT Central?
Thanks
There is no definitive answer to that question, neither are "better", but most of the times one will fit your use case more than the other.
If you want a complete, managed way of connecting devices to the cloud and create dashboards (within the product's limits), a Software as a Service solution like Azure IoT Central can be a match. Think about the requirements of the project you're looking to do, and if it's all supported by IoT Central, go for it! If there are some features you can build by leveraging data export from IoT Central, it might still be a great fit.
If you want to build bi-directional communication and device registration for IoT devices into your own cloud platform, IoT Hub comes into play. Maybe you need better control of the data, or maybe the data insights you need aren't supported by IoT Central. There are a lot of cases where it might not be the best choice. IoT Hub gives you a lot more flexibility that you can use to create almost any IoT scenario.
Both are not directly comparable, there are specific advantages of IoT Central which you may need to consider.
IoT Hub is a PaaS service which can be used with other services to create an IoT solution while IoT Central is IoT Application platform which can be used as-is or extended via companion application. Even addressing basic functionality in IoT Central you will need over dozen other services and you own responsibility to design, manage and administer the orchestration yourself.
IoT Central internally uses multiple IoT Hubs (HA/DR) and bunch of services to bring the functionality that you see in the application. This includes App Service to host the UX, Rules Engine, Fast Storage, API layer, Data Export, RBAC, in-app Multi-tenancy , etc. etc. The key advantages you get -
Full featured IoT solution with high availability, security, scalability that is available in < 10 secs under 99.9% SLA
Simplification, easy to connect any device or simulate basic capabilities using the built-in plug-n-play support. Just select any device from the pnp catalog and try it out even before purchasing the devices.
Create user or app level dashboards with device specific views. Device specific view can be auto-generated with PnP devices.
Rule creation, alerting and integration with other applications via Logic Apps, Functions
Data Export functionality to Event Hub, Service Bus, Blob Storage or Web hooks
Rich Job's interface allowing updating device configurations or firmware
RBAC in combination with Organizations allow giving specific permissions to user.
The big advantage is all this is available with a very simpler per device per month pricing that starts as low as 8 cents per device per month ($2 a year) + additional messages https://azure.microsoft.com/en-us/pricing/details/iot-central/
In general unless you already have UX, Storage, Rules engine, etc. elements required for IoT Solution and need to add IoT Hub to ingest and manage IoT devices it will make more sense to start with IoT Central and build with it. It will save time, efforts and you can focus on specific differentiation than build the underlying plumbing and owning the management and sustenance. It is difficult to come to that price point given the high cost of cloud engineers required to support and maintain it.
It is recommended that all customers begin their IoT journey with our aPaaS offering Azure IoT Central. IoT Central is a ready-made environment for IoT solution development. As an aPaaS offering it is built to simplify and accelerate IoT solution assembly and operations, by preassembling PaaS services from the IoT Platform (including IoT Hub and the IoT Hub Device Provisioning Service) and across Azure. A customer that starts with IoT Central builds valuable expertise regardless of whether they go to production with IoT Central, or later build a custom solution to meet complex business needs using PaaS services. To learn more about onboarding to Azure IoT check out this documentation: https://aka.ms/azureiotarch and stay tuned for a session at Microsoft Ignite Nov3-4th Entitled Onboarding to Azure IoT
So, I am working on an IoT solution on Azure, we have been using a partner solution where we had the partner's devices linked to his cloud solution that exposes the data to us Via REST services. Right now we want to have our own IoT Cloud Solution on Azure.
At first, I am planning to build a Bridge between our IoT Solution and the partner's cloud solution via its REST Services that will link to our IoT Hub in order to ingest the data to our cloud.
Also, the data will not be only telemetry data but we'll have to send commands as well to those devices.
My question: I would like to know what would be the appropriate technology/solution to use a gateway (Data Grid, Azure Function, Azure WebJob)
The numbers in the picture represent the step that I am considering to tackle this problem.
1- First we are implementing an Application gateway that will have to get the data from the partner's system and sending commands to their system. It will allow us to first build the other components of our system and make sure that it can handle what is in place right now.
2- Second, the partner's devices will connect directly to a device gateway that is connected to our IoT Hub. In this case, we will not be using the gateway made in 1 anymore.
3- Finally, we will have our own devices connected to our IoT Hub, the partner's devices will always be connected to our IoT Hub via the gateway built in 2.
Let me try to answer your questions in the order you have asked.
For application gateway, where you are trying to pull data through
REST, you can use Azure functions and then you use Cosmos DB or any
storage to save data. I see , after getting device data from Partner
network, you are routing it to IoT-Hub (I would not say, its
incorrect), however once we pull data through Rest, we can directly
put into DB. So my Answer is to use Azure functions to pull data
from Partner solutions and put into DB.
If partner device is capable of running Azure IoT sdks or can be
provisioned to send data to IoT Hub directly, this will ease lot of
things and you would be able to send D2C and C2D messages easily.
further, here you can route data to DB by using configuration from
IoT Hub.
For your devices you can use IoT Hub Directly or can use Azure
IoT Edge (device gateway as you pointed ), both are fine , depends
on use case and also if we want to do some edge computation or
analytics at device side. And one important suggestions, use Azure
functions where ever you find that you have to integrate devices
data through Rest. Most cost effective in such scenarios.
Let me know if it clears your doubts.
After some time working on the subject, I did implement an AZURE Function app for the following reasons :
Supports Continuous Deployment and Integration Even though Azure Functions is serverless architecture, it still supports Continuous Deployment and Continuous Integration
Capabilities for implementing code - Being event-driven, the application platform has capabilities to implement code triggered by events occurring in any third-party service or on-premise system.
Compute-on-demand: This delivery model ensures that computing resources are available to the users as per their demand.
I have also used Azure Table Storage as database storage technology.
We are looking to use Windows Azure to host our existing SaaS platform and extend our functionality and capability. WE will be taking adavantage of both the data storage and application and web service functionality of Azure.
My question is as follows:
Some of our clients will not want Public CLoud access. Since our datastore stores sensitive client data many of them will require our whole system to be hosted internally on their own network and servers.
If we setup a full Azure setup of database and connected applications and processes how difficult is it to be able to duplicate that system for a specific client on their own servers and network using existing Microsoft technologies?
I know its a vague question and I also have a liminted understanding of Azure so whatever information you can provide here would be most appreciated.
Thank you
It sounds like you need the flexibility of a hybrid cloud/on-prem solution. Likely the best solution is the Windows Azure Service Bus. Essentially, you configure a WCF web service in the cloud (SOAP, REST, etc) that performs asynchronous brokered messaging between your on-premise application and your web application. This can be performed using queue messages, for example:
The web application (cloud) requests resources from the brokering service (cloud) by sending a queue message
The service handles the queue message and makes it available to the consuming (on-prem) service
On-prem service checks for new messages from the brokering service, gets the request for data, and returns desired data from DB
On-prem service sends message to brokering service with desired data
Web app (cloud) checks for new messages from the brokering service, then uses the data from on-prem service
Service bus is secure, asynchronous, fault-tolerant, and ensures that both components are decoupled.
Another method is to use Windows Azure Connect, which is a VPN solution that sets up network-level connnectivity. I recommend Service Bus because it promotes a more robust and scalable architecture, and fault-tolerance is high.