I'm working in IoT enterprise application, where we have created all resources in South Central US. Recently (9/4/18) I noticed South Central US was down for long business hours due to any reason.
Now I'm trying to find best possible solution for high availability when a complete region down.
We are using following Azure resources.
EventHub (telematic data ingestion)
Azure Functions (EventHub, CosmosDB, ServicesBus Trigger)
Web App & WebJob (Schedule and continuous)
ServiceBus (Queue & Topic)
Application Insight (Application logs)
Storage Account (EventHub checkpointing and other data)
Cosmos DB
VSTS (CI/CD)
For Cosmos DB I know the solution, what should I do for other resources?
I don't see any way to create EventHub or ServiceBus multi-region cluster.
There's no cluster arrangement for ServiceBus & EventHub but can set up a fail-over flow for both.
Please refer to these articles on MS Docs:
Azure Event Hubs Geo-disaster recovery
Best practices for insulating applications against Service Bus outages and disasters
Let me know if that helps!
Azure provides Availability Zones and Geo Disaster Recovery support for both Service Bus and Event Hubs.
Here is the link for Availability Zones for Service Bus and Event Hubs
For Geo Disaster Recovery, look into Service Bus DR, Event Hubs DR
Related
I am building an application in Azure, and I really like the application Insights "Application Map". I would like to integrate IoTHub to the map, so I can check the entire stack from device to webinterface in one central place.
What I have tried:
I tried creating a diagnostic setting directly in the IoTHub.
For destination I used the same Log Analytics workspace as my application insights is using.
Results:
I can see logs using the "Logs" tab under the IoTHub, but nothing in the application insights.
The actual question:
How do I connect the IoTHub to the application insights? If this is not possible, what do you normally do to check if everything is as it should be inside the IoTHub?
Application Insights is a feature of Azure Monitor that provides extensible application performance management (APM) and monitoring for live web apps.
What metrics do you plan to track through Application Insights?
When you have critical applications and business processes relying on Azure resources, you want to monitor those resources for their availability, performance, and operation. You can use the features of Azure Monitor to analyze, monitor data generated by Azure IoT Hub and setup alerts.
Azure IoT Hub creates monitoring data using Azure Monitor, which is a full stack monitoring service in Azure that provides a complete set of features to monitor your Azure resources in addition to resources in other clouds and on-premises.
Start with the article Monitoring Azure resources with Azure Monitor, which describes the following concepts:
What is Azure Monitor?
Costs associated with monitoring
Monitoring data collected in Azure
Configuring data collection
Standard tools in Azure for analyzing and alerting on monitoring data
Please see Monitoring Azure IoT Hub and Set up and use metrics and logs with an IoT hub for more details.
I have an Azure Event Hubs in WEST US with Geo-Recovery enabled to sync with EAST US. I don't want to automatically failover to EAST US in case of the Primary outage, can we disable the automatic failover? I want to manually do the failover post business confirmation.
There is no automatic failover from Microsoft end when you have set up the Geo-disaster recovery for the event hub. You need to manually do the failover either from azure portal/REST API's. The failover can also be automated based on your business scenario where your custom application will monitor the resource and based on your business conditions your custom application will be calling the event hub REST endpoint to initiate the failover.
according to MS there is a Azure Service bus component to on-prem data gateway , however when you install the Azure component no service bus resource created. we need to understand the impact on pricing i.e. is the price of service bus hidden within Azure on prem data gateway ? there is no pricing guidelines for Azure On premise gateway
The On-premises data gateway relies on Azure Service Bus for cloud connectivity and correspondingly establishes outbound connections to its associated Azure region, which means an Azure Service Bus implicitly created when a data gateway is installed.
Created Azure Service Bus will not be shown explicitly. There is no extra cost for it. All you have to pay is for the on-prem data gateway which includes all infrastructure costs(Azure Service Bus).
There is not a licensing cost for the data gateway and data flowing into Azure is free. You pay egress costs which can be calculated using https://azure.microsoft.com/en-us/pricing/details/bandwidth/
Do Logic Apps have some sort of built in geo-replication like the Azure Scheduler or Key Vaults? I can't seem to find any information about it.
I have seen some implementations using API management but that is for Logic App that use HTTP triggers, in my case I'm using Service Bus triggers.
If there is no geo-replication how would a disaster recovery implementation look like for my scenario?
I think you are asking three questions - How do I get a geo-redundant Logic Apps deployment and How do I get a geo-redundant Service Bus Messaging deployment and how do I use them in combination.
I would start with the Service Bus Messaging side as it is the foundation for the LA process. In order to have a geo-redundant Service Bus Messaging queue you have to use the Premium SKU and this article goes into detail on how it works: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-geo-dr
For the Logic Apps side you would setup an LA in each region (primary and secondary) and point the Logic Apps to the alias for Service Bus Queue. You would then disable the Logic App in the secondary region and only enable it when the primary region's Logic App was not operational. This would have to be done with some endpoint monitoring scripting and then switch over to the secondary and disable the primary.
Like you said, there are other more automated options (Traffic Manager) when Logic Apps is being triggered by HTTP traffic but since you are reading queues the recovery is more complex.
I'm struggling at understanding if and what needs to be done to provide for high availability of two different types of Azure resources:
Azure Service Bus
Function Apps
Service Bus guarantee at least 99.9% of the time for most of the service (includes Relay, Queues and Topics, Notification Hubs). Besides, replication and partitioning messaging entities (Each partitioned queue or topic consists of multiple fragments. Each fragment is stored in a different messaging store. If the corresponding messaging store is unavailable, Service Bus writes the message to a different fragment, if possible.) could be used as common solution for high availability.
The following article would be helpful, please read it.
High Availability and Disaster Recovery for Azure Service
Bus
For Function Apps running on App Service Plans Microsoft guarantee that the associated Functions compute will be available 99.95% of the time. So if possible, you could run your Function App on an App Service Plan and enable the Always On setting.