Situation:
I hit a throttling limitation of free tier IoT Hub where deployments are limited to 10. This limitation is described here
Failed deployments manifests itself
when Azure DevOps CI pipeline is used
Azure Portal
Problem(s):
The error is pretty clear, but what I'm struggling to understand is whether the limit of 10 gets reset at some point or is that it - no more automatic deployments for this Edge device, ever? The documentation seems to suggest that paid for IoT Hubs have 100 deployment limit, what do developers do after that?
I have created a deployment manifest and attempted to deploy it manually through VS Code. This has worked, but then within very short period of time (~5min) the Hub reverts to previous configuration. Is that because of this limit or its a property of the Hub to e.g. revert to previous configuration if newly deployed modules fail for one reason or another?
You should not create a deployment for every device. Rather target deployments to a set of devices: https://learn.microsoft.com/en-us/azure/iot-edge/how-to-deploy-at-scale?view=iotedge-2020-11
To modify a single device, follow the steps outlined here: https://learn.microsoft.com/en-us/azure/iot-edge/how-to-deploy-modules-portal?view=iotedge-2020-11
This allows you to change every one of your max 1 million devices per IoT Hub individually. But is this really needed?
Best practice: Use layered deployments (https://learn.microsoft.com/en-us/azure/iot-edge/module-deployment-monitoring?view=iotedge-2020-11) and change settings via Module Twin, if needed.
Related
I'm new to both Terraform and Azure. In a timespan of 6 weeks I have to create an IoT solution. I've only got 3 weeks left.
I have to provision devices in an IoT hub using a DPS and if 'm not wrong, a DPS uses azure functions to do so.
My question is, how can I add azure functions to a dps using terraform or is this not possible and do I have it all wrong?
DPS does not require an Azure Function for basic functionality. That's only needed if you're doing custom allocation policies. If all you want to do is have an IoT Hub assigned from a pool of one or more Hubs, then one of the built-in allocation policies will work, as described in the overview:
Multiple allocation policies to control how DPS assigns devices to IoT
hubs in support of your scenarios: Lowest latency, evenly weighted
distribution (default), and static configuration via the enrollment
list. Latency is determined using the same method as Traffic Manager.
The documentation explains how to select this in the Portal, and since you mention you're using Terraform, the provider does this using the allocation_policy setting according to the provider documentation.
I understand you're under a small time crunch but you might find the self-paced training at Microsoft Learn for the AZ-220 exam worth your time. There is a learning path about device provisioning, and the information on load balancing is covered in that (for example, at this lesson).
We are testing a set of PS scripts to create several Azure artifacts:
- Storage Accounts
- SQL Servers
- Service Bus
- App Services
- AppInsights
- IOT Hub
The performance is very variable. For example, sometimes creating the StorageAccounts take a second or three (we create 2), and sometimes it takes 20 minutes+. All our resources are in US East; we are using the New-AzureRmxxx PS commands.
Is this typical? How do others investigate such issues in addition to confirming there are no outages via the Azure Health dashboards?
First off all, Azure had this problem with storage account creation lately, but it appears to be gone (at least at the time of writing).
The other problem you might be facing - creating storage accounts (or any resources for that matter) in place of the ones you've just deleted. So you have a storage account named mystorageacct, you deleted it in the script and right after that you are starting to create it with the same name, that can take a while because of how Azure works.
And generally you don't troubleshoot behavior like this in the cloud. Cloud is meant to throw timeouts every now and then. If you would be creating unique named storage accounts every time this wouldn't be an issue. And if Azure has got some problems, those are temporary and nothing you can do about them (but knowing about them helps). Here's the RSS feed.
I have observed this behavior when using DevOps to recreate a Resource Group.
If I purge(Remove) the resources and then attempt to recreate them using the same name, this causes slowness.
I found if I delete the Resource Group in it's entirety, then recreate it and the accompanying resources, the slowness does not bite as often.
I have been researching on Microsoft Azure Event Hubs. My goal is to figure out a way to provide automatic scalability. This is a experimental work and I am really only trying to know what can I do with Azure event hubs. I do not have access to the Azure platform to test test anything :( .
Well, so far, I found that through REST API and Service Bus Powershell I can add Throughput Units (to increase performance - I am relying on this: Scale Azure Service Bus through Powershell or API) and increase or decrease Event's Expiration time (which might influence capacity - https://msdn.microsoft.com/en-us/library/azure/dn790675.aspx).
The problem is that, presuming that the previous techniques work and I am able to scale event hubs' performance automatically, I still need a way to know when to trigger scalability mechanisms. To know when and how to trigger scalability, I need to work on some functions that rely upon the event hub's metrics (or a way to monitoring it). The problem is that I can't really find any metrics. The only thing that I find is this: https://azure.microsoft.com/en-us/documentation/articles/cloud-services-how-to-monitor/ - Which actually does not solve my problem because although it may present some interesting metrics, it does not serve the purposes of my "application" (which will come if I can prove that I can successfully scale Azure automatically); and this Azure service bus statistics/Monitoring - which's links are not working.
Surely I can find more information about Service Bus Explorer, and surely it may provide some interesting insights over the event hub metrics, I am just wondering if there is something like this: https://github.com/HBOCodeLabs/incubator-storm/blob/master/STORM-UI-REST-API.md that allow me to access some kind of metrics, rather than creating my own metrics
Thanks in advance
Best regards
You can retrieve metrics about Event Hubs (an Event Hub is a Service Bus Entity) using the Service Bus Entity Metrics REST APIs(https://msdn.microsoft.com/library/azure/dn163589.aspx). Using this you can retrieve the same metrics displayed in the portal such as:
Number of incoming messages
Incoming throughput
Outgoing throughput
These should help you determine when you need to scale your application up or down.
This video is useful for getting started https://channel9.msdn.com/Blogs/Subscribe/Service-Bus-Namespace-Management-and-Analytics
If 3rd party services are an option, look into CloudMonix # http://cloudmonix.com
It can monitor Event Hubs (among gazillion other Azure-related things) and execute Azure Automation runbooks (among gazillion other actions) as a reaction to load conditions/throughout of a whole hub or individual partitions and optionally based on any other metrics in your environment.
Your Azure Automation runbooks could have the logic to execute increases in your EH's throughout, etc.
Disclaimer: I'm affiliated with the product.
HTH
Service Bus Explorer is great. I actually use this.
ServiceBus Explorer
We currently have a window service which send some notification emails to users after doing some processing on database(SQL database). Runs once in day.
We want to move this on azure cloud. One alternate is to put it on Azure VM as is. but I am finding some other best possible solution for that.
I study about recurring and on demand Web jobs but I am not sure is this is best solution.
Also is there any possibility to update configuration of service code in App.config without re-deploy the code of service on cloud. I means we can manage configuration from Azure portal.
Thanks in advance.
Update 11/4/2016
Since this was written, there are 2 additional features available in Azure that are both excellent choices depending on what functionality you need:
Azure Functions (which was based on the WebJobs described below): Serverless code that can be trigger/invoked in various ways, and has scaling support.
Azure Service Fabric: Microservice platform, with support for actor model, stateful and stateless services.
You've got 3 basic options:
Windows service running on VM
WebJob
Cloud service
There's a lot of information out there on the tradeoffs between these choices, but here's a brief summary.
VM - Advantages: you can move your service basically as it is without having to change much or any of your code. They also have the easiest connectivity with other resources in Azure (blob storage, virtual networks, etc). The disadvantage is you're giving up all the of PaaS advantages and are still stuck managing your own VM infrastructure
WebJob - Advantages: Multiple invocation options (queues, blobs, manually, queue receive loops, continuous while-loop style, etc), scheduled (would cover your case). Easy to deploy (can go with website, as a console app, automatically through Kudu), has some built in logging in Azure portal - and yes, to answer your question, you can alter the configuration in the portal itself for connection strings and app settings.
Disadvantages - you'll need to update code, you don't have access to underlying resources (if you need that), and more of something to keep in mind than a disadvantage - it uses the same resources as the webapp it's deployed with.
Web Jobs are the newest of the options, but at the same time appear to have active development going on to increase the functionality and usefulness.
Cloud Service - like a managed VM, has some deployment options, access to underlying VM if needed. Would require some code changes from your existing service.
There's nothing you've mentioned in your use case that makes me think a Web Job shouldn't be first thing you try.
(Edit: Troy Hunt has a great and relatively recent blog post illustrating most of the points I've mentioned about Web Jobs above: http://www.troyhunt.com/2015/01/azure-webjobs-are-awesome-and-you.html)
Is there a way do do dynamic elasticity in Windows Azure? If my workers begin to get overloaded, or queues start to get too full, or too many workers have no work to do, is there a way to dynamically add or remove workers through code or is that just done manually (requires human intervention) right now? Does anyone know of any plans to add that if its not currently available?
Microsoft shipped the Autoscaling Application Block (Wasabi) to provide dynamic scaling. Some of the supported scenarios:
Autoscaling both web and worker roles in Windows Azure by dynamically changing instance counts or performing application throttling.
Autoscaling Windows Azure roles based on timetables.
Autoscaling Windows Azure roles based on metrics collected from the application and/or Windows Azure but constrained by upper and lower bounds on the instance count per role.
Preventing fast oscillations in the number of role instances with the stabilizer. The stabilizer can also help to optimize costs by limiting scaling up operations to the beginning of the hour and scaling down operations to the end of the hour.
Monitoring and logging autoscaling activity.
Sending notifications to preview any scaling operations before they take place.
Encrypting the rules and other configuration in Windows Azure blob storage or in local file storage.
Managing the autoscaler configuration by using Windows PowerShell.
A comprehensie sample application (Tailspin Surveys) showcasing all these features is provided (installation instructions are available here). Also, check out the Developer's Guide and the Channel9 video walkthrough.
The block is available as standalone download of binaries, source or via NuGet.
Here are a couple of talks/demos showing Wasabi in action:
CloudCover Episode on autoscaling
p&p symposium talk "Windows Azure app scaling to need"
There's a Service Management API, and you can use that to scale your application (from code running in Windows Azure or from code running outside of Windows Azure).
http://msdn.microsoft.com/en-us/library/ee460799.aspx and http://code.msdn.microsoft.com/Release/ProjectReleases.aspx?ProjectName=windowsazuresamples&ReleaseId=3233.
Windows Azure has just added the autoscaling feature built into the platform. Now it's trivially easy to configure your autoscaling rules right in the management portal:
See the announcement and the demo. I've also written a post comparing Windows Azure Autoscale to Wasabi and outlining the path forward.
Create a queue named autoscale.[your_role_name].instance_count
In the Management Portal, set the autoscale to Queue.
Set the Target Count field to 1.
Now you can use standard enqueue and dequeue operations on that queue to control the number of worker role instances. You've got 7 days to process a message before it expires, so you might want to create a worker role that can ensure that the number of messages in the queue is tracking your target instance count.
If you're after dynamic elasticity, you've probably already got a worker-role-based controller in mind already, so that's probably not a problem.
Lokad.Cloud open source project for Windows Azure contains distributed executor framework. Among other things it provides auto-scaling with VM provisioning feature.