I have a query to setup multiple environments at a time so that we can discreetly test multiple projects at once. Ideally we should be able to spin these environments up and down as necessary.
We have microservice based architecture and are mostly using azure PAAS services in our infrastructure.
Currently i have tried to automate our infrastructure through terraform its almost done but next step is deployment of code as services are not containerized so tried using azure pipelines but its a huge task, can i get any better idea for this that how we could do this.
Should look at leveraging Azure Pipeline Templates Once this is defined then can reuse it everywhere. For instance with terraform created a template for doing the plan and apply that just needs to be fed in the directory the terraform is located in. This saved time across all projects as we just need to reference our template and the rest was taken care of.
In terms of your other question with the ability to spin up and spin down this can be easily done if the application is architected with that in mind. Keep in mind for deployment certain things where names must be unique: storage account, app service and things that are potentially shared: i.e. network.
The other piece to consider is how to ensure these ad hoc environments are actually being spun down. Would recommend something like a tagging strategy or process that cleans up resources that haven't been deployed in x amount of days.
Related
Currently, we are hosting five websites on a Linux VM. The websites reside in their separate directories and are hosted by Nginx. The SSL is terminated at Azure Application gateway which sends the traffic to the VM. If a file is updated in a remote repository, the local copy is updated by a cron task which is a simple Bash script running git pull and few additional lines. Not all five websites need to be updated at the same time.
We created the image of the VM and provisioned a VMSS set up.
What could be the easiest or standard way of deploying the codes to the VMSS? The codes also need some manual changes each time due to client's requirements.
Have a look into Azure Durable Functions as an active scripted deployment manager.
You can configure your Durable Function to be triggered via a cron schedule, then it can orchestrate a series of tasks, monitoring for responses from the deployment targets for acceptable response before continuing each step or even waiting for user input to proceed.
By authoring your complex workflow using either of C#/JavaScript/Python/PowerShell you are only limited by your own ability to transform your manual process into a scripted one.
Azure Functions is just one option of many, it really comes down to the complexity of your workflow and the individual tasks. Octopus Deploy is a common product used to automate Azure application deployments and may have templates that match your current process, I go straight to Durable Functions when I find it too hard to configure complex steps that involve waiting for specific responses from targets before proceeding to the next step, and I want to use C# to evaluate those responses or perhaps reuse some of my application logic as part of the workflow.
Currently, I'm storing all key/value pairs in Application Settings, but I'm not happy with this approach. What is the recommended way to store settings for dev, test, stage, and prod? I need to make sure that prod settings are not visible to developers. Is there a way to create 4 different JSON files and define access permissions on them? Or do I need to create 4 different Function apps (or subscriptions)?
Azure App Configuration is a relatively new service that sounds like it could help in terms of managing the config values centrally with more control than individual instance App Settings.
Beyond that, you could perhaps build segregation by limiting devs to pushing code only and not accessing the hosting environment (Azure portal, etc). The layer in between would be something like Azure DevOps or Github Actions that has access to Azure, while devs are limited to pushing code that triggers deployment.
Also worth reminding ourselves that devs ultimately have a lot of access by virtue of writing the code. If they want to get at runtime data, they can, somehow. If you consider the devs untrusted, you may have bigger problems. If it's just a matter of preventing mistakes, a solid devops process is the key.
I thought one of the advantages of Azure was that I could turn services on and off depending on when I want them to be available.
However I cant see how to pause my App Service Plan.
Is it possible?
I want to use the S1 tier so that I can play with what it offers. However I want to be able to pause the cost accumulation when I am not using it.
I see from the app service pricing help that an app will still be billed for even though it is in the stopped state.
Yet the link also clearly states that I only pay for what I use. So how does that work?
If you put your hosting plan onto the free tier, you will stop being charged for it. However if you have things like deployment slots and certificates these will be deleted.
The ability to turn services on and off, is more to do with being able to scale services, so if you need 50 servers for an hour you can easily do that.
What you can do to make your solution temporary is to create a deployment script, using Powershell or Resource manager Templates then you can deploy your solution for exactly as long as you need it and then delete it again when you don't. In this sense you can turn your services on and off at a whim.
Azure provides building blocks for you to create the solution you need, it is up to you to figure out how to best use those building blocks to create the solution you seek.
Edited to answer extended question.
If you want to use the S1 pricing plan, and not have it charge when you are not using it, the only way of achieving that is by using automation. Fortunately, this is reasonably trivial to achieve.
If you look at this template it is pretty much all configured to deploy a website from Github to Azure on demand. If you edit that to configure it to your needs you can have a new Azure website online with 2 minutes of running the script.
Then you would have another script that deleted it once you had finished.
Doing it this way you would loose no functionality, and probably learn quite a bit about what is possible with Azure along the way.
App Service Plan
An app service plan is the hardware that a web app runs on. In the free and shared tier your web apps share an instance with other web apps. In the other tiers you have a dedicated virtual machine. It is this virtual machine that you pay for. In that case it is irrelevant whether or not you have web apps running on your app service or not, you still have a virtual machine running and you will be charged for that.
To change the App Service Plan via PowerShell, you can run the following command
Set-AzureRmAppServicePlan -ResourceGroupName $rg -Name $AppServicePlan -Tier Free
I was able to accomplish this using the dashboard by selecting the App Service Plan, clicking Scale up (App Service Plan), and then from there if you click Dev/Test you can select the Free tier.
As others have mentioned, you need to script this. Fortunately, I created a repository with one-click deployment to your Azure resources.
https://github.com/jraps20/jrap-AzureVerticalScaling
The steps are intended to be as simple and generic as possible:
Execute the one-click deployment from the repo readme
Select the subscription, resource group etc.
Deploy resource to Azure
Set up your schedule to scale up and scale down as-needed
The scripting relies on runbooks and variables to maintain the previous state of each App Service Plan and App Services within those plans. Some App Services cannot be scaled due to specific settings being used (AlwaysOn, Use32BitWOrkerProcess, ClientCertEnabled, etc.). In those cases, the previous values are stored as variables prior to down scaling and then the original values are reapplied when the services are scaled up.
For more clarity, I have written a blog post that goes into detail. The post is pertaining to Sitecore, but applies to any App Service setup- Drastically Reduce Azure PaaS Hosting Costs in Non-Prod Environments With Scheduled Vertical Scaling. It also includes a brief video tutorial to show its use case.
Myself and others have been using this repository/approach for well over a year and it works great. I mostly use it for POC's to reduce costs when I'm not actively working on something. However, its main intention was for targeting non-prod environments to save costs during non-work hours.
Azure App Service Plan is just an logical concept of a set of features and capacity that you can share across multiple apps. I don`t think you can "pause" a plan, instead you can pause your service. and depends on billing model of each service, you might or might not get charged.
Pausing = Delete or lower tier.
Scripting is the key.
Design Diagram
Use scripts to create (also consider shared resources)
Delete using scripts
Use scripts to recreate.
eg: If we use resource group properly per environment then
Export-AzureRmResourceGroup will create a template for us (everything in the resource group will be pulled out as script). So we can delete it and recreate it anytime.
To pause a VM and stop billing you need to shut is down and deallocate it. Just shutting down still has the capacity reserved as if its running.
Storage can't be shutdown - it can be moved to lower cost tiers.
We have written a high scalable Cloudservice for MS Azure with two roles: "WebsiteRole" and "WebsiteWorkerRole". For better performance we deploy this Cloudservice in multiple regions (2x US, 2x EU, 1x JP). We have different configuration files for each region (EuWestProductive.azurePubxml, ServiceConfiguration.CloudEuWest.cscfg, Web.ReleaseEuWest.config).
Now the Problem: In each Region we have running the "WebsiteRole" and "WebsiteWorkerRole". But the "WebsiteWorkerRole" has only very small tasks, so that one extra small instance in one region is more than enough.
We tried to set the Role instance count to zero (ServiceConfiguration.CloudEuWest.cscfg). But this is not allowed:
Azure Feedback: Allow a Role instance count of 0
Is there an other way to remove a role when deploy the Cloudservice?
No, as you've discovered, a cloud service does not allow for scale to zero. You have to effectively remove the deployment. To have the minimum change to what you already have in place you could separate the two roles into two different deployments. Then have an Azure Automation Script, or set of scripts run elsewhere, that handles deploying the worker role when needed and decommissioning when it's not needed.
Depending on the type of workload that worker is doing you could also look at taking another route of using something like Azure Automation to perform the work. This is especially true if it's a small amount of processing that occurs only a few times a day. You're charged by the minute for the automation script, so just make sure it's going to run less than the actual current instance does.
It really boils down to what that worker is doing, how much processing it really needs to do, how much resources it needs and how often it needs to be running. There are a lot of options, such as Azure Automation, another thread on the web role, a separate cloud service deployment, etc. Each with their own pros and cons. One option might even to look at the new Azure Functions they just announced (in preview and charged by the execution).
The short answer is separate the worker from the WebSiteRole deployment, then decide the best hosting mechanism for that worker role making sure that the option includes the ability to only run when you need it to.
Thanks #MikeWo, your idea to separate the deployments was great!
I have verified this with an small example project and it works just fine. Now it is also possible to change the VM size and other configurations per region.
(Comments do not allow images)
We currently have a window service which send some notification emails to users after doing some processing on database(SQL database). Runs once in day.
We want to move this on azure cloud. One alternate is to put it on Azure VM as is. but I am finding some other best possible solution for that.
I study about recurring and on demand Web jobs but I am not sure is this is best solution.
Also is there any possibility to update configuration of service code in App.config without re-deploy the code of service on cloud. I means we can manage configuration from Azure portal.
Thanks in advance.
Update 11/4/2016
Since this was written, there are 2 additional features available in Azure that are both excellent choices depending on what functionality you need:
Azure Functions (which was based on the WebJobs described below): Serverless code that can be trigger/invoked in various ways, and has scaling support.
Azure Service Fabric: Microservice platform, with support for actor model, stateful and stateless services.
You've got 3 basic options:
Windows service running on VM
WebJob
Cloud service
There's a lot of information out there on the tradeoffs between these choices, but here's a brief summary.
VM - Advantages: you can move your service basically as it is without having to change much or any of your code. They also have the easiest connectivity with other resources in Azure (blob storage, virtual networks, etc). The disadvantage is you're giving up all the of PaaS advantages and are still stuck managing your own VM infrastructure
WebJob - Advantages: Multiple invocation options (queues, blobs, manually, queue receive loops, continuous while-loop style, etc), scheduled (would cover your case). Easy to deploy (can go with website, as a console app, automatically through Kudu), has some built in logging in Azure portal - and yes, to answer your question, you can alter the configuration in the portal itself for connection strings and app settings.
Disadvantages - you'll need to update code, you don't have access to underlying resources (if you need that), and more of something to keep in mind than a disadvantage - it uses the same resources as the webapp it's deployed with.
Web Jobs are the newest of the options, but at the same time appear to have active development going on to increase the functionality and usefulness.
Cloud Service - like a managed VM, has some deployment options, access to underlying VM if needed. Would require some code changes from your existing service.
There's nothing you've mentioned in your use case that makes me think a Web Job shouldn't be first thing you try.
(Edit: Troy Hunt has a great and relatively recent blog post illustrating most of the points I've mentioned about Web Jobs above: http://www.troyhunt.com/2015/01/azure-webjobs-are-awesome-and-you.html)