I would like to perform the following steps on schedule (presumably using Azure Automation):
Provision a VM in Azure
Run a powershell script on that VM
Deprovision VM
Actually I have more steps but left only 3 for simplicity.
I am new to IaC and appreciate your general guidance and advice.
Is it scope of Azure Automation or I need something else?
I would like to code everything in text format and put in Git and update automatically via Pull Requests
Should I use Runbooks or DSC?
Regarding step 2, I cannot figure out how I can upload my powersehll script into newly created VM and run it locally. The script downloads some files and updates some remote resources.
Thanks,
Ruslan
there are a lot of options and tools to achieve your goal.
If you will be working strictly in the Azure cloud, The following tools are most commonly used for building an environment.
Azure-powershell
Azure-CLI
ARM-templates
each of them very similar but all a little different with their own benefits to them, but they are all tools for building your virtual infrastructure. For configuring your resources there are other tools. Like you mentioned yourself, DSC is a tool to configure virtual machines.
if you are planning to use github to push your code, i would recommend using ARM-templates. You can very easily use your own or other templates by referencing in your code. However this might be the 'hardest' solution to learn and understand the syntax in comparison to the cli and powershell. But also the most frequently used.
It is possible to build your environment and configure it in the same script using the Azure-CLI, Azure-Powershell or an other opensource solution like Terraform, But this is not best practice.
A lot of starter scripts are publicly available on github and in the Microsoft docs.
if you have any specific questions you can always send me a message, i am currently working on azure automation myself.
To deploy my infrastructure I need to deploy a VM with a custom script extension. The only purpose of the VM, is to execute the script. After the execution of the script the VM should be deleted automatically.
How can this be done?
Additional information:
This is an azure resource manager deployment
the deletion should work in the azure marketplace environment as well.
this probably means you are doing something wrong, you can use Azure Container Instance to run the script and shutdown. it should work with marketplace as well (as far as I know you can have custom container in marketplace offerings).
Marketplace only allows you to use arm templates to deploy stuff, so you cannot really do what you are asking with an arm template. well, you might be able to hack something like that with nested deployments and complete mode, but I doubt that will pass moderation in marketplace.
technically, you can make vm delete itself as a part of the script. again, not something I would advise.
I am investigating ways to automate deployment of a specific build of a product to a specific Azure Cloud Service or VM.
The following steps would be automated, with as little manual intervention as possible:
Create a Cloud Service or VM
Install a specific build of the product (as a standalone exe or
Windows service, not IIS)
Tweak the configuration files(s)
Set up user account(s)
Run the exe/service
The code is currently in Visual Studio Online / TFS. We have Cruise Control .NET CI set up and we are looking at moving to TeamCity.
This will be used for the usual QA & Production type environments, but also for ad-hoc deployment e.g. if a trial feature has been added to the product and we want to deploy that to a new VM for a specific customer to play around with. Ideally we would be able to use the command line or a UI to pick the build, create the VM and specify any configuration changes.
One possible solution might be Octopus Deploy although I don't think this would be able to actually create an Azure VM. I will probably also look at the Azure API, and also TFS deploy.
Basically is this feasible, and are there any proven alternatives that I'm missing, in order to narrow down my research?
Thanks in advance!
While Octopus Deploy can do many things, in this particular scenario of yours, you're asking it to do three types of work - release management, automated provisioning and configuration management. It's a fine line between automation awesomeness and a really sticky situation.
Of the tasks you're asking, almost all of them can be done within Octopus today. I'd argue that it may be possible to Create a cloud service or VM. If there's some PowerShell cmdlet/library that allows you to spin up VMs with authentication, odds are you can do it Octopus - but it may not be the right tool to do that job today. Why?
In my opinion, it distorts the barrier between Developers, DevOps and SysAdmins. Whether you use Chef, Puppet, Salt, etc. whatever configuration management you have, that needs a whole layer of users with the expertise to back it up - often said expertise of system which the very developers who want such flexibility may not have. Secondly, right now this isn't a focus within Octopus (yet). I'd be hard pressed to say whether to use a tool such as Octopus on what it can do vs what it should do or not.
It's really nice that Azure now has support for preinstalling the Octopus tentacle for VMs. But that requires additional info such as, the Server thumbprint, port other supplementary configuration info in order to automate vm provisioning. That configuration management - should it be under Octopus's control, or something like Chef or Puppet? I honestly don't have an answer to this but my feeling as of now is not Octopus. Someday, perhaps, but until this is really ready and fully tested and vetted, I'd wait it out (a little) at least with Octopus.
If you're the adventurous type, then by all means try out Octopus. I may do a PoC (proof of concept) of this infrastructure automation later this year, but to rely on it today for business/production usage as the primary means of infrastructure automation will be risky and require a lot of work and experimentation. Again, I'm not saying it cannot be done, I'm questioning whether it should be done within Octopus as of this response today.
If anything, from the Octopus Deploy side of things is this feasible? Yes - it just hasn't quite been worked out yet. Looking at what you want to do, I'd say it's a two-phase process: 1. spinning up the new VM, attaching the tentacle to the environment and 2. running the deployment process on that new VM.
I'd also recommend checking out the Octopus blog. They're publicly talking about infrastructure automation. You can read about it here: http://octopusdeploy.com/blog/rfc-cloud-and-infrastructure-automation-support
I hope this response helps in some way.
The solution to the automated deployment in Azure is use ElasticBox.
I will skip the details of all the configuration options for Azure supported by ElasticBox, as they are detailed in the documentation section: http://elasticbox.com/documentation/deploying-and-managing-instances/using-azure/.
You only need to create a box (abstraction unit that ElasticBox uses to define the installation and configuration of the deployment of a service or application in any cloud) that takes care of the steps you need to be automated. So finally you will deploy the vm with near no manual intervention, just one click or a command with some parameters.
A box includes the variables necessary for your deployment and your scripts (In this case probably PowerShell, but they could be bash, python, perl, java, etc.)
When you deploy the box you create to deploy your application, ElasticBox will:
Create a Cloud Service or VM. (ElasticBox takes care of provision the vm in your Azure provider, or any of your preferred cloud provider).
Install a specific build of the product (as a standalone exe or Windows service, not IIS) -> This should be your install event script.
Tweak the configuration files(s) -> This should be part of your configure event script.
Set up user account(s) -> This should be part of your configure event script.
Run the exe/service -> This should be part of your start event script.
ElasticBox has a command line tool that enables to do VM deployments of your boxes and also you can manage your deployed vms with it: https://pypi.python.org/pypi/ebcli
It also support automatic termination of the vm after a custom time value.
This is quite a broad question, but certainly the goal is achieveable via one of a number of methods. While a bit old, Tom Hollander's blog on automated deployments is a good starting place. I've seen a lot of OctopusDeploy used as well as TeamCity but they all ultimately rely on Azure's PowerShell Cmdlets, Management Libraries in custom code or pure REST API calls.
Just an FYI; One option is to do everything by using the Azure Management API. I also like to reference the Azure Client Libraries in a VS project and do everything is C# code.
I have this situation where I need to install and configure various things in the OS. I know that I can use VMs for this, but I need the code deployment functionality and wonder if there is a way to have a custom OS image and also be able to do code deployments using visual studio publish.
Custom OS images is not option for Web/Worker roles (i.e. True PaaS Cloud Services). However you add OS customizations via a Startup Task. I can hardly think of anything that is not possible via a startup task.
I have two Azure VM's running in a cloud service. They contains almost the same thing. Some TCP port's are also opened between them.
Is it possible to create a deploy package from this existing setup so that at a later time can deploy this setup in an easy way. I.e. I want to be able to do this:
1. Create deploy package from existing setup *
2. Delete whole existing cloud service including VM's
3. Deploy the package from step 1 and have everything created again.
*I can save one of the VM's to my Azure storage and use it as template for both of them if that is easier.
How to accomplish this if it is possible?
Yes, you can take what you have as a template and use it to stand up multiple silos. But in IaaS, there isn't a notion of a deployment package. There's a few things you'll need to do...
1) understand how to take an existing VM and turn it into an image
2) use Powershell or another DevOps style automation suite (Chef/Puppet/etc..) to define deploy your silo.
You seem specifically interested in how to create an image so I'd recommend using the tutorial we have published on this. http://www.windowsazure.com/en-us/documentation/articles/virtual-machines-capture-image-windows-server/ This does of course presume you're running Windows Server. But a Linux version it can be found at: http://www.windowsazure.com/en-us/documentation/articles/virtual-machines-linux-capture-image/
The automation of a deployment depends on a great many things, so I'd suggest at a starting point, familiarizing yourself with the management API: http://msdn.microsoft.com/en-us/library/windowsazure/ee460799.aspx
With the implementation of Resource Manager, you can now easily use JSON template to deploy and redeploy resources in Azure. There are also starter templates available - https://azure.microsoft.com/en-us/documentation/templates/