The subject line is what I am looking to accomplish in a nutshell. The testing is for a Windows client that connects to a locally hosted server. I need the CodedUI test to run on as many VMs as possible.
I am new to Azure and all of the terminology associated with it, but have been doing a bit of research and it looks like Azure Pipelines may help me accomplish what I need. My company's Azure admin is not familiar with Pipelines and has asked if I may need to use the Microsoft administered Azure DevOps for that.
I am hoping that someone who knows what they're talking about could help me with this. Is what I am trying to do feasible? What are all the pieces that I will need? Is there an upper limit on how many VMs I can run a test on simultaneously?
Thanks in advance!
Azure DevOps Pipelines can help you accomplish this. There are some considerations though.
The standard way to UI test a web app would be to create a build that includes your app and tests, then create a new release definition with the built-in "Visual Studio Test" tasks, and run the release on a number of Microsoft-hosted agents (VMs).
First problem, since you are using a windows client, Microsoft-hosted agents probably won't work because they don't have connectivity to your network. You can use self-hosted agents, but that means you have VMs to manage now.
Second problem, pricing is not based on minutes, but on number of concurrent jobs. If you want to be able to run tests on 20 agents at once, you have to pay for 20 concurrent jobs, even if you only run your tests for 5 minutes a month.
Putting on my creative thinking hat... Here's a solution that would work with a single Microsoft-hosted agent. You could create an ARM template that does the following:
Stand up as many VMs as you want
Use VM extensions and Powershell DSC and/or other scripts to configure the VM (You can install Windows features, connect to your domain, install Chrome, etc)
Run powershell scripts to download your application, configuration, and testing tools from somewhere (ex. file server or azure storage), then run your tests and publish the results
Your release pipeline would deploy the ARM template to a new resource group, wait, and then delete everything when the tests are done.
This solution has the benefit of running on VMs in your network without making you maintain or pay for VMs long term.
Related
I would like to perform the following steps on schedule (presumably using Azure Automation):
Provision a VM in Azure
Run a powershell script on that VM
Deprovision VM
Actually I have more steps but left only 3 for simplicity.
I am new to IaC and appreciate your general guidance and advice.
Is it scope of Azure Automation or I need something else?
I would like to code everything in text format and put in Git and update automatically via Pull Requests
Should I use Runbooks or DSC?
Regarding step 2, I cannot figure out how I can upload my powersehll script into newly created VM and run it locally. The script downloads some files and updates some remote resources.
Thanks,
Ruslan
there are a lot of options and tools to achieve your goal.
If you will be working strictly in the Azure cloud, The following tools are most commonly used for building an environment.
Azure-powershell
Azure-CLI
ARM-templates
each of them very similar but all a little different with their own benefits to them, but they are all tools for building your virtual infrastructure. For configuring your resources there are other tools. Like you mentioned yourself, DSC is a tool to configure virtual machines.
if you are planning to use github to push your code, i would recommend using ARM-templates. You can very easily use your own or other templates by referencing in your code. However this might be the 'hardest' solution to learn and understand the syntax in comparison to the cli and powershell. But also the most frequently used.
It is possible to build your environment and configure it in the same script using the Azure-CLI, Azure-Powershell or an other opensource solution like Terraform, But this is not best practice.
A lot of starter scripts are publicly available on github and in the Microsoft docs.
if you have any specific questions you can always send me a message, i am currently working on azure automation myself.
Recently I've started experimenting and getting familiar with some of the Azure offerings. I made a simple app, connected it with azure functions and azure storage as well as some other offerings like service bus for example.
So far so good, the app is working great and I got my feet wet with some great Azure services.
But now I'm unsure on how best to proceed because what I have so far is a development version of my app. If I wanted to make a prod version do I have to provision a different set of all the azure resources used for the dev version?
So basically, I would have mydevsite.azurewebsites.net and myprodsite.azurewebsites.net. Is this correct? I can restrict mydevsite.azurewebsites.net with some IP address restrictions so that is not publicly available but I still feel this is a hacky way of doing this and that there should be a better way.
Is there a common approach to a scenario like this?
This is a bit of a broad question, but I can tell how I have done it before.
A common setup would be three environments, Dev, Test and Production.
Dev mostly runs on the developer's machine (as much as it can). We use a local IIS installation to run the web app, and a local SQL Server as a database. Azure Storage and Cosmos DB can also be emulated locally. Certain services like Search for example can't be run locally so you would have to run those in Azure anyway.
Test and Production are basically two identical resource groups with the same resources, just configured slightly differently. So double the App Service Plans, SQL databases etc.
Depending on how you want to do it though, you can of course share resources across environments. But it is a good idea to somehow make sure they do not accidentally use the other environment's stuff. And the definite bad side of this is that you are putting production data in the same place as test data, which frankly should not be together.
I know some organizations run a Dev environment fully in Azure. There can be a couple reasons for this: very heavy environment which can't really run on dev machines, or they want to test ARM template deployment at dev stage too.
Having duplicated services allows you to use ARM templates for automatically deploying and updating the infrastructure, which is pretty nice.
If you are on Standard or higher, you might think to use Deployment Slots in App Service for different environments, but they are really not meant for that purpose. We use them to reduce application downtime when deploying a new version, and as a fallback if the update turns out bad. So the deployment goes to a "staging" deployment slot, which gets swapped with the other one, and the new version is live. We then stop the deployment slot so we are not running the older version in the background unnecessarily.
But otherwise we have a separate App Service Plan with separate Web Apps with their own staging slots.
Deployment slots documentation: https://learn.microsoft.com/en-us/azure/app-service/web-sites-staged-publishing
I am investigating ways to automate deployment of a specific build of a product to a specific Azure Cloud Service or VM.
The following steps would be automated, with as little manual intervention as possible:
Create a Cloud Service or VM
Install a specific build of the product (as a standalone exe or
Windows service, not IIS)
Tweak the configuration files(s)
Set up user account(s)
Run the exe/service
The code is currently in Visual Studio Online / TFS. We have Cruise Control .NET CI set up and we are looking at moving to TeamCity.
This will be used for the usual QA & Production type environments, but also for ad-hoc deployment e.g. if a trial feature has been added to the product and we want to deploy that to a new VM for a specific customer to play around with. Ideally we would be able to use the command line or a UI to pick the build, create the VM and specify any configuration changes.
One possible solution might be Octopus Deploy although I don't think this would be able to actually create an Azure VM. I will probably also look at the Azure API, and also TFS deploy.
Basically is this feasible, and are there any proven alternatives that I'm missing, in order to narrow down my research?
Thanks in advance!
While Octopus Deploy can do many things, in this particular scenario of yours, you're asking it to do three types of work - release management, automated provisioning and configuration management. It's a fine line between automation awesomeness and a really sticky situation.
Of the tasks you're asking, almost all of them can be done within Octopus today. I'd argue that it may be possible to Create a cloud service or VM. If there's some PowerShell cmdlet/library that allows you to spin up VMs with authentication, odds are you can do it Octopus - but it may not be the right tool to do that job today. Why?
In my opinion, it distorts the barrier between Developers, DevOps and SysAdmins. Whether you use Chef, Puppet, Salt, etc. whatever configuration management you have, that needs a whole layer of users with the expertise to back it up - often said expertise of system which the very developers who want such flexibility may not have. Secondly, right now this isn't a focus within Octopus (yet). I'd be hard pressed to say whether to use a tool such as Octopus on what it can do vs what it should do or not.
It's really nice that Azure now has support for preinstalling the Octopus tentacle for VMs. But that requires additional info such as, the Server thumbprint, port other supplementary configuration info in order to automate vm provisioning. That configuration management - should it be under Octopus's control, or something like Chef or Puppet? I honestly don't have an answer to this but my feeling as of now is not Octopus. Someday, perhaps, but until this is really ready and fully tested and vetted, I'd wait it out (a little) at least with Octopus.
If you're the adventurous type, then by all means try out Octopus. I may do a PoC (proof of concept) of this infrastructure automation later this year, but to rely on it today for business/production usage as the primary means of infrastructure automation will be risky and require a lot of work and experimentation. Again, I'm not saying it cannot be done, I'm questioning whether it should be done within Octopus as of this response today.
If anything, from the Octopus Deploy side of things is this feasible? Yes - it just hasn't quite been worked out yet. Looking at what you want to do, I'd say it's a two-phase process: 1. spinning up the new VM, attaching the tentacle to the environment and 2. running the deployment process on that new VM.
I'd also recommend checking out the Octopus blog. They're publicly talking about infrastructure automation. You can read about it here: http://octopusdeploy.com/blog/rfc-cloud-and-infrastructure-automation-support
I hope this response helps in some way.
The solution to the automated deployment in Azure is use ElasticBox.
I will skip the details of all the configuration options for Azure supported by ElasticBox, as they are detailed in the documentation section: http://elasticbox.com/documentation/deploying-and-managing-instances/using-azure/.
You only need to create a box (abstraction unit that ElasticBox uses to define the installation and configuration of the deployment of a service or application in any cloud) that takes care of the steps you need to be automated. So finally you will deploy the vm with near no manual intervention, just one click or a command with some parameters.
A box includes the variables necessary for your deployment and your scripts (In this case probably PowerShell, but they could be bash, python, perl, java, etc.)
When you deploy the box you create to deploy your application, ElasticBox will:
Create a Cloud Service or VM. (ElasticBox takes care of provision the vm in your Azure provider, or any of your preferred cloud provider).
Install a specific build of the product (as a standalone exe or Windows service, not IIS) -> This should be your install event script.
Tweak the configuration files(s) -> This should be part of your configure event script.
Set up user account(s) -> This should be part of your configure event script.
Run the exe/service -> This should be part of your start event script.
ElasticBox has a command line tool that enables to do VM deployments of your boxes and also you can manage your deployed vms with it: https://pypi.python.org/pypi/ebcli
It also support automatic termination of the vm after a custom time value.
This is quite a broad question, but certainly the goal is achieveable via one of a number of methods. While a bit old, Tom Hollander's blog on automated deployments is a good starting place. I've seen a lot of OctopusDeploy used as well as TeamCity but they all ultimately rely on Azure's PowerShell Cmdlets, Management Libraries in custom code or pure REST API calls.
Just an FYI; One option is to do everything by using the Azure Management API. I also like to reference the Azure Client Libraries in a VS project and do everything is C# code.
Deploying apps to Windows Azure Websites feels incredibly more convenient compared to the initial WebRole option. Being able to push through Git, and get the app restarted in ~20s is a massive improvement over the 15min role redeploy.
Thus, I am considering using this option for what used to be hosted in WorkerRole as well. Indeed, it's possible to allocate a full VM to run WA website.
Are there any gotcha to be aware of when attempting this? Obviously, as the name suggests, WA websites are not intended for backoffice processing.
In an upcoming feature for Windows Azure Websites, the scenario you're referring to will be supported:
http://github.com/projectkudu/kudu/wiki/Web-jobs
The following will allow you to have 2 types of processes to run aside your website:
Triggered - Start your process on a scheduled (or manual) basis.
Continuous - Your process will always be on (if it goes, it brings it back up).
Regarding the differences between Azure Webrole and Azure Website, there's a different question:
What is the difference between an Azure Web Site and an Azure Web Role
Cloud Services gives you two different environments: staging and production. You can also use Continous deployment with Git, Tfs, Codeplex, Dropbox too. But if you don't need this two environments, you can go with websites.
Using a Virtual Machine, you'll be responsible for the operating system, runtime, data and also your app (obviously). Just be aware that you'll have to apply the service packs / security packs by yourself. If your app doesn't use 3rd party components I don't see a reason to use a VM for that.
I would like use a continuous integration system in my projects. However, I don't want to use a server sitting in my office, instead I'd like my continuous integration server to run on Windows Azure. Has anyone set this up? Are there recipes to host Hudson or CruiseControl.Net (or any other CI system) within Azure?
We use the Build Manager in Team Foundation Server to push our automated builds to Azure. We set up our Azure hosted services to allow for Web Publish through remote desktop using this plugin - http://dunnry.com/blog/2010/12/20/UsingWebDeployWithWindowsAzure.aspx
You do not need to involve Team Foundation Server to use the plugin - you can set up your instances to publish to azure instantaneously through Visual Studio Web Publish without CI.
I think there are two parts to the Azure CI solution - automated builds from your source control and actually pushing the bits to azure. The plugin makes the publish to Azure much, much faster. So if you want real time access to changes on Azure, you will need to look into the plugin.
It sounds like you are trying to use Azure as an infrastructure provider, which runs a VM where your builds happen.
This is not a particularly good use-case for Azure. Azure is a platform to run your custom-built applications. While it does provide VM's similarly to Amazon or other IaaS cloud providers, those VM's are "stateless", can go up and down at will and meant to act as application servers where more than one can be up at any given time.
You can probably get this to work on an Azure VM but I am not sure if the pain would be worth it.
Azure's instances are application servers, not "windows servers".
HTH
I have installed Jenkins on Windows Azure, it works very well for me.
http://blogs.msdn.com/b/gongcheng/archive/2013/02/27/jenkins-on-windows-azure-the-missing-manual.aspx