I'm able to register my existing Azure VM as build agent in VSTS. Build works fine but I would like to turn the machine on and off only if there is any work for it. Is there any way how I can turn on the VM if I see there is some work for it in a queue and then turn it off if the queue is idle for let's say 5 minutes?
Add a hosted agent job to the start to start the VM and add an agent job or additional step to the end to shut down the VM when finished.
Add an Agent job to run this as hosted to start the VM. Then use all build steps as self-hosted(private).
Add your Azure subscription
Choose Inline Script to add the following to start it with VM name and resource group
start-AzureRmVM -Name ""-ResourceGroupName ""
Add another Agent Job or additional step as the last Step
To shut down the VM when the build has finished.
With the following Script
stop-AzureRmVM -Name ""-ResourceGroupName "" -Force
I'm wondering if this could be helpful for you because this is not exactly what you asked for. I used REST API calls before the build and after the build to start and then stop a specific VM. See how to start and power off a VM in the API documentation.
I created an agentless job as the first step, with a task entitled Invoke REST API. The task lets you authenticate to your Azure account, so you don't have to handle that manually. All you have to do is specify the URL suffix. For instance, to start a VM named MyVm, you add a suffix similar to the following one:
/subscriptions/subscription-id/resourceGroups/resource-group-name/providers/Microsoft.Compute/virtualMachines/MyVm/start?api-version=2018-06-01
where the subscription ID and resource group can be verified in the Overview page of your virtual machine in Azure.
After the build you can add another agentless job, but this time with a REST call of the powerOff endpoint:
/subscriptions/subscription-id/resourceGroups/resource-group-name/providers/Microsoft.Compute/virtualMachines/MyVm/powerOff?api-version=2018-06-01
There isn’t the good way to start/stop azure VM when build in VSTS. Regarding WebHook, there isn’t the event for queue build, for other events, they aren’t meet many requirements (e.g. non-CI build, queue build manually)
I recommend you use Hosted agent, with free Hosted Pipeline, you get 4 hours (240 minutes) per month and maximum duration of 30 minutes per build or deployment in Team Services.
How to buy more pipeline capacity for builds and releases in Visual Studio Team Services
Workaround:
Queue build at the specify time, for example, 7:00 am on Monday and Thursday, then you can auto-start and auto-shutdown the VM
Run on Hosted agent (Execute Powershell to start VM)=> Run on private agent
More information, you can refer to: How to Create a Monster Build Agent in Azure for Cheap
You could try using a WebHook in VSTS, and select the most appropriate event (like code pushed, or release created). Open the admin page for the team project in Visual Studio Team Services, and on the Service Hooks tab, run the subscription wizard, select the event you're interested in, and fill in the relevant criteria.
You could then use an Azure Automation Runbook to subscribe to the WebHook and, using PowerShell, either use the build VM if it's already started or else start it up. You could use another WebHook to signal that a build has been completed, or poll VSTS from the VM via its various APIs.
You might want to delay shutting the VM down after a build, as there is a time cost associated with starting up / shutting down.
Related
We have created a pipeline to restart multiple VM services in correct order. Within Azure Logic App I can create a release, but want to deploy the same release daily. How can I do this in Logic Apps?
I current use logic apps to shutdown and start VMs daily as they are not used during silent hours but need to deploy restart services pipeline regularly.
From the picture you can see it only creates release not deploys it. I need it to only deploy to CL04 stage daily not all three stages and really don't need a new release every day, just redeploy release 7 to the same environment that the start VM logic app is running on daily.
but want to deploy the same release daily.
You can achieve this if you use Recurrence Trigger in your Logic app workflow.
For complete information you can go through this document.
I am using third party software for master data management. This software does not have API's to perform deployment across environment. It has command line utility which needs to be installed on machine and call the commands to perform the installation. I am looking to automate this solution using Azure Devops pipeline. What I am trying to do is as below
Create windows VM in Azure and install command line utility on it.
Store script in some folder in the VM.
Use Azure pipeline to call this script which is stored in my VM.
I don't even know if it possible to do such things. I tried looking on internet on how to call script stored in VM via Azure pipeline but didn't find any useful link.
If any one has done such activity or have an idea how can it be achieved please help.
this can help you
Running the command line in a Azure Virtual Machine is fully supported in Azure Pipelines. You could install a self-hosted agent in the VM.
Before that, I recommend that you could create a new agent pool for the self-hosted agents. Please go to the organization settings -> Agent pools-> click the "Add pool" -> choose the "self-hosted" type.
Then, you could refer to this document to complete the installation. When you configure the agent, you could choose the new created agent pool.
After installation, you could create a pipeline, choose the new created agent pool and add a Command Line task to run the command line. Of course, if you have many agents in the same agent pool, you could also set the demands in the pipeline to specify one agent.
reference : Is it possible to run a command line in a Azure Virtual Machine from Azure DevOps pipeline?
I have TFS 2015 and i was able to automated the build process from the branch and get the files from the drop folder as shown below:
It has release for multiple projects like Web API and Windows Service
I want Azure VM on which i want to automate the deployment process - continuous delivery.
Deploy the Web API on IIS on Azure VM
Deploy the Windows Services On Azure VM.
Run Scripts SQL.
I have credentials of Azure VM. How i can perform the three above steps.
I have worked on a similar problem in the past so can probably help you out (MSFT, if it helps).
Web Api on IIS on Azure VM
This is almost completely automated in the form of WinRM - IIS Web App Deployment task that you can find and add in your release definition. The link provides complete instructions on what parameters to provide and tweaks to be done for Azure VM compared to on-premise ones. There are a few prerequisites to running this task, like installing and configuring IIS on the VM which the documentation discusses in detail. As a necessary input to this task, you need to provide the web deploy package which I am assuming was generated as your build output. If not, you can refer to this SO post to get the required output. If you have parameters like connection strings that you wish to modify at deploy time, using a parameters.xml file in the above task.
Windows Service on Azure VM
There is no completely automated task for this requirement, but it is pretty straight-forward. It can be achieved by using the PowerShell on Target Machines task along with Azure File Copy task. For the first task, all that is required as input is the .exe of the windows service that you wish to deploy, which should be generated as the output of your build process (build artifacts). Much of the remote machine inputs for this task is similar to the previous one so you should not have any problem there. You will need to check-in the Powershell script that does the actual windows service installation, in your source code as part of the same windows service project (copy local = True). This will ensure that as the build output, you will have access to the powershell script which you can use in the second task. Azure File Copy is required to copy your powershell script to the Azure VM so that the Powershell task can execute it. Let's assume you copied the powershell script to a folder C:\Data\ on the Azure VM.
$serviceName = "MyWindowsService"
$exeFullName = "path\\to\\your\\service.exe"
$serviceDisplayName = "MyWindowsService"
$pss = New-Service $serviceName $exeFullName -DisplayName $serviceDisplayName
-StartupType Automatic
Add this content to the checked in powershell file and name it installWindowsService.ps1. Then in the powershell task provide the path of the powershell file to execute as C:\Data\installWindowsService.ps1.
Run SQL Scripts on Azure VM
I haven't personally worked on this so the best I can do is point you in the right direction. If you are using DACPAC for your SQL deployment, you can use the WinRM - SQL Server Database Deployment task. If you just intend to execute scripts, use the remote powershell task from above and refer this post that will help you with running SQL commands through powershell script
Seems you want the CD release process picks up the artifacts published by your CI build and then deploys them to your IIS servers/Windows Services on Azure VM.
If you've just completed a CI build, then you should create a new release definition that's automatically linked to the build definition.
Open the Releases tab of the Build & Release hub, open the + drop-down in the list of release definitions, and choose Create release definition.
For 2, write a powershell script to handle this, ensure build outputs
were available to copy from the ‘Drop’ folder on the build and that
they are copied to C:\xxx\ on the target VM(s). More detail steps
please refer this blog.
For 3, you could use Azure SQL Database Deployment task. Either
select the SQL Script file on the automation agent or on a UNC path
that is accessible to the automation agent. Or directly enter the
InLine SQL Script to run against the Azure SQL Server Database. Also take a look at the tutorial.
Maybe not all the task is fully Compatible with TFS2015 version, you could upgrade your TFS version to get more new features or customize your own build/release task to handle it.
In Visual Studio Team Services, I would like to specify the Azure Subscription via a variable (see image below). The reason for this is that our ops team maintains the NON-PROD and PROD environments on different subscriptions.
However, when I do this I get the following in the log:
##[section]Starting: Azure App Service Deploy: XXXXfunc
==============================================================================
Task : Azure App Service Deploy
Description : Update Azure App Service using Web Deploy / Kudu REST APIs
Version : 2.1.10
Author : Microsoft Corporation
Help : [More Information](https://aka.ms/azurermwebdeployreadme)
==============================================================================
[RESOURCE_NAME] exists false
##[warning]Can\'t find loc string for key: CouldnotfetchacccesstokenforAzureStatusCode
##[error]CouldnotfetchacccesstokenforAzureStatusCode 401 Unauthorized
##[section]Finishing: Azure App Service Deploy: XXXXfunc
If I set the subscription using the drop down, then it all works fine.
Is this possible to do? If so, how?
UPDATE:
I am using App Service here as a simple example, but I also have a collection of Azure PowerShell tasks joined into a single Task Group. Each of these tasks need to have the subscription specified. If I can't pass a Azure sub in, then I'll have to either clone the Task Group or just list each step individually.
Its easier (and fits with how the system has been designed), if instead of this, you just use the Releases workflow to setup your dev/test environment, with the app service deploy pointing at that subscription, then clone that environment, call the new one "prod", and modify the release step in that environment to point at the other subscription. You can then also add governance (ie. email approval processes) around the build and deploy, and ensure the exact same build artifact is deployed to both environments, but when you want them to. I wrote a blog post last year to take you through this step by step, see https://russellyoung.net/2016/11/09/continuous-deployment-of-a-asp-net-core-app-to-azure-using-vsts/
We have an app we deploy to Azure. It involves deploying several cloud services with a mix of web roles and worker roles. Some of the worker roles pick items up off a queue and process them. We also have some scheduled jobs that run periodically (backing up Azure table storage, etc).
Every time we deploy, we have to watch the Staging environment in the portal and manually stop the roles from starting. We have to do this because we don't want the Staging and Production slots both processing information at the same time (e.g. pulling from the same queue but processing it differently, or both running the same scheduled job simultaneously, etc).
The only way I've found to have a deployment go into Staging in a stopped state is to leave the last deployment there also stopped. Downside is you're charged for those instances, even when they're not running.
So, how do you deploy to an empty staging slot in Azure without the deployment starting up?
EDIT: We kick off the builds through Visual Studio or Visual Studio Online (i.e. TFS). Don't usually use powershell.
There is no way to create a deployment but not have it start. What you can do instead is have a setting in your csdef/cscfg that your code would read during OnStart.
For example, you would have a setting called "ShouldRun" set to False. In OnStart you would have a loop that checked that setting and exits the loop if ShouldRun==True. After you deploy you would then go to the portal and change that setting to True whenever you are ready for it to start processing. Once the loop exits then the OnStart method will finish which will cause Azure to call your Run method and bring your instances to the Ready state.
In addition you could add a Changed event handler to stop processing messages when the setting was changed to False. This would let you first stop your production deployment and then start your staging deployment.
For me, you need to separate even your queue's and configs. Another option, you can create a powershell script to stop your cloud service after publish to it.
http://msdn.microsoft.com/en-us/library/dn495211.aspx