I'm using Azure for hosting and Azure Pipelines for CI/CD operations
I have an image build and deploy operations defined like that:
- stage: Package
displayName: 'Package app'
jobs:
- job:
steps:
- task: Docker#2
displayName: 'Build image'
inputs:
containerRegistry: '$(containerRegistry)'
repository: '$(containerRepository)'
command: 'build'
Dockerfile: './Dockerfile'
buildContext: '.'
tags: |
$(Build.BuildId)
- task: Docker#2
displayName: 'Push image'
inputs:
command: push
containerRegistry: '$(containerRegistry)'
repository: '$(containerRepository)'
tags: |
$(Build.BuildId)
- stage: Deploy
displayName: 'Deploy'
jobs:
- job:
steps:
- task: AzureWebAppContainer#1
inputs:
azureSubscription: $(subscription)
appName: $(appName)
What should I do to execute some operations in my container after task AzureWebAppContainer is finished? I have to make some database updates after the deploy operation.
I've tried to find documentation for Azure and search for some SO topics, but didn't find any solutions yet, except usage of entrypoint / cmd for database updates, which is not working for me
I think there should be some Azure pipelines mechanism to perform such actions
You can use the startup command in the AzureWebAppContainer#1 or the AzureAppServiceSettings to manage the afterward operation.
By the way, you could also refer to this doc for Azure Web App for Containers to get more details.
Related
I have pushed a new image in the azure container registry
But i am unable to find the image in azure container registry
Please find the deplyment yaml file
name: Dotnet Code Build and Push
jobs:
- job: Job_1
displayName: Agent job 1
pool:
vmImage: ubuntu-18.04
steps:
- checkout: self
- task: Docker#0
displayName: Build an image
inputs:
azureSubscription: 'sc-***dp'
azureContainerRegistry:
loginServer: *****aksacr.azurecr.io
id: "/subscriptions/4f76bb2f-c521-45d1-b311-b87bf**747ff/resourceGroups/eus-***dp-rg/providers/Microsoft.ContainerRegistry/registries/*****aksacr"
imageName: ***dpacr.azurecr.io/ims-***dp/$(Build.Repository.Name):$(Build.BuildId)
- task: Docker#2
displayName: Login to ACR
inputs:
command: login
containerRegistry:'sc-***dp'
- task: Docker#2
displayName: Push an image
inputs:
azureSubscription: 'sc-***dp'
repository: ims-***dp
azureContainerRegistry: '{"loginServer":"*****aksacr.azurecr.io", "id" : "/subscriptions/4f76bb2f-c521-45d1-b311-b87bf**747ff/resourceGroups/eus-***dp-rg/providers/Microsoft.ContainerRegistry/registries/*****aksacr"}'
action: Push an image
I would recommend to create a new service connection with the container registry if one does not exist.
and select Azure Container Registry.
Then you can push in your registry using the below task. Keep in mind that ${{container}} should have a notation as xxx.azurecr.io/imagename:tag
- task: Docker#2
displayName: pushing image ${{container}}
inputs:
containerRegistry: 'serviceConnectionName'
repository: '${{container}}'
command: 'push'
tags: |
mytag
Try to push the code in azure devops pipeline so you can directly push image in azure container registry
Here we can docker task it can build up and push the image with help of setting and it take the image from repository and then it can push into the azure container registry
task: Docker#2
displayName: Push an image
inputs:
command: buildAndpush
repository: $(imageRepository)
dockerfile: $(dockerfilepath)
ContainerRegistry: $(dockerRepositoryserviceconnectio)
tags:
$(tag)
Save and run.
Note : After running you can't see Repository please refresh again because in azure pipeline the job may be still running once the job is completed & successfully run and refresh you can able see image is pushed
Please refer this document by rajaniesh kaushikk has given detail information
Please check the repository and containerRegistry value in your Docker#2 task, make sure they points to target resource. My script for your reference below. Please note the parameters in Docker#2 task, no azureContainerRegistry and command needed.
variables:
dockerRegistryServiceConnection: acrconn1
imageRepository: 'acrlearn'
containerRegistry: 'test.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'windows-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
I'm used to using the classic Devops "Release" pipelines for deploying code changes to Kubernetes clusters. Recently I've been looking into switching to using Azure Pipelines "deployment" jobs together with "Environments". It seems to work really well and I like a lot of the features, like being able to inspect the Kubernetes entities associated with your deployments, and track the deployment history.
Something that I'm accustomed to from the classic Release pipelines is rolling back to an old deployment if it is discovered that a bug has been released (to production for example). Since Release pipelines are based on build artifacts, you simply run the deployment on the old artifact in the Releases UI.
Now using deployments under the Environments tab, I'm not sure how to run a rollback, short of actually making a code change to revert back to the old state (and run through CI builds again needlessly). Another option is, since the deployment is done relative to the code (or commit) rather than an artifact, one could manually run a new pipeline and target the given commit - but this is quite cumbersome to achieve in the Devops UI, and seems prone to errors. In my opinion rolling back should be really easy to achieve, and not prone to errors.
Any ideas how to do this? Here is a sample of my yaml file
trigger:
batch: true
branches:
include:
- master
pr:
branches:
include:
- master
variables:
azureContainerRegistry: <registryUrl>
azureContainerRegistryServiceConnection: <serviceConnection>
kubernetesConfigPath: kubernetes
kubernetesNamespace: <my-namespace>
major: 0
buildNumber: $(major).$(Build.BuildId)
imageName: "$(azureContainerRegistry)/<my-app>:$(buildNumber)"
stages:
- stage: Bake
displayName: "Build and Push image"
jobs:
- job: Validate
displayName: "Build image"
pool:
name: "Docker"
steps:
- script: docker build -t $(imageName) .
displayName: Build App
- job: Publish
displayName: "Push image"
dependsOn: Validate
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
name: "Docker"
steps:
- task: Docker#2
displayName: Login to Container Registry
inputs:
command: login
containerRegistry: $(azureContainerRegistryServiceConnection)
- script: docker push $(imageName)
displayName: PUSH $(imageName)
- stage: DeployTest
displayName: "Deploy TEST"
dependsOn: Bake
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
jobs:
- deployment: Deploy
environment: <my-test-env>.$(kubernetesNamespace)
pool:
name: "Docker"
strategy:
runOnce:
deploy:
steps:
- task: qetza.replacetokens.replacetokens-task.replacetokens#3
displayName: "Replace tokens"
inputs:
targetFiles: $(kubernetesConfigPath)/base/*.yaml
escapeType: none
tokenPrefix: "{"
tokenSuffix: "}"
- task: Kubernetes#1
displayName: "kubectl apply"
inputs:
namespace: $(kubernetesNamespace)
command: apply
arguments: -k $(kubernetesConfigPath)/test
versionSpec: 1.7.0
checkLatest: true
- task: Kubernetes#1
displayName: "kubectl rollout status"
inputs:
namespace: $(kubernetesNamespace)
command: rollout
arguments: "status deployments/<my-app>"
versionSpec: 1.7.0
checkLatest: true
- stage: DeployProd
displayName: "Deploy PROD"
dependsOn: DeployTest
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
jobs:
- deployment: Deploy
environment: <my-prod-env>.$(kubernetesNamespace)
pool:
name: "Docker"
strategy:
runOnce:
deploy:
steps:
- task: qetza.replacetokens.replacetokens-task.replacetokens#3
displayName: "Replace tokens"
inputs:
targetFiles: $(kubernetesConfigPath)/base/*.yaml
escapeType: none
tokenPrefix: "{"
tokenSuffix: "}"
- task: Kubernetes#1
displayName: "kubectl apply"
inputs:
namespace: $(kubernetesNamespace)
command: apply
arguments: -k $(kubernetesConfigPath)/prod
versionSpec: 1.7.0
checkLatest: true
- task: Kubernetes#1
displayName: "kubectl rollout status"
inputs:
namespace: $(kubernetesNamespace)
command: rollout
arguments: "status deployments/<my-app>"
versionSpec: 1.7.0
checkLatest: true
Redeploying from the older version of the code is the way to do it.
one could manually run a new pipeline and target the given commit -
but this is quite cumbersome to achieve in the Devops UI, and seems
prone to errors
This is you need a well-organised source control branching and tagging policy. If you previously deployed from branch "releases/release-20212710.01", then deployed from branch "releases/release-20212710.02", you don't need to make any code changes. Rolling back just means selecting the older branch – which still exists, with the same code as before – and deploying.
I am building a Docker image in my Azure pipeline. Now, I want to push this image to multiple aws accounts(dev, stage, prod) depending on the configuration parameters. The problem is, image is not available in publish artifact. I came across this and this article during my research. I am confused about the solution regarding saving the docker image so it can be available in publish artifact. I have two specific questions:
How will I use the docker save command in Azure pipeline task after docker build. The available docker task doesn't have this command.
Is there any better way of doing this apart from saving an image.
How will I use the docker save command in Azure pipeline task after docker build. The available docker task doesn't have this command.
This related to the task version.
Steps: Add task docker->switch the task version to 0->select the option Run a Docker command, then we could run the docker save command, check the pic below.
Is there any better way of doing this apart from saving an image.
We recommend that you use this to upload the docker image as an artifact.
Nowadays the preferred way of writing pipelines is with yaml.
Please see my code on how to do it in a multistage pipeline with Artifactory docker registry.
If we are in master the the MY_ADDITIONAL_TAG will be also pushed.
trigger:
- master
resources:
- repo: self
variables:
tag: '$(Build.BuildId)'
stages:
- stage: Build
displayName: Build DevOps base image
jobs:
- job: Build_and_Push
steps:
- task: Docker#2
displayName: Build
inputs:
containerRegistry: ''
repository: 'MY_REPO/IMAGE'
command: 'build'
Dockerfile: 'PATH_TO_MY_DOCKERFILE'
tags: $(tag)
- task: Bash#3
displayName: Save Docker Image
inputs:
targetType: 'inline'
script: |
docker save MY_DOCKER_REPO_IMAGE_NAME:$(tag) -o $(Pipeline.Workspace)/MY_IMAGE_FILE.tar
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)/MY_IMAGE_FILE.tar'
artifact: 'MY_ARTIFACT'
publishLocation: 'pipeline'
- stage: Push
displayName: Push DevOps base image
jobs:
- job: Push
steps:
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'current'
artifactName: 'MY_ARTIFACT'
targetPath: '$(Pipeline.Workspace)'
- task: Bash#3
displayName: Load Docker Image
inputs:
targetType: 'inline'
script: |
docker load --input $(Pipeline.Workspace)/MY_IMAGE_FILE.tar
docker tag MY_DOCKER_REPO_IMAGE_NAME:$(tag) MY_DOCKER_REPO_IMAGE_NAME:MY_ADDITIONAL_TAG
- task: Docker#2
displayName: push development tags
inputs:
containerRegistry: 'MY_DOCKER_REGISTRY'
repository: 'MY_REPO/IMAGE'
command: 'push'
tags: |
$(tag)
- task: Docker#2
condition: eq(variables['Build.SourceBranch'], 'refs/heads/master')
displayName: push 0-devops tag
inputs:
containerRegistry: 'MY_ARTIFACT'
repository: 'MY_REPO/IMAGE'
command: 'push'
tags: |
MY_ADDITIONAL_TAG
After a lot of reserach I found this article which resolved my issue https://dev.to/n3wt0n/container-image-promotion-across-environments-build-artifact-5887
Note: We can manually add commands which are not present in the Docker task snippet available in Azure DevOps. This is also mentioned in the article and steps to do this.
So I've finished my backend and frontend part of the project.
Now big aspect of my project is scraper function, which is implemented in the backend side of the code. Right now, I need to open VS code every day, and run a function which will trigger the scrapers. Now I've researched about, and Azure has a function apps which has a scheduled function.
Now what I want is: I want just to call a file inside my Azure repo. My backend and frontend are in the different repos, and I want to run file scraping-service.js inside scraping folder in order to scrape data and insert the data into the db.
Now normally I run pipeline with azure-service.yml which has its own configuration for running my project. Is any way to implement this function to run just scraping-service.js at certain time of the day?
Thanks!
Is any way to implement this function to run just scraping-service.js at certain time of the day?
In Azure Pipeline, you could set schedules for pipelines.
Here is a doc about the detailed info: Configure schedules for pipelines.
For example: In yaml pipeline , you could set cron .
trigger:
- main
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: 'xxx'
# Web app name
webAppName: 'stanbackapp'
# Environment name
environmentName: 'stanbackapp'
# Agent VM image name
vmImageName: 'ubuntu-latest'
schedules:
- cron: "0 0 * * *"
displayName: Daily midnight build
branches:
include:
- main
stages:
- stage: Build
displayName: Build stage
condition: ne(variables['Build.Reason'], 'Schedule')
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm run build --if-present
npm run test --if-present
displayName: 'npm install, build and test'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Build1
displayName: Build1 stage
condition: eq(variables['Build.Reason'], 'Schedule')
jobs:
- job: Build1
displayName: Build1
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- task: CmdLine#2
inputs:
script: ' node $(build.sourcesdirectory)/scraping/scraping-service.js'
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: $(environmentName)
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: stanbackapp'
inputs:
azureSubscription: $(azureSubscription)
appType: webAppLinux
appName: $(webAppName)
runtimeStack: 'NODE|10.10'
package: $(Pipeline.Workspace)/drop/$(Build.BuildId).zip
startUpCommand: 'npm run dev'
In this case, when the pipeline is triggered by schedules, it will run Single file, or it will run the whole pipeline.
You can also add condition to task level.
The normal way is to create an azure devops pipeline that deploys the code you want to run to an azure function when the code changes. So put the code of scraping-service.js in the function or have the function call the method in scraping-service.js. See the docs
Although there might be ways to run the code in an Azure Devops pipeline I don't think it is meant to run application code. You won't have the monitoring capabilities an azure function gives you, nor the availability of scaling, configuration and all the thing Azure Functions provides.
I´m struggling to make my MultiStage pipelines to run a .exe file in a hosted agent running in a Azure VM.
My .yaml file is:
trigger:
- develop
stages:
- stage: build
displayName: Build
jobs:
- job: buildJob
pool:
vmImage: 'ubuntu-16.04'
variables:
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
inputs:
versionSpec: '5.5.0'
- task: DotNetCoreCLI#2
displayName: 'Dotnet Build $(buildConfiguration)'
inputs:
command: 'build'
arguments: '--configuration $(buildConfiguration)'
projects: '**/TestProj.csproj'
- task: DotNetCoreCLI#2
displayName: "Publish"
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/TestProj.csproj'
arguments: '--no-restore --configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: false
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: Container
- stage: Release
displayName: Release
dependsOn: build
jobs:
- deployment: AzureVMDeploy
displayName: agentDeploy
environment:
name: AzureDeploy
resourceName: vmName
resourceType: VirtualMachine
tags: develop
This VM is on the azure pipelines Environment. After I run this pipeline, the folder is downloaded into the VM, but I cannot find how to automate the execution of the output .exe file in this folder.
I think the way is to create a job with a task to do it, but I cannot figure out how to set the agent installed on the VM to run this task.
How can I do that?
If I understood you correctly, you want to execute your artifact file which was deployed to VM.
I think that PowerShell on Target Machines task should do the job for you. Yoy can write simple inline script to exeute your file. However, you need to have remoting confogured on VM. This article may help you with this.
You could specify tasks in strategy: Deployment job For example:
YAML
stages:
- stage: build
jobs:
- job: buildJob
pool:
vmImage: 'Ubuntu-16.04'
steps:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
publishLocation: 'pipeline'
- stage: deploy
dependsOn: build
jobs:
- deployment: DeployWeb
displayName: deploy Web App
environment:
name: vm1
resourceType: virtualmachine
strategy:
runOnce:
deploy:
steps:
- script: echo my first deployment
- task: CmdLine#2
inputs:
script: 'more README.md'
workingDirectory: '$(Pipeline.Workspace)/build.buildJob/s'
For this YAML pipeline, I publish all files in pipeline workspace to artifact in build stage, then this artifact will be download to target virtual machine of vm1 environment in deploy stage (folder name will be {stage name}.{job name}), then run command line task to get a file's content. (The script and command line tasks will be run on that virtual machine)