I am trying to set the deployment variable for a step using a variable that is passed to a custom pipeline. The idea is not to have to duplicate the custom pipeline as the only change is the deployment variables that are read from bitbucket settings.
The definition looks as follows, but throws an error
pipelines:
custom:
my-pipeline:
- variables:
- name: deployment
- step:
deployment: $deployment
script:
- ...
Am I missing something here, or is the deployment key not allows to accept a variable?
Unfortunetly you can't use variables in the deployment field. Variables are only available in script fiels. However, your problem is easily solved in another way: anchors
For example:
definitions:
steps:
- step: &Test-step
name: Run tests
script:
- npm install
- npm run test
- step: &Deploy-step
name: Deploy to staging
script:
- npm install
- npm run build
- fab deploy
pipelines:
default:
- step: *Test-step
- step:
<<: *Deploy-step
name: Deploy to Staging
deployment: staging
trigger: manual
custom:
Staging Deployment
- step: *Test-step
- step:
<<: *Deploy-step
deployment: staging
Production Deployment
- step: *Test-step
- step:
<<: *Deploy-step
deployment: production
Related
I'm currently trying to setup my complete build/release pipeline with yaml files.
First I tried with different stages (dev/staging/prod) and it worked.
Now I wanted to add an approval that the deploy doesn't not happen automatically on each system.
Therefore I added an environment in the TFS with an approval check.
But when I try to setup the yaml file I always get an error.
I don't know how to setup this properly.
This is my main yaml file called release-pipeline.yaml
trigger:
- master
pool:
name: POOL
stages:
- stage: BuildSolution
jobs:
- job: BuildSolutionJob
displayName: Build
workspace:
clean: all
steps:
- template: yaml/BuildSolution.yml
- template: yaml/CopyFiles.yml
- template: yaml/PublishArtifact.yml
- stage: DeployOn_STAGING_System
dependsOn: BuildSolution
jobs:
- job: Deploy_STAGING
- template: yaml/Deploy.yml
parameters:
Environment: 'SITE'
Staging: 1
- stage: Deploy_DEV_System
dependsOn: BuildSolution
jobs:
- deployment: Deploy_DEV
environment: ENVCHECK_DEV
strategy:
runOnce:
deploy:
steps:
- template: yaml/Deploy.yml
parameters:
Environment: 'SITE'
ViewDeploy: 1
This is my Deploy.yml file which i want to execute (only some snips):
parameters:
- name: Environment
type: string
- name: ProdSystem
type: number
default: 0
- name: Staging
type: number
default: 0
- name: ViewDeploy
type: number
default: 0
jobs:
- job:
variables:
artifactName: $[stageDependencies.BuildSolution.BuildSolutionJob.outputs['SetVariables.artifactName']]
version: $[stageDependencies.BuildSolution.BuildSolutionJob.outputs['SetVariables.version']]
steps:
- task: PowerShell#2
displayName: Display given parameters
inputs:
targetType: inline
script: >
Write-Host "ArtifaceName: " $(artifactName)
Write-Host "Environment for Deploy: " ${{ parameters.Environment }}
Write-Host "##vso[task.setvariable variable=isStaging]${{ parameters.Staging }}"
failOnStderr: true
When I try to execute I get the following error:
/release-pipeline.yml: Unexpected value 'parameters'.
How do I need to change this that it will work with the template in both cases, with and without the environment approval check?
I tried https://samlearnsazure.blog/2020/02/05/approvals-in-environments/
and of course different structure for the calling. But nothing helped.
In the tutorials they always have steps below the "deploy" but because I have different sites and environments I want/need the template file to save work.
I found another post which goes in the same direction.
I reworked my complete template so that I can use the approach of this: DevOps template with conditional manual approval job
It's not the same as I wanted, but it works.
My goal was that I don't want to create a environment if I have no checks on this site. Only for sites where I wanted the approval check.
With the above mentioned solution I need to create an environment for each site, independent if I have checks or not.
I could successfully deploy my project into the production environment using the provided documentation https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/static-web-apps/bitbucket.md
pipelines:
branches:
main:
- step:
name: Deploy to test
deployment: test
script:
- pipe: microsoft/azure-static-web-apps-deploy:main
variables:
APP_LOCATION: '$BITBUCKET_CLONE_DIR'
OUTPUT_LOCATION: '$BITBUCKET_CLONE_DIR/dist'
API_TOKEN: $deployment_token​
But there are no information about how to deploy to other environments than production, e.g: staging, qa, release ...
With Azure pipeline, the value can be set with the deployment_environment parameter.
Does anyone have a solution for it?
If you are using azure static web apps with multiples slots like prod,staging/dev then you have to specify DEPLOYMENT_ENVIRONMENT in your bitbucket-pipelines.yml file
Example :
pipelines:
branches:
develop:
- step:
name: Deploy to staging
deployment: staging
script:
- pipe: microsoft/azure-static-web-apps-deploy:main
variables:
APP_LOCATION: '$BITBUCKET_CLONE_DIR/dist'
API_TOKEN: $deployment_token
DEPLOYMENT_ENVIRONMENT:Staging
master:
- step:
name: Deploy to production
deployment: production
script:
- pipe: microsoft/azure-static-web-apps-deploy:main
variables:
APP_LOCATION: '$BITBUCKET_CLONE_DIR/dist'
API_TOKEN: $deployment_token
DEPLOYMENT_ENVIRONMENT:Prod
If you are using the multiple Static Web App resources then you can follow the answer provided by David Torres, you have to create tokens for each web and mention them in variables.
Thanks & Regards
Just got this working earlier today. I ran into the same issue, looks like they only support Staging environments when using Github or Azure Pipelines. I was able to get this to work by creating multiple Static Web App resources in Azure Portal. Prod gets the "paid" version, develop branch gets the "free" version.
Here is what the pipelines YAML looks like:
pipelines:
branches:
develop:
- step:
name: Deploy to staging
deployment: staging
script:
- pipe: microsoft/azure-static-web-apps-deploy:main
variables:
APP_LOCATION: '$BITBUCKET_CLONE_DIR/dist'
API_TOKEN: $deployment_token_beta
master:
- step:
name: Deploy to production
deployment: production
script:
- pipe: microsoft/azure-static-web-apps-deploy:main
variables:
APP_LOCATION: '$BITBUCKET_CLONE_DIR/dist'
API_TOKEN: $deployment_token
Then I added a new deployment_token_beta to the repo deployment variables using the key from the second web app
azure-pipeline.yml
trigger:
- master
parameters:
- name: config
displayName: Execution Environment
type: string
default: QA
values:
- QA
- PreProd
- Prod
pool:
vmImage: 'windows-latest'
The above works perfectly, so in Azure the Execution Environment parameter is shown when I run the pipeline.
If however I attempt to put the parameters in a template as follows:
azure-pipeline.yml
trigger:
- master
extends:
template: parameters.yml
pool:
vmImage: 'windows-latest'
parameters.xml
parameters:
- name: config
displayName: Execution Environment
type: string
default: QA
values:
- QA
- PreProd
- Prod
Then when I run the pipeline the parameter is not shown.
In summary I'm trying to re-use a parameters.yml in different pipelines but extends: template: does not seem to work even though per this link it should:
https://learn.microsoft.com/en-us/azure/devops/pipelines/security/templates?view=azure-devops#set-required-templates
Runtime parameters are something different than templates parameters and having the second in your pipeline will not cause them to show on the UI. There is no way to template runtime parameters. You need to repeat them in each pipeline you expect to have them.
I have a simple pipeline config:
image: python:3.7.3
pipelines:
branches:
Server:
- step:
name: Test
script:
- pytest --ignore .
which yields the following error:
We didn't find the deployment keyword in your bitbucket-pipelines.yml file
What should i do?
I just figured out, that i did not have "bitbucket-pipelines" enabled in the repository in the settings.
Here are examples to run python with bitbucket pipelines.
You can change unit test according to yours.
First make sure pipelines are enabled.
Generate SSH.
Add host key.
Add variable to Deployment menu.
I'm also attach running pipelines with tags and branch.
Generate SSH from :
https://bitbucket.org/<WORKSPACE>/<REPOSITORY_NAME>/admin/addon/admin/pipelines/ssh-keys
and put it to your server ~/.ssh/authorized_keys
definitions:
steps:
# Build
- step: &build
name: Install and Test
image: python:3.7.2
trigger: automatic
script:
- pip install -r requirements.txt
- python3 test.py test
# Deployment
- step: &deploy
name: Deploy Artifacts
trigger: automatic
deployment: test
script:
# Deploy New Artifact
- pipe: atlassian/scp-deploy:0.3.11
variables:
USER: <REMOTE_USER>
SERVER: <REMOTE_HOST>
REMOTE_PATH: <REMOTE_PATH>
LOCAL_PATH: $BITBUCKET_CLONE_DIR/**
# Runner
pipelines:
# Running by tags
tags:
v*:
- step: *build
- step:
<<: *deploy
deployment: test
trigger: manual
# Running by branch
branches:
master:
- step: *build
- step:
<<: *deploy
deployment: test
trigger: automatic
I read the environments documentation here and the issues opened under the environment resource, however I find it impossible to achieve my goal:
I would like to use a parametrized yaml template in order to deploy to multiple environments like below:
parameters:
pool_name: ''
aks_namespace: ''
environment: ''
jobs:
- job: preDeploy
displayName: preDeploy
pool:
name: $(pool_name)
steps:
- template: cd_step_prerequisites.yml
- deployment: Deploy
displayName: Deploy
dependsOn: preDeploy
condition: succeeded()
variables:
secret_name: acrecret
pool:
name: dockerAgents
**environment: '$(environment).$(aks_namespace)'**
strategy:
runOnce:
deploy:
steps:
- template: cd_step_aks_deploy.yml
- job: postDeploy
displayName: postDeploy
dependsOn: Deploy
condition: succeeded()
pool:
name: $(pool_name)
steps:
- template: cd_step_postrequisites.yml
I would like to use this approach so that I only host a minimal pipeline.yml next to my code, and then I would have all the templates in a different repo and call them from the main pipeline, as such:
resources:
repositories:
- repository: self
- repository: devops
type: git
name: devops
- stage: CD1
displayName: Deploy to Alpha
jobs:
**- template: pipeline/cd_job_api.yml#devops**
parameters:
pool_name: $(pool_name)
aks_namespace: $(aks_namespace)
app_name: $(app_name)
app_image_full_name: $(app_image_full_name)
environment: alpha
Then I would be able to pass the $environment variable in order to manipulate multiple deployment targets (AKS clusters/ groups of namespaces) from one template.
Currently this seems to be impossible as the default AzureDevOps parser fails when I try to run my pipeline, with the message "$(environment) environment does not contain x namespace" which tells me that the variable doesn't get expanded.
Is this planning to be implemented anytime soon? If not, are there any alternatives to use only one parametrized job template to deploy to multiple environments?
I think you would need to either parse the files and do a token replace with a script or there should be steps for that.
Your main alternative would be helm. It allows to create templates and pass in variables to render those templates.
Maybe I'm a bit late to the party, but I was also struggling with this problem and found this open thread.
I found this "closed" issue on github. The key points for me in the issue are this comment with a partial solution and this other comment pointing to the explanation of why is not working. Quoting Microsoft's article:
It also answers another common issue: why can't I use variables to resolve service connection / environment names? Resources are authorized before a stage can start running, so stage- and job-level variables aren't available. Pipeline-level variables can be used, but only those explicitly included in the pipeline. Variable groups are themselves a resource subject to authorization, so their data is likewise not available when checking resource authorization.
Regarding the solution, based on the first comment I reference, I ended up creating a new Variable Group with variables with the following naming convention: product.environment.varname. Then I added this group to the beginning of the pipeline (global scope) and then referenced the variables using macro syntax: $(var)
Quick example:
variables:
- group: Product.Pipelines.Environments
jobs:
- job: preDeploy
displayName: preDeploy
pool:
name: $(pool_name)
steps:
- template: cd_step_prerequisites.yml
- deployment: Deploy
displayName: Deploy
dependsOn: preDeploy
condition: succeeded()
variables:
secret_name: acrecret
pool:
name: dockerAgents
environment: $(product.dev.environmentname) #this is the variable within the variable group
strategy:
runOnce:
deploy:
steps:
- template: cd_step_aks_deploy.yml
The variable group will contain among other variables:
product.dev.environmentname: Development
product.stg.environmentname: Staging
product.prd.environmentname: Production