I am trying to create a pipeline in concourse, which is going to trigger on github updates on a remote branch, and use that branch to plan, apply and destroy a terraform deployment.
- name: terraform-repo
type: git
icon: github
source:
uri: https://github.com/....
#docker image
- name: terraform-0-13-7
type: registry-image
source:
repository: hashicorp/terraform
tag: 0.13.7
jobs:
- name: terraform-deplyoment
plan:
- get: terraform-0-13-7
- get: terraform-repo
trigger: true
- task: terraform-init
image: terraform-0-13-7
config:
inputs:
- name: terraform-repo
outputs:
- name: terraform-repo
platform: linux
run:
path: terraform
dir: terraform-repo
args:
- init
- task: terraform-plan
image: terraform-0-13-7
config:
inputs:
- name: terraform-repo
outputs:
- name: terraform-repo
platform: linux
run:
path: terraform
dir: terraform-repo
args:
- plan
params:
variable1: "test"
variable2: "test2"
This is erroring out on the concourse GUI when triggering the pipeline mentioning that the vars are not available. Am I doing something wrong with the syntax?
The params are exposed to the task as environment variables so you should use them as input variables
- task: terraform-plan
image: terraform-0-13-7
config:
inputs:
- name: terraform-repo
outputs:
- name: terraform-repo
platform: linux
run:
path: terraform
dir: terraform-repo
args:
- plan
params:
TF_VAR_variable1: "test"
TF_VAR_variable2: "test2"
Related
I have on my azure pipelines a libary varrible named:
deployHostMachine
and in it a variable named:
Host
I need to run a task using if with condition on Host How do i do it?
My code:
parameters:
trigger:
- none
variables:
- group: DeploymentHost
- name: localForHost
value: $(Host)
- name: testEmpty
value: ''
stages:
- stage: "ManageDeployMachines"
jobs:
- job: "ManageDeployMachines"
pool:
vmImage: 'windows-latest'
steps:
- ${{ if eq(parameters.paramHost,'rnd-3.sepio.systems') }}:
- task: ManageVariableGroupTask#0
displayName: 'Set deploy host url'
inputs:
pat: 'yxtloxscphyvuevrzafozok533orcv3pqon4a7e3kzpc4je23zha'
project: '70e3803b-2a31-4f00-83a1-9fc825f4447b'
vgname: 'DeploymentHost'
orgname: 'SepioDevOfficial'
op: 'Update'
varops: '= Host => "rnd-4.sepio.systems"'
Inside Github Action I'm using Anchore+grype to scan a container image, using the job below:
name: "CI"
on:
push:
pull_request:
branches:
- main
jobs:
image-analysis:
name: Analyze image
runs-on: ubuntu-18.04
needs: build
steps:
- name: Scan operator image
uses: anchore/scan-action#v3
id: scan
with:
image: "qserv/qserv-operator:2022.1.1-rc1"
acs-report-enable: true
In order to ignore a false-positive during image scan, I want to create the file $HOME/.grype.yaml (see content below) before launching the image scan:
ignore:
# False positive, see https://github.com/anchore/grype/issues/558
- vulnerability: CVE-2015-5237
fix-state: unknown
package:
name: google.golang.org/protobuf
version: v1.26.0
type: go-module
location: "/manager"
Could you please show me how to create this file inside Github Action?
you could do something as simple as creating the file and then writing to it like this:
- name: Create grype.yaml
run: |
touch grype.yaml
echo "
ignore:
# False positive, see https://github.com/anchore/grype/issues/558
- vulnerability: CVE-2015-5237
fix-state: unknown
package:
name: google.golang.org/protobuf
version: v1.26.0
type: go-module
location: "/manager"" > ~/grype.yaml
This one works and has been tested successfully on Github Actions:
name: "CI"
on:
push:
pull_request:
branches:
- main
jobs:
image-analysis:
name: Analyze image
runs-on: ubuntu-18.04
permissions:
security-events: write
needs: build
steps:
- name: Create grype configuration
run: |
cat <<EOF > $HOME/.grype.yaml
ignore:
# False positive, see https://github.com/anchore/grype/issues/558
- vulnerability: CVE-2015-5237
fix-state: unknown
package:
name: google.golang.org/protobuf
version: v1.26.0
type: go-module
location: "/manager"
EOF
- name: Scan operator image
uses: anchore/scan-action#v3
id: scan
with:
image: ""qserv/qserv-operator:2022.1.1-rc1""
acs-report-enable: true
fail-build: false
Need a small help , can someone point me what is the issue here .
What i am trying to achieve , the pipeline has to run checkout irrespective of the branch i am in , but in the build stage if the pipeline run from master it has execute some templet if from other branch different templet , i tried many option no luck .
Any pointer much appreciated .
- name: release
displayName: ReleaseVersion
type: string
default: ' '
- name: Deployment
displayName: DeploymentVersion
type: string
default: ' '
- name: Library
displayName: Library Release
type: boolean
default: True
trigger:
- none
pool:
name: DOTAzure-Ubuntu-20.04
resources:
repositories:
- repository: azureTemplates
type: githubenterprise
name: AAAAA/azure-pipelines
endpoint: BBBBBB
# You should add below variables to work complete pipeline
variables:
- name: servicename
value: XXXXX
- name: countryCode
value: YYYYY
- name: targetPort
value: 8113
stages:
- stage: Checkout
displayName: Micro Service checkout and Secrets
jobs:
- job:
steps:
- template: templates/repocheckout.yml#azureTemplates
- template: templates/retrieving_secrets.yml#azureTemplates
- template: templates/Ingesting_secrets.yml#azureTemplates
- stage: Build
displayName: Microservice Build
jobs:
- job: Master
condition: eq(variables['Build.SourceBranchName'], 'master')
steps:
- template: templates/mavenbuild_new.yml#azureTemplates
- template: templates/push_artifact_jfrog.yml#azureTemplates
parameters:
service_name: '$(countryCode)-$(servicename)'
release_version: $(release)
- job: Develop
dependsOn: Master
condition: eq(variables['Build.SourceBranchName'], 'develop')
steps:
- template: templates/mavenbuild.yml#azureTemplates
- template: templates/push_artifact_jfrog.yml#azureTemplates
parameters:
service_name: '$(countryCode)-$(servicename)'
release_version: $(release)
This doesn't look like valid yaml.
Have a look at this: https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema
It looks like you are missing this at the top of the file:
stages:
i.e. your yaml should be:
stages:
- stage: Checkout
displayName: Micro Service checkout and Secrets
jobs:
etc
I have template
dockerbuild.yml
steps:
- task: Bash#3
displayName: Build an image to container registry
inputs:
script : echo $(PATH)
then a variable file
var.build.yml
- name: PATH
value: 'docker/path'
- name: PATH1
value: 'docker/oldpath'
- name: PATH2
value: 'docker/newpath'
azurepipeline1.yml
resources:
repositories:
- repository: templates
type: git
name: components/pipeline_templates
trigger:
- none
pool:
name: PoolA
variables:
- template: variabletemplates/var.build.yml#templates
jobs:
steps:
- template: CI-CD/dockerbuild.yml#templates # Template reference
parameters:
PATH: ${{ variables.PATH }}
- template: CI-CD/dockerbuild.yml#templates # Template reference
parameters:
PATH: ${{ variables.PATH1 }}
- template: CI-CD/dockerbuild.yml#templates # Template reference
parameters:
PATH: ${{ variables.PATH2 }}
The above code works
But if I change the parameter to
parameters:
PATH: ${{ variables.PATH1 }}
OR
parameters:
PATH: ${{ variables.PATH2 }}
In the output it shows
Output:
docker/path
$(PATH1)
$(PATH2)
I need to reuse the template with many times within the pipeline with different paths. Please help in resolving the issue
Your template needs a parameters block.
parameters:
- name: PATH
type: string
steps:
- task: Bash#3
displayName: Build an image to container registry
inputs:
script : echo ${{ parameters.PATH }}
I have a Azure DevOps YAML Pipeline to execute a Terraform deployment using the Terraform by MS DevLabs extension and an Azure Resource Manager service connection.
The last working state was using a pipeline template yaml file however I had to configure a parameter within the template and call the variable using the template expression syntax.
...
...
stages:
- stage: Plan
displayName: Terrafom Plan
jobs:
- job: DEV PLAN
displayName: Plan (DEV)
pool:
vmImage: "ubuntu-latest"
variables:
az_service_connection: "MyServiceConnection"
tf_environment: "DEV"
tf_state_rg: "DEV"
tz_state_location: "canadacentral"
tf_state_stgacct_name: "mystorageaccuontname1231231"
tf_state_container_name: "tfstate"
steps:
- template: templates/terraform-plan.yml
parameters:
az_service_connection: ${{ variables.az_service_connection }}
...
...
steps:
- task: terraformInstaller#0
displayName: "Install Terraform $(tf_version)"
inputs:
terraformVersion: $(tf_version)
- task: TerraformTaskV1#0
displayName: "Run > terraform init"
inputs:
command: "init"
commandOptions: "-input=false"
backendServiceArm: ${{ parameters.az_service_connection }}
...
...
I believe the reason why this works is because the template expression syntax ${{ variables.varname}} evaluates at compile time vs. runtime. If I didn't do it this way, i'd either get $(az_service_connection) passed into the backendServiceArm input or an empty value.
With the introduction of variable groups, i'm now facing similar behavior. I expect that the variable group evaluates after the template expression variable which causes ${{ variables.az_service_connection }} to have an empty value. I am unsure how to get this working.
How can I use variable groups with a pipeline template that uses a service connection?
I used $() syntax to pass arm connection to template:
Template file:
parameters:
- name: 'instances'
type: object
default: {}
- name: 'server'
type: string
default: ''
- name: 'armConnection'
type: string
default: ''
steps:
- task: TerraformTaskV1#0
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '${{ parameters.armConnection }}'
backendAzureRmResourceGroupName: 'TheCodeManual'
backendAzureRmStorageAccountName: 'thecodemanual'
backendAzureRmContainerName: 'infra'
backendAzureRmKey: 'some-terrform'
- ${{ each instance in parameters.instances }}:
- script: echo ${{ parameters.server }}:${{ instance }}
Main file:
trigger:
branches:
include:
- master
paths:
include:
- stackoverflow/09-array-parameter-for-template/*
# no PR triggers
pr: none
pool:
vmImage: 'ubuntu-latest'
variables:
- group: my-variable-group
- name: my-passed-variable
value: $[variables.myhello] # uses runtime expression
steps:
- template: template.yaml
parameters:
instances:
- test1
- test2
server: $(myhello)
armConnection: $(armConnection)
Note: Group my-variable-group contains armConnection variable