creating ARM template for storage devops - azure

I try to create a storage account via a devops pipeline.
So I have this yaml file:
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
pool:
vmImage: ubuntu-latest
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: 'spn-azure--contributor-002'
subscriptionId: 'fea4c865-1e54-44b3-ba1d-07315468f083'
action: 'Create Or Update Resource Group'
resourceGroupName: 'rg-idn-'
location: 'West Europe'
templateLocation: 'Linked artifact'
csmFile: '**/template.json'
csmParametersFile: '**/parameters.json'
deploymentMode: 'Incremental'
- task: AzureResourceManagerTemplateDeployment#3
inputs:
azureResourceManagerConnection: 'spn-azure--contributor-002'
subscriptionId: 'fea4c865-1e54-44b3-ba1d-07315468f083'
resourceGroupName: 'rg-idn-'
location: 'West Europe'
csmFile: ARMTemplates/storage/azuredeploy.json
csmParametersFile: ARMTemplates/storage/azuredeploy.parameters.json

It looks like you have an Azure Policy blocking your deployment here - meaning that your ARM template does not meet the rules of this policy. Someone from your organization most probably implemented this one to make sure that you are complying to specific security / architecture standards.
You can see it from the error message : Error Type: PolicyViolation, Policy Definition Name : ESLZ Storage Account set to minimum TLS and Secure transfer should be enabled, Policy Assignment Name : ALZ_DeployEncrTLS.. You should be able to retrieve it under your Subscription / Resource Group, in the Policy blade.
Basically, and this is an assumption as we would need to look into that specific policy, but you most likely have to specify an TLS version here (probably 1.2, to be checked in the policy) and enable secure transfer.
For this you need to set minimumTlsVersion to the correct version and supportsHttpsTrafficOnly to true within your ARM template. You can have a look at Storage Account ARM specs.

Related

AzureResourceManagerTemplateDeployment fail in Azure DevOps pipeline

I have the following task that runs an Arm Template that is tested and working in another system for creating a Management group in Azure. Everything is checked and it should work, but I keep giving this error:
##[error]Check out the troubleshooting guide to see if your issue is addressed: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
##[error]Error: Could not find any file matching the template file pattern
What I am doing wrong?
parameters:
- name: ParentId
type: string
- name: NameId
type: string
- name: DisplayName
type: string
steps:
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: "Management Group"
azureResourceManagerConnection: ServiceConnectionName
location: "West Europe"
csmFile: "$(System.DefaultWorkingDirectory)/**/ManagementGroup.json"
csmParametersFile: "$(System.DefaultWorkingDirectory)/**/ManagementGroup.parameters.json"
overrideParameters: "-NameId ${{ parameters.NameId }} -DisplayName ${{ parameters.DisplayName }} -ParentId ${{ parameters.ParentId }}"
deploymentMode: "Incremental"
deploymentOutputs: armOutputs
I have had recently similar issue. I have noticed you are using capital letters for naming your arm template. As far as I know it should be small letters. So to solve this particular issue, you need to correct following lines:
csmFile: "$(System.DefaultWorkingDirectory)/**/ManagementGroup.json"
csmParametersFile: "$(System.DefaultWorkingDirectory)/**/ManagementGroup.parameters.json"
to
csmFile: "$(System.DefaultWorkingDirectory)/**/managementgroup.json"
csmParametersFile: "$(System.DefaultWorkingDirectory)/**/managementmroup.parameters.json"
And make your files with small letters in your repository.
That should solve the problem.

Deploy packages to multiple webapp service by azure pipeline single stage

I have more than 100 webapp service in azure. I want to deploy packages in 100 webapps by azure pipeline with one pipeline yml file. But I couldn't find any documentation like this. I got one microsoft documentation and they prefer to increase pipeline steps. If I have 100 webapps service, then have to add 100 steps for each deployment. This is not an efficient way and its time consuming. I want just like this step.
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy'
inputs:
azureSubscription: '$(Parameters.connectedServiceName)'
appType: webApp
ResourceGroupName: $(group)
appName: 'JustGoTestAgain, justgotesttwo, webapp123, webapp555, webapp777 and so on ........'
package: '$(build.artifactstagingdirectory)/**/*.zip'
This yaml file is showing error. I couldn't find any essential extensions to fix it. I also couldn't find any azure powershell deployment command regarding to this issue. How can I get the solution?
You will not be able to do this like this. However you can use Azure Cli task:
- task: AzureCLI#2
displayName: Azure CLI
inputs:
azureSubscription: '$(Parameters.connectedServiceName)'
scriptType: ps
scriptLocation: inlineScript
inlineScript: |
$apps= #('JustGoTestAgain, justgotesttwo, webapp123, webapp555, webapp777 and so on ........')
foreach ($app in $apps) {
az webapp deployment source config-zip -g $(group) -n $app --src '$(build.artifactstagingdirectory)/SOME_FOLDER/Artifact.zip'
}
And here you have more details about deployment itself
Annother approach with multiple task bu continuation if one fail is:
parameters:
- name: apps
type: object
default:
- JustGoTestAgain
- justgotesttwo
- and so on
steps:
- ${{ each app in parameters.apps}}:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy ${{ app }}'
continueOnError: true
inputs:
azureSubscription: '$(Parameters.connectedServiceName)'
appType: webApp
ResourceGroupName: $(group)
appName: ${{ app }}
package: '$(build.artifactstagingdirectory)/**/*.zip'
Thete was issue with space. Now is fine. Apart from that there is only one issue with connectedServiceName
Job Job: Step input azureSubscription references service connection $(Parameters.connectedServiceName) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz. Job Job: Step input azureSubscription references service connection $(Parameters.connectedServiceName) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz. Job Job: Step input azureSubscription references service connection $(Parameters.connectedServiceName) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.
Which I skipped here as you already have it on your solution.

AzureResourceManagerTemplateDeployment fails to find template using pattern when executed in Deployment Job

I have been experimenting with Azure Logic Apps and wanted to figure out a way to codify the deployment process so that I could setup a CI/CD pipeline with secrets and all the good stuff.
So I set out with a yml file with multiple ways to deploy the same Logic App.
Hardcoding the values of the input params to the task like Connected Service, Subscription, Resource Group etc. in a step inside a regular job.
Doing the same thing but inside a Deployment job.
Use Pipeline variables to extract these values and repeat as 1 and 2.
1 and 2 again but this time using Pipeline Variables that are marked as Secrets
so on and so forth.
However, everytime I run the AzureResourceManagerTemplateDeployment#3 inside a deployment job, it fails to find the ARM template file.
Why is the deployment job unable to find the ARM Template using the pattern that works when the it is not run as a deployment job?
Do deployment jobs not have access to the build directory?
How do I help the deployment job to find the file? Should I be giving it a link to the template file instead of a pattern?
Everytime I search for the AzureResourceManagerTemplateDeployment task docs, I get the docs page of AzureResourceGroupDeployment task which is very similar but not the same
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
As I was about to post this question, I did more searching online and came across the original docs of the AzureResourceManagerTemplateDeployment which states that if the file is part of a repository then one must specify the path to the ARM template using the help of system variables.
csmFile: "$(Build.Repository.LocalPath)/**/LogicApp.json"
csmParametersFile: "$(Build.Repository.LocalPath)/**/LogicApp.parameters.json"
I can confirm that this did not work either.
What could I be missing?
stages:
- stage: 'HardcodedJobStage'
displayName: 'HardcodedJobStage'
jobs:
- job: 'HardcodedJob'
displayName: HardcodedJob
pool:
vmImage: ubuntu-latest
workspace:
clean: all
steps:
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
ConnectedServiceName: 'Subscription (e6d1dg8c-bcd6-4713-b2f1-c9a0375d687d)'
subscriptionName: 'e6d1dg8c-bcd6-4713-b2f1-c9a0375d687d'
action: 'Create Or Update Resource Group'
resourceGroupName: 'AzureLogicApp'
location: 'UK South'
templateLocation: 'Linked artifact'
csmFile: '**/LogicApp.json'
csmParametersFile: '**/LogicApp.parameters.json'
deploymentMode: 'Incremental'
- stage: 'HardCodedDeployJobStage'
displayName: 'HardCodedDeployJobStage'
jobs:
- deployment: 'HardCodedDeployJob'
displayName: HardCodedDeployJob
pool:
vmImage: ubuntu-latest
workspace:
clean: all
environment: development
strategy:
runOnce:
deploy:
steps:
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
ConnectedServiceName: 'Subscription (e6d1dg8c-bcd6-4713-b2f1-c9a0375d687d)'
subscriptionName: 'e6d1dg8c-bcd6-4713-b2f1-c9a0375d687d'
action: 'Create Or Update Resource Group'
resourceGroupName: 'AzureLogicApp'
location: 'UK South'
templateLocation: 'Linked artifact'
csmFile: '**/LogicApp.json'
csmParametersFile: '**/LogicApp.parameters.json'
deploymentMode: 'Incremental'
The problem here was that I had to publish the templates as artifacts and share it between the stages.
So I copied the ARM template json files to a folder using CopyFiles task and then used the PublishPipelineArtifact task to publish the contents as a pipeline artifact. This can then be referenced later by the following stage by using the DownloadPipelineArtifact task.
Now my YAML looks something like:
stages:
- stage: 'HardcodedJobStage'
displayName: 'HardcodedJobStage'
jobs:
- job: 'HardcodedJob'
displayName: HardcodedJob
pool:
vmImage: ubuntu-latest
workspace:
clean: all
steps:
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
ConnectedServiceName: 'Subscription (e6d1dg8c-bcd6-4713-b2f1-c9a0375d687d)'
subscriptionName: 'e6d1dg8c-bcd6-4713-b2f1-c9a0375d687d'
action: 'Create Or Update Resource Group'
resourceGroupName: 'AzureLogicApp'
location: 'UK South'
templateLocation: 'Linked artifact'
csmFile: '**/LogicApp.json'
csmParametersFile: '**/LogicApp.parameters.json'
deploymentMode: 'Incremental'
- task: CopyFiles#2
inputs:
Contents: $(Build.SourcesDirectory)/AzureLogicApp/**/*.json
targetFolder: $(Build.ArtifactStagingDirectory)
- task: PublishPipelineArtifact#1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
artifactName: armtemplate
- stage: 'HardCodedDeployJobStage'
displayName: 'HardCodedDeployJobStage'
jobs:
- deployment: 'HardCodedDeployJob'
displayName: HardCodedDeployJob
pool:
vmImage: ubuntu-latest
workspace:
clean: all
environment: development
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact#2
inputs:
artifact: armtemplate
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
ConnectedServiceName: 'Subscription (e6d1dg8c-bcd6-4713-b2f1-c9a0375d687d)'
subscriptionName: 'e6d1dg8c-bcd6-4713-b2f1-c9a0375d687d'
action: 'Create Or Update Resource Group'
resourceGroupName: 'AzureLogicApp'
location: 'UK South'
templateLocation: 'Linked artifact'
csmFile: $(Pipeline.Workspace)/armtemplate/**/LogicApp.json
csmParametersFile: $(Pipeline.Workspace)/armtemplate/**/LogicApp.parameters.json
deploymentMode: 'Incremental'
Assuming your yml pipeline is defined in the same git repository as the LogicApp json files, you could use csmFile as an absolute path with the 'root' being the git repo root folder. For example, if your logicapp files are in /app/logicapp/LogicApp.json, and your yml pipeline is in the same git repo but maybe in /pipelines/pipeline.yml, you could then set csmFile's value to app/logicapp/LogicApp.json (and same for csmParametersFile).

Referencing Azure Key Vault secrets from CI/CD YAML

We have a multi-stage YAML pipeline that does CI/CD to an existing set of Azure Resources
The stages are
Build
Deploy to Development and Run Tests
If Previous succeeded - Deploy to Production and Run Tests
We use the AzureRmWebAppDeployment task during the deployment stages and we use the AppSettings argument to that task to specify environment-specific settings. For example
- task: AzureRmWebAppDeployment#4
displayName: 'Deploy Azure App Service'
inputs:
azureSubscription: '$(azureSubscriptionEndpoint)'
appType: '$(webAppKind)'
WebAppName: 'EXISTING__AZURE_RESOURCENAME-DEV'
Package: '$(Pipeline.Workspace)/**/*.zip'
AppSettings: >
-AzureAd:CallbackPath /signin-oidc
-AzureAd:ClientId [GUID was here]
-AzureAd:Domain [domain was here]
-AzureAd:Instance https://login.microsoftonline.com/
-AzureAd:TenantId [Id was here]
-EmailServer:SMTPPassword SECRETPASSWORD
-EmailServer:SMTPUsername SECRETUSERNAME
There are two settings in that set, EmailServer: SMTPUsername and EmailServer: SMTPPassword that I want to pull from an Azure KeyVault. I know how to reference the KV secret from Azure Portal using the syntax
#Microsoft.KeyVault(SecretUri=https://our.vault.azure.net/secrets/SendGridPassword/ReferenceGuidHere)
but how do I reference the value from the YAML pipeline so it is set in Azure?
As pointed out by Thomas in this comment, Referencing Azure Key Vault secrets from CI/CD YAML
I can explicitly set the value in the YAML file like this:
-EmailServer:SMTPPassword #Microsoft.KeyVault(SecretUri=https://our.vault.azure.net/secrets/SendGridPassword/ReferenceGuidHere)
You need to set an AzureKeyVault#1 task with RunAsPreJob to true, this will make your key vault values available as CI/CD jobs environment variables so you can use it as $(KEY-OF-SECRET-VALUE) on the rest of your stages in the job.
The following piece of yaml file is a working example.
We set for python unittest a set of env variable provided from Azure key-vault
trigger:
batch: true # disable concurrent build for pipeline
branches:
include:
- '*' # CI start for all branches
pool:
vmImage: ubuntu-16.04
stages:
- stage: Test
jobs:
- job: sample_test_stage
steps:
- task: AzureKeyVault#1
inputs:
azureSubscription: 'YOUR SUBSCRIPTION HERE'
KeyVaultName: 'THE-KEY-VAULT-NAME'
SecretsFilter: '*'
RunAsPreJob: true
- task: UsePythonVersion#0
inputs:
versionSpec: '3.7'
- script : python -m unittest discover -v -s tests
displayName: 'Execute python unittest'
env: { MY-ENV-VAL-1: $(SECRET-VALUE-1), MY-ENV-VAL-2: $(SECRET-VALUE-2)}
Note that sometimes you need to approve connection beetween AzureDevops and another Azure service like KeyVault

Azure Devops YAML multi staged pipeline artifact issue

My Azure Devops YAML multi staged pipeline is giving me an error when doing an ARM deployment. The issue is downloading the artifact from the build. See the error here:
Error
It looks like the artifact is not being downloaded, in the jobs before they are being downloaded. The difference is that the production deployment needs to be approved and therefore it is inside a -deployment and not in a -job.
See the code here:
- stage: Deploy_Prod
dependsOn: Deploy_Acc
# Only deploy when build is from master
condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/master'))
jobs:
- deployment: 'Deploy_Prod'
pool:
vmImage: 'ubuntu-latest'
# Set envrironment for approval, see https://dev.azure.com/dept/DTNL%20-%20CBRE/_environments/5?view=resources
environment: cbre_prod
strategy:
runOnce:
deploy:
steps:
# Download build artifact
- download: current
artifact: Templates
# Deploy production infra
- task: AzureResourceManagerTemplateDeployment#3
displayName: 'Deploy production infrastructure'
inputs:
deploymentScope: 'Resource Group'
ConnectedServiceName: '***'
subscriptionName: '***'
action: 'Create Or Update Resource Group'
resourceGroupName: '***'
location: 'West Europe'
templateLocation: 'Linked artifact'
csmFile: 'azuredeploy.json'
csmParametersFile: 'azuredeploy-parameters-prod.json'
deploymentMode: 'incremental'
Does anybody know how I can download the artifact from a multi staged pipeline, while using -deployment instead of -job?
Working version with job, just for reference:
- stage: Deploy_Acc
dependsOn: Deploy_Test
# Only deploy when build is from master
condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/master'))
jobs:
- job: 'Deploy_Acc'
pool:
vmImage: 'ubuntu-latest'
steps:
# Download build artifact
- download: current
artifact: Templates
# Deploy acceptation infra
- task: AzureResourceManagerTemplateDeployment#3
displayName: 'Deploy acceptation infrastructure'
inputs:
deploymentScope: 'Resource Group'
ConnectedServiceName: '***'
subscriptionName: '***'
action: 'Create Or Update Resource Group'
resourceGroupName: '***-acc'
location: 'West Europe'
templateLocation: 'Linked artifact'
csmFile: 'azuredeploy.json'
csmParametersFile: 'azuredeploy-parameters-acc.json'
deploymentMode: 'incremental'
The documentation on deployment jobs suggests that you may not need to instruct it to download the artifact, that it happens automatically?
deploy: Used to run steps that deploy your application. Download artifact task will be auto injected only in the deploy hook for deployment jobs. To stop downloading artifacts, use - download: none or choose specific artifacts to download by specifying Download Pipeline Artifact task.
I don't see anything wrong with your code. In your artifact creation, have you used "Publish Pipeline Artifact"?

Resources