How to pass KeyVault secrets to a template or a script file in Azure Pipelines? - azure

I have this YAML file:
steps:
- task: AzureKeyVault#2
displayName: Get secret from AzureVault
inputs:
azureSubscription: 'subName'
KeyVaultName: 'vaultName'
SecretsFilter: 'mySecret'
RunAsPreJob: true
- template: \pipelines\templates\vm_setup.yml
parameters:
os_pass: $(mySecret)
How do I use mySecret inside myScript.ps1 or inside myTemplate.yml?
I tried to pass it as an argument, or map it to an env variable then pass that env variable as an argument but neither worked!
My myTemplate.yml looks like this:
parameters:
- name: os_pass
type: string
steps:
- task: PowerShell#2
displayName: Trial
inputs:
targetType: 'filepath'
filePath: '${{ parameters.workingDirectory }}\myScript.ps1'
arguments: >-
- OS_Pass ${{ parameters.os_pass }}
And this is myScript.ps1
param (
[Parameter(Mandatory = $true)]
[string]$OS_Pass
)
$password = ConvertTo-SecureString -String $OS_Pass -AsPlainText -Force
$cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList 'admin', $password
Write-Host '---------'
Write-Host $OS_Pass
Doing so the secret is now a string! How do I pass it without changing its type?

Take a look at the official documentation here: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-secret-variables?view=azure-devops&tabs=yaml%2Cbash.
There are a few different ways to accomplish what you want to do. Without knowing exactly what you want to do, it is hard to recommend a specific method to use. Take a look above and let us know if that resolves your issue.

Related

Azure SQL Server CICD Bicep and Key Vault

I have built a CICD Pipeline to deploy an Azure SQL Server and DB and part of this process is to obtain the username and password from secrets in a Key Vault. The Key Vault task in YAML works and can access the Vault, but when it calls an Azure CLI Task to execute the Bicep, it fails with the following error:
ERROR: Unable to parse parameter: **
My code:
steps:
- task: AzureKeyVault#2
displayName: 'Download Key Vault Secrets'
inputs:
connectedServiceName: ${{ parameters.AzureResourceManagerConnection }}
keyVaultName: ${{ parameters.keyVaultName }}
secretsFilter: '*'
- task: AzureCLI#2
displayName: '${{ parameters.deploymentType }}: ${{ parameters.targetEnvironment }} ${{ parameters.product }}'
inputs:
azureSubscription: ${{ parameters.AzureResourceManagerConnection }}
scriptType: 'ps'
scriptLocation: 'inlineScript'
inlineScript: |
$tags = "${{ parameters.releaseFolder }}/${{ parameters.targetEnvironment }}/tags.json"
$products = "${{ parameters.releaseFolder }}/${{ parameters.targetEnvironment }}/products.json"
$productDeploymentFile = "${{ parameters.releaseFolder }}/SQLServer.bicep"
az --version
az deployment group ${{ parameters.deploymentType }} --name ${{ parameters.targetEnvironment }}-${{ parameters.product }}-products-deployment --resource-group ${{ parameters.resourceGroup }} --template-file $productDeploymentFile --parameters $tags $products "$(sql-admin-username)" "$(sql-admin-password)" --mode ${{ parameters.deploymentMode }}
And the Bicep file:
param tags object
param sqlServers array
param sqlDatbases array
param sqlAdminUserName string
#secure()
param sqlAdminPassword string
resource sqlServer 'Microsoft.Sql/servers#2021-08-01-preview' = [for sql in sqlServers: {
name: sql.Name
location: sql.location
properties: {
administratorLogin: sqlAdminUserName
administratorLoginPassword: sqlAdminPassword
administrators: {
azureADOnlyAuthentication: false
administratorType: sql.administratorType
principalType: sql.principalType
login: sql.login
sid: sql.sid
tenantId: sql.tenantId
}
}
tags: tags
}]
Is there anything obvious that is incorrect?
Looking at your yaml file, the way you are passing inline parameters is not correct (see documentation), you would need to specify the parameter names as well:
az deployment group `
...
--parameters `
sqlAdminUserName="$(sql-admin-username)" `
sqlAdminPassword="$(sql-admin-password)" `
...

Only SPN credential auth scheme is supported for non windows agent

I have a linux agent where I want to run a PowerShell script with an azcli command, using Azure Resource Manager service connection.
This is the task I am using :
- task: AzurePowerShell#5
displayName: 'Add webapp OutboundIPs into SA FW'
inputs:
azureSubscription: ${{ parameters.serviceConnection }}
ScriptType: 'FilePath'
ScriptPath: '$(path)/update-SA-firewall.ps1'
ScriptArguments: '-webappOutboundIPs "$(webappOutboundIPs)" -SAName $(SAName) -RG ${{ parameters.resourceGroupName }}'
azurePowerShellVersion: 'LatestVersion'
And this is the script:
Param(
[string] [Parameter(Mandatory=$true)] $webappOutboundIPs,
[string] [Parameter(Mandatory=$true)] $SAName,
[string] [Parameter(Mandatory=$true)] $RG
)
# get the Array of IPs from the given string
$IPs = $webappOutboundIPs.Split(",")
# Add these IPs into the SA Firewall
foreach ($ip in $IPs) {
az storage account network-rule add -g $RG --account-name $SAName --ip-address $ip | out-null
}
The error I get is :
Line | 106 | throw ("Only SPN credential auth scheme is
supported for non wind …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Only SPN credential auth scheme is supported for non windows
| agent.
##[error]PowerShell exited with code '1'.
Am I missing something here :/ ?

Delete the Azure Resources if any of the task in azure pipeline fails

I have Azure Pipeline which creates VMs, Storage , NICs etc. And i want that these resources should be deleted if any of the task in the pipeline got failed. How this can be achieved, do I need to use script in my YAML or is there any extension available?
A sample for you. You just need to output the vars and then use condition to check:
trigger:
- none
pool:
vmImage: windows-latest
steps:
- task: AzurePowerShell#5
displayName: Create Storage1
name: createstorage1
inputs:
azureSubscription: 'xxx'
ScriptType: 'InlineScript'
Inline: |
$resourceGroup = "xxx"
$location = "westus"
$accountName = "bowman08191"
Write-Host "##vso[task.setvariable variable=resourceGroup;isoutput=true]$resourceGroup"
Write-Host "##vso[task.setvariable variable=location;isoutput=true]$location"
Write-Host "##vso[task.setvariable variable=accountName;isoutput=true]$accountName"
New-AzStorageAccount -ResourceGroupName $resourceGroup `
-Name $accountName `
-Location $location `
-SkuName Standard_RAGRS `
-Kind StorageV2
azurePowerShellVersion: 'LatestVersion'
- task: AzurePowerShell#5
displayName: Create Storage2
name: createstorage2
inputs:
azureSubscription: 'xxx'
ScriptType: 'InlineScript'
Inline: |
$resourceGroup = "xxx"
$location = "westus"
$accountName = "bowman08192"
Write-Host "##vso[task.setvariable variable=resourceGroup;isoutput=true]$resourceGroup"
Write-Host "##vso[task.setvariable variable=location;isoutput=true]$location"
Write-Host "##vso[task.setvariable variable=accountName;isoutput=true]$accountName"
New-AzStorageAccount -ResourceGroupName $resourceGroup `
-Name $accountName `
-Location $location `
-SkuName Standard_RAGRS `
-Kind StorageV2
azurePowerShellVersion: 'LatestVersion'
- task: AzurePowerShell#5
displayName: This will be failed
inputs:
azureSubscription: 'xxx'
ScriptType: 'InlineScript'
Inline: |
xxx
azurePowerShellVersion: 'LatestVersion'
- task: AzurePowerShell#5
displayName: Create Storage1
name: createstorage3
inputs:
azureSubscription: 'xxx'
ScriptType: 'InlineScript'
Inline: |
xxx
$resourceGroup = "xxx"
$location = "westus"
$accountName = "bowman08193"
Write-Host "##vso[task.setvariable variable=resourceGroup;isoutput=true]$resourceGroup"
Write-Host "##vso[task.setvariable variable=location;isoutput=true]$location"
Write-Host "##vso[task.setvariable variable=accountName;isoutput=true]$accountName"
New-AzStorageAccount -ResourceGroupName $resourceGroup `
-Name $accountName `
-Location $location `
-SkuName Standard_RAGRS `
-Kind StorageV2
azurePowerShellVersion: 'LatestVersion'
- task: AzurePowerShell#5
condition: failed()
continueOnError: true
inputs:
azureSubscription: 'xxx'
ScriptType: 'InlineScript'
Inline: |
Remove-AzStorageAccount -Name $(createstorage1.accountName) -ResourceGroupName $(createstorage1.resourceGroup) -Force
Remove-AzStorageAccount -Name $(createstorage2.accountName) -ResourceGroupName $(createstorage2.resourceGroup) -Force
Remove-AzStorageAccount -Name $(createstorage3.accountName) -ResourceGroupName $(createstorage3.resourceGroup) -Force
azurePowerShellVersion: 'LatestVersion'
The above is Storage service, other service are similar.
By the way, you can deploy all of the services to a new resource group, if failed, just delete the whole group.
You haven't mentioned which scripting language you are using to deploy these resources. It depends on both scripting language and logic. In this case YAML pipeline can be more useful like having stages or jobs. Also use 'continue on error' as true
Set a variable to find your job executed successfully or not
echo "##vso[task.setvariable variable=ISVALIDBUILD;isOutput=true]True"
Create a new job if previous job fails. here InfraBuild is previous job.
- job: RunOnceifFailed
dependsOn: InfraBuild
variables:
PrintResults: $[ dependencies.InfraBuild.outputs['DetermineResult.PrintResults'] ]
condition: eq(dependencies.InfraBuild.outputs['DetermineResult.ISVALIDBUILD'], 'False')
Write the tasks to delete the resources in the new job.
Likewise you can also have a job if 'Infrabuild' Job executed successfully.
Please refer MS documents to get to know more about this, hope it helps you get started.

How can I know if the pipeline is using a Microsoft-hosted agent or self-hosted agent

While running an azure build pipeline (yml version) is there any programmatic way (inside the pipeline itself) to know if the current pipeline is running on the ms-hosted agent or self-hosted agent?
We have one pre-defined variable called 'Agent.Name' which gives the agent name.
But it keeps changing the agent name (Hosted, Azure)
Agent.Name=Hosted, Agent.Name=Azure
Is there any way to determine the type of agent at the pipeline run time.
- task: Bash#3
displayName: Show Agent Name
inputs:
targetType: 'inline'
script: |
echo $(Agent.Name)
No built-in feature to achieve your requirements.
But we can 'for-each' the 'azure pipeline agent pool' to get all of the Microsoft-agent names in it. And then compare.
trigger:
- none
# pool:
# name: VMAS
# 1
stages:
- stage: s1
displayName: get the Microsoft host agents' names
jobs:
- job: testJob
steps:
- task: PowerShell#2
name: setvar
inputs:
targetType: 'inline'
script: |
# logic here. For example you get the vars and put it into this format:
$PAT = "<Your Personal Access Token>"
$org_name = "<Your Organization Name>"
$pool_id = <The microsoft hosted agent pool id>
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "Basic "+$PAT)
$url = "https://dev.azure.com/"+$org_name+"/_apis/distributedtask/pools/"+$pool_id+"/agents?api-version=6.0"
$response = Invoke-RestMethod $url -Method 'GET' -Headers $headers
$response | ConvertTo-Json
$str = "";
foreach ($item in $response.value) {
$str += $item.name
$str += ","
}
Write-Host $str
Write-Host "##vso[task.setvariable variable=outputvars;isOutput=true]$str"
# 2
- stage: s2
displayName: check whether current agent name is one of the above
dependsOn: s1
variables:
vars: $[ stageDependencies.s1.testJob.outputs['setvar.outputvars'] ]
jobs:
- job:
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$varsArr = "$(vars)".Split(',')
$microsofthostagent = 0
foreach ($var in $varsArr)
{
if ($var -eq "$(Agent.Name)")
{
$microsofthostagent = 1
}
else
{
}
}
if( $microsofthostagent -eq 1){
Write-Host "This pipeline is based on Microsoft Host Agent."
}else{
Write-Host "This pipeline is based on Self Host Agent."
}
By default, the self host agent will not have the same name as the Microsoft host agent.
You just need to be careful not to name the self host agent the same as the agent in the Microsoft agent pool (eg "Hosted Agent", "Azure Pipelines ")
On my side, it works:
Microsoft hosted agent
Self hosted agent
As we are using both the ms-hosted & self-hosted so as a solution I started verifying the names of my self-hosted agents (these names are already known to us) & based on this I am able to pick the MS-hosted agents

How to retrieve certificate from a keyvault and import it into another without saving it?

In an Azure pipeline there are following tasks
AzureResourceManagerTemplateDeployment#3 deploys a Key Vault from an ARM template
Then a AzurePowerShell#5 checks if the Key vault contains a "my-self-signed-cert" and if not - imports it into the Key Vault
Finally another AzureResourceManagerTemplateDeployment#3 deploys a Service Fabric cluster and configures the SF cluster and its VMSS to use the certificate
Here are the tasks:
- task: AzureResourceManagerTemplateDeployment#3
displayName: 'Deploy Keyvault'
inputs:
deploymentScope: 'Resource Group'
subscriptionId: '${{ parameters.SubscriptionId }}'
azureResourceManagerConnection: '${{ parameters.ArmConnection }}'
action: 'Create Or Update Resource Group'
resourceGroupName: '${{ parameters.resourceGroupName }}'
location: 'West Europe'
templateLocation: 'Linked artifact'
csmFile: '$(Build.SourcesDirectory)/pipelines/templates/keyvault.json'
csmParametersFile: '$(Build.SourcesDirectory)/pipelines/templates/keyvault-params.json'
deploymentMode: 'Incremental'
- task: ARM Outputs#5
displayName: 'Collect Keyvault output'
inputs:
ConnectedServiceNameSelector: 'ConnectedServiceNameARM'
ConnectedServiceNameARM: '${{ parameters.ArmConnection }}'
resourceGroupName: '${{ parameters.resourceGroupName }}'
whenLastDeploymentIsFailed: 'fail'
- task: AzurePowerShell#5
displayName: 'Import certificate'
inputs:
azureSubscription: '${{ parameters.ArmConnection }}'
ScriptType: 'InlineScript'
azurePowerShellVersion: '3.1.0'
Inline: |
$Cert = Get-AzKeyVaultCertificate -VaultName my-kv -Name my-self-signed-cert
if (!$Cert) {
$Base64 = 'MIIWMgIBA___3000_chars_here____o7WqDoWm5I7fg=='
$Cert = Import-AzKeyVaultCertificate -VaultName my-kv -Name my-self-signed-cert -CertificateString $Base64
}
# set the pipeline variables Thumbprint and SecretId - needed for SF deployment
echo "##vso[task.setvariable variable=Thumbprint]$($Cert.Thumbprint)"
echo "##vso[task.setvariable variable=SecretId]$($Cert.SecretId)"
# deploy SF cluster by ARM template and use the SF Cluster certificate thumbsprint as admin cert
- task: AzureResourceManagerTemplateDeployment#3
displayName: 'Deploy SF cluster'
inputs:
deploymentScope: 'Resource Group'
subscriptionId: '${{ parameters.SubscriptionId }}'
azureResourceManagerConnection: '${{ parameters.ArmConnection }}'
action: 'Create Or Update Resource Group'
resourceGroupName: '${{ parameters.resourceGroupName }}'
location: 'West Europe'
templateLocation: 'Linked artifact'
csmFile: '$(Build.SourcesDirectory)/pipelines/templates/sfcluster.json'
csmParametersFile: '$(Build.SourcesDirectory)/pipelines/templates/sfcluster-params.json'
overrideParameters: '-certificateThumbprint $(Thumbprint) -sourceVaultResourceId $(KeyvaultId) -certificateUrlValue $(SecretId)'
deploymentMode: 'Incremental'
This works well, but now I am trying to replace the self-signed certificate by a real certificate, hosted at another Key Vault.
My plan is to download the new certificate contents (including the key) from the other Key Vault, then Base64-encode it (to avoid creating any temporary files) - and finally Import-AzKeyVaultCertificate ... -CertificateString $Base64 into my Key Vault (see the "Step 2" in my list of tasks).
My problem is that I am stuck in retrieving the certificate contents.
I am able to retrieve the "real" certificate with the following PowerShell commands:
$Cert = Get-AzKeyVaultCertificate -VaultName the-company-kv -Name the-real-cert
$Secret = Get-AzKeyVaultSecret -VaultName the-company-kv -Name the-real-cert
They above commands return some metadata, but there is nothing resembling the contents that I would be able to (if not already base64-encoded):
$Base64 = [System.Convert]::ToBase64String($Bytes)
Import-AzKeyVaultCertificate -VaultName my-kv -Name my-self-signed-cert -CertificateString $Base64
Here a solution for how to copy a certificate from one Key Vault to another (here: the-company-kv -> my-kv) without saving it into a temporary file:
$Cert = Get-AzKeyVaultCertificate -VaultName my-kv -Name the-real-cert
if (!$Cert) {
$OrigCert = Get-AzKeyVaultCertificate -VaultName the-company-kv -Name the-real-cert
$Secret = Get-AzKeyVaultSecret -VaultName the-company-kv -Name $OrigCert.Name
$Cert = Import-AzKeyVaultCertificate -VaultName my-kv -Name $OrigCert.Name -CertificateString $Secret.SecretValueText
}
I didn't realize, that PowerShell is not showing all properties, when I enter $Secret at the command prompt and thus I didn't see the $Secret.SecretValueText at first.

Resources