I have created a variable group and set a variable there CustomArtifactName and have given the scope as Release
The value of this variable is set in build pipeline so it generates a unique name for artifacts
Write-Host "##vso[task.setvariable variable=CustomArtifactName]$(Get-Date -Format yyyyMMddhhmmss)"
I can use this fine in build pipelline in yaml in PublishPipelineArtifact#1 tasks like this
'drop-pom-''$(CustomArtifactName)'
But when I use the below in release pipeline maven tasks in Maven POM File field
$(System.DefaultWorkingDirectory)/cucumber-java-junit5-webapp/drop-pom-$(CustomArtifactName)/s/target/pom.xml
I get
##[error]Unhandled: Not found mavenPOMFile:
I also tried to use this variable in Download Pipeline Artifact task but got similar result
How do I use variables from variable group in release tasks fields/properties? According to this msdoc we simply have to use $(variableName) but thats not working .
Based on your description, when you reference the variable group in the build pipeline, you set the variable with new value.
The new value only applies to the build pipeline, it does not update the value of the variable in the variable group, so it will still use the old variable in the release pipeline. This could be the root cause of this issue.
To solve this issue , you need to add a Powershell task to run Rest API(Variablegroups - Update) to update the variable in variable group.
Here is the Powershell Script example:
$token = "PAT"
$url="https://dev.azure.com/{organizationname}/{projectname}/_apis/distributedtask/variablegroups/{variablegroupid}?api-version=5.0-preview.1"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$JSON = #'
{
"variables": {
"CustomArtifactName": {
"value": "$(CustomArtifactName)"
}
},
"type": "Vsts",
"name": "{variablegroupname}",
"description": "Updated variable group"
}
'#
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method PUT -Body $JSON -ContentType application/json
Result: The variable in variable group will update based on each build.
Update:
variable group id:
You could open the variable group in website and check the URL.
When the Powershell script runs successfully, the value of the variable will be passed to the variable in Variable Group.
Related
I have a Powershell script that will make a GET request to SonarQube's REST API to check if the Quality Gate passes or fails. If the Quality Gate fails, the pipeline will fail. I am able to make this work when only looking at the master branch however, I am trying to look at all branches and pull requests.
My pipeline Powershell script:
- job:
pool:
name: 'POEM-GBT-Agent'
variables:
- group: SonarQube
displayName: 'SonarQube API'
steps:
- checkout: none
- powershell: |
$token = [System.Text.Encoding]::UTF8.GetBytes("$(SONARQUBE_API_TOKEN)" + ":")
$base64 = [System.Convert]::ToBase64String($token)
$basicAuth = [string]::Format("Basic {0}", $base64)
$headers = #{ Authorization = $basicAuth }
if ($(System.PullRequest.PullRequestId)) {
$param = "pullRequest=$(System.PullRequest.PullRequestId)"
}
else {
$param = "branch=$env:$BRANCH_NAME"
}
$result = Invoke-RestMethod -Method Get -Uri https://sonarqube.tjx.com/api/qualitygates/project_status?projectKey=$(sonarProjectKey)"&"$param -Headers $headers
$result | ConvertTo-Json | Write-Host
if ($result.projectStatus.status -ne "OK") {
Write-Host "##vso[task.logissue type=error]Quality Gate Failed"
Write-Host "##vso[task.complete result=Failed]"
}
env:
BRANCH_NAME: replace('$(Build.SourceBranch)', 'refs/heads/', '')
This results in an error saying:
+ $param = "branch=$env:$BRANCH_NAME"
+ ~~~~~
Variable reference is not valid. ':' was not followed by a valid variable name
character. Consider using ${} to delimit the name.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : InvalidVariableReferenceWithDrive
After receiving this error I changed my conditional statement to:
if ($(System.PullRequest.PullRequestId)) {
$param = "pullRequest=$(System.PullRequest.PullRequestId)"
}
else {
$param = "branch=${env}:$BRANCH_NAME"
}
After changing my conditional I get this error:
System.PullRequest.PullRequestId : The term 'System.PullRequest.PullRequestId'
is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify
that the path is correct and try again.
At D:\GBT\agent\Workspace\_temp\0a756446-474a-4d58-94ff-ad25e38c3c7a.ps1:9
char:7
+ if ($(System.PullRequest.PullRequestId)) {
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (System.PullRequest.PullRequestI
d:String) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : CommandNotFoundException
I am trying to set $param to pullRequest=1234 if the Git PR is affected, otherwise, I want to set $param to something like branch=feature/my-branch-name.
You have two problems here. I'll delve into each below.
The term 'System.PullRequest.PullRequestId' is not recognized
Under your job step, define some environment variables. I have limited experience with Azure DevOps (will reference as ADO from here on) but you will need two under your powershell environment variable definitions (you already have BRANCH_NAME but I will include it below):
- job:
steps:
- powershell: |
YOUR
CODE
HERE
env:
BRANCH_NAME: replace('$(Build.SourceBranch)', 'refs/heads/', '')
PULL_REQUEST_ID: $(System.PullRequest.PullRequestId)
You need to define PULL_REQUEST_ID as an env var because your rendered PowerShell script won't render $() as an inserted value from ADO. This is likely by design as $() is syntax used in other programming languages, including PowerShell. This is the crux of your issue where System.PullRequest.PullRequestId can't be found as a command; PowerShell literally tries to use that as a program name which it can't find.
Within your script, you can then reference the pull request ID with:
$env:PULL_REQUEST_ID
Variable reference is not valid. ':' was not followed by a valid variable name character
The issue is distinct from the issue above, but the solution is the same. Just reference the environment variable as:
$env:BRANCH_NAME
You don't need the extra $, as this just confuses the parser.
Now, you're not asking for this, but if you needed a conditional environment variable name (e.g. the name of the environment variable is sourced from some other ADO variable), you need to use a different syntax to access the environment variable, as the $ confuses the parser when accessed via the $env: syntax. Read on if you are curious about this.
If you did need to access an environment variable whose name is conditional
PowerShell doesn't allow special characters in variable names without specifying the variable name as a string. The notable exceptions are _ (no special meaning) and :, the latter of which is used as a separator for variables defined within a PSDrive. Environment variables are accessible via the Environment provider, which is accessed under the Env PSDrive. You can see this by running Get-PSDrive, there is an Environment provider exposed under the Env drive.
You can reference PSProvider variables in a few ways, the most common way is $drive:path (the variable name is technically considered a path node under the provider). So to reference the UserProfile variable, you can use:
$env:UserProfile # ====> Returns the path to your user profile directory
The problem with your code is you have the following:
$env:$VariableName
In this case the intent is to get an environment variable value whose name is based on another variable's value, but this confuses the parser for this syntax, as $ is now being translated as a literal portion of the variable name, which is invalid. Normally you would use the ${} syntax to avoid this... but you can't here, because the confused portion should be rendered as part of that same variable. In this case, you need to use an alternative approach to access the environment variable, go through the provider directly.
To use UserProfile as an example again:
$envVarName = 'UserProfile'
$envVarValue = (Get-ChildItem Env:/$envVarName).Value
This is the other way to get a value from a PSProvider; traverse its contents as a drive. This works somewhat like a filesystem, though Get-ChildItem will return different properties for different provider types than the FileInfo or DirectoryInfo objects you may be used to from the Filesystem provider. It's a bit cumbersome, but luckily it's not a scenario one often needs to account for.
Currently, we use Veracode in our ci enviroment(Azure Pipelines) for security check. Veracode has a azuredevops plugin, that creates flaws as work items(either bug, task etc) in azure boards. However, we created a certain epic, "Security Issues", and we expect that there's a way to link via pipeline, a already created work item to it as a parent/child.
I've been searching for methods (first time working with Azure API's), and I didnt found any specific way to achieve this integration( I did see this, but in this case, they create a work item, instead of linking 2 that already exist How to add work item as child to parent?).
I did talk to Veracode team, however this feature isn't currently available. We can add custom Tags however.
TL:DR: How can I, via pipeline automation, query for certain work items in azureboards, and link them as a parent/child to a certain created epic.
Thanks in advance!
You can use update work item REST API to add a link between two work items. Since you are using REST API in the pipeline, you can use run your REST API in a PowerShell task. Here is the sample of adding a child link to an existing work item:
$organization = "{Organization name}"
$project = "{Project name}"
$pat = "{PAT}"
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $pat)))
$url = "https://dev.azure.com/$organization/$project/_apis/wit/workitems/{Work item id}?api-version=5.1"
$contentType = "application/json-patch+json"
$body= #'
[
{
"op": "add",
"path": "/relations/-",
"value": {
"rel": "System.LinkTypes.Hierarchy-Forward",//Add a child link
##"rel": "System.LinkTypes.Hierarchy-Reverse",//Add a parent link
"url": "https://dev.azure.com/{Organization name}/{Project name}/_apis/wit/workItems/{Work item id}"
}
}
]
'#
Invoke-RestMethod -Uri $url -Method PATCH -ContentType $contentType -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Body $body
You can find the Work Item Relation Types with this API.
If you want to query the work items, you can try to use Wiql. Please refer to this example.
I have a Powershell script that is executed in my YAML pipeline during the release stage.
I can reference group variables using $(myVar). However now I have a need to reference a variable dynamically given its name.
For instance, if I have the string 'myVar', I want to search the variable group for a variable named 'myVar' and get its value.
Is this possible to achieve ?
In the end I did it like this. First I installed the Devops module for powershell so I can get group variables through devops api :
Add-AzureDevOpsAccount -OrganisationName "myor" -ProjectName "myproj" -Token "mytoken"
Write-Host -ForegroundColor DarkGreen "Getting variables group based on name"
$group = Get-AzureDevOpsVariableGroup -SearchString "dev"
Then to get a variable based on its name :
$tokenValue = $group.variables | Select-Object -ExpandProperty $someName | Select-Object -ExpandProperty Value
Requirement: Need to capture the current or running build pipeline result using REST API at the end of the same build pipeline.
I have 3 build pipelines for 3 different environments and each build having 3 different stages(Stage1, Stage2, Stage3).
I need to get the current running build's final stage (Stage3) results (whether succeeded/failed).
I need to get that result information post final stage, i would like to run PS script as next task/job or Post job to capture the result of final stage whether it has passed/failed using Rest API.
I have PS script ready and I would like to know exact API which can be used for this scenario.
Challenge: I'm at half stage, I'm having challenge in getting the final stage' result for that particular running build at the end of the same build pipeline.
Example code snippet:
$personalAccessToken=(Get-AzureKeyVaultSecret -VaultName $keyVaultName -Name $secretname).SecretValueText
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($personalAccessToken)"))
$header = #{Authorization=("Basic {0}" -f $token)}
$projectsUrl = "https://dev.azure.com/$AzureDevopsAccount/$Project/_apis/build/builds?api-version=5.0&resultFilter=all&definitions=$definition"
$projects = Invoke-RestMethod -Uri $projectsUrl -Method Get -Headers $header
Write-Host "Pipeline = $($projects.value.result| ConvertTo-Json -Depth 1)"
Using this code, I'm able to capture the result for all pipelines. I just need to know how to get the status of the running build at the end of pipeline completion.
Note: As I have 3 different build pipelines, I need to be able to separately capture this result for all 3 builds at the end of each build pipeline.
Any suggestions will be appreciated. Thanks.
Get the Current/Running Build final stage results at the end of the
pipeline as post job/task using REST API
To get the stage results, please use below api which does not documented and you can get it from F12:
Get https://dev.azure.com/{org}/{project}/_build/results?buildId=$(Build.BuildId)&__rt=fps&__ver=2
Powershell script:
$token = "{token}"
$url =" https://dev.azure.com/{org}/{project}/_build/results?buildId=$(Build.buildid)&__rt=fps&__ver=2"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Get
Write-Host "results = $($response | ConvertTo-Json -Depth 100)"
Then you can get the stage results from its response.
In our system, we use number to represent the result: 0: succeeded, 1: succeded with issues, 2: failed, 3: canceled, 4: skipped, 5: abandoned
Note: As what you want, you just want to get the final stage result instead of all stages of current pipeline. I need to say, until now, there no directly way can achieve that. You must specify the stage name to filter out the result code of final stage. Here has a sample on how to filter.
as post job/task
As you know, post-job is a system build-in task which used to clean the environments. If you want to add a similar task and set as a post job, you need add a customized extension: Use a decorator to inject steps into a pipeline.
I have developed this extension for myself, and upload it into my github so that you can refer to my repos(its just a simple sample).
In its definition, you just need to paste the above powershell script into my-decorator.yml file.
At this time, the powershell script which used to prompt the final stage result can set as a post job in your pipeline.
Hope my extension can help you.
I have an ARM template that has and outputs section like the following:
"outputs": {
"sqlServerFqdn": {
"type": "string",
"value": "[reference(concat('Microsoft.Sql/servers/', variables('sqlserverName'))).fullyQualifiedDomainName]"
},
"primaryConnectionString": {
"type": "string",
"value": "[concat('Data Source=tcp:', reference(concat('Microsoft.Sql/servers/', variables('sqlserverName'))).fullyQualifiedDomainName, ',1433;Initial Catalog=', variables('databaseName'), ';User Id=', parameters('administratorLogin'), '#', variables('sqlserverName'), ';Password=', parameters('administratorLoginPassword'), ';')]"
},
"envResourceGroup": {
"type": "string",
"value": "[parameters('hostingPlanName')]"
}
}
I have a Azure Resource Group Deployment task that uses the template. I then want to use the variable $(sqlServerFqdn) in the next task for configuration. The variable doesn't seem to just populate and I cannot find anywhere that tells me how to use 'outputs' values on release.
What do I need to do to get the variable to populate for use in configuring tasks after this ARM template runs? An example would be in the parameters to a powershell script task or another ARM template.
VSTS Azure Resource Group Deployment task has outputs section now (since January 2018). So you can set variable name in Deployment outputs of Azure Resource Group Deployment task to, for example, ResourceGroupDeploymentOutputs and add PowerShell Script task with the following inline script:
# Make outputs from resource group deployment available to subsequent tasks
$outputs = ConvertFrom-Json $($env:ResourceGroupDeploymentOutputs)
foreach ($output in $outputs.PSObject.Properties) {
Write-Host "##vso[task.setvariable variable=RGDO_$($output.Name)]$($output.Value.value)"
}
And in subsequent tasks you can use your template variables. So, for example, if you have sqlServerFqdn variable in your template it will be available as $(RGDO_sqlServerFqdn) after PowerShell Script task is completed.
Capturing this answer because I always end up at this question when searching for the solution.
There is a marketplace task which makes ARM template output parameters available further down the pipeline. But in some cases you don't have permission to purchase marketplace items for your subscription, so the following PowerShell will do the same thing. To use it you add it as a powershell script step immediately following the ARM template resource group deployment step. It will look at the last deployment and pull the output variables into pipeline variables.
param(
[string] $resourceGroupName
)
$lastDeployment = Get-AzureRmResourceGroupDeployment -ResourceGroupName $resourceGroupName | Sort Timestamp -Descending | Select -First 1
if(!$lastDeployment) {
throw "Deployment could not be found for Resource Group '$resourceGroupName'."
}
if(!$lastDeployment.Outputs) {
throw "No output parameters could be found for the last deployment of Resource Group '$resourceGroupName'."
}
foreach ($key in $lastDeployment.Outputs.Keys){
$type = $lastDeployment.Outputs.Item($key).Type
$value = $lastDeployment.Outputs.Item($key).Value
if ($type -eq "SecureString") {
Write-Host "##vso[task.setvariable variable=$key;issecret=true]$value"
}
else {
Write-Host "##vso[task.setvariable variable=$key;]$value"
}
}
Note that the environmental variables won't be available in the context of this script, but will in subsequent tasks.
The output value shown on the UI for the Visual Studio Team Services task for Azure Resource Group Deployment only seems to work for the scenario described in Eddie's answer, which is for VMs. In fact, if your deployment doesn't include VMs, you will get an error something like:
No VMs found in resource group: 'MY-RESOURCE-GROUP-NAME'. Could not
register environment in the output variable: 'myVariableName'.
For non-VM examples, I created a powershell script that runs after the RG deployment. This script, as an example, takes input variables for resource group $resourceGroupName and the name of the output variable you need $rgDeploymentOutputParameterName. You could customize and use something similar:
#get the most recent deployment for the resource group
$lastRgDeployment = Get-AzureRmResourceGroupDeployment -ResourceGroupName $resourceGroupName |
Sort Timestamp -Descending |
Select -First 1
if(!$lastRgDeployment)
{
throw "Resource Group Deployment could not be found for '$resourceGroupName'."
}
$deploymentOutputParameters = $lastRgDeployment.Outputs
if(!$deploymentOutputParameters)
{
throw "No output parameters could be found for the last deployment of '$resourceGroupName'."
}
$outputParameter = $deploymentOutputParameters.Item($rgDeploymentOutputParameterName)
if(!$outputParameter)
{
throw "No output parameter could be found with the name of '$rgDeploymentOutputParameterName'."
}
$outputParameterValue = $outputParameter.Value
# From here, use $outputParameterValue, for example:
Write-Host "##vso[task.setvariable variable=$rgDeploymentOutputParameterName;]$outputParameterValue"
In Nov 2020, after this commit - https://github.com/microsoft/azure-pipelines-tasks/commit/1173324604c3f61ce52cdcc999f6d4d7ea9ab8f9 , the variables could directly be used in the subsequent tasks in the pipeline (No powershell scripts required!!)
This is what the steps look like -
In the ARM template deployment task, give any reference name to the Deployment Outputs section under Advanced drop down. In my case I have given armOutputVariable.
See image for visual description
Now to use the value of sqlServerFqdn in the subsequent tasks, simply use it in this manner $(armOutputVariable.sqlServerFqdn.value)
For example, let's say I want to use it to override a parameter in my test task which follows the deployment so I can use it in the following manner -
Example image
To summarize all the outputs in the ARM could be used in the further steps directly in this manner (make sure you assign a reference name in the ARM template deployment step) -
$(armOutputVariable.sqlServerFqdn.value)
$(armOutputVariable.sqlServerFqdn.type)
$(armOutputVariable.primaryConnectionString.value)
$(armOutputVariable.primaryConnectionString.type)
$(armOutputVariable.envResourceGroup.value)
$(armOutputVariable.envResourceGroup.type)
First you define the Azure Resource Deployment Task and in this context the Deployment outputs
In the next step you create a PowerShell Task that takes the Deployment outputs defined above as input arguments
The PowerShell script looks as follows and assigns for each output defined in the ARM template a separate VSTS environment variable with the same name as defined in the ARM template output section. These variables can then be used in subsequent tasks.
param (
[Parameter(Mandatory=$true)]
[string]
$armOutputString
)
Write-Host $armOutputString
$armOutputObj = $armOutputString | convertfrom-json
Write-Host $armOutputObj
$armOutputObj.PSObject.Properties | ForEach-Object {
$type = ($_.value.type).ToLower()
$key = $_.name
$value = $_.value.value
if ($type -eq "securestring") {
Write-Host "##vso[task.setvariable variable=$key;issecret=true]$value"
Write-Host "Create VSTS variable with key '$key' and value '$value' of type '$type'!"
} elseif ($type -eq "string") {
Write-Host "##vso[task.setvariable variable=$key]$value"
Write-Host "Create VSTS variable with key '$key' and value '$value' of type '$type'!"
} else {
Throw "Type '$type' not supported!"
}
}
In a subsequent task you can access the environment variables either by passing them as argument via '$(varName)' (this works for SecureString too) or e.g. in a PowerShell script via $env:varName (this does not work for SecureString)
VSTS allows setting variables in powershell scripts which you can use in other tasks.
The syntax is
Write-Host "##vso[task.setvariable variable=myvariable;]myvalue"
You can have an inline Powershell script which can set the required variable to consume in yet to be executed tasks.You can access it like $(myvariable).
You may need to system.debug variable to true to use this.
Read more details here.
You just need to add an output variable name for "Azure Resource Group Deployment" task like following:
And then use the variable in "PowerShell on Target Machines" task:
"PowerShell on Target Machines" task will use the resource configured in "Azure Resource Group Deployment" task:
Output variables:
Create/update action of the Azure Resource Group task now produces an
output variable during execution. The output variable can be used to
refer to the resource group object in the subsequent tasks. For
example "PowerShell on Target Machine" task can now refer to resource
group output variable as '$(variableName)' so that it can execute the
powershell script on the resource group VM targets.
Limitation: Output variable produced during execution will have
details about VM hostname(s) and (public) ports, if any. Credentials
to connect to the VM host(s) are to be provided explicitly in the
subsequent tasks.
Refer to this link for more details: Azure Resource Group Deployment Task