I have a Powershell script that will make a GET request to SonarQube's REST API to check if the Quality Gate passes or fails. If the Quality Gate fails, the pipeline will fail. I am able to make this work when only looking at the master branch however, I am trying to look at all branches and pull requests.
My pipeline Powershell script:
- job:
pool:
name: 'POEM-GBT-Agent'
variables:
- group: SonarQube
displayName: 'SonarQube API'
steps:
- checkout: none
- powershell: |
$token = [System.Text.Encoding]::UTF8.GetBytes("$(SONARQUBE_API_TOKEN)" + ":")
$base64 = [System.Convert]::ToBase64String($token)
$basicAuth = [string]::Format("Basic {0}", $base64)
$headers = #{ Authorization = $basicAuth }
if ($(System.PullRequest.PullRequestId)) {
$param = "pullRequest=$(System.PullRequest.PullRequestId)"
}
else {
$param = "branch=$env:$BRANCH_NAME"
}
$result = Invoke-RestMethod -Method Get -Uri https://sonarqube.tjx.com/api/qualitygates/project_status?projectKey=$(sonarProjectKey)"&"$param -Headers $headers
$result | ConvertTo-Json | Write-Host
if ($result.projectStatus.status -ne "OK") {
Write-Host "##vso[task.logissue type=error]Quality Gate Failed"
Write-Host "##vso[task.complete result=Failed]"
}
env:
BRANCH_NAME: replace('$(Build.SourceBranch)', 'refs/heads/', '')
This results in an error saying:
+ $param = "branch=$env:$BRANCH_NAME"
+ ~~~~~
Variable reference is not valid. ':' was not followed by a valid variable name
character. Consider using ${} to delimit the name.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : InvalidVariableReferenceWithDrive
After receiving this error I changed my conditional statement to:
if ($(System.PullRequest.PullRequestId)) {
$param = "pullRequest=$(System.PullRequest.PullRequestId)"
}
else {
$param = "branch=${env}:$BRANCH_NAME"
}
After changing my conditional I get this error:
System.PullRequest.PullRequestId : The term 'System.PullRequest.PullRequestId'
is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify
that the path is correct and try again.
At D:\GBT\agent\Workspace\_temp\0a756446-474a-4d58-94ff-ad25e38c3c7a.ps1:9
char:7
+ if ($(System.PullRequest.PullRequestId)) {
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (System.PullRequest.PullRequestI
d:String) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : CommandNotFoundException
I am trying to set $param to pullRequest=1234 if the Git PR is affected, otherwise, I want to set $param to something like branch=feature/my-branch-name.
You have two problems here. I'll delve into each below.
The term 'System.PullRequest.PullRequestId' is not recognized
Under your job step, define some environment variables. I have limited experience with Azure DevOps (will reference as ADO from here on) but you will need two under your powershell environment variable definitions (you already have BRANCH_NAME but I will include it below):
- job:
steps:
- powershell: |
YOUR
CODE
HERE
env:
BRANCH_NAME: replace('$(Build.SourceBranch)', 'refs/heads/', '')
PULL_REQUEST_ID: $(System.PullRequest.PullRequestId)
You need to define PULL_REQUEST_ID as an env var because your rendered PowerShell script won't render $() as an inserted value from ADO. This is likely by design as $() is syntax used in other programming languages, including PowerShell. This is the crux of your issue where System.PullRequest.PullRequestId can't be found as a command; PowerShell literally tries to use that as a program name which it can't find.
Within your script, you can then reference the pull request ID with:
$env:PULL_REQUEST_ID
Variable reference is not valid. ':' was not followed by a valid variable name character
The issue is distinct from the issue above, but the solution is the same. Just reference the environment variable as:
$env:BRANCH_NAME
You don't need the extra $, as this just confuses the parser.
Now, you're not asking for this, but if you needed a conditional environment variable name (e.g. the name of the environment variable is sourced from some other ADO variable), you need to use a different syntax to access the environment variable, as the $ confuses the parser when accessed via the $env: syntax. Read on if you are curious about this.
If you did need to access an environment variable whose name is conditional
PowerShell doesn't allow special characters in variable names without specifying the variable name as a string. The notable exceptions are _ (no special meaning) and :, the latter of which is used as a separator for variables defined within a PSDrive. Environment variables are accessible via the Environment provider, which is accessed under the Env PSDrive. You can see this by running Get-PSDrive, there is an Environment provider exposed under the Env drive.
You can reference PSProvider variables in a few ways, the most common way is $drive:path (the variable name is technically considered a path node under the provider). So to reference the UserProfile variable, you can use:
$env:UserProfile # ====> Returns the path to your user profile directory
The problem with your code is you have the following:
$env:$VariableName
In this case the intent is to get an environment variable value whose name is based on another variable's value, but this confuses the parser for this syntax, as $ is now being translated as a literal portion of the variable name, which is invalid. Normally you would use the ${} syntax to avoid this... but you can't here, because the confused portion should be rendered as part of that same variable. In this case, you need to use an alternative approach to access the environment variable, go through the provider directly.
To use UserProfile as an example again:
$envVarName = 'UserProfile'
$envVarValue = (Get-ChildItem Env:/$envVarName).Value
This is the other way to get a value from a PSProvider; traverse its contents as a drive. This works somewhat like a filesystem, though Get-ChildItem will return different properties for different provider types than the FileInfo or DirectoryInfo objects you may be used to from the Filesystem provider. It's a bit cumbersome, but luckily it's not a scenario one often needs to account for.
Related
Is there some special variable available in Terraform configuration files which would point to current file name?
I'd like to use it for description fields in various resources, so that someone seeing these resources in the systems would know where is the master definition for them.
e.g.
in myinfra.tf
resource "aws_iam_policy" "my_policy" {
name = "something-important"
description = "Managed by Terraform at ${HERE_I_WOULD_LIKE_TO_USE_THE_VARIABLE}"
policy = <<EOF
[...]
EOF
}
And I would hope the description becomes:
description = "Managed by Terraform at myinfra.tf"
I tried ${path.module} but that only gives "filesystem path of the module where the expression is placed", so pragmatically speaking - everything but the file name I want.
Here's what I can share. Use the data external resource to call an external script that would get the directory/file name and then return it back as a string or any other type that your resources require. Obviously it's not exactly what you wanted as you'll get the dir/file name indirectly but hopefully it helps for others or even yourself for use-cases.
We use that only for azurerm and for very complex integrations that are not yet supported with the current provider versions. I have have not tested it specifically for AWS but since it's a core Terraform resource provider, I'm guessing it might work across the board.
data "external" "cwd" {
program = ["./script.sh"]
query = {
cwd = "${path.cwd}"
}
}
resource "aws_iam_policy" "my_policy" {
name = "something-important"
description = "Managed by Terraform at ${data.external.dir_script.result.filename}"
policy = <<EOF
[...]
EOF
This is how my script looks like:
#!/bin/sh
#echo '{"cwd":"for_testing"}' | ./dir_name.sh | xargs
PIPED=`cat`
errPrint "INFO: Got PIPED data:\n$PIPED"
DIR=`jq -r .cwd <<< $PIPED`
cd $DIR
filename=`ls | grep \.tf$ | xargs`
errPrint "INFO: Returning this as STDOUT:${filename}"
echo "{\"name\":\"$filename\"}"
You need to be that the data from the script needs to return a valid JSON object.
The program must then produce a valid JSON object on stdout, which will be used to populate the result attribute exported to the rest of the Terraform configuration. This JSON object must again have all of its values as strings. On successful completion it must exit with status zero.
Unfortunately, like the others mentioned, there's no other way to get the current file name being 'applied'.
I think you might benefit from using something like yor from Bridge Crew.
From the project's README:
Yor is an open-source tool that helps add informative and consistent tags across infrastructure-as-code frameworks such as Terraform, CloudFormation, and Serverless.
Yor is built to run as a GitHub Action automatically adding consistent tagging logics to your IaC. Yor can also run as a pre-commit hook and a standalone CLI.
So basically, it updates your resources tags with things like:
tags = {
env = var.env
yor_trace = "912066a1-31a3-4a08-911b-0b06d9eac64e"
git_repo = "example"
git_org = "bridgecrewio"
git_file = "applyTag.md"
git_commit = "COMMITHASH"
git_modifiers = "bana/gandalf"
git_last_modified_at = "2021-01-08 00:00:00"
git_last_modified_by = "bana#bridgecrew.io"
}
Maybe that would be good enough to provide what you're trying to do?
As far as my testimony, I have not used yor since my tagging uses a different approach. Instead of having "raw" tags, we use a label module that builds the tags for us and then merges in local tags.
Just sharing this info FYI in case it helps.
I have a Powershell script that is executed in my YAML pipeline during the release stage.
I can reference group variables using $(myVar). However now I have a need to reference a variable dynamically given its name.
For instance, if I have the string 'myVar', I want to search the variable group for a variable named 'myVar' and get its value.
Is this possible to achieve ?
In the end I did it like this. First I installed the Devops module for powershell so I can get group variables through devops api :
Add-AzureDevOpsAccount -OrganisationName "myor" -ProjectName "myproj" -Token "mytoken"
Write-Host -ForegroundColor DarkGreen "Getting variables group based on name"
$group = Get-AzureDevOpsVariableGroup -SearchString "dev"
Then to get a variable based on its name :
$tokenValue = $group.variables | Select-Object -ExpandProperty $someName | Select-Object -ExpandProperty Value
I have almost 20 resources in azure, 4 of them have been given Tags #
{"Office1work"="work"}
{"Office2practice"="Practice"}
{"Office3practice"="Practice"}
{"Office4practice"="Practice"}
Now I want to get the resources whose Tag names start with the keyword "Office".
I know to get a resource by a TagName,for example "hello", I simply use the following command,
get-azureRmResource -TagName "Hello"
How can I use the -Tagname property of get-azurermresource to give me all resources whose tags are starting with the keyword "Office" ?
Or is there any other good method to get all resources whose Tags start with a particular string?
Thanks :)
You can use this code snippet:
$resources = Get-AzureRmResources
$resources.foreach{ if ($PSItem.tags.keys -match '^Office') { $PSItem } }
First you get all the resources in the subscription, then you filter out all the resource whose tags do not match the 'Office' "expression".
as #LotPings points out, it would probably make more sense to filter without saving to a temporary variable:
$resources = Get-AzureRmResources|Where-Object {$_.tags.keys -match "^Office"}
Also, I didnt notice you were asking for a starts with filter, so you should use ^Office as a more strict filter (if you need to).
While deploying a build to multiple environments like UAT, Production. I want to replace one file config.uat.json or config.prod.json with config.json. Is there any option available? just like we have XML Transformation.
I am aware of Json Variable substitution but that doesn't serve my purpose as the variable list is long (almost 50 entries)
Thanks in Advance!
JSON variable substitution should be a good option, if you don't want to add variables one by one on Variables tab, you can update them with VSTS REST API, or add a powershell script to set the variables.
Otherwise, you may need to delete the config.uat.json or config.prod.json, and copy the config.json to the target machine.
Inside your Program.cs file, you can get an environment variable which will represent your current environment:
var builder = WebHost.CreateDefaultBuilder(args);
var currentEnv = builder.GetSetting("environnement");
using this currentEnv value, you will be able to load the file config.{currentEnv}.json
builder.AddJsonFile($"config.{currentEnv}.json", optional: false, reloadOnChange: true);
EDIT:
If you want to do this in powershell, you can make a transofmration of your configuration file with a default: appsettings.json containing keys, and appsettings.env.json containing overriding.
To transform your configuration, you can do something like this:
Param(
[Parameter(Mandatory=$true)][string]$SpecificConfig
)
$defaultConfig = "AppSettings.json";
$settingsContent = ConvertFrom-Json $defaultConfig;
$specificContent = ConvertFrom-Json $SpecificConfig;
# Do this on each <property> to override
if (![string]::IsNullOrEmpty($specificContent.<property>))
{
$settingsContent.<property> = $specificContent.<property>;
}
Write-Host $settingsContent > $defaultConfig;
I'm setting up a Powershell script to provide some user notifications. The notification is legal in nature and may be updated/changed from time to time so it must be fairly easy to locate. It also has a few 'fill in the blank' variables that depend on the person receiving the notification.
I wanted to have a secondary Powershell file that contained the copy (text) to be used, so something like...
$body = "By accessing this system, you agree that your name ($currentUserName) and IP address ($currentUserIPAddr) will be recorded and stored for up to ($currentUserRetentionPeriod)."
The file could be updated as needed without actually opening the script, finding the line to edit, and potentially messing up other items/just being difficult. However, I'm looping through several thousand users in a single execution, so all the $currentUser... variables will be re-used frequently. This poses a problem because $body tries to get the variables immediately and acts as a static string instead of evaluating the variable contents each time it's invoked.
Is there a clever way for me to define $body a single time (i.e. not inside a loop) but still allow for redefinition of internal variables? I'd also rather not split the string up into multiple parts so it became $part1 + $var1 + part2 + var2....n+1 times.
A simple approach would be to just dot-source the script containing the copy whenever you need the variable "re-compiled":
BodyDef.ps1:
$body = "By accessing this system, you agree that your name ($currentUserName) and IP address ($currentUserIPAddr) will be recorded and stored for up to ($currentUserRetentionPeriod)."
Send-Notification.ps1
$bodyDefPath = (Join-Path $PSScriptRoot BodyDef.ps1)
foreach($user in Get-Users){
$currentUserName = $user.UserName
$currentUserIPAddr = $user.IPAddress
$currentUserRetentionPeriod = $user.RententionPeriod
. $bodyDefPath
Send-MailMessage -Body $body
}
The above would work just fine, but it's not very powershell-idiomatic, and kind of silly, reading the file over and over again.
As suggested in the comments, you should define a second function (or just a scriptblock) if you want to reuse the same template with different values:
Send-Notification.ps1
# You could as well define this as a function, doesn't make much difference
$NotificationSender = {
param($User)
$body = "By accessing this system, you agree that your name ($($user.UserName)) and IP address ($($user.IPAddress)) will be recorded and stored for up to $($user.RetentionPeriod)."
Send-MailMessage -Body $body
}
foreach($user in Get-Users){
& $NotificationSender -user $user
}