My team is working to integrate an infrastructure-as-code scanning solution into our build pipelines and we've discovered that the string "GCP" is being replaced with three asterisks when tasks are being executed in our build pipelines. This isn't unique to one task either whereas I created a bash script to execute and list our our repository and all directories that start with "GCP" are replaced by the three asterisks. The only variable set using the "GCP" value is the "system.teamProject" variable and we are not using any secret values that I know of and there are no variable groups used.
Any help would be greatly appreciated. Thanks!
Bash Asterisk Output "ls -a"
IaC Scanning Asterisk Task Failure
If you have set any secret variables in your pipeline, or have linked any variable groups that contain secret variables (include the secrets from the connected external and remote services services), generally the values of these secrets will be masked as asterisks.
When you try to print the values of the secrets to the output logs, the values will display as asterisks in the logs. If you try to output the values into a text file, the values will still display as asterisks in the file.
In addition, if a string that is not set as secret but its substrings are the values of some existing secrets in the pipeline, these substring parts may be masked as asterisks when trying to output this string.
If you do not set any secrets, for us to investigate this issue further, would you like to share us with the actual value that was masked as asterisks in the the logs? We well investigate and evaluate whether this string contains some special or sensitive characters that may be automatically identified as secrets by Azure DevOps.
Related
I currently have a dev.auto.tfavrs file with a few dozen variables and their values for my application such as:
DB_NUM_RECORDS_PER_EXECUTION = "Something here"
QUEUE_API_KEY = "Something here"
application = "My Appplication"
Application_Secret_Key = "Some Secret Key here"
All variables are defined in a variables.tf file, with the sensitive ones assigned a blank value.
I run terraform plan in this manner:
terraform apply -var-file=env/dev.auto.tfvars
(I have also qa.auto.tfvars and prod.auto.tfvars for the other environments)
I want to inject certain sensitive values to certain keys such as :
Application_Secret_Key via The Terraform Cloud Variable sets so developers don't have to.
I have added Application_Secret_Key in the TF Cloud Variable set.
but when i run the above terraform plan, Terraform Cloud is not injecting the value stored in the variable set...instead it assigns the blank value as defined in my configuration.
It is my understanding that the auto.tfvars files take precedence and over write the terraform.tfavars file in Terraform Cloud. Hence the sensitive values are Blank
I do not want to add dozens of variables in the TF Cloud Variable set...only certain sensitive ones.
Is this possible?
Thanks in advance
We have a masked variable for example called SECRET_JAMES_BOND with value /vault/abc. During the pipeline execution, the log shows as [MASKED]. When we changed the value to ABCDE, the value was shown. Even we don't change it, we can easily get the value from the log if we perform SECRET_JAMES_BOND = $(echo $SECRET_JAMES_BOND) | base64). By echoing the base64 value, it will display on the UI because the value has changed. When we copy the encoded base64 string and decode it, we get the actual secret. How can we prevent a masked variable from echoing if its value has been changed ? Shouldn't masked variables show [MASKED] even the value is different from its original value ?
How can we prevent a masked variable from echoing if its value has been changed
In short: you can't if someone with access is determined to do so.
Masking is not meant to prevent developers with access to control CI steps from revealing secrets through jobs. It is meant to prevent disclosure by accident. There are endless ways that a value could be exfiltrated through the job output it would be impossible to cover them all.
If you wanted to prevent the base64 representation of the secret from being shown, you can register that value as another masked variable, but it must be done in advance.
Shouldn't masked variables show [MASKED] even the value is different from its original value ?
Good arguments could be made that GitLab should mask some common variations of masked variables, such as base64, url-encoded, backwards, etc. Other CI services (Travis CI for example) do this.
However, this would still be ONLY for the purposes of accidental disclosure.
For example, another way I've seen developers accidentally reveal secrets is by using curl with the -v flag.
script:
- curl -v https://myusername:${SECRET_JAMES_BOND}#myhost.example.com/secret
In the above example, the output of curl will show (in part) an output like:
> Authorization: Basic bXl1c2VybmFtZTpzZWNyZXQ=
So, if GitLab also masked base64 variations of a password, it would have prevented this accidental disclosure.
But like I mentioned, there's endless ways to output a secret: url-encoding, Caesar cipher, rot-N (rot16, rot24, rot32, etc) or even custom ways like just echoing each character one line at a time, potentially in reverse order.
So, really, you can only realistically prevent accidents, not malicious exfiltration by a user with access to execute code in a job.
I'm running a devops release pipeline and i pull back the keyvaults secrets via this task
steps:
task: AzureKeyVault#1
My secret value looks something like this. abc$def&ghi
I'm updating a database record with this secret via Powershell.
The value that ends up in the database is "abcghi" The $ and characters up to, and including the & are excluded.
Do I need to escape the secrets coming from keyvault somehow?
-Randy
Are you trying to use the secret value inside of a double-quoted string in your PowerShell database update script? Something like this:
$sqlCommand = "update table set value='abc$def&ghi' where id=1";
If so, PowerShell is attempting to expand $def as a variable. It's probably not really a variable and will expand into an empty string.
You can escape the dollar sign in your key vault by using a backtick `$, but you have to be careful if you need to use that secret in a non-PowerShell scenario.
Alternatively, you could use single-quote characters around your string instead; although you'll then need to escape any single-quote characters used inside that string:
$sqlCommand = 'update table set value=''abc$def&ghi'' where id=1';
See the PowerShell docs about quoting rules for more information.
I am using DevOps pipeline to build and deploy to different environments
For one environment I am encountering this issue where i am using a Pipeline Variable with $$ in the value
For Example:
Password pipeline variable with value = $omeCla$$Password
When i deploy it fails and when i check the logs the password is displayed as $omeCla$Password. So basically when $$ are together it drops one $
For all variable i am using regex __VaraibleValue__ and its working fine
I have tried:
$omeCla$\$Password to try and escape and it displays as $omeCla$\$Password . So basically \ doesn't work.
I tried '$omeCla$$Password' to try and escape and it displays as '$omeCla$Password'
I want to keep this value as a normal pipeline variable before review
So basically how can I escape this?
Or should I add a Secret Token here in the replace token task (see screenshot below)? and then make the pipeline variable secret? If so, what should I set for Secret Token? Also, in app.config in my repo what should I use instead of the regex __VariableName__ that I use for normal variables?
The solution was to use 4 $. So if you have $$ together you need to add $$$$
Example: $someCla$$$$Password
#JaneMa-MSFT as requested
https://developercommunity.visualstudio.com/content/problem/1296808/azure-pipeline-how-to-escape-special-characters-in.html
I have below variables.
variable value
stage dev
admin $($(stage)-admindata)
dev-admindata 4000
But these multiple substitutions are not working for variable admin.
Please let me know how to solve this.
admin values should be 4000 when I use it in yaml or in json file, Currently I am getting $(dev-admindata)
At this moment, the value of nested variables (like $($(stage)-admindata)) are not yet supported in the build/release pipelines.
If you want to give different values to the admin variable depending on the value of stage varibale. As a work around you can write a script in the powershell task to judge, e.g. If the value of the stage variable is dev, then assign the value of the dev-admindata variable to the admin variable.
if ($(stage) -eq "dev"){
$admin = $(dev-admindata)
}
else{
xxxxx
}
For the similar issue ,you can refer to this case.