All,
Is it possible to convert my server.cert.template file in to server.cert using consul template and consul KV.
In my Consul am defining a KV pair as follows:
SERVER_SSL_CERT = 4r4ffqh;fhq2fhwhhqwpihpvhqv'in3v3rvnrqpnrvnnvvqn
And my server.cert.template looks as follows:
-----BEGIN CERTIFICATE-----
${SERVER_SSL_CERT}
-----END CERTIFICATE-----
Earlier I was using local files for this substitution. Can consul handle this and if so how ? How do I query consul to dump this value here ?
Thanks
So I got this working as follows. Sharing it in case some one comes across a similar situation. Contents of my template file are as follows:
{{$host := "hostname" -}}
{{$keyname := printf "Project/%s/Certs/SERVER_SSL_CERT" $host -}}
{{ key $keyname }}
When we run consul-template on it, it will print the value as part of the last line.
Related
I have a next.js application where I have a local .pem file for JWT public/private key authentication. My host does not allow me to upload secret files, so I am trying to access the .pem from a muli-line string .env variable.
I have the contents of the .pem stored like this:
PUB_KEY="-----BEGIN RSA PUBLIC KEY----- [newline]
YTuFE/BOU+pF6T2nYuyYQugqJHUZ62b3LJAeZYyBIjW+LZLeHGkUOZfqHnwSbmd9[newline]
isMbGtdNN/wujOftX1GSDApHX0LyNb+covtN4X4mcFFqyKuotFftUjOLzvb2AeJe[newline]
s9285pXbhAKVcStyAn26pI5...[newline]
-----END RSA PUBLIC KEY-----"
I get the error: Error, secretOrPrivateKey must be an asymmetric key when using RS256
I have tried to convert the .env variable from a multi line string to a single line string, preserving the newlines with this function:
function makeSingleLineKey(key) {
return key.replace(/(\r\n|\n|\r)/gm, '\\n');
}
but I get the same error.
There is an issue with the formatting of the .pem key I think.
Any ideas what might be going wrong?
Thanks
I was able to fix this by storing the .pem files in .env variables where the multi line strings were converted to a single line with the \n line breaks preserved e.g.
`LOCAL_AUTH_PUB_KEY={"PUB_KEY":"-----BEGIN RSA PUBLIC KEY-----\nMIICCgKCAgEAyLKk4ldt/a9dItmU2GkCVUJ1nTIh137JLTGLVyfPHkBaEBpMSm0j\nYT...`
I then accessed the variables in my code like this:
const { PUB_KEY } = JSON.parse(process.env.LOCAL_AUTH_PUB_KEY)
https://dev.to/cfofiu/how-to-store-a-long-private-key-in-vercel-s-environment-variables-46f5
Is it possible to print all defined secrets (or at least their keys) in a gitlab_ci.yml script?
Pseudocode:
deploy:
stage: deploy
script: |
for KEY in $SECRETS
echo $KEY
done
I believe getting the secrets alone is only possible if you already know which secrets exist in the pipeline. But in most images, such as the default ruby image, the env command exists, which lists all env variables, both key and value.
All secrets should be contained within these variables, or at least be listed as a file from within there.
If you have control over how the secrets are named, you can also achieve your original result of only getting secrets, by filtering them. For instance, if you can name every secret with a postfix like _SECRET, you could call
env | grep _SECRET=
and if, as in your original example you only want to list the values, you can call:
env | grep _SECRET= | cut -d = -f2-
I've got a .env file with public/private multiline keys in it, but only some keys are parsing. While the PUBLIC_KEY variable parses correctly with the dotenv package, PRIVATE_KEY is parsed as '-----BEGIN RSA PRIVATE KEY-----.
We've tried swapping their positions in the file, swapping their values, but nothing we do changes the way the file parses--regardless of how we change the file, the PUBLIC_KEY alone parses correctly.
PUBLIC_KEY='-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAuEdNotdGbDKZO1o7oATA
...
BvnP5uVCRrHigT1b8kvpJ/ptVw3hy2yE9h6V0Lolqq8XJ4kydLrOym5fVCdQlGBV
a9R7j5Z/03IUU34BwHNy648CAwEAAQ==
-----END PUBLIC KEY-----
'
PRIVATE_KEY='-----BEGIN RSA PRIVATE KEY-----
MIIJKQIBAAKCAgEAuEdNotdGbDKZO1o7oATAn6PsIoN3DhVLAZBDjg2JZ54ZDVc/
...
jUId69Z3cNk9QM2q0y26qo8uhEZ+yHCkxC3tBfWJ45YrP+Mj3FsPR044rhmu
-----END RSA PRIVATE KEY-----
'
We've got a working solution to manually input line breaks into the remaining keys, but we're still curious if anyone knows what could be causing this.
Thank you!
I've got kv with secret dbname = "DatabaseName".
In Azure DevOps, I'm using two tasks one for reading secrets from Key Vault and second one is Azure Powershell where is my code line Write-Host $(dbname).
Output in logs is ***.
And I want to get DatabaseName as output.
The only thing that I found is
$DWH = "$(dbname)"
Write-Host "$($DWH.ToCharArray() -join ' ' )"
And the output is D a t a b a s e N a m e.
Is there a possibility to get DatabaseName instead of *** or D a t a b a s e N a m e?
According to the docs, Microsoft mask the secret variables from the logs, so you can't print it normally.
You can install the Read Secrets from Key Vault extension that read the secrets from Key Vault and store them in a regular variables (and not in secret variables like the Microsoft built-in task).
I am afraid that you cannot get what you want.
For security reason, you cannot show secrets directly in logs. It is designed as that.
But it doesn't not affect your use of it. For example, I just downloaded secrets from Azure Keyvault, and I can check them in next PowerShell task:
Result
According to the documentation, one can split the secrets and output substrings:
We make an effort to mask secrets from appearing in Azure Pipelines
output, but you still need to take precautions. Never echo secrets as
output. Some operating systems log command line arguments. Never pass
secrets on the command line. Instead, we suggest that you map your
secrets into environment variables.
We never mask substrings of secrets. If, for example, "abc123" is set
as a secret, "abc" isn't masked from the logs. This is to avoid
masking secrets at too granular of a level, making the logs
unreadable. For this reason, secrets should not contain structured
data. If, for example, "{ "foo": "bar" }" is set as a secret, "bar"
isn't masked from the logs.
Here is an example in bash, which can potentially be transferred to Powershell:
- task: Bash#3
inputs:
targetType: inline
script: |
# let's say the secret is Passw0rd
# Direct output is masked:
echo $MYSECRET
# OUTPUTS "***"
# Concatenated output is masked:
echo "ABC$MYSECRET DDD"
# outputs "ABC*** DDD"
# Experimenting with substrings:
firstPart=${MYSECRET::-2}
secondPart=${MYSECRET: -2}
# Substrings are displayed:
echo $firstPart
# outputs "Passw0"
echo $secondPart
# outputs "rd"
# Substrings concatenated with other strings are displayed:
echo "$firstPart-$secondPart"
# outputs "Passw0-rd"
# Directly concatenated substrings are masked:
echo "$firstPart$secondPart"
# outputs "***"
# Secrets can be written to a file:
echo "$MYSECRET" > test.txt
# Secrets are even masked when being displayed as part of a file:
cat test.txt
# outputs "***"
env:
MYSECRET: $(my_secret) # This is defined in a variable group
- task: PublishPipelineArtifact#1
inputs:
targetPath: 'test.txt' # This pipeline artifact contains the secret unmasked
artifact: 'TestArtifact'
publishLocation: 'pipeline'
displayName: 'Publish PipelineRunData artifact'
We therefore have two options to get the secret:
Output the secret partially and concatenate manually
Download pipeline artifact
Probably not ideal from a security perspective, but it is extremely important to understand the limitations of the system: Whoever has access to modifying a pipeline can potentially get access to the secrets used by the pipeline.
Using this library..
https://github.com/motdotla/dotenv
I've added a private RSA key to .env. Using it to sign a JWT.
When I sign the JWT, this error greats me.
error:0906D066:PEM routines:PEM_read_bio:bad end line
My RSA key is just a basic multi line, key.
-----BEGIN RSA PRIVATE KEY-----
MIIEpgIBAAKCAQE.......
........
The RSA key is multi-line.
DotEnv can only read one-liners, the unfortunate work around (AFIAK) are to add a \n to the end of every line. And then make this string a one liner. But this error keeps occuring..
Has anyone else experienced this and have solution?
Or perhaps know of another env file manager that can handle multi-lines?
Add surrounding double quotes to the value in the .env file and replace line breaks with \n like so:
KEY="-----BEGIN RSA PRIVATE KEY-----\nMIIE..."