eyaml is not decrypted on Puppet agent - puppet

Question

We running a Puppet master server on CentOS 7.6 in combination with the repo from puppet (http://yum.puppetlabs.com/puppet6/el/7)
When I write am eyaml where I define a variable, I just see the encrypted value on the agent (Windows 2016).
On the Puppet master, I can edit the eyaml, everthing looks fine. The only difference to a standard installation is, that everything is in a different environment 'myenv'.
hiera.yaml:
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
### Encrypted eyaml files
- name: "Secret data: per-node, per-datacenter, common"
lookup_key: eyaml_lookup_key
path: "/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc/%{::trusted.certname}.eyaml"
options:
pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
# Environments
- name: "env2"
glob: "env/test/*/%{::trusted.certname}.yaml"
- name: "env1"
glob: "env/test/%{::trusted.certname}.yaml"
# Components
- name: "cmp2"
glob: "cmp/test/*/%{::trusted.certname}.yaml"
- name: "cmp1"
glob: "cmp/test/%{::trusted.certname}.yaml"
# Others
- name: "Other YAML hierarchy levels"
paths:
- "common.yaml"
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ puppetserver -v
puppetserver version: 6.3.0
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ hiera -v
3.5.0
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ eyaml version
[hiera-eyaml-core] hiera-eyaml (core): 3.0.0
[ root #pup-mst-srv-10:/etc/yum.repos.d]-$ puppetserver gem list -e hiera-eyaml
*** LOCAL GEMS ***
hiera-eyaml (3.0.0)
[ root #pup-mst-srv-10:/etc/eyaml]-$ cat config.yaml
pkcs7_private_key: '/etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem'
pkcs7_public_key: '/etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem'
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ ls -l /etc/puppetlabs/puppet/eyaml/
total 8
-r--r--r--. 1 puppet puppet 1679 Jul 11 15:39 private_key.pkcs7.pem
-r--r--r--. 1 puppet puppet 1050 Jul 11 15:39 public_key.pkcs7.pem
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ eyaml edit rdc.eyaml
---
classes:
- win_ad_abcd
win_ad_abcd::testpassword : DEC(1)::PKCS7[test12]!
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ cat rdc.eyaml
---
classes:
- win_ad_abcd
win_ad_abcd::testpassword : ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAS0E/Y3+QzFhRVZM+F+5kQ8ZQrvGddUno5sDeg3Np9P1/8I5Xetemrx5DTKQaD5C4DS3kgvxjrSqVk/GCCMtZUW5Ynlym1yvylHA7zXmn+g6pYbe5XW88y2Xv1IzdxHwPmgOlFAXJCRoieTrfph+Y4mQBWi2uyrTphHM/o31JcDREfzOeucTSGaHnq8SHeP7t5O7w5ZFG4++hasBLUTubG2ZOAgQRTlksmTK3oOJ0eLRDab4LpgBMaL/VaZgFiu3qmMb3IPtHlaSEAiTRQzdJW7WeHTJUqPSBNni1WmPXA3lFqmp8PFomxsLBTv7i9/gw7SQ2FHwpu5izH6iKwzmEcjA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBRpizv6doUY5DzpFaBg45lgBCJeK3Yi9qSUCulkHzBDzx6]
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/modules/win_ad_abcd/manifests]-$ cat init.pp
class win_ad_abcd (
$testpassword = "Not Set",
)
{
notify{"eyaml --> ${testpassword} <--":}
file { 'C:\Windows\Temp\test.out':
content => $testpassword,
}
}
I expect an output in the test.out file on the agent with "test12", but I got always the encrypted output.
Would be great to have some hints.
Ivo

Answer 1

I spent a bit of time unsuccessfully trying to reproduce this. On the basis of this information, I also can't think of any reason why this would occur.
On attempting to create a minimal, complete example, I simplified the setup, and you could try using something similar to my setup to see if you can still reproduce the issue. Then, add in additional complexity similar to your original problem until you find the minimal, complete, verifiable example. We might be able to help further if you do this.
Simpler hiera.yaml:
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Secret data: per-node, per-datacenter, common"
lookup_key: eyaml_lookup_key
path: "encrypted.yaml"
options:
pkcs7_private_key: ./keys/private_key.pkcs7.pem
pkcs7_public_key: ./keys/public_key.pkcs7.pem
spec/fixtures/hiera/data/encrypted.yaml:
---
test: >
ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw
DQYJKoZIhvcNAQEBBQAEggEAedQX8FExcat6yk0zsUzNbzQ/07w8ghPOw4eY
ycrfz0H7Cr7KnuBMY0yloFmtWuhYcjXETfaU3U3zGr9IOl4Aiy7yD3ZIvH0Y
HoEWKiJeUzNGrpaH/QFk378cEbpd6LXG46nMzw6w21uhASmvVt3KmZBJwY29
sEk2MpZm32H4JxQQosns4SDMQ6tA5h1xSrgpBTKd1x5vKSTsNnLAahjW31aH
JbK7Se+hHJ4zi9P0/ZjT07OTq1X2rwnfNK8wgKJa/VEDSH+KoNub+4TDHfj/
CWyGQx3Y5U1J2R2/6P5Vp2zRaAf/0BT43Ud/M8H25BIjYosuGtDVCVbbxMNK
mXZITDA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBDEPKno3R1K0XNat4a9
uCuDgBAdp579qk9MbgWLXyXSBD80]
Created keys:
▶ eyaml createkeys
▶ ls keys
private_key.pkcs7.pem public_key.pkcs7.pem
Create a secret:
▶ eyaml encrypt -l 'test' -s 'mySecretString' >> \
spec/fixtures/hiera/data/encrypted.yaml
Which I then cleaned up by further editing that file manually to get the content shown above.
I tested using lookup:
▶ puppet lookup \
--hiera_config=spec/fixtures/hiera/hiera.yaml test
--- mySecretString
Theories:
The information above doesn't truly represent your setup.
You could have actually encrypted the encrypted string?
Things I ruled out:
An error would be seen if there was some problem accessing your eyaml keys. Or if the eyaml library wasn't available. The fact that no error is seen suggests the keys are ok, the eyaml_lookup_key function is found, and eyaml is working basically.
Otherwise, see if you can boil this down to a truly minimal, verifiable, complete example.

Answer 2

Thanks Alex, I create a new VM with puppet and set them up like you described. This helped me to find a stupid mistake.
The eyaml with the encrypted password was rdc.eyaml and I had a symlink to this file like myHostname.yaml
So: myHostname.yaml -> rdc.eyaml
After moving myHostname.yaml to myHostname.eyaml everything works fine.