I'm trying (and struggling) to get a (multiple) role model implemented in Hiera.
I've worked in the last 2 years with exact the same model as a user and now want to rebuild the same structure on my own. For example, my node.yaml should contain only the roles I want to apply onto the host:
/etc/puppetlabs/code/environments/production/nodes/my.host.de.yaml
classes:
- ydixken_baseinstall
- additional_modules
[...]
For me it's way more intuitive, to place a yaml in the roles/ directory, with the name of the role, and avoid dealing with profiles:
/etc/puppetlabs/code/environments/production/roles/ydixken_baseinstall.yaml
classes:
- apt
- unattended_upgrades
- [...]
apt::update:
frequency: 'daily'
loglevel: 'debug'
[...]
Placing the role definitions as a node fact is not practicable for me. It's also nice-to-have to allow a customization of the already defined values inside of the node configuration, if needed.
Right now my directory, hiera.yaml & file-structure looks like this:
/etc/puppetlabs/puppet/hiera.yaml
version: 5
defaults:
datadir: /etc/puppetlabs/code/environments/production
data_hash: yaml_data
hierarchy:
- name: "Per-node data (yaml version)"
paths:
- "nodes/%{fqdn}.yaml"
- "roles/%{role}.yaml"
- common
/etc/puppetlabs/code/environments/production/hiera.yaml
version: 5
defaults:
hierarchy:
- name: "FQDN"
path: "nodes/%{fqdn}.yaml"
- name: "Roles"
path: "roles/%{role}.yaml"
- name: "Common Data"
path: "common.yaml"
/etc/puppetlabs/code/environments/production/manifests/site.pp
hiera_include('classes')
How can i achieve this?
My current error:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, Could not find class ::ydixken_baseinstall for my.host.de (file: /etc/puppetlabs/code/environments/production/manifests/site.pp, line: 1, column: 1) on node my.host.de
I've found exactly, what i was looking for: r10k
Related
We running a Puppet master server on CentOS 7.6 in combination with the repo from puppet (http://yum.puppetlabs.com/puppet6/el/7)
When I write am eyaml where I define a variable, I just see the encrypted value on the agent (Windows 2016).
On the Puppet master, I can edit the eyaml, everthing looks fine. The only difference to a standard installation is, that everything is in a different environment 'myenv'.
hiera.yaml:
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
### Encrypted eyaml files
- name: "Secret data: per-node, per-datacenter, common"
lookup_key: eyaml_lookup_key
path: "/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc/%{::trusted.certname}.eyaml"
options:
pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
# Environments
- name: "env2"
glob: "env/test/*/%{::trusted.certname}.yaml"
- name: "env1"
glob: "env/test/%{::trusted.certname}.yaml"
# Components
- name: "cmp2"
glob: "cmp/test/*/%{::trusted.certname}.yaml"
- name: "cmp1"
glob: "cmp/test/%{::trusted.certname}.yaml"
# Others
- name: "Other YAML hierarchy levels"
paths:
- "common.yaml"
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ puppetserver -v
puppetserver version: 6.3.0
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ hiera -v
3.5.0
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ eyaml version
[hiera-eyaml-core] hiera-eyaml (core): 3.0.0
[ root #pup-mst-srv-10:/etc/yum.repos.d]-$ puppetserver gem list -e hiera-eyaml
*** LOCAL GEMS ***
hiera-eyaml (3.0.0)
[ root #pup-mst-srv-10:/etc/eyaml]-$ cat config.yaml
pkcs7_private_key: '/etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem'
pkcs7_public_key: '/etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem'
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ ls -l /etc/puppetlabs/puppet/eyaml/
total 8
-r--r--r--. 1 puppet puppet 1679 Jul 11 15:39 private_key.pkcs7.pem
-r--r--r--. 1 puppet puppet 1050 Jul 11 15:39 public_key.pkcs7.pem
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ eyaml edit rdc.eyaml
---
classes:
- win_ad_abcd
win_ad_abcd::testpassword : DEC(1)::PKCS7[test12]!
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ cat rdc.eyaml
---
classes:
- win_ad_abcd
win_ad_abcd::testpassword : ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAS0E/Y3+QzFhRVZM+F+5kQ8ZQrvGddUno5sDeg3Np9P1/8I5Xetemrx5DTKQaD5C4DS3kgvxjrSqVk/GCCMtZUW5Ynlym1yvylHA7zXmn+g6pYbe5XW88y2Xv1IzdxHwPmgOlFAXJCRoieTrfph+Y4mQBWi2uyrTphHM/o31JcDREfzOeucTSGaHnq8SHeP7t5O7w5ZFG4++hasBLUTubG2ZOAgQRTlksmTK3oOJ0eLRDab4LpgBMaL/VaZgFiu3qmMb3IPtHlaSEAiTRQzdJW7WeHTJUqPSBNni1WmPXA3lFqmp8PFomxsLBTv7i9/gw7SQ2FHwpu5izH6iKwzmEcjA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBRpizv6doUY5DzpFaBg45lgBCJeK3Yi9qSUCulkHzBDzx6]
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/modules/win_ad_abcd/manifests]-$ cat init.pp
class win_ad_abcd (
$testpassword = "Not Set",
)
{
notify{"eyaml --> ${testpassword} <--":}
file { 'C:\Windows\Temp\test.out':
content => $testpassword,
}
}
I expect an output in the test.out file on the agent with "test12", but I got always the encrypted output.
Would be great to have some hints.
Ivo
I spent a bit of time unsuccessfully trying to reproduce this. On the basis of this information, I also can't think of any reason why this would occur.
On attempting to create a minimal, complete example, I simplified the setup, and you could try using something similar to my setup to see if you can still reproduce the issue. Then, add in additional complexity similar to your original problem until you find the minimal, complete, verifiable example. We might be able to help further if you do this.
Simpler hiera.yaml:
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Secret data: per-node, per-datacenter, common"
lookup_key: eyaml_lookup_key
path: "encrypted.yaml"
options:
pkcs7_private_key: ./keys/private_key.pkcs7.pem
pkcs7_public_key: ./keys/public_key.pkcs7.pem
spec/fixtures/hiera/data/encrypted.yaml:
---
test: >
ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw
DQYJKoZIhvcNAQEBBQAEggEAedQX8FExcat6yk0zsUzNbzQ/07w8ghPOw4eY
ycrfz0H7Cr7KnuBMY0yloFmtWuhYcjXETfaU3U3zGr9IOl4Aiy7yD3ZIvH0Y
HoEWKiJeUzNGrpaH/QFk378cEbpd6LXG46nMzw6w21uhASmvVt3KmZBJwY29
sEk2MpZm32H4JxQQosns4SDMQ6tA5h1xSrgpBTKd1x5vKSTsNnLAahjW31aH
JbK7Se+hHJ4zi9P0/ZjT07OTq1X2rwnfNK8wgKJa/VEDSH+KoNub+4TDHfj/
CWyGQx3Y5U1J2R2/6P5Vp2zRaAf/0BT43Ud/M8H25BIjYosuGtDVCVbbxMNK
mXZITDA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBDEPKno3R1K0XNat4a9
uCuDgBAdp579qk9MbgWLXyXSBD80]
Created keys:
▶ eyaml createkeys
▶ ls keys
private_key.pkcs7.pem public_key.pkcs7.pem
Create a secret:
▶ eyaml encrypt -l 'test' -s 'mySecretString' >> \
spec/fixtures/hiera/data/encrypted.yaml
Which I then cleaned up by further editing that file manually to get the content shown above.
I tested using lookup:
▶ puppet lookup \
--hiera_config=spec/fixtures/hiera/hiera.yaml test
--- mySecretString
Theories:
The information above doesn't truly represent your setup.
You could have actually encrypted the encrypted string?
Things I ruled out:
An error would be seen if there was some problem accessing your eyaml keys. Or if the eyaml library wasn't available. The fact that no error is seen suggests the keys are ok, the eyaml_lookup_key function is found, and eyaml is working basically.
Otherwise, see if you can boil this down to a truly minimal, verifiable, complete example.
Thanks Alex, I create a new VM with puppet and set them up like you described. This helped me to find a stupid mistake.
The eyaml with the encrypted password was rdc.eyaml and I had a symlink to this file like myHostname.yaml
So: myHostname.yaml -> rdc.eyaml
After moving myHostname.yaml to myHostname.eyaml everything works fine.
I have look at other questions like Using hiera to set class parameters? and others which discusses hiera 3. I am using hiera 5.
Here is my hiera.yaml
[root#e64a2e5c7c79 fisherman]# cat /fisherman/fisherman/hiera/hiera.yaml
---
version: 5
defaults: # Used for any hierarchy level that omits these keys.
datadir: data # This path is relative to hiera.yaml's directory.
data_hash: yaml_data # Use the built-in YAML backend.
hierarchy:
- name: "Apps" # Uses custom facts.
path: "apps/%{facts.appname}.yaml"
I also have this hiera data file:
[root#e64a2e5c7c79 fisherman]# cat /fisherman/fisherman/hiera/apps/HelloWorld.yaml
---
fisherman::create_new_component::component_name: 'HelloWord'
But when I run my puppet agent like so ...
export FACTER_appname=HelloWorld
hiera_config=/fisherman/fisherman/hiera/hiera.yaml
modulepath=/fisherman/fisherman/modules
puppet apply --modulepath=$modulepath --hiera_config=$hiera_config -e 'include fisherman'
... I get this error ...
Error: Evaluation Error: Error while evaluating a Function Call, Class[Fisherman::Create_new_component]: expects a value for parameter $component_name (file: /fisherman/fisherman/modules/fish
erman/manifests/init.pp, line: 12, column: 9) on node e64a2e5c7c79
I tried debugging hiera with puppet lookup like so:
[root#e64a2e5c7c79 /]# export FACTER_appname=HelloWorld
[root#e64a2e5c7c79 /]# hiera_config=/fisherman/fisherman/hiera/hiera.yaml
[root#e64a2e5c7c79 /]# modulepath=/fisherman/fisherman/modules
[root#e64a2e5c7c79 /]# puppet lookup --modulepath=$modulepath --hiera_config=$hiera_config --node agent.local --explain fisherman::create_new_component::component_name
Searching for "lookup_options"
Global Data Provider (hiera configuration version 5)
Using configuration "/fisherman/fisherman/hiera/hiera.yaml"
Hierarchy entry "Apps"
Path "/fisherman/fisherman/hiera/data/apps/.yaml"
Original path: "apps/%{facts.appname}.yaml"
Path not found
Environment Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"
Merge strategy hash
Hierarchy entry "Per-node data (yaml version)"
Path "/etc/puppetlabs/code/environments/production/data/nodes/.yaml"
Original path: "nodes/%{::trusted.certname}.yaml"
Path not found
Hierarchy entry "Other YAML hierarchy levels"
Path "/etc/puppetlabs/code/environments/production/data/common.yaml"
Original path: "common.yaml"
Path not found
Module data provider for module "fisherman" not found
Searching for "fisherman::create_new_component::component_name"
Global Data Provider (hiera configuration version 5)
Using configuration "/fisherman/fisherman/hiera/hiera.yaml"
Hierarchy entry "Apps"
Path "/fisherman/fisherman/hiera/data/apps/.yaml"
Original path: "apps/%{facts.appname}.yaml"
Path not found
Environment Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"
Hierarchy entry "Per-node data (yaml version)"
Path "/etc/puppetlabs/code/environments/production/data/nodes/.yaml"
Original path: "nodes/%{::trusted.certname}.yaml"
Path not found
Hierarchy entry "Other YAML hierarchy levels"
Path "/etc/puppetlabs/code/environments/production/data/common.yaml"
Original path: "common.yaml"
Path not found
Module data provider for module "fisherman" not found
Function lookup() did not find a value for the name 'fisherman::create_new_component::component_name'
I noticed this in the above output:
Hierarchy entry "Apps"
Path "/fisherman/fisherman/hiera/data/apps/.yaml"
Original path: "apps/%{facts.appname}.yaml"
Path not found
It looks like facts.appname is empty and not HelloWorld as I had expected.
What am I doing wrong here?
Thanks
Based on the information in the question I can't reproduce this. Here is my setup if it helps:
# init.pp
class test (
String $component_name,
) {
notify { $facts['appname']:
message => "Component name: $component_name for fact appname of ${facts['appname']}"
}
}
# hiera.yaml
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Apps" # Uses custom facts.
path: "apps/%{facts.appname}.yaml"
# data/apps/HelloWorld.yaml
---
test::component_name: 'MyComponentName'
# spec/classes/test_spec.rb
require 'spec_helper'
describe 'test' do
let(:hiera_config) { 'spec/fixtures/hiera/hiera.yaml' }
let(:facts) {{ 'appname' => 'HelloWorld' }}
it {
is_expected.to contain_notify("HelloWorld")
.with({
'message' => "Component name: MyComponentName for fact appname of HelloWorld"
})
}
end
Tested on Puppet version:
▶ bundle exec puppet -V
6.6.0
Output:
▶ bundle exec rake spec
I, [2019-07-07T16:42:51.219559 #22140] INFO -- : Creating symlink from spec/fixtures/modules/test to /Users/alexharvey/git/home/puppet-test
/Users/alexharvey/.rvm/rubies/ruby-2.4.1/bin/ruby -I/Users/alexharvey/.rvm/gems/ruby-2.4.1/gems/rspec-core-3.8.2/lib:/Users/alexharvey/.rvm/gems/ruby-2.4.1/gems/rspec-support-3.8.2/lib /Users/alexharvey/.rvm/gems/ruby-2.4.1/gems/rspec-core-3.8.2/exe/rspec --pattern spec/\{aliases,classes,defines,functions,hosts,integration,plans,tasks,type_aliases,types,unit\}/\*\*/\*_spec.rb
test
should contain Notify[HelloWorld] with message => "Component name: MyComponentName for fact appname of HelloWorld"
Finished in 0.1444 seconds (files took 0.9699 seconds to load)
1 example, 0 failures
You also can query the Hiera hierarchy directly using puppet lookup like this:
▶ FACTER_appname=HelloWorld bundle exec puppet lookup \
--hiera_config=spec/fixtures/hiera/hiera.yaml test::component_name
--- MyComponentName
I'm having some trouble with Puppet and Hiera, in that Hiera seems to not be using the facts I ask for in the templates.
I have a /etc/puppetlabs/puppet/hiera.yaml file, with a simple sample hierarchy in it:
---
version: 5
defaults:
datadir: "/etc/puppetlabs/code/environments/%{::environment}/hieradata"
data_hash: yaml_data
hierarchy:
- name: "Per-node data"
path: "nodes/%{trusted.certname}.yaml"
- name: "Per-OS defaults"
path: "os/%{facts.os.family}.yaml"
- name: "Common data"
path: "common.yaml"
I then have a puppet agent which has run and sent it's facts to Puppet. When I test a lookup on it, the variables all expand out to empty strings:
> puppet lookup profiles --environment production --node puppet.example.local --explain
Searching for "lookup_options"
Global Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/puppet/hiera.yaml"
Merge strategy hash
Hierarchy entry "Per-node data"
Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/test-.yaml"
Original path: "nodes/test-%{trusted.certname}.yaml"
Path not found
Hierarchy entry "Per-OS defaults"
Path "/etc/puppetlabs/code/environments/production/hieradata/os/.yaml"
Original path: "os/%{facts.os.family}.yaml"
Path not found
Hierarchy entry "Common data"
Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"
Original path: "common.yaml"
No such key: "lookup_options"
Searching for "profiles"
Global Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/puppet/hiera.yaml"
Hierarchy entry "Per-node data"
Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/.yaml"
Original path: "nodes/test-%{trusted.certname}.yaml"
Path not found
Hierarchy entry "Per-OS defaults"
Path "/etc/puppetlabs/code/environments/production/hieradata/os/.yaml"
Original path: "os/%{facts.os.family}.yaml"
Path not found
Hierarchy entry "Common data"
Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"
Original path: "common.yaml"
No such key: "profiles"
Function lookup() did not find a value for the name 'profiles'
The value isn't found, but I figure that's irrelevant because it isn't even looking for any real files, that are all showing as nodes/.yaml and os/.yaml.
If I run the command again with --debug, I can see a dump of information about the node that includes the facts:
Debug: Facter: fact "osfamily" has resolved to "RedHat".
...
Debug: Facter: fact "os" has resolved to {
architecture => "x86_64",
family => "RedHat",
hardware => "x86_64",
name => "CentOS",
release => {
full => "7.4.1708",
major => "7",
minor => "4"
},
Puppet: 5.4.0
Have your facts file specified in the following path
/etc/facter/facts.d/${your_component_facts}.sh
and give required permissions:
chmod +x /etc/facter/facts.d/${your_component_facts}.sh
I am pretty new to puppet. I configured a hiera file, whose path is /etc/puppetlabs/puppet/hiera.yaml, as so
version: 5
hierarchy: []
backends:
- yaml
yaml:
- datadir: /etc/puppetlabs/puppet/some_dir
and I get this error
Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://docs.puppet.com/puppet/5.3/reference/deprecated_language.html
(file & line not available)
Error: Evaluation Error: Error while evaluating a Function Call, Lookup of key 'user_dir' failed: The Lookup Configuration at '/etc/puppetlabs/puppet/hiera.yaml' has wrong type, unrecognized key 'backends'
The Lookup Configuration at '/etc/puppetlabs/puppet/hiera.yaml' has wrong type, unrecognized key 'yaml' at /etc/puppetlabs/code/environments/production/manifests/site.pp:30:17 on node puppet,some_cluster_DNS.internal
Initially, I had those kinds of format for the keys :backends: :yaml: but it seemed it is not regular one for the 5 version, so that I deleted the : sign
Someone has an idea ?
First, in terms of that warning, you should definitely switch over to the Puppet lookup function from the Hiera hiera functions if you are using Hiera >= 4: https://puppet.com/docs/puppet/4.10/hiera_use_function.html
Second, in terms of that error, I would consult the documentation on how to setup a Hiera 5 config file: https://puppet.com/docs/puppet/4.10/hiera_config_yaml_5.html
Using the proper format, your config file would look like:
# /etc/puppetlabs/puppet/hiera.yaml
version: 5
defaults:
- data_hash: yaml_data
- datadir: /etc/puppetlabs/puppet/some_dir
hierarchy: []
What you are trying to do on the last line (specify a specific datadir for the yaml_data backend) is not allowed in Hiera 5. If you want to specify a datadir for a specific backend, then you need to specify a level of the hierarchy for just that backend (or just that backend's datadir; you can customize in several depths of matrices with it) and specify a datadir there. For example:
hierarchy:
- name: yaml data
data_hash: yaml_data
datadir: /etc/puppetlabs/puppet/some_dir
paths:
- "%{trusted.certname}.yaml"
- common.yaml
I need to be able to manage user accounts using pe_accounts and hiera. This page:
http://docs.puppetlabs.com/pe/latest/accounts_class.html#usage-example
says I can create a module like site_accounts and add this text
class {'pe_accounts':
data_store => yaml,
}
to site_accounts/manifests/init.pp
and then install this syntax
---
sysop:
locked: false
comment: System Operator
uid: '700'
gid: '700'
groups:
- admin
- sudonopw
into a file named /var/lib/hiera/pe_accounts_users_hash.yaml
and this will be enough to add the user sysop. But it's not, of course.
I did add the below syntax to hiera.conf
:hierarchy:
- "common"
- "pe_accounts_users_hash"
- "pe_accounts_group_hash"
The error I keep getting is:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find class pe_accounts for vmtest2.lala.kaka.org