Iterate over a Puppet Hiera hash in a puppet manifest - puppet

Update 1
I've changed the structure of the Hiera data a little and trying a different manifest style.
I'm trying to iterate over the following Hiera hash in a Puppet manifest:
windows-10.yaml:
---
message: "This node is using Windows 10 data"
#profile::hiera_test::backups_enabled: false
profile::hiera_test::firewall:
rule1:
groupName: 'Cortana'
profileNames: ['Private','Public']
action: 'Deny'
rule2:
groupName: 'Microsoft Photos'
profileNames: 'Public'
action: 'Deny'
Although I've updated the data structure and puppet lookup... returns what appears to be valid, I'm not entirely confident in the structure.
I have tried multiple permutations of the manifest. The latest of which looks like the following (based on this answer given by Matt Schuchard):
hiera_test.pp:
class profile::hiera_test (
Hash $data = lookup('profile::hiera_test::firewall', "merge" => 'hash'),
){
$data.each | String $key, Hash $value = {}|{
notify {
default:
name => "Demo_${key}",
message => 'Item DEFAULT',
;
$key:
* => $value,
}
}
}
And the error / output from the above:
PS C:\Users\LocalAdmin> puppet agent -t
Notice: Local environment: 'production' doesn't match server specified node environment 'development', switching agent to 'development'.
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: no parameter named 'groupName' (file: /etc/puppetlabs/code/environments/development/site-modules/profile/manifests/hiera_test.pp, line: 31) on Notify[rule1] (file: /etc/puppetlabs/code/environments/development/site-modules/profile/manifests/hiera_test.pp, line: 31) on node winnode1.domain.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Ideally, I want it to work inside a class declaration (why?, because that's about as far as my Puppet learning has got, but happy to further my learning. I'm also using Puppet Enterprise (2019.0.2)).
There are several similar questions around the internet, but they are either out of date (Hiera <5), have incomplete examples including this, or I can't work out how to transpose them into what I need. Apparently create_resources is due for depreciation
?
If anyone can tell me where I'm going wrong that would be great.

I have got a working solution, but hopefully someone can chime in with a better way.
With the example in the question, the line * => $value substitutes * with keys from the Hiera hash. These keys are used as parameters of the notify resource and as per the error, there is no such parameter groupName for notify. I'd either have to change the keys in my Hiera data to match the parameters of notify which wouldn't make much sense or learn how to use the Hiera data keys as parameters...
Anyway, the Hiera data is the same as in the question, but the class is as follows:
class profile::hiera_test (
Hash $data = lookup('profile::hiera_test::firewall', "merge" => 'hash'),
){
$data.each | String $key, Hash $value = {}|{
notify {
default:
name => "Demo_${key}",
message => 'Item DEFAULT',
;
$key:
name => "Demo_${key}",
message => "Output: GroupName: ${value['groupName']}, Profile Names: ${value['profileNames']}, Action: ${value['action']}",
}
}
}
As you can see, I've replaced * with valid parameters of the notify resource.
And the output:
Notice: Output: GroupName: Cortana, Profile Names: [Private, Public], Action: Deny
Notice: Output: GroupName: Microsoft Photos, Profile Names: Public, Action: Deny
Now to replace Notify with the real exec resource. Converting the profileNames array to PowerShell will be fun, I have no doubt.

Related

How to use a single puppet erb template to create two different files with different contents

I'm trying to use a single template file to create the Solaris 11 /etc/hosts.deny and /etc/hosts.allow files, with the hiera data in data/Solaris.json.
How can I have the hosts.deny file contain only "ALL: ALL", while the hosts.allow contain the "ALL: ALL" and the variable content from the Solaris.json file?
It should be noted that my org is using Puppet 5.5 Open Source, with ruby templates and json instead of yaml.
Here's the manifest.
class etc::solaris_hosts_files (
Array[String] $entry,
) {
file { 'hosts.allow': # Creates /etc/hosts.allow using template.
ensure => 'present',
content => template('etc/solaris_hosts_template.erb'),
owner => 'root',
group => 'other',
path => '/etc/hosts.allow',
mode => '0444',
}
file { 'hosts.deny': # Creates /etc/hosts.deny using template.
ensure => 'present',
content => template('etc/solaris_hosts_template.erb'),
owner => 'root',
group => 'other',
path => '/etc/hosts.deny',
mode => '0444',
}
}
Here is the template file.
ALL: ALL
<% unless $etc::solaris_hosts_files::hosts.allow -%>
<%= #entry.join(' ') %>
Here is the data/Solaris.json variable info.
"etc::solaris_hosts_files::entry": [
"<some_net>:<some_mask>",
"<some_net>:<some_mask> #<some_hostname>" ],
In the interest of being thorough, yes, the #entry variable is included in data/common.json. We assign the value in Solaris.json.
"etc::solaris_hosts_files::entry": "",
I am a noob puppeteer. Any help will be much appreciated. Thank you!!
Posting an answer to Matt Schuchard's question.
hosts.deny
ALL: ALL
hosts.allow:
ALL: ALL
<ip>:<mask>
<ip>:<mask>
The ip:mask comes from the hiera data.
The rendered result is an error:
Info: Retrieving locales
Info: Loading facts
Error: Could not retrieve catalog from remote server: No content type in http response; cannot parse
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Puppet nested resources create_resources, can't convert string into hash

trying to build a DNS with this module: ref. But getting this error:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, can't convert String into Hash.
I have nested YAML, but not sure if it's correctly formatted or not or problems with something else within my code.
This is my dns profile dns.pp:
class profile::bind {
validate_hash($conf)
$conf = hiera_hash('bind::zone', undef)
create_resources('profile::bind::make::zone', $conf)
}
This is how I define my zone with make_zone.pp:
define profile::bind::make::zone (
$hash_data,
$zone,
$ensure,
$zone_contact,
$zone_ns,
$zone_serial,
$zone_ttl,
$zone_origin,
) {
validate_hash($hash_data)
bind::zone { $zone :
ensure => $ensure,
zone_contact => $zone_contact,
zone_ns => [$zone_ns],
zone_serial => $zone_serial,
zone_ttl => $zone_ttl,
zone_origin => $zone_origin,
}
}
This is my host1.yaml data:
---
version: 5
bind::zone:
zone: test.ltd
ensure: present
zone_contact: 'contact.test.ltd'
zone_ns:
-'ns0.test.ltd'
-'ns1.test.ltd'
zone_serial: '2018010101'
zone_ttl: '767200'
zone_origin: 'test.ltd'
hash_data:
"newyork":
owner: "11.22.33.44"
"tokyo":
owner: "22.33.44.55"
"london":
owner: "33.44.55.66"
bind::cname:
ensure: present
record_type: master
There are a number of mistakes and misunderstandings in the code. I fixed them up so that the code at least compiles and ended up with this.
Changes to profile::bind:
class profile::bind {
include bind
$conf = lookup('bind::zone')
create_resources(profile::bind::make::zone, $conf)
}
Changes to profile::bind::make::zone:
define profile::bind::make::zone (
Enum['present','absent'] $ensure,
String $zone_contact,
Array[String] $zone_ns,
String $zone_serial,
String $zone_ttl,
String $zone_origin,
Hash[String, Hash[String, String]] $hash_data,
) {
bind::zone { $name:
ensure => $ensure,
zone_contact => $zone_contact,
zone_ns => $zone_ns,
zone_serial => $zone_serial,
zone_ttl => $zone_ttl,
zone_origin => $zone_origin,
}
}
Changes to host1.yaml:
---
bind::zone:
'test.ltd':
ensure: present
zone_contact: 'contact.test.ltd'
zone_ns:
- 'ns0.test.ltd'
- 'ns1.test.ltd'
zone_serial: '2018010101'
zone_ttl: '767200'
zone_origin: 'test.ltd'
hash_data:
"newyork":
owner: "11.22.33.44"
"tokyo":
owner: "22.33.44.55"
"london":
owner: "33.44.55.66"
Some explanation:
immediate problem:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, can't convert String into Hash.
This error was caused because your Hiera data was not correctly structured as a Hash[String, Hash[String, String]]. Notice in the yaml I have removed your key "zone" and created a nested Hash there.
must include the bind class
The camptocamp BIND module requires the bind class to also be declared. See its documentation.
validate_hash function is legacy and in the wrong place
As John Bollinger mentioned in the comment, you had the validate_hash on the wrong line. I think that was a cut/paste issue, because you would have got a different error message if that was really your code. Anyway, since you're using Puppet 5 (I guess that by the version => 5 in your Hiera), don't use the legacy validate functions ; use Puppet's data type validation. So I just deleted that line.
use lookup() instead of hiera_hash()
Again, since you're using Puppet 5, use the lookup() function instead of the deprecated hiera_hash() function.
version 5 belongs in hiera.yaml, not host1.yaml
It won't cause you any problems, but the line version: 5 won't do anything here, and it belongs in your hiera.yaml file. I used a hiera.yaml file as follows for testing:
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Host 1"
paths:
- host1.yaml
zone_ns type confusion
You had 2 problems with the zone_ns - firstly, a typo in your YAML (no space after the -) ; and secondly, you passed in an Array of zone NS's and then tried to coerce the array to an array in your defined type.
zone parameter should be the name var
Notice I had to delete the $zone parameter in your defined type, and I used the special $name variable instead, to get the name from the title.
refactored to use data type validation
Notice how I used Puppet's data type validation on your inputs in the defined type, and then I had no further need for the legacy validate_hash function and other related validate functions. Read more about that here.
I think that's all. Hope that helps!

Puppet: Adding multiple AD users to local group

I am trying to add multiple AD users on a Windows Server 2012 to the Administrators group, but it's throwing an error. If I specify only a single user in the params.pp file, then it works fine.
params.pp
$user_to_add = [
'ad8\iisuser',
'ad8\dbuser',
],
$group_name = 'Administrators',
add_user_to_local_group.pp
class common::add_user_to_local_group (
$user_to_add = $common::params::user_to_add,
$group_name = $common::params::group_name,
) inherits common::params {
$user_to_add.each |$user_name| {
group { "Add $user_name to local group":
ensure => present,
name => $group_name,
members => [ $user_name ],
}
}
}
Error:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: {"message":"Server Error: Evaluation Error: Error while eval
uating a Resource Statement, Cannot alias Group[Add ad8\\dbuser to local group] to [\"Administrators\"] at /etc/puppetlabs/code/en
vironments/automation/modules/common/manifests/add_user_to_local_group.pp:6; resource [\"Group\", \"Administrators\"] already declared
at /etc/puppetlabs/code/environments/automation/modules/common/manifests/add_user_to_local_group.pp:6 at /etc/puppetlabs/code/environme
nts/automation/modules/common/manifests/add_user_to_local_group.pp:6:9 on node lab.ad8.com","issue_kind":"RUNTIME_ERROR","stacktrace
":["Warning: The 'stacktrace' property is deprecated and will be removed in a future version of Puppet. For security reasons, stacktrac
es are not returned with Puppet HTTP Error responses."]}
You are trying to circumvent resource uniqueness/multiple declarations by providing a different title for the two resources, but resources must also have unique namevars https://docs.puppet.com/puppet/4.9/lang_resources.html#namenamevar. The namevar for the group resource is name, which is aliased from the title if not specified in the attributes (hence the error message output being what it is) https://docs.puppet.com/puppet/latest/type.html#group-attribute-name.
Thus, when you declare two resources for
group { "Add $user_name to local group":
ensure => present,
name => $group_name,
members => [ $user_name ],
}
with the same name attribute like you are doing when you iterate over the hash (since $group_name is the same for both), then you will have a multiple declaration error thrown. This is also why it works for you when you specify only a single user, since you then have namevar uniqueness.
To fix this, you need to have only one group resource that adds both users simultaneously instead of sequentially.
class common::add_user_to_local_group (
$user_to_add = $common::params::user_to_add,
$group_name = $common::params::group_name,
) inherits common::params {
group { $group_name:
ensure => present,
members => $user_to_add,
}
}
I would also recommend pluralizing the use of the word 'user' for clarification ($user_to_add --> $users_to_add). Another improvement may be to allow passing in multiple groups and iterating over those with an associated member hash, but you can decide that for yourself.

Puppet: configuring with augeas a set of [keys:values] via create_resources from hiera

I am trying to create an interface to pseudo-loop over a set of keys:values from my hiera yaml to update a config file with augeas
define augeas_config (
$key,
$value
)
{
augeas{ "/var/MYCONF/MYCONF.def":
lens => "/var/lib/puppet/lib/augeas/lenses/MYCONF.aug",
incl => "/var/MYCONF/MYCONF.def",
context => "/var/MYCONF/MYCONF.def",
changes => [ "set $key $val" ],
}
}
$augeas_files = hiera_hash('lib_BOX::MYCONF::config', {} )
validate_hash($augeas_files)
create_resources('augeas_config', $augeas_files)
where in my yaml keys:values to be updated are supposed to be in a hash like
lib_BOX::MYCONF::config:
SITE_NAME: "TEST-SITE"
OTHER_STUFF: "DEBUG"
So, the idea is to apply my augeas lense (not sure, if I really need 'context', when 'incl' has to be used with 'lens') for the pairs from my yaml.
However, puppet fails currently complaining about a string instead of an expected hash
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Evaluation Error: Error while evaluating a Function Call, can't convert String into Hash at /etc/puppet/environments/development/modules/lib_BOX/manifests/config.pp:28:3 on node MY.NODE.FOO
where line 28 is the one with "create_resources('augeas_config'...". Since I get a hash from hiera, I suppose something in my resource definition is broken, but I do not see what??
Maybe somebody has an idea for me?
Cheers and thanks,
Thomas
Data in your yaml file is invalid. Change it to something like:
lib_BOX::MYCONF::config:
first_aug:
key: SITE_NAME
value: "TEST-SITE"
second_aug:
key: OTHER_STUFF
value : "DEBUG"
In addition you do not have to use hiera_hash. You can use just hiera.
Please read about differences between hiera lookup functions and follow examples about lookup types.
Probably you will also have to remove line validate_hash($augeas_files).

Puppet: unable to get hiera variable

I've been using hiera for several weeks now and all was working fine til few days ago when i started to get that kind of message:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find data item nom in any Hiera data file and no default supplied on node d0puppetclient.victor-buck.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
So i tried to make a very simple test to check if the problem came from my last code changes and i'm still getting this message. I can't get hiera variable anymore.
Below the test i made:
hiera.yaml:
---
:backends:
- yaml
:yaml:
:datadir: /etc/puppet/hieradata
:hierarchy:
- common
site.pp:
# /etc/puppet/manifests/site.pp
case $operatingsystem {
'Solaris': { include role::solaris }
'RedHat', 'CentOS': { include redhat::roles::common }
/^(Debian|Ubuntu)$/: { include role::debian }
# default: { include role::generic }
}
case $hostname {
/^d0puppetclient/: { include test }
}
test.pp:
class test{
$nom = hiera('nom')
file {"/root/test.txt":
ensure => file,
source => "/etc/puppet/test.txt.erb",
}
}
test.txt.erb:
<%= nom %>
Any idea about to fix this? I thought this could be an file access right issue, so i tried to grante access on some files (755) and it's not working...
You need to define nom in your common.yaml in order for it to hold a value. You can set a default value and conditionally create the file if you don't plan on setting it.
class test {
$nom = hiera('nom', false)
if $nom {
file { '/root/test.txt':
ensure => file,
content => template('test/test.txt.erb')
}
}
}
Notice how i used content instead of source. When using erb templates you need to specify the content using the template() function.
Using Templates
If you use source it is expecting a file rather than an erb template.
Hope this helps.

Resources