Puppet Hiera Data Objects - puppet

I'm trying to pass a list of hashes into a class. The need is to pass in one or more objects into my class - each object has a small set of data that is used to generate a templated script with the data points found in the hash.
I've successfully used Hiera Arrays & Hashes:
mymods::keyvaultcerts::array::certNames: ['cert1','cert2','cert3']
mymods::chocopackages::hash::packages:
dotnetfx : 4.8
urlrewrite : 2.1
azure-cli : 2.31
vcredist : 12.0`
The above has worked great. I can access the Hash data very easily
class mymods::chocopackages::hash (
Hash $packages
){
$packages.each | $packageName, $packageVersion | {
package{$packageName:
ensure => $packageVersion,
}
}
What I'm trying to do now, is building off the above. I need to pass in a list of objects to my class. So I'm trying to use an array of hash objects:
profile::octopus::worker::workerConfigurations:[
{
Port: 10933,
Pool: "General Worker Pool",
User: "Dummy1",
DisplaySuffix: "",
},
{
Port: 10934,
Pool: "General Dev Pool",
User: "Dummy2",
DisplaySuffix: "Dev",
},
{
Port: 10935,
Pool: "General QA Pool",
User: "Dummy3",
DisplaySuffix: "_QA",
}
]
Then, in my class I'm trying to iterate through the list and access the data like so ...
class profile::octopus::worker(
Array $workerConfigurations
){
$workerConfigurations.each | Integer $workerIndex, Hash $workerConfigurationData | {
$workerConfigurationData.each | $workerPort, $workerPool, $workerUser, $workerDisplaySuffix | {
notify{"\$workerPort ${$workerPort}" :}
}
}
}
What I was expecting, was to see notifications of the ports (10933, 10934, 10935). However, I'm receiving an error on the client-side:
Error while evaluating a Method Call, 'each' block expects between 1 and 2 arguments, got 4
My question is: Is this possible? If not, are there any recommendations on how to pass in a list of one or more objects into my class?

The built-in each function accepts a lambda that takes either one or two parameters. The significance of those parameters depends on whether one or two are given and on whether the first argument to each() is an array or a hash.
In no event may the block accept four parameters:
$workerConfigurationData.each | $workerPort, $workerPool, $workerUser, $workerDisplaySuffix | {
notify{"\$workerPort ${$workerPort}" :}
}
Nor is it clear what you expect the each to be doing for you there. Rather than iterating, you seem to want to select and use one of the mappings from the hash. You do that by subscripting the hash with the wanted key(s):
notify { "Port: ${workerConfigurationData['Port']}" :}
That has nothing in particular to do with Hiera, by the way. You can declare and use hashes (and arrays) without Hiera being involved.

Related

Iterate a puppet resource collector

I am trying to develop a puppet class with a defined resource which creates the configuration for a website.
One of the things that the defined resource has to do is assign the IP address of the website to a dummy interface. Due to constraints of the project this is done with NetworkManager.
So I have to generate a file like
[connection]
id=dummydsr
uuid=50819d31-8967-4321-aa34-383f4a658789
type=dummy
interface-name=dummydsr
permissions=
[ipv4]
method=manual
#IP Addresses come here
ipaddress1=1.2.3.4/32
ipaddress2=5.6.7.8/32
ipaddress3=8.7.6.5/32
[ipv6]
method=ignore
There is to be a line ipaddressX=... for every instance of the defined resource.
My problem is how do I track the number of times the defined resource has been instantiated so I can somehow increment a counter and generate the ipaddress lines.
Or for each instantiated defined resource, append the IP address to an array which I can later use to build the file
If I understand you, and I'm not certain that I do, but I think you would want to do something like this:
define mytype(
Integer $count,
...
) {
file { 'some_network_manager_file':
content => template(...)
}
}
And then you would have a loop:
$mystuff.each |$count, $data| {
mytype { ...:
count => $count,
...
}
}
Key insight here may be that the each function has some magic in it that allows you to get the index if you need it, see also this answer.
Now I think that's how it will work, without me spending time researching NetworkManager. If you provide more of your code, I may be able to update this to be more helpful.
This is less than ideal since I would prefer to have it inside the defined resource, but since I instantiate the defined resource with the data from a hash I use said hash to iterate that part.
class xxx_corp_webserver (
Hash $websites ={}
){
create_resources('xxx_corp_webserver::website', $websites)
# This would be nicer inside the defined class, but I did not find any other way
# Build and array with the IP addresses which are for DSR
$ipaddresses = $websites.map | $r | {
if $r[1]['enabledsr'] {
$r[1]['ipaddress']
}
}
# For each DSR address add the line
$ipaddresses.each | Integer $index , String $ipaddress | {
$num = $index+1
file_line{"dummydsr-ipaddress${num}":
ensure => present,
path => '/etc/NetworkManager/system-connections/dummydsr',
line => "address${num} = ${ipaddress}/32",
match => "^address.* = ${ipaddress}/32",
after => '# IP Addresses come here',
notify => Service['NetworkManager'],
require => File['/etc/NetworkManager/system-connections/dummydsr'],
}
}
}

How to read keys into array?

I am trying to read keys from a hiera json file into an array.
The json is as follows:
{
"network::interfaces": {
"eth0": {
"ip": "10.111.22.10"
},
"eth1": {
"ip": "10.111.22.11"
},
"eth2": {
"ip": "10.111.22.12"
}
}
}
In my Puppet code, I am doing this:
$network_interfaces = hiera_array('network::interfaces')
notice($network_interfaces)
Which results in the following:
Notice: Scope(Class[Role::Vagrant]): {eth0 => {ip => 10.111.22.10}, eth2 => {ip => 10.111.22.11}, eth3 => {ip => 10.111.22.12}}
But what I want are just the interfaces: [eth0, eth1, eth2]
Can someone let me know how to do this?
The difference between hiera_array() and plain hiera() has to do with what happens when the requested key (network::interfaces in your case) is present at multiple hierarchy levels. It has very little to do with what form you want the data in, and nothing to do with selecting bits and pieces of data structures. hiera_array() requests an "array-merge" lookup. The more modern lookup() function refers to this as the "unique" merge strategy.
It seems unlikely that an array-merge lookup is in fact what you want. In that case, the easiest thing to do is read the whole hash and extract the keys:
$network_interfaces = keys(hiera('network::interfaces'))
In Puppet 4 you'll need to use the keys() function provided by the puppetlabs/stdlib module. From Puppet 5 on, that function appears in core Puppet.

Puppet nested resources create_resources, can't convert string into hash

trying to build a DNS with this module: ref. But getting this error:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, can't convert String into Hash.
I have nested YAML, but not sure if it's correctly formatted or not or problems with something else within my code.
This is my dns profile dns.pp:
class profile::bind {
validate_hash($conf)
$conf = hiera_hash('bind::zone', undef)
create_resources('profile::bind::make::zone', $conf)
}
This is how I define my zone with make_zone.pp:
define profile::bind::make::zone (
$hash_data,
$zone,
$ensure,
$zone_contact,
$zone_ns,
$zone_serial,
$zone_ttl,
$zone_origin,
) {
validate_hash($hash_data)
bind::zone { $zone :
ensure => $ensure,
zone_contact => $zone_contact,
zone_ns => [$zone_ns],
zone_serial => $zone_serial,
zone_ttl => $zone_ttl,
zone_origin => $zone_origin,
}
}
This is my host1.yaml data:
---
version: 5
bind::zone:
zone: test.ltd
ensure: present
zone_contact: 'contact.test.ltd'
zone_ns:
-'ns0.test.ltd'
-'ns1.test.ltd'
zone_serial: '2018010101'
zone_ttl: '767200'
zone_origin: 'test.ltd'
hash_data:
"newyork":
owner: "11.22.33.44"
"tokyo":
owner: "22.33.44.55"
"london":
owner: "33.44.55.66"
bind::cname:
ensure: present
record_type: master
There are a number of mistakes and misunderstandings in the code. I fixed them up so that the code at least compiles and ended up with this.
Changes to profile::bind:
class profile::bind {
include bind
$conf = lookup('bind::zone')
create_resources(profile::bind::make::zone, $conf)
}
Changes to profile::bind::make::zone:
define profile::bind::make::zone (
Enum['present','absent'] $ensure,
String $zone_contact,
Array[String] $zone_ns,
String $zone_serial,
String $zone_ttl,
String $zone_origin,
Hash[String, Hash[String, String]] $hash_data,
) {
bind::zone { $name:
ensure => $ensure,
zone_contact => $zone_contact,
zone_ns => $zone_ns,
zone_serial => $zone_serial,
zone_ttl => $zone_ttl,
zone_origin => $zone_origin,
}
}
Changes to host1.yaml:
---
bind::zone:
'test.ltd':
ensure: present
zone_contact: 'contact.test.ltd'
zone_ns:
- 'ns0.test.ltd'
- 'ns1.test.ltd'
zone_serial: '2018010101'
zone_ttl: '767200'
zone_origin: 'test.ltd'
hash_data:
"newyork":
owner: "11.22.33.44"
"tokyo":
owner: "22.33.44.55"
"london":
owner: "33.44.55.66"
Some explanation:
immediate problem:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, can't convert String into Hash.
This error was caused because your Hiera data was not correctly structured as a Hash[String, Hash[String, String]]. Notice in the yaml I have removed your key "zone" and created a nested Hash there.
must include the bind class
The camptocamp BIND module requires the bind class to also be declared. See its documentation.
validate_hash function is legacy and in the wrong place
As John Bollinger mentioned in the comment, you had the validate_hash on the wrong line. I think that was a cut/paste issue, because you would have got a different error message if that was really your code. Anyway, since you're using Puppet 5 (I guess that by the version => 5 in your Hiera), don't use the legacy validate functions ; use Puppet's data type validation. So I just deleted that line.
use lookup() instead of hiera_hash()
Again, since you're using Puppet 5, use the lookup() function instead of the deprecated hiera_hash() function.
version 5 belongs in hiera.yaml, not host1.yaml
It won't cause you any problems, but the line version: 5 won't do anything here, and it belongs in your hiera.yaml file. I used a hiera.yaml file as follows for testing:
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Host 1"
paths:
- host1.yaml
zone_ns type confusion
You had 2 problems with the zone_ns - firstly, a typo in your YAML (no space after the -) ; and secondly, you passed in an Array of zone NS's and then tried to coerce the array to an array in your defined type.
zone parameter should be the name var
Notice I had to delete the $zone parameter in your defined type, and I used the special $name variable instead, to get the name from the title.
refactored to use data type validation
Notice how I used Puppet's data type validation on your inputs in the defined type, and then I had no further need for the legacy validate_hash function and other related validate functions. Read more about that here.
I think that's all. Hope that helps!

GraphQL string concatenation or interpolation

I'm using GitHub API v 4 to learn GraphQL. Here is a broken query to fetch blobs (files) and their text content for a given branch:
query GetTree($branch: String = "master") {
repository(name: "blog-content", owner: "lzrski") {
branch: ref(qualifiedName: "refs/heads/${branch}") {
name
target {
... on Commit {
tree {
entries {
name
object {
... on Blob {
isBinary
text
}
}
}
}
}
}
}
}
}
As you see on line 3 there is my attempt of guessing interpolation syntax, but it does not work - I leave it as an illustration of my intention.
I could provide a fully qualified name for a revision, but that doesn't seem particularly elegant. Is there any GraphQL native way of manipulating strings?
I don't think there's anything in the GraphQL specification that specifically outlines any methods for manipulating string values within a query.
However, when utilizing GraphQL queries within an actual application, you will provide most of the arguments for your query by utilizing variables that are passed alongside your query inside your request. So rather than being done inside your query, most of your string manipulation will be done within your client code when composing the JSON that will represent your variables.

Rspec Puppet: Defined Type iteration

Using Puppet 3
Testing using rspec-puppet
Iterating over an array of hashes using a Defined Type
Getting an Error, telling me that my parameter (which defaults to the value of $title) cannot be accessed the way I am because it is not an Array or Hash
I'm using old-style iteration in a puppet module, creating a defined type to iterate over an array of hashes. I'm trying to write a test for this define in rspec-puppet, attempting to assign a hash to the :title using let(). The $title is then supposed to be set to my variable called $daemon, yet my tests keep throwing errors saying that $daemon is not a hash or array.
Here's how I'm creating my defined type:
define my_module::daemon_install ($daemon = $title) {
package {"${daemon['package_name']}":
ensure => "${daemon['package_version']}",
}
file {"${some_fact}/${daemon['binary']}.conf":
ensure => file,
content => "blah"
notify => Service["${daemon['name']}"],
}
service {"${daemon['name']}":
ensure => running,
enable => true,
}
}
And here's how I'm trying to set the title:
describe 'my_module::daemon_install' do
context 'with foo' do
let(:title) {
{
"name" => "foo",
"package_name" => "bar",
"package_version" => "1.0.1",
"binary" => "food",
}
}
# ...
end
end
And here's the error:
daemon is not a hash or array when accessing it with package_version
I'm actually abit new to using defined types for iteration, and very new at rspec-puppet, so I'm not sure if I'm missing something obvious here or not.
But why is it only complaining about package_version and not package_name? And more importantly: why is it not a hash, when (I believe) I'm setting it to a hash correctly in the spec file.
I should mention that another test, of a class which uses this defined type, completes successfully. So it seems related to how I'm trying to set the title when directly testing the define, if I were to guess.
Rspec always converts title into String.
Use $name in define() instead of $title and add the following into tests:
let :title do
{ ... }
end
let :params do
{ :name => title }
end
Please note$name should be equal of $title.

Resources