How to use properly exported resources with Puppet - puppet

I've been thinking for hours on how to solve a problem with Puppet 4.
Here is my case :
I have a module "Cassandra", and I have 3 machines
Cluster1 (Hostname : CassandraCluster1)
Cluster2 (Hostname : CassandraCluster1)
Cluster3 (Hostname : CassandraCluster1)
I want to collect the hostnames of three clusters in an array, so I can pass them to the cassandra configuration file (which I'm using as a template epp) :
cassandra.yaml.epp
- seeds: "<%= $cluster::hostnames %>"
So the solution is Exported Resources I came up with :)
I've been playing with all day, but no idea how to make up this work. here is what I've tried :
on each cluster I add this code to collect the hostnames :
##file {"${hostname}":
content => 'epp(puppet://modules/cassandra/cassandra.yaml.epp)',
}
# Collect:
File <<| |>>
But I'm not sure if this is a good idea ?!

I've found a temporary solution that I'm using now :
I have a role fact which I'm adding to each machine. and I'm retrieving the hosts from puppetdb using puppetdbquery module.
and then on puppet code I use this query to find the nodes with the role attached
$hosts = query_nodes("role=apache")

To the extent that the question is how to create the wanted file with use of exported resources, it's important to understand that collecting exported resources causes them to be added to the target node's catalog. Exporting and collecting resources is not merely a data-transfer excercise. Your example exports several File resources; collecting these will result in the same number of separate files being managed on the target (or else a duplicate resource error if you happen to have a resource title collision).
For a long time, the usual way to build a file based on pieces contributed by multiple nodes was to use the puppetlabs/concat module, with each contributing node exporting a concat::fragment resource. For example:
##concat::fragment { "${hostname} Cassandra seed":
target => '/path/to/file',
content => " ${hostname}",
order => '10',
tag => 'cassandra'
}
Those would be collected into the catalog for the target node, to be used in conjunction with a corresponding concat resource declared there:
concat { '/path/to/file':
ensure => 'present',
# ...
}
concat::fragment { "Cassandra seed start":
target => '/path/to/file',
content => ' - seeds: "'
order => '05',
}
Concat::Fragment <<| tag == 'cassandra' |>>
concat::fragment { "Cassandra seed tail":
target => '/path/to/file',
content => '"'
order => '15',
}
That still works just fine, but it is becoming more common to rely instead on querying puppetdb, as you demonstrate in your own answer.

Related

Puppet - How to write yaml files based on Role/Profile method

I've added our infrastructure setup to puppet, and used roles and profiles method. Each profile resides inside a group, based on their nature. For example, Chronyd setup and Message of the day are in "base" group, nginx-related configuration is in "app" group. Also, on the roles, each profile is added to the corresponding group. For example for memcached we have the following:
class role::prod::memcache inherits role::base::debian {
include profile::app::memcache
}
The profile::app::memcached has been set up like this :
class profile::app::memcache {
service { 'memcached':
ensure => running,
enable => true,
hasrestart => true,
hasstatus => true,
}
}
and for role::base::debian I have :
class role::base::debian {
include profile::base::motd
include profile::base::chrony
}
The above structure has proved to be flexible enough for our infrastructure. Adding services and creating new roles could not been easier than this. But now I face a new problem. I've been trying to separate data from logic, write some yaml files to keep the data there, using Hiera version 5. Been looking through internet for a couple of days, but I cannot deduct how to write my hiera files based on the structure I have. I tried adding profile::base::motd to common.yaml and did a puppet lookup, it works fine, but I could not append chrony to common.yaml. Puppet lookup returns nothing with the following common.yaml contents :
---
profile::base::motd::content: This server access is restricted to authorized users only. All activities on this system are logged. Unauthorized access will be liable to prosecution.'
profile::base::chrony::servers: 'ntp.centos.org'
profile::base::chrony::service_enable: 'true'
profile::base::chrony::service_ensure: 'running'
Motd lookup works fine. But the rest, no luck. puppet lookup profile::base::chrony::servers returns with no output. Don't know what I'm missing here. Would really appreciate the community's help on this one.
Also, using hiera, is the following enough code for a service puppet file?
class profile::base::motd {
class { 'motd':
}
}
PS : I know I can add yaml files inside modules to keep the data, but I want my .yaml files to reside in one place (e.g. $PUPPET_HOME/environment/production/data) so I can manage the code with git.
The issue was that in init.pp file inside the puppet module itself, the variable $content was assigned a value. Removing the value fixed the problem.

Triggering dependent resources in a interation loop

I'm using Puppet to set up workstations and I want to modify the default (NTUSER.DAT) HKLM registry before the user logs on, which involves loading and unloading the hive. I have written some PowerShell scripts to facilitate the load/unload. Although I have three distinct actions, it appears that Puppet is trying to unload the hive before the registry module can make all the changes. I believe I need to add some dependencies using subscribe and refreshonly.
This question is very similar to this one, with the exception that my data is in Hiera, therefore I want to iterate over the data.
$temp_hive_name = $base_windows::temp_hive_name
# LOAD REGISTRY HIVE
exec { 'load_registry_hive' :
command => template('base_windows/Load-RegHive.ps1.erb'),
unless => template('base_windows/Test-HiveLoadState.ps1.erb'),
provider => powershell,
logoutput => true,
}
# MODIFY REGISTRY, ITERATING OVER HIERA DATA
$base_windows::registry.each | $key, $value | {
registry::value { "registry_${key}" :
key => "${value['key']}\\${temp_hive_name}\\${value['subkey']}",
type => $value['type'],
data => $value['data'],
value => $value['value'],
}
}
# UNLOAD REGISTRY HIVE
exec { 'unload_registry_hive' :
command => template('base_windows/Unload-RegHive.ps1.erb'),
onlyif => template('base_windows/Test-HiveLoadState.ps1.erb'),
provider => powershell,
logoutput => true,
}
This works fine when there are one or two Hiera entries.
I guess I could put the load / unload exec resources into an .each loop and add subscribe and refreshonly, however, it seems rather inefficient to do that for each item.
If anyone has any ideas, I'd be grateful if you could share?
T.I.A.
I believe I need to add some dependencies using subscribe and refreshonly.
I'm not so sure that you need to add dependencies, because without explicit dependencies, resources should be applied in the relative order in which they appear in the manifest. Additionally, refreshonly does not declare a dependency, and subscribe is probably not appropriate for this particular task. Furthermore, although refreshonly works in conjunction with dependencies, it's probably not appropriate for this task, either, because notify / subscribe is not right for it.
In a general sense, the key issues are these:
the hive must be loaded before you can attempt to sync any registry entries, so you cannot know whether any given registry resource is out of sync without loading the hive first;
if the hive is loaded then it must also be unloaded;
but the hive must not be unloaded before all the registry entries are synced.
You cannot make Exec['load_registry_hive'] refreshonly because there is no resource that would signal it. You can, however, check whether $base_windows::registry has any elements as a precondition for doing any of the work. If it does, then you definitely need to load the hive.
You can set up explicit dependencies, and I'm generally inclined to do that, as it protects against surprises when a resource is affected by dependency edges that are not apparent at the point of its declaration. So I would suggest this:
$temp_hive_name = $base_windows::temp_hive_name
if ! $base_windows::registry.empty() {
# LOAD REGISTRY HIVE
exec { 'load_registry_hive' :
command => template('base_windows/Load-RegHive.ps1.erb'),
unless => template('base_windows/Test-HiveLoadState.ps1.erb'),
provider => powershell,
logoutput => true,
}
# MODIFY REGISTRY, ITERATING OVER HIERA DATA
$base_windows::registry.each | $key, $value | {
registry::value { "registry_${key}" :
key => "${value['key']}\\${temp_hive_name}\\${value['subkey']}",
type => $value['type'],
data => $value['data'],
value => $value['value'],
require => Exec['load_registry_hive'],
before => Exec['unload_registry_hive'],
}
}
# UNLOAD REGISTRY HIVE
exec { 'unload_registry_hive' :
command => template('base_windows/Unload-RegHive.ps1.erb'),
onlyif => template('base_windows/Test-HiveLoadState.ps1.erb'),
provider => powershell,
logoutput => true,
}
}
Note that you will necessarily both load and unload the hive on each Puppet run, because you cannot determine whether any entries need to be updated without doing so.

How can I use Foreman host groups with Puppet?

I have this manifest:
$foremanlogin = file('/etc/puppetlabs/code/environments/production/manifests/foremanlogin.txt')
$foremanpass = file('/etc/puppetlabs/code/environments/production/manifests/foremanpass.txt')
$query = foreman({foreman_user => "$foremanlogin",
foreman_pass => "$foremanpass",
item => 'hosts',
search => 'hostgroup = "Web Servers"',
filter_result => 'name',
})
$quoted = regsubst($query, '(.*)', '"\1"')
$query6 = join($quoted, ",")
notify{"The value is: ${query6}": }
node ${query6} {
package { 'atop':
ensure => 'installed',
}
}
When I execute this on agent I got error:
Server Error: Could not parse for environment production: Syntax error at ''
Error in my node block
node ${query6} {
package { 'atop':
ensure => 'installed',
}
}
I see correct output from notify, my variable looks like this:
"test-ubuntu1","test-ubuntu2"
Variable in correct node manifest format.
I don't understand whats wrong? variable query6 is correct.
How to fix that?
I just want to apply this manifest to foreman host group, how to do this right?
On the Puppet side, you create classes describing how to manage appropriate subunits of your machines' overall configuration, and organize those classes into modules. The details of this are far too broad to cover in an SO answer -- it would be analogous to answering "How do I program in [language X]?".
Having prepared your classes, the task is to instruct Puppet which ones to assign to each node. This is called "classification". Node blocks are one way to perform classification. Another is external node classifiers (ENCs). There are also alternatives based on ordinary top-level Puppet code in your site manifest. None of these are exclusive.
If you are running Puppet with The Foreman, however, then you should configure Puppet to use the ENC that Foreman provides. You then use Foreman to assign (Puppet) classes to nodes and / or node groups, and Foreman communicates the details to Puppet via its ENC. That does not require any classification code on the Puppet side at all.
See also How does host groups work with foreman?

Checking for custom file using puppet

Part of my puppet manifest checks for the existence of a custom sshd_config. If one is found, I use that. If it's not then I use my default. I'm just wondering if there is a more "puppet" way of doing this
if file("/etc/puppetlabs/puppet/files/${::fqdn}/etc/ssh/sshd_config", '/dev/null') != '' {
$sshd_config_source = "puppet:///private/etc/ssh/sshd_config"
} else {
$sshd_config_source = "puppet:///public/etc/ssh/sshd_config"
}
file { '/etc/ssh/sshd_config':
ensure => 'present',
mode => '600',
source => $sshd_config_source,
notify => Service['sshd'],
}
This code works but it's a little odd as file I have to give it the full path on the puppet master but when assigning $sshd_config_source I have to use the puppet fileserver path (puppet:///private/etc...).
Is there a better way of doing this?
It's a little known feature of the file type that you can supply multiple source values.
From the docs:
Multiple source values can be specified as an array, and Puppet will use the first source that exists. This can be used to serve different files to different system types:
file { "/etc/nfs.conf":
source => [
"puppet:///modules/nfs/conf.$host",
"puppet:///modules/nfs/conf.$operatingsystem",
"puppet:///modules/nfs/conf"
]
}
So you should just specify both the specific and the generic file URL, in that order, and Puppet will do the right thing for you.

What is ## called in puppet, and what does it do?

I'm learning puppet, and have come across a part of haproxy configuration which looks like the following
##haproxy::balancermember { $::fqdn:
listening_service => 'puppet00',
server_names => $::hostname,
ipaddresses => $::ipaddress,
ports => '8140',
options => 'check',
}
I'm trying to work out what the ## is called and what it does in this config
The manifest is declaring an exported resource for use by another node.
This allows you to create dynamic configurations that adapt to the shifting set of nodes in a given setup.

Resources