Puppet: Can $hostname be checked against a master file before running manifest head? - puppet

I've seen someone doing a check on whether an agent's MAC address is on a specific regular expression before it runs the specified stuff below. The example is something like this:
if $is_virtual == "true" and $kernel == "Linux" and $macaddress =~ /^02:00:0A/ {
include nmonitor
include rootsh
include checkmk-agent
include backuppcacc
include onecontext
include sysstatpkg
include ensurekvmsudo
include cronntpdate
}
That's just it in that particular manifest file. Similarly another manifest example but via regular expression below:
node /^mi-cloud-(dev|stg|prd)-host/ {
if $is_virtual == 'false' {
include etchosts
include checkmk-agent
include nmonitor
include rootsh
include sysstatpkg
include cronntpdate
include fstab-ds-dev
}
}
I've been asked of whether can that similar concept be applied upon checking the agent's hostname with a master file of hostnames allowed to be run or otherwise.
I am not sure whether it can be done, but the rough idea goes around something like:
file { 'hostmasterfile.ini'
ensure => present,
source => puppet:///test/hostmaster.ini,
content => $hostname
}
$coname = content
#Usually the start / head of the manifest
if $hostname == $coname {
include <a>
include <b>
}
Note: $fqdn is out of the question.
To my knowledge, I have not seen any such sample manifest that matches the request. Whats more, it goes against a standard practice of keeping things easier to manage and not putting all eggs in a basket.
An ex-colleague of mine claims that idea above is about self-provisioning. However that concept is non-existent in Puppet (he posed that question at a workshop a few months back). I am not sure how true is that though.
If that thing above can be done, any suggestion of how can it be done? Or is it best to go back to the standard one manifest per node for easy maintenance?
Thanks very much.
M

Well, you can replace your node blocks with if constructs.
if $hostname == 'host1' {
# manifest for host1 here
}
You can combine this with some sort of inifile (e.g., using the generate) function. If the <a> and <b> for the include statements are then fetched from your ini file as well, you have constructed a crude ENC.
Note that this has security implications - any agent can claim to have any host name. It's even very simple to do:
FACTER_hostname=kerberos01 puppet agent --test
Any node can receive the catalog for kerberos01 this way. (node blocks rely on $certname instead, which cannot be forged.)
I could not decipher your precise intent from your question, but I suspect that you really want an ENC or a Hiera based approach.
Edit after feedback from your first comment:
To make the master read contents from local files, you should
get rid of the file { 'hostmasterfile.ini': } - it only allows you to set contents, not retrieve them
initialize the variable content using the file function (this will make all nodes fail if the file is not readable)
The code could look like this (assuming that there can be multiple host names in the ini file).
$ini_data = file('/etc/puppet/files/test/hostmaster.ini')
Next step would be a regex lookup like this:
if $ini_data =~ /name=$hostname/ {
Unfortunately, this does not work! Puppet will not expand variable values in regular expressions, apparently.
You can use this (kind of silly) workaround:
$ini_lookup = regsubst($ini_data, "name=$hostname", '__FOUND__')
if $ini_lookup =~ /__FOUND__/ {
...
}
Final remark about security: If your team is adamant about not using $certname for this lookup (although it should be easy to map host names to cert names), you should consider adding the host name to your trusted facts.

Related

Terraform - Rendering Local Values and Variables to speed up development

Is there a way to render the values of Local Values and Variables? As an example, I have this file
variable "foo" {
type = string
default = "bar"
}
locals {
my_var = "here is ${var.foo}"
}
# Rendered:
# my_var = "here is bar"
Is there a quick and easy way to do it? I've tried terraform console but it's hard to use it with complicated use-cases, such as templatefile, jsonencode, jsondecode, merge and the list goes on and on.
I need this capability to test functions that I'm not yet familiar with. Doing terraform apply for just checking how my functions are working is something that I'm trying to avoid.
I needed something like https://www.katacoda.com/courses/terraform/playground but for local usage with fast results.
I created this project - https://github.com/unfor19/tfcoding - which brings up a Docker container that watches for changes in the file tfcoding.tf and renders the Local Values automatically upon saving this file.
Demo:

Puppet : How to load file from agent - Part 2

I am trying to load the contents of a json file and assign them to variables.
My json file looks like this :
{ "master":{ "key1":"value1", "key2":"value2", "key3":"value3" } }
On my local machine, I was able to use the following manifest to load the json file and parse it ; it worked just fine.
$master_hash=loadjson('some_file.json')
$key1=$master_hash['master']['key1']
$key2=$master_hash['master']['key2']
$key3=$master_hash['master']['key3']
However, when I move it to the Puppet master, this fails as it looks for the json file on the Puppet master ! In my earlier request, Puppet : How to load file from agent, I was told to use a function and that worked fine for one fact, but in this case I need to generate a number of them depending on the contents of the json file. How can I achieve this ?
Functions like loadjson() execute on the machine which is compiling the catalog. In the majority of cases this means that the function executes on the master. Since some_file.json doesn't exist on the master it won't load the file.
If you want to transfer information from the agent to the master then you need to use a fact to do so. Facts are synced to the agent machine and executed at the start of the run, and their values are sent back to the master.
The answer to your previous question was a good base, but I'll expand on it a bit here:
# module_name/lib/facter/master_hash.rb
require 'json'
Facter.add(:master_hash) do
setcode do
# return content of foo as a string
f = File.read('/path/to/some_file.json')
master_hash = JSON.parse(f)
master_hash
end
end
The last line of the setcode block gets returned as the value of the fact. In this case it would expose a $::master_hash fact which would contain a hash from the parsed json.

Puppet create variable names using hiera

I want Puppet to create a different variable name depending on the hiera file associated with the environment. I want to do this because I want Puppet to use the ip address associated with a specific network interface. Ideally, the network interface will be in the hiera file. That way you could concatenate the ip_address variable name with the network interface defined in the hiera file, which would look something like.
::ipaddress_{$network_interface_from_hiera_file}
Is this possible?
Right now I have an the following, but I think there is a better implementation. If the network interfaces change I would have to add another case.
if $environment == 'production' {
$client_address = $::ipaddress_enp130s0f0
} else {
$client_address = $::ipaddress_eth2
}
It sounds like you're after an eval in Puppet, like you have in shell and Perl other languages, and as far as I know, there isn't one.
I would probably just use a custom fact that always returns the IP address I care about. Of course, then you need to solve the problem of how to get the custom facts out to your fleet.
Another solution might be to use Hiera's hierarchical lookup:
In hiera.yaml:
:hierarchy:
- %{::node_environment}
- common
In common.yaml:
---
myclass::client_address: "%{::ipaddress_eth2}"
In production.yaml:
---
myclass::client_address: "%{::ipaddress_enp130s0f0}"
Finally, be aware that you can look up values from within Hiera, see here. Possibly that could be helpful.

How to edit a file and pass values into it through Puppet

I want to pass limit values in /etc/security/limits.conf in all servers available in environment through puppet. Need to be automated this process whenever i create new instance, this limit values directly append to new boxes
There's an official puppet module for modifying limits.
You could also use file_line for this.
file_line { 'append_limits_conf':
path => '/etc/security/limits.conf',
match => 'variable_name',
line => 'vriable_name = foobar',
}
The match will be used to determine if the variable is already declared within the file. If yes - it will change the value to whatever you define at line. If it's not in there it will just append the line.
More sophisticated file editing can be done with augeas - but might be overkill for you case right now.

Perl program structure for parsing

I've got question about program architecture.
Say you've got 100 different log files with different formats and you need to parse and put that info into an SQL database.
My view of it is like:
use general config file like:
program1->name1("apache",/var/log/apache.log) (modulename,path to logfile1)
program2->name2("exim",/var/log/exim.log) (modulename,path to logfile2)
....
sqldb->configuration
use something like a module (1 file per program) type1.module (regexp, logstructure(somevariables), sql(tables and functions))
fork or thread processes (don't know what is better on Linux now) for different programs.
So question is, is my view of this correct? I should use one module per program (web/MTA/iptablat)
or there is some better way? I think some regexps would be the same, like date/time/ip/url. What to do with that? Or what have I missed?
example: mta exim4 mainlog
2011-04-28 13:16:24 1QFOGm-0005nQ-Ig
<= exim#mydomain.org.ua** H=localhost
(exim.mydomain.org.ua)
[127.0.0.1]:51127 I=[127.0.0.1]:465
P=esmtpsa
X=TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32
CV=no A=plain_server:spam S=763
id=1303985784.4db93e788cb5c#mydomain.org.ua T="test" from
<exim#exim.mydomain.org.ua> for
test#domain.ua
everything that is bold is already parsed and will be putted into sqldb.incoming table. now im having structure in perl to hold every parsed variable like $exim->{timstamp} or $exim->{host}->{ip}
my program will do something like tail -f /file and parse it line by line
Flexability: let say i want to add supprot to apache server (just timestamp userip and file downloaded). all i need to know what logfile to parse, what regexp shoud be and what sql structure should be. So im planning to have this like a module. just fork or thread main process with parameters(logfile,filetype). Maybe further i would add some options what not to parse (maybe some log level is low and you just dont see mutch there)
I would do it like this:
Create a config file that is formatted like this: appname:logpath:logformatname
Create a collection of Perl class that inherit from a base parser class.
Write a script which loads the config file and then loops over its contents, passing each iteration to its appropriate handler object.
If you want an example of steps 1 and 2, we have one on our project. See MT::FileMgr and MT::FileMgr::* here.
The log-monitoring tool wots could do a lot of the heavy lifting for you here. It runs as a daemon, watching as many log files as you could want, running any combination of perl regexes over them and executing something when matches are found.
I would be inclined to modify wots itself (which its licence freely allows) to support a database write method - have a look at its existing handle_* methods.
Most of the hard work has already been done for you, and you can tackle the interesting bits.
I think File::Tail is a nice fit.
You can make an array of File::Tail objects and poll them with select like this:
while (1) {
($nfound,$timeleft,#pending)=
File::Tail::select(undef,undef,undef,$timeout,#files);
unless ($nfound) {
# timeout - do something else here, if you need to
} else {
foreach (#pending) {
# here you can handle log messages depending on filename
print $_->{"input"}." (".localtime(time).") ".$_->read;
}
(from perl File::Tail doc)

Resources