I am converting one of my masterless modules to use Consul. How do I use external facts to pass in the Consul host and Consul token? These change in every environment and are not managed by Puppet. I am using the Puppet module 'lynxman-hiera_consul', '0.1.2'. Note that I had to downgrade my hiera.yaml to version 3 to use it with this module.
Before my Puppet masterless run I export some facts
export FACTER_CONSULHOST=consul-randomid..us-west-2.elb.amazonaws.com
export FACTER_MYTOKEN=some-token
I can test this works with
facter mytoken; puppet facts --debug|grep mytoken
facter consulhost;puppet facts --debug|grep consulhost
My hiera.yaml looks like this Hiera.yaml Gist. This works fine if I replace the fact interpolation with strings.
With the basic issue being with the fact interpolation on line 15
:token: "%{facts.mytoken}"
This is my example manifest for testing this
Consul.pp Gist
Related
Background
I am using puppet5 & hiera5 on a puppet master from separate git repos.
The code is deployed using r10k into their respective locations using the branch to environment directory mapping.
Due to the environment conflict limitation in r10k (apologies if this is no longer the case) I use a separate location for my hieradata:
Puppet code - /etc/puppetlabs/code/envirnments/
Hiera code - /etc/puppetlabs/code/hieradata/
I have a global hiera.yaml file in /etc/puppetlabs/puppet/hiera.yaml and an environment specific hiera.yaml file in my hiera branch / environment location.
e.g. /etc/puppetlabs/code/hieradata/production/hiera.yaml
Problem
My environment hiera.yaml is ignored due to its location, it is expected to be where my puppet <ENVIRONMENT> code is:
/etc/puppetlabs/code/environments/production/hiera.yaml
taken from https://docs.puppet.com/puppet/5.0/hiera_config_yaml_5.html#location
but instead it is:
/etc/puppetlabs/code/hieradata/production/hiera.yaml
Solution
The only solution that I have been able to find is to locate a hiera.yaml in my puppet code repo with a datadir set to
datadir: ../../hieradata/%{::environment}/hieradata
(it must be relative)
Which works ok but is not ideal so I am looking for other solutions.
Even if I set the global hiera.yaml datadir to:
/etc/puppetlabs/code/hieradata/%{::environment}/
(it can be a full path)
and do not have a hiera.yaml file in my puppet repo, hiera works, but the environmental hiera.yaml located there is ignored.
Another idea was to swap the locations but then my puppet code does not work.
So ideally what I need to do is set the environment layer hier.yaml location as it seems to be hard coded to <ENVIRONMENT>, which is where my puppet code is.
I still want to use r10k and I want to host my environmental hiera.yaml in my hiera repo.
my working code:
datadir: "/etc/puppetlabs/code/environments/%{environment}/hiera"
but path should be correct.
I am trying out a masterless process for setting up an OpenStack cloud. I use the puppetlabs-haproxy (v1.5.0) module for setting up HAProxy. When I run my puppet manifest for the controller node like so ...
puppet apply -e 'include controller'
I get this warning:
Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored at /etc/puppetlabs/code/environments/production/modules/haproxy/manifests/balancermember/collect_exported.pp:7:3
I have tried setting storeconfigs to true like this:
puppet config set storeconfigs true
But that results in this error:
Error: Cached facts for aos-mgmt-01 failed: Could not find terminus puppetdb for indirection facts
Error: Could not find terminus puppetdb for indirection facts
Error: Could not run: Could not retrieve facts for aos-mgmt-01: Could not find terminus puppetdb for indirection facts
I guess I can just ignore the warning but does anyone know what I can do to correct or suppress the warning?
Thanks
The puppetlabs haproxy module is collecting exported ressource (like this : https://github.com/puppetlabs/puppetlabs-haproxy/blob/master/manifests/balancermember/collect_exported.pp#L7)
You need a puppetdb to collect exported resources, so your warning is normal.
You can either ignore it, or use a module that doesn't need exported resources.
I have an agent/master setup. I have created a new environment in /etc/puppetlabs/code/environments/ called master.
The content of environment.conf for the master directory environment is
modulepath = site:modules:$basemodulepath
manifest = manifests/site.pp
and when I try puppet agent -t --environment master I am getting some error
Notice: Local environment: 'master' doesn't match server specified node environment 'production', switching agent to 'production'.
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for node1.localpuppet.com
Info: Applying configuration version '1490712072'
Notice: Applied catalog in 0.67 seconds
I am new to puppet. What changes do I need?
PE Console Config
This is a "really fun" quirk of Puppet Enterprise that showed up in the last couple of years. You have to specify the nodes in the PE Classifier that are allowed to specify their directory environment in the puppet.conf or in the puppet agent -t --environment arguments.
In the agent-specified environment tab in the Classifier (you see it at the bottom of your picture above), you can enable it for all nodes. Do this by adding a rule, selecting the name fact, using a regular expression (~), then using the regexp for matching all characters (.*). After you fill this out, the PE Classifier will give you a number of matching nodes. It should be all that are subscribed to your master. Remember to click in the bottom right to update your rules. Your nodes will now be able to use master instead of production from the config file or CLI arguments.
That being said, if you are doing this to avoid naming your default Git branch production in your control repository when working with Code Manager, you should really just rename the branch as that is much easier.
just testing out hiera and I'd like to be able to view all the available data (variable=value pairs) in the hierarchy for a given node.
My Hiera hierarchy is configured as:
---
:backends:
- yaml
- json
:yaml:
:datadir: C:\Puppet\hieradata
:hierarchy:
- "Env/%{::env}"
- common
I can run the following to return node1's value for 'some-common' variable :
>hiera some-common ::env=node1
data
What I'd like to be able to see is all the variable=value pairs available to node1 in the hierarchy, is this possible?
Thanks
I'm afraid this is not possible. Closest thing you could do is dump facts for specific node:
facter -y > node.yml
And then use them for look for specific keys:
hiera -y node.yml my_class:arg -d
this way you will be able to access Hiera keys based on operating system, domain, etc. (depends on your hierarchy defined in hiera.yaml).
Yet another option is to ssh into puppet master node. And use puppet lookup (should be available since Puppet 4). lookup is using by default Hiera backend (again requires hiera.yaml config file).
puppet lookup resolv_conf::nameservers --node mynode.example.net
or more verbose version:
puppet lookup resolv_conf::nameservers --merge deep --environment production --explain --node mynode.example.net
I found another workaround - you can add top level key to your yaml data:
node-data:
hosts:
- localhost:3367
- company.com
dns: 8.8.8.8
policy:
retries: 3
timeout: 5
and do puppet lookup with merge for this top level key, e.g.:
puppet lookup --merge hash/deep node_data
I am using masterless Puppet. And My situation is that I am using a custom Hiera backend called hiera-regex. The process of using it is that you have to have hiera-regex installed on your target machine.
If sometime for some reason, hiera-regex is not installed, hiera still looks up for the key in hierarchy and starts using that key which it was not suppose to use in the presence of hiera-regex. Ideally, I would want Puppet to exit when it does not find hiera-regex backend installed. But as of now, Puppet only prints a notice() which can be easily missed and can leave your node in an unwanted state.
Is it possible to somehow configure Puppet/hiera for masterless setup to not continue at all if the backend hiera is configured to use is not installed?
Here is my hiera.yaml file:
---
:backends:
- regex # this is the custom backend (hiera-regex)
- yaml
:regex:
:datadir: /etc/puppet/hiera
:yaml:
:datadir: /etc/puppet/hiera
:hierarchy:
- "%{fqdn}"
- base