I am using masterless Puppet. And My situation is that I am using a custom Hiera backend called hiera-regex. The process of using it is that you have to have hiera-regex installed on your target machine.
If sometime for some reason, hiera-regex is not installed, hiera still looks up for the key in hierarchy and starts using that key which it was not suppose to use in the presence of hiera-regex. Ideally, I would want Puppet to exit when it does not find hiera-regex backend installed. But as of now, Puppet only prints a notice() which can be easily missed and can leave your node in an unwanted state.
Is it possible to somehow configure Puppet/hiera for masterless setup to not continue at all if the backend hiera is configured to use is not installed?
Here is my hiera.yaml file:
---
:backends:
- regex # this is the custom backend (hiera-regex)
- yaml
:regex:
:datadir: /etc/puppet/hiera
:yaml:
:datadir: /etc/puppet/hiera
:hierarchy:
- "%{fqdn}"
- base
Related
I recently upgraded puppet version 3 to version 5. all is working fine with the new version but hiera configurations for puppet 5 is not working as expected. I think I missing something which would deploy changes in the remote node. Please advise what should I do here. below are the configurations for my setup.
1) Hiera.yaml
cat /etc/puppetlabs/code/environments/hiera.yaml
version: 5
hierarchy:
- name: "Master"
path: "environments/%{environment}/data/%{trusted.certname}.yaml"
data_hash: yaml_data
datadir: /etc/puppetlabs/code/
2) And my Environment YAML files are kept at
cat /etc/puppetlabs/code/environments/staging/data/puppetsr7.demo.com.yaml
demo::configuration::phpini::memory_limit: '64'
3) but when I run the command on my remote node, nothing is changing
/opt/puppetlabs/bin/puppet agent
4) In order to troubleshoot I tried to run the command
puppet lookup --explain demo::configuration::phpini::memory_limit --environment staging --node puppetsr7.demo.com
and got below output
Searching for "lookup_options"
Global Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/code/environments/hiera.yaml"
Hierarchy entry "Master"
Path "/etc/puppetlabs/code/environments/staging/data/puppetsr7.demo.com.yaml"
Original path: "environments/%{environment}/data/%{trusted.certname}.yaml"
Found key: "lookup_options" value: nil
Module data provider for module "demo" not found
Searching for "demo::configuration::phpini::memory_limit"
Global Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/code/environments/hiera.yaml"
Hierarchy entry "Master"
Path "/etc/puppetlabs/code/environments/staging/data/puppetsr7.demo.com.yaml"
Original path: "environments/%{environment}/data/%{trusted.certname}.yaml"
Found key: "demo::configuration::phpini::memory_limit" value: "64"
It's showing the proper value when running from CLI i.e 64 which I need to be get applied on a remote node in php.ini and change the value from 512 to 64.
But don't know how to proceed further from here as I struck now. please help to troubleshoot this.
What I did is I kept the required class in site.pp file as well which I want to get executed through hieradata.
"demo::configuration::phpini::memory_limit: '64'" in hiera file and "demo::configuration::phpini::memory_limit in site.pp.
Hoping that some one could get help from it.
Background
I am using puppet5 & hiera5 on a puppet master from separate git repos.
The code is deployed using r10k into their respective locations using the branch to environment directory mapping.
Due to the environment conflict limitation in r10k (apologies if this is no longer the case) I use a separate location for my hieradata:
Puppet code - /etc/puppetlabs/code/envirnments/
Hiera code - /etc/puppetlabs/code/hieradata/
I have a global hiera.yaml file in /etc/puppetlabs/puppet/hiera.yaml and an environment specific hiera.yaml file in my hiera branch / environment location.
e.g. /etc/puppetlabs/code/hieradata/production/hiera.yaml
Problem
My environment hiera.yaml is ignored due to its location, it is expected to be where my puppet <ENVIRONMENT> code is:
/etc/puppetlabs/code/environments/production/hiera.yaml
taken from https://docs.puppet.com/puppet/5.0/hiera_config_yaml_5.html#location
but instead it is:
/etc/puppetlabs/code/hieradata/production/hiera.yaml
Solution
The only solution that I have been able to find is to locate a hiera.yaml in my puppet code repo with a datadir set to
datadir: ../../hieradata/%{::environment}/hieradata
(it must be relative)
Which works ok but is not ideal so I am looking for other solutions.
Even if I set the global hiera.yaml datadir to:
/etc/puppetlabs/code/hieradata/%{::environment}/
(it can be a full path)
and do not have a hiera.yaml file in my puppet repo, hiera works, but the environmental hiera.yaml located there is ignored.
Another idea was to swap the locations but then my puppet code does not work.
So ideally what I need to do is set the environment layer hier.yaml location as it seems to be hard coded to <ENVIRONMENT>, which is where my puppet code is.
I still want to use r10k and I want to host my environmental hiera.yaml in my hiera repo.
my working code:
datadir: "/etc/puppetlabs/code/environments/%{environment}/hiera"
but path should be correct.
I have an agent/master setup. I have created a new environment in /etc/puppetlabs/code/environments/ called master.
The content of environment.conf for the master directory environment is
modulepath = site:modules:$basemodulepath
manifest = manifests/site.pp
and when I try puppet agent -t --environment master I am getting some error
Notice: Local environment: 'master' doesn't match server specified node environment 'production', switching agent to 'production'.
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for node1.localpuppet.com
Info: Applying configuration version '1490712072'
Notice: Applied catalog in 0.67 seconds
I am new to puppet. What changes do I need?
PE Console Config
This is a "really fun" quirk of Puppet Enterprise that showed up in the last couple of years. You have to specify the nodes in the PE Classifier that are allowed to specify their directory environment in the puppet.conf or in the puppet agent -t --environment arguments.
In the agent-specified environment tab in the Classifier (you see it at the bottom of your picture above), you can enable it for all nodes. Do this by adding a rule, selecting the name fact, using a regular expression (~), then using the regexp for matching all characters (.*). After you fill this out, the PE Classifier will give you a number of matching nodes. It should be all that are subscribed to your master. Remember to click in the bottom right to update your rules. Your nodes will now be able to use master instead of production from the config file or CLI arguments.
That being said, if you are doing this to avoid naming your default Git branch production in your control repository when working with Code Manager, you should really just rename the branch as that is much easier.
just testing out hiera and I'd like to be able to view all the available data (variable=value pairs) in the hierarchy for a given node.
My Hiera hierarchy is configured as:
---
:backends:
- yaml
- json
:yaml:
:datadir: C:\Puppet\hieradata
:hierarchy:
- "Env/%{::env}"
- common
I can run the following to return node1's value for 'some-common' variable :
>hiera some-common ::env=node1
data
What I'd like to be able to see is all the variable=value pairs available to node1 in the hierarchy, is this possible?
Thanks
I'm afraid this is not possible. Closest thing you could do is dump facts for specific node:
facter -y > node.yml
And then use them for look for specific keys:
hiera -y node.yml my_class:arg -d
this way you will be able to access Hiera keys based on operating system, domain, etc. (depends on your hierarchy defined in hiera.yaml).
Yet another option is to ssh into puppet master node. And use puppet lookup (should be available since Puppet 4). lookup is using by default Hiera backend (again requires hiera.yaml config file).
puppet lookup resolv_conf::nameservers --node mynode.example.net
or more verbose version:
puppet lookup resolv_conf::nameservers --merge deep --environment production --explain --node mynode.example.net
I found another workaround - you can add top level key to your yaml data:
node-data:
hosts:
- localhost:3367
- company.com
dns: 8.8.8.8
policy:
retries: 3
timeout: 5
and do puppet lookup with merge for this top level key, e.g.:
puppet lookup --merge hash/deep node_data
I am trying to create a custom provider for package but for some reasons I keep on getting
err: Could not run Puppet configuration client: Parameter provider
failed: Invalid package provider 'piprs' at
/usr/local/src/ops/services/puppet/modules/test/manifests/init.pp:5
I have added pluginsync=true in puppet.conf in both client and server. I have created the following rb file in module/test/lib/puppet/provider/package/piprs.rb. I am basically trying to create a custom provider for package resource type
#require 'puppet/provider/package'
Puppet::Type.type(:package).provide(:piprs,
:parent => ::Puppet::Provider::Package) do
commands : pip => "/usr/local/bin/pip"
desc "Python packages via `pip`."
def create
pip "freeze"
end
def destroy
end
def exists?
end
end
In the puppet.conf, there is the following source attribute
pluginsource = puppet://puppet/plugins
I am not sure what it is. If you need anymore details, please do post a comment.
First things first - you do realize there is already a Python pip provider in core?
https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb
If that isn't what you want - then lets move on ...
For starters - try your module without a Puppet master - this is going to be better for development anyway. You need to make sure Ruby can find the library path:
export RUBYLIB=<path_to_module>/lib
Then, try writing a small test in a .pp file:
package { "mypackage": provider => "piprs" }
And run it locally:
puppet apply mytest.pp
This will rule out a code bug in your provider versus a plugin sync issue.
I notice there is a space between the colon and the command - that isn't your problem is it?
commands : pip => "/usr/local/bin/pip"
If you can get this working without a puppetmaster, your problem is sync related.
There are a couple of things that can go wrong - make sure the file is sync'd properly on the client:
ls /var/lib/puppet/lib/puppet/provider/package
You should see the piprs.rb file there. If it is, you may need to make sure your libdir is set correctly:
puppet --configprint libdir
This should point to /var/lib/puppet/lib in most cases.