I recently upgraded puppet version 3 to version 5. all is working fine with the new version but hiera configurations for puppet 5 is not working as expected. I think I missing something which would deploy changes in the remote node. Please advise what should I do here. below are the configurations for my setup.
1) Hiera.yaml
cat /etc/puppetlabs/code/environments/hiera.yaml
version: 5
hierarchy:
- name: "Master"
path: "environments/%{environment}/data/%{trusted.certname}.yaml"
data_hash: yaml_data
datadir: /etc/puppetlabs/code/
2) And my Environment YAML files are kept at
cat /etc/puppetlabs/code/environments/staging/data/puppetsr7.demo.com.yaml
demo::configuration::phpini::memory_limit: '64'
3) but when I run the command on my remote node, nothing is changing
/opt/puppetlabs/bin/puppet agent
4) In order to troubleshoot I tried to run the command
puppet lookup --explain demo::configuration::phpini::memory_limit --environment staging --node puppetsr7.demo.com
and got below output
Searching for "lookup_options"
Global Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/code/environments/hiera.yaml"
Hierarchy entry "Master"
Path "/etc/puppetlabs/code/environments/staging/data/puppetsr7.demo.com.yaml"
Original path: "environments/%{environment}/data/%{trusted.certname}.yaml"
Found key: "lookup_options" value: nil
Module data provider for module "demo" not found
Searching for "demo::configuration::phpini::memory_limit"
Global Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/code/environments/hiera.yaml"
Hierarchy entry "Master"
Path "/etc/puppetlabs/code/environments/staging/data/puppetsr7.demo.com.yaml"
Original path: "environments/%{environment}/data/%{trusted.certname}.yaml"
Found key: "demo::configuration::phpini::memory_limit" value: "64"
It's showing the proper value when running from CLI i.e 64 which I need to be get applied on a remote node in php.ini and change the value from 512 to 64.
But don't know how to proceed further from here as I struck now. please help to troubleshoot this.
What I did is I kept the required class in site.pp file as well which I want to get executed through hieradata.
"demo::configuration::phpini::memory_limit: '64'" in hiera file and "demo::configuration::phpini::memory_limit in site.pp.
Hoping that some one could get help from it.
Related
I have a repository with several separate configs which share some modules, and reference those modules using relative paths that look like ../../modules/rabbitmq. The directories are setup like this:
tf/
configs/
thing-1/
thing-2/
modules/
rabbitmq/
cluster/
...
The configs are setup with a remote backend to use TF Cloud for runs and state:
terraform {
backend "remote" {
hostname = "app.terraform.io"
organization = "my-org"
workspaces {
prefix = "config-1-"
}
}
}
Running terraform init works fine. When I try to run terrform plan locally, it gives me an error saying:
Initializing modules...
- rabbitmq in
Error: Unreadable module directory
Unable to evaluate directory symlink: lstat ../../modules: no such file or
directory
...as if the modules directory isn't being uploaded to TF Cloud or something. What gives?
It turns out the problem was (surprise, surprise!) it was not uploading the modules directory to TF Cloud. This is because neither the config nor the TF Cloud workspace settings contained any indication that this config folder was part of a larger filesystem. The default is to upload just the directory from which you are running terraform (and all of its contents).
To fix this, I had to visit the "Settings > General" page for the given workspace in Terraform Cloud, and change the Terraform Working Directory setting to specify the path of the config, relative to the relevant root directory - in this case: tf/configs/config-1
After that, running terraform plan displays a message indicating which parent directory it will upload in order to convey the entire context relevant to the workspace. 🎉
Update #mlsy answer with a screenshot. Using Terraform Cloud with free account. Resolving module source to using local file system.
terraform version
Terraform v1.1.7
on linux_amd64
Here is the thing I worked for me. I used required_version = ">= 0.11"
and then put all those tf files which have provider and module in a subfolder. Kept the version.tf which has required providers at root level. Somehow I have used the same folder path where terraform.exe is present. Then Built the project instead of executing at main.tf level or doing execution without building. It downloaded all providers and modules for me. I am yet to run on GCP.
enter image description here - Folder path on Windows
enter image description here - InteliJ structure
enter image description hereenter image description here
Use this source = "mhmdio/rabbitmq/aws
I faced this problem when I started. Go to hashicorp/terraform site and search module/provider block. They have full path. The code snippets are written this way. Once you get path run Terraform get -update Terraform init - upgrade
Which will download modules and provider locally.
Note: on cloud the modules are in repo but still you need to give path if by default repo path not mapped
I have similar issue, which I think someone might encounter.
I have issue where in my project the application is hosted inside folder1/folder2. However when I run terraform plan inside the folder2 there was an issue because it tried to load every folder from the root repository.
% terraform plan
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.
Preparing the remote plan...
The remote workspace is configured to work with configuration at
infrastructure/prod relative to the target repository.
Terraform will upload the contents of the following directory,
excluding files or directories as defined by a .terraformignore file
at /Users/yokulguy/Development/arepository/.terraformignore (if it is present),
in order to capture the filesystem context the remote workspace expects:
/Users/yokulguy/Development/arepository
â•·
│ Error: Failed to upload configuration files: Failed to get symbolic link destination for "/Users/yokulguy/Development/arepository/docker/mysql/mysql.sock": lstat /private/var/run/mysqld: no such file or directory
│
│ The configured "remote" backend encountered an unexpected error. Sometimes this is caused by network connection problems, in which case you could retry the command. If the issue persists please open a support
│ ticket to get help resolving the problem.
╵
The solution is sometime I just need to remove the "bad" folder whic is the docker/mysql and then rerun the terraform plan and it works.
Background
I am using puppet5 & hiera5 on a puppet master from separate git repos.
The code is deployed using r10k into their respective locations using the branch to environment directory mapping.
Due to the environment conflict limitation in r10k (apologies if this is no longer the case) I use a separate location for my hieradata:
Puppet code - /etc/puppetlabs/code/envirnments/
Hiera code - /etc/puppetlabs/code/hieradata/
I have a global hiera.yaml file in /etc/puppetlabs/puppet/hiera.yaml and an environment specific hiera.yaml file in my hiera branch / environment location.
e.g. /etc/puppetlabs/code/hieradata/production/hiera.yaml
Problem
My environment hiera.yaml is ignored due to its location, it is expected to be where my puppet <ENVIRONMENT> code is:
/etc/puppetlabs/code/environments/production/hiera.yaml
taken from https://docs.puppet.com/puppet/5.0/hiera_config_yaml_5.html#location
but instead it is:
/etc/puppetlabs/code/hieradata/production/hiera.yaml
Solution
The only solution that I have been able to find is to locate a hiera.yaml in my puppet code repo with a datadir set to
datadir: ../../hieradata/%{::environment}/hieradata
(it must be relative)
Which works ok but is not ideal so I am looking for other solutions.
Even if I set the global hiera.yaml datadir to:
/etc/puppetlabs/code/hieradata/%{::environment}/
(it can be a full path)
and do not have a hiera.yaml file in my puppet repo, hiera works, but the environmental hiera.yaml located there is ignored.
Another idea was to swap the locations but then my puppet code does not work.
So ideally what I need to do is set the environment layer hier.yaml location as it seems to be hard coded to <ENVIRONMENT>, which is where my puppet code is.
I still want to use r10k and I want to host my environmental hiera.yaml in my hiera repo.
my working code:
datadir: "/etc/puppetlabs/code/environments/%{environment}/hiera"
but path should be correct.
I have an agent/master setup. I have created a new environment in /etc/puppetlabs/code/environments/ called master.
The content of environment.conf for the master directory environment is
modulepath = site:modules:$basemodulepath
manifest = manifests/site.pp
and when I try puppet agent -t --environment master I am getting some error
Notice: Local environment: 'master' doesn't match server specified node environment 'production', switching agent to 'production'.
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for node1.localpuppet.com
Info: Applying configuration version '1490712072'
Notice: Applied catalog in 0.67 seconds
I am new to puppet. What changes do I need?
PE Console Config
This is a "really fun" quirk of Puppet Enterprise that showed up in the last couple of years. You have to specify the nodes in the PE Classifier that are allowed to specify their directory environment in the puppet.conf or in the puppet agent -t --environment arguments.
In the agent-specified environment tab in the Classifier (you see it at the bottom of your picture above), you can enable it for all nodes. Do this by adding a rule, selecting the name fact, using a regular expression (~), then using the regexp for matching all characters (.*). After you fill this out, the PE Classifier will give you a number of matching nodes. It should be all that are subscribed to your master. Remember to click in the bottom right to update your rules. Your nodes will now be able to use master instead of production from the config file or CLI arguments.
That being said, if you are doing this to avoid naming your default Git branch production in your control repository when working with Code Manager, you should really just rename the branch as that is much easier.
I am playing around with puppet and am trying to copy a file from my local directory (my laptop) on to my puppet agent. I have two VM's running, one is puppet master and one is puppet agent. I looked up at this answer here but it seems like it was an older version on puppet. I am running puppet 3.4.3 . I have gone through the pro puppet book and the puppet tutorials but find them way to confusing (the former having very glaring typos). It would be BIG help if someone helped me out with the process in simple steps. This is what I have till now.
I created a folder named my_module in /etc/puppet/.
In /etc/puppet/my_module is created two folders files, manifests and a file init.pp .
Init.pp looks like this:
class myfile {
file { "/home/me/myfolder/file.py":
mode => "0440",
owner => 'root',
group => 'root',
source => 'puppet:///modules/module_name/datas.xls',
}
}
I then copied the file file.py to the files folder I created above. I am unsure how to proceed after this step. Any help?
please read this documentation regarding creating your own modules. The module you created is in the wrong location right now. Should be /etc/puppet/modules or wherever the modulepath in /etc/puppet/puppet.conf points to on the puppet master.
The file given with source => 'puppet:///modules/module_name/datas.xls' is the one which will be placed in /home/me/myfolder/file.py on the client where you run the puppet agent -t command to rollout your changes.
Another good source for examples how to use the standard builtin puppet features is Type Reference of puppetlabs.
I am trying to create a custom provider for package but for some reasons I keep on getting
err: Could not run Puppet configuration client: Parameter provider
failed: Invalid package provider 'piprs' at
/usr/local/src/ops/services/puppet/modules/test/manifests/init.pp:5
I have added pluginsync=true in puppet.conf in both client and server. I have created the following rb file in module/test/lib/puppet/provider/package/piprs.rb. I am basically trying to create a custom provider for package resource type
#require 'puppet/provider/package'
Puppet::Type.type(:package).provide(:piprs,
:parent => ::Puppet::Provider::Package) do
commands : pip => "/usr/local/bin/pip"
desc "Python packages via `pip`."
def create
pip "freeze"
end
def destroy
end
def exists?
end
end
In the puppet.conf, there is the following source attribute
pluginsource = puppet://puppet/plugins
I am not sure what it is. If you need anymore details, please do post a comment.
First things first - you do realize there is already a Python pip provider in core?
https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb
If that isn't what you want - then lets move on ...
For starters - try your module without a Puppet master - this is going to be better for development anyway. You need to make sure Ruby can find the library path:
export RUBYLIB=<path_to_module>/lib
Then, try writing a small test in a .pp file:
package { "mypackage": provider => "piprs" }
And run it locally:
puppet apply mytest.pp
This will rule out a code bug in your provider versus a plugin sync issue.
I notice there is a space between the colon and the command - that isn't your problem is it?
commands : pip => "/usr/local/bin/pip"
If you can get this working without a puppetmaster, your problem is sync related.
There are a couple of things that can go wrong - make sure the file is sync'd properly on the client:
ls /var/lib/puppet/lib/puppet/provider/package
You should see the piprs.rb file there. If it is, you may need to make sure your libdir is set correctly:
puppet --configprint libdir
This should point to /var/lib/puppet/lib in most cases.