I have written a manifest that performs a number of tasks. The very first task is that it loads the contents of a file into a variable. The file will exist on the target node ( or managed node or the one running the Puppet agent).
However, when I triggered the manifest via a puppet run, I realized that it was expecting to find the file on the master, not the agent!
$some_var = file("path_to_file")
How do I fix this so that it loads the file from the agent?
Indeed functions execute only on the master. Therefore, you need either an external or custom fact for this to execute on the node. Here is a custom fact ready to go for this purpose of returning the contents of a file foo:
# module_name/lib/facter/foo_content.rb
Facter.add(:foo_content) do
setcode do
# return content of foo as a string
File.read('/path/to/foo')
end
end
You can then use this thusly:
# facter 3
$some_var = $facts['foo_content']
# facter 2
$some_var = $::foo_content
Note this solution assumes foo is not some extremely enormous file.
https://docs.puppet.com/facter/3.6/custom_facts.html
Related
Within our pipeline's we would like to set a variable based on some user defined capabilities. For example, agent-1 may store all python versions under "C:/Python" whereas agent-2 may store all python versions under "C:/Documents/Python" and a script may need to know of all the contents stemming from this folder. So, to fix this, we set some user capabilities of where it's stored.
Agent 1: PYTHON_DIR = C:/Python
Agent 2: PYTHON_DIR = C:/Documents/Python
We would like to extract these from in our azure-pipelines.yml for use in future script steps.
We initially tried using the syntax:
variables:
PYTHON_EXE: $(PYTHON_DIR)\Python38\...\python.exe
but this simply echos out as
$(PYTHON_DIR)\Python38\...\python.exe even after an agent reboot.
I am trying to force a Puppet class that creates a file to be processed before another class that needs this file to exist to run properly. Following the Puppet article Language: Containment of resources I am using contain.
My code does not work and I do not understand why. It gives this error:
Error: Evaluation Error: Error while evaluating a Function Call, Failed to parse template testing/def.erb:
Filepath: /root/local/testing/templates/def.erb
Line: 1
Detail: No such file or directory # rb_sysopen - /tmp/abc
at /root/local/test2.pp:16:16 on node example.com
Here is the code (stripped down):
### test2.pp
class klass1 {
file { '/tmp/abc':
content => 'xxx',
}
}
# Stage 0 creates the file /tmp/abc.
class stage0 {
contain klass1
}
# Stage 1 uses the contents of /tmp/abc to create the
# file /tmp/def.
class stage1 {
file { '/tmp/def':
content => template('testing/def.erb'),
}
}
# Try to force stage0 to be loaded before stage1.
include stage0
class { 'stage1':
require => Class['stage0']
}
### testing/templates/def.erb
Contents: <%= File.read("/tmp/abc") %>
I am using Puppet 5.3.3.
The issue here does not relate to containment, but to the dependency in your template at compile time on the call to File.read("/tmp/abc").
Ordinarily, compilation occurs on the Puppet Master a.k.a. Puppet Server, and the template function also runs at this time. Thus, your template def.erb attempts to read from a nonexistent file at compile time on the Puppet Master.
A better solution is likely to be define the content of file /tmp/abc in Puppet itself as data or a variable and then pass that variable to the template function, and so remove the dependency on reading from the file on disk altogether.
Without fully understanding why you were trying to separate this file content into multiple classes in the first place, I can't really comment any further.
Puppet is a declarative language that is typically used to ensure certain state of resources. But if you really need make a decision based on local code evaluation, you're basically left with 2 options:
Firstly, use facter, in some module create lib/facter/my_fact.rb:
#!/usr/bin/env ruby
require 'facter'
Facter.add(:load1) do
confine kernel: 'Linux'
setcode do
Facter::Util::Resolution.exec("cat /proc/loadavg | awk '{print $1}'")
end
end
Then your "function" result will be available via facter load1 (from shell) or $facts['load1'] from Puppet code. Such code is always evaluated before applying Puppet's catalog. If your function doesn't take arguments, this might be a good option. Note this is a silly example, the load is already available via facter load_averages.1m (though the usefulness of such fact is questionable). Using too many facts isn't good idea, it would prolong time required for applying a Puppet catalog. There's a soft limit for number of facts on puppetserver.
Second option would be using Deferred function. Evaluation of such function is delayed into later phase of catalog application (won't be evaluated on compile server).
Puppet code looks like this:
$value = Deferred("mymodule::load", ["1m"])
and the actual implementation should be in a Puppet module in a Ruby function, e.g. lib/puppet/functions/load.rb:
Puppet::Functions.create_function(:'mymodule::load') do
dispatch :load do
param 'String', :load_type
return_type 'String'
end
def load(load_type)
case load_type
when '1m'
avgs[0]
when '5m'
avgs[1]
else
raise "#{load_type} not supported"
end
end
end
The advantage is the latter approach is that you can pass multiple arguments. Although returning more complex types than String or Numeric doesn't seem to currently supported.
Actually I am downloading list of files from ftp and from the downloaded path I am reading all the list of filenames for processing.
In exec{"download from ftp ${value}" I am downloading directories and sub directories with files from ftp to local. From that path am getting the list using custom facts $facts['listdirectory']
My problem is that Facts['listdirectory'] is executed before being downloaded from ftp.
How to add dependency to $datadir=$facts['listdirectory'] or how to make this facts get executed after download?
class classname{
exec{"download from ftp ${value}":
command => "wget -r --user=${ftp_username} --
password=${ftp_password} ${value}/* -P ${patch_download_path}",
path => ['/usr/bin', '/usr/sbin',],
timeout => 1800,
user =>'root',
}
$datadir=$facts['listdirectory']
}
My problem is that Facts['listdirectory'] is executed before being downloaded from ftp.
It looks like you mean that the fact's value is determined before the directory contents (not the fact implementation) is downloaded. Certainly that's what will happen, in any case.
All facts that will inform a given catalog-building run are evaluated first, then delivered as a group to the catalog builder (which typically runs remotely on a puppet master). This gives the catalog builder a consistent snapshot of machine state to work from as it computes the desired target state by evaluating your manifests in light of the facts presented. The result is delivered in the form of a catalog of classes and resources, which the local Puppet then applies.
Only at the catalog-application stage will the command specified by your Exec resource run. This is after the whole catalog has been built, and long after fact evaluation. If you want to adapt dynamically to what has been downloaded then you must either do so on the next Puppet run, or script it and run the script via the same or another Exec resource, or write a custom type and provider that encompass the whole process (probably including the download, too).
Using the functions in my earlier queries (see reference below), I am able to pull the file from the agent and perform the necessary tasks. However, this is affecting all the users on the system as it throws an exception stating that the file is not found. Is there anyway I can add some logic like unless file_exists .... to this ruby function ?
My hierarchy is shown below. I am not following why it affects other users who are not even in "mymodules".
Root
modules
mymodules
lib
facter
ruby_function1.rb
ruby_function2.rb
modules_by_userxx1
modules_by_userxx2
modules_by_userxx3
....
Reference :
Puppet : How to load file from agent
Puppet : How to load file from agent - Part 2
As requested by Dominic, adding reference code :
# module_name/lib/facter/master_hash.rb
require 'json'
Facter.add(:master_hash) do
setcode do
# return content of foo as a string
f = File.read('/path/to/some_file.json')
master_hash = JSON.parse(f)
master_hash
end
end
I'll assume you're talking about the custom fact from the previous answers rather than a Ruby function, in which case, add a File.exist? conditional:
# module_name/lib/facter/master_hash.rb
require 'json'
Facter.add(:master_hash) do
setcode do
if File.exist?('/path/to/some_file.json')
# return content of foo as a string
f = File.read('/path/to/some_file.json')
master_hash = JSON.parse(f)
master_hash
end
end
end
Please include the full error message (with the filename/line number it provides) and the source code when asking questions.
Custom facts are shipped within modules, but are all synced to agents as they're used to discover data.
Facts are synced and run on agents before the node is classified because they can be used to make classification and catalog decisions, e.g. based on hostname or OS. Because facts run before classification, it isn't yet possible to know which classes (and perhaps modules) are going to be applied, so should be safe to run on every node in the environment.
I am using the MultiJob Project in order to implement a process the runs every time there is a push to a certain Git branch.
How can I pass parameters between 2 different jobs (each job is located in a separate MultiJob Phase)
What i tried to do is:
Job A: (Run on the Master - windows)
echo 2 parameters into a new file (Called parameters.properties) that i placed in a shared location (not in the workspace of Job A)
so this file's contnet looks like:
currentBuild=2012-11-27_09-20-50
currentBranch=master
Job B: (Run on a Linux Slave)
The option of "This build is parametrized" is on.
Added "File Parameter" and only the file name as I also set a custom workspace to the shared location where the file is located.
The i have a shell script that tries to use this parametrs but it doesnt get it.
Please Assist,
Doron
The Solution:
in the MultiJob Main Project:
Set "This build is parametrized"
Add 2 Text Parameters (different names since the Job Build is based on Jenkins variable Build_Id which is changed and i want later to set the currentBuild to a constant value and not to a new one)
JobBuild
JobBranch
On each Job under each MultiJob Phase:
Add Predefined Parameters:
currentBuild=${JobBuild}
currentBranch=${JobBranch}
untick the "Current job parameters"
untick the Exposed SCM"