I have a Vagrantfile that provisions a VM by running a chef recipe. The first resource in the chef recipe is:
directory "/downloads" do
owner "root"
group "root"
mode "0755"
action :create
end
# check that it worked:
raise "/downloads doesn't exist!" unless File.exists? "/downloads"
When I run this at work, it works fine.
When I run it at home, it fails, the exception is raised when I check to see if /downloads exists.
I'm not sure why this is happening. I would expect it to behave identically, since the underlying Vagrant box is the same both at work and at home. I am a chef newb so perhaps there is something I am not understanding about the order in which the resources are run within my recipe? I would expect them to run in sequential order...
I also tried putting a notifies call inside the directory block, where I call another execute block :immediately. That works, but inside the second execute block I test to see whether /downloads has been created and it hasn't.
Clearly I'm missing something very basic.
Chef has two phases of execution: a compile phase and a converge phase.
In the compile phase, any resource declarations you write (like directory) are compiled but not executed. Any bare Ruby code you write is also executed at this time. In the converge phase, any compiled resources are then converged in a test-and-set operation.
I'm going to assume that at work, you already have a /downloads directory pre-existing, so the resource is a no-op during converge, and the raise doesn't happen during compile.
If you want arbitrary Ruby code to execute at converge time, put it in a ruby_block resource.
Related
Actually I am downloading list of files from ftp and from the downloaded path I am reading all the list of filenames for processing.
In exec{"download from ftp ${value}" I am downloading directories and sub directories with files from ftp to local. From that path am getting the list using custom facts $facts['listdirectory']
My problem is that Facts['listdirectory'] is executed before being downloaded from ftp.
How to add dependency to $datadir=$facts['listdirectory'] or how to make this facts get executed after download?
class classname{
exec{"download from ftp ${value}":
command => "wget -r --user=${ftp_username} --
password=${ftp_password} ${value}/* -P ${patch_download_path}",
path => ['/usr/bin', '/usr/sbin',],
timeout => 1800,
user =>'root',
}
$datadir=$facts['listdirectory']
}
My problem is that Facts['listdirectory'] is executed before being downloaded from ftp.
It looks like you mean that the fact's value is determined before the directory contents (not the fact implementation) is downloaded. Certainly that's what will happen, in any case.
All facts that will inform a given catalog-building run are evaluated first, then delivered as a group to the catalog builder (which typically runs remotely on a puppet master). This gives the catalog builder a consistent snapshot of machine state to work from as it computes the desired target state by evaluating your manifests in light of the facts presented. The result is delivered in the form of a catalog of classes and resources, which the local Puppet then applies.
Only at the catalog-application stage will the command specified by your Exec resource run. This is after the whole catalog has been built, and long after fact evaluation. If you want to adapt dynamically to what has been downloaded then you must either do so on the next Puppet run, or script it and run the script via the same or another Exec resource, or write a custom type and provider that encompass the whole process (probably including the download, too).
everyone,
when I deploy my package to a linux environment, I met this error:
.../Linux-2.6c2.5-i686/Ncurses/Ncurses-15766.0-0/lib/libncurses.so.5 is encountered a second time at /apollo/_env/FBAMerchantAutoRemovalDaemon-swit1na.1755067.237551097.1107633519/perl/lib/perl5.8-dist/File/Find.pm line 542.
though I read the perl script, I have no idea what is wrong. I suspect my environment is tainted. Does anyone have idea what is wrong and how can I debug this problem? Thanks a lot in advance!
Zhe
From perldoc File::Find
follow
Causes symbolic links to be followed. Since directory trees with symbolic links (followed) may contain files more than once and may even have cycles, a hash has to be built up with an entry for each file. This might be expensive both in space and time for a large directory tree. See "follow_fast" and "follow_skip" below. If either follow or follow_fast is in effect:
It is guaranteed that an lstat has been called before the user's wanted() function is called. This enables fast file checks involving _. Note that this guarantee no longer holds if follow or follow_fast are not set.
There is a variable $File::Find::fullname which holds the absolute pathname of the file with all symbolic links resolved. If the link is a dangling symbolic link, then fullname will be set to undef.
So, if, for the purposes of your application, if it is OK to follow symlinks, invoke find with the follow option set:
find({ wanted => \&process, follow => 1 }, $dir);
Or, consider if one of the other follow_skip behaviors is more appropriate for your application:
follow_skip
follow_skip==1, which is the default, causes all files which are neither directories nor symbolic links to be ignored if they are about to be processed a second time. If a directory or a symbolic link are about to be processed a second time, File::Find dies.
follow_skip==0 causes File::Find to die if any file is about to be processed a second time.
follow_skip==2 causes File::Find to ignore any duplicate files and directories but to proceed normally otherwise.
It may be that follow_skip => 2 is more appropriate for your application. Only you can make that decision.
disclaimer: pretty new to chef and I've inherited a bunch of chef cookbooks. The methods below are sub-optimal but it is what I have to work with for now. Be gentle, please. :) Also, please bear with me as I try to describe what I need.
Please note that we are using chef-client 11.16.4. Updating to 12.x, for now, is not an option.
tl;dr
Is there a way to specify a guard from the state of the current running block:
...
only_if { this_block_did_something }
notifies :run, 'bash[deploy-custom-docker-container]', :immediately
OK....
Take this chunk of code in a recipe I inherited and need to refactor a little...
# The identities of the innocent have been changed for their
# protection. Please ignore odd things in this example:
application app[:name] do
path app[:deploy_path]
enable_submodules true
repository app[:repository]
owner OWNER
group GROUP
symlinks({
"file.py" => "path/file.py"
})
revision app[:branch]
deploy_key data_bag_item('deployment_keys', 'keyname')['private_key']
end
link "/path/to/file.py" do
to "/path/to/settings-%s.py" % [file]
end
# This is where I need some direction...I think.
# note that CMD is a valid constant and the custom docker
# container does not follow any industry standard docker
# conventions due to our strange use-case. So I had to resort
# using a bash block to call our custom start/stop/restart script
bash 'deploy-custom-docker-container' do
code <<-EO
#{CMD} restart
EO
# currently a subscribes but I've tried other methods which
# don't achieve what I'm trying to accomplish
subscribes :run, 'application[%s]' % [app[:name]]
end
The application app[:name] deploys source code onto the target node whenever the repo has new code to be synced. The bash block restarts a very custom and non-industry standard docker container which uses the code.
In its current form, which is undesirable, the bash[deploy-custom-docker-container] block always gets executed irrespective of whether application app[:name] has to deploy code to a git repo or not (IE repo is up to date vs not up to date) on the target node. I'm sure I could create some code that determines if the repo was updated, touch a state/lock file, and then guard execution of the bash block by checking if that lockfile exists. To me, that would be a sub-optimal way to achieve my goal. What would be optimal is to use chef's state of the update as the method of setting the guard. Is that possible?? Read on...
In other words, when application app[:name] is hit during chef-client runtime, and a repo has been updated (and thus deployed on the node), chef-client reports the steps of application app[:name] deploying the new code. If the repo didn't need to be updated, chef-client happily skips the block with a "(up to date)" message. If the repo needed to be updated, chef-client shows the steps taken to deploy the code. So chef-client knows the state of the block of code it just ran.
Also, my observations of how chef-client runs in our environment has shown me that it doesn't matter if I put a notifies block in application app[:name] for bash[deploy-custom-docker-container] or use the subscribes method (pasted above); the bash block gets run irrespective of the state of the application app[:name]. I'd prefer that if the application app[:name] doesn't have an update to perform then the bash block doesn't run.
What I fear is that I will have to use a state file to determine the state of the update of the repo from the application app[:name] block. I'd rather just guard off the state of the run-time from chef's perspective of the application app[:name] block.
FIXED CODE
As pointed out by zts, my actions were wrong or missing. The following code is what I was able to come up with that resolved my issue.
application app[:name] do
...
notifies :run, 'link[%s]' % [filetolink], :immediately
end
link filetolink do
to file
notifies :run, 'bash[deploy-custom-docker-container]', :immediately
end
bash 'deploy-custom-docker-container' do
code <<-EO
#{CMD} restart
EO
action :nothing
end
This works for me now.
Notifications only fire if the notifying resource has changed (and the other way around, subscriptions only fire if the resource you're subscribing to has changed).
The reason the bash block runs irrespective of the notification is that, by default, bash blocks will run. If you only want a resource to run when notified, make sure to include action :nothing.
ie:
bash 'deploy-custom-docker-container' do
code <<-EO
#{CMD} restart
EO
action :nothing
subscribes :run, 'application[%s]' % [app[:name]]
end
In the official Puppet docs it says that there are two chaining arrows:
https://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html
-> (ordering arrow)
Causes the resource on the left to be applied before the resource on the right. Written with a hyphen and a greater-than sign.
~> (notification arrow)
Causes the resource on the left to be applied first, and sends a refresh event to the resource on the right if the left resource changes. Written with a tilde and a greater-than sign.
Can someone clarify the difference between these two?
The document you mentioned has given the best explanation. If you try to understand it by simple way, using the exist sample.
Package['ntp'] -> File['/etc/ntp.conf'] ~> Service['ntpd']
For File['/etc/ntp.conf'], puppet needs to make sure that the package ntp has been installed before it creates or updates the file ntp.conf. There is no restart request.
But for Service['ntpd'], ntp.conf needs to exist first - that's the same order as ->. * But if puppet finds the file ntp.conf has any changes (whether it is created or updated), service ntp needs to be restarted. That's the difference*.
For more reading about ordering in puppet, please see these documents:
Learning Puppet — Resource Ordering
And do some testing by yourself to understand how it works.
set Package['ntp'], File['/etc/ntp.conf'] ,Service['ntpd'] with the order.
run puppet apply to make sure, Package/File/Service are ready on the system.
make a change in the file ntp.conf.
enable the --debug option with the puppet apply command. The debug log will give you detail in the background - for example, you should see that the file gets updated and the ntpd service gets restarted.
I need to write a Puppet script to manage the directory /foo/bar such that:
the file mode on /foo/bar is 777, but the permissions of everything within the directory are not managed by Puppet.
the owner/group on /foo/bar and everything within it is baz.
That is, the first requirement is non-recursive, but the second attribute is recursive.
Puppet provides a single recursive attribute, which affects the behavior of owner, group, and mode simultaneously. This means that I cannot specify the desired behavior using a single resource declaration.
I tried using two resource declarations, but then I get the error
Error: Duplicate declaration: File[/foo/bar] is already declared in file /my/puppet/file.pp at line XX; cannot redeclare
Yes, this will not work. Mind that Puppet is not a scripting engine, but a tool to model your desired state.
You will therefor have to decide how you want to manage your directory: As a single file system entry (recurse => false) or a whole tree (recurse => true). In the latter case, Puppet will always manage all properties for which you are passing values.
In your situation, you will likely have to fall back to the workaround of managing the permissions of the directory itself through a different resource, likely an exec resource that calls chmod, independently of the file resource. The latter must not pass a value for mode in this constellation, otherwise the two resources will always work against one another.
It's no ideal, but Puppet is not well equipped to deal with your specific requirements.