disclaimer: pretty new to chef and I've inherited a bunch of chef cookbooks. The methods below are sub-optimal but it is what I have to work with for now. Be gentle, please. :) Also, please bear with me as I try to describe what I need.
Please note that we are using chef-client 11.16.4. Updating to 12.x, for now, is not an option.
tl;dr
Is there a way to specify a guard from the state of the current running block:
...
only_if { this_block_did_something }
notifies :run, 'bash[deploy-custom-docker-container]', :immediately
OK....
Take this chunk of code in a recipe I inherited and need to refactor a little...
# The identities of the innocent have been changed for their
# protection. Please ignore odd things in this example:
application app[:name] do
path app[:deploy_path]
enable_submodules true
repository app[:repository]
owner OWNER
group GROUP
symlinks({
"file.py" => "path/file.py"
})
revision app[:branch]
deploy_key data_bag_item('deployment_keys', 'keyname')['private_key']
end
link "/path/to/file.py" do
to "/path/to/settings-%s.py" % [file]
end
# This is where I need some direction...I think.
# note that CMD is a valid constant and the custom docker
# container does not follow any industry standard docker
# conventions due to our strange use-case. So I had to resort
# using a bash block to call our custom start/stop/restart script
bash 'deploy-custom-docker-container' do
code <<-EO
#{CMD} restart
EO
# currently a subscribes but I've tried other methods which
# don't achieve what I'm trying to accomplish
subscribes :run, 'application[%s]' % [app[:name]]
end
The application app[:name] deploys source code onto the target node whenever the repo has new code to be synced. The bash block restarts a very custom and non-industry standard docker container which uses the code.
In its current form, which is undesirable, the bash[deploy-custom-docker-container] block always gets executed irrespective of whether application app[:name] has to deploy code to a git repo or not (IE repo is up to date vs not up to date) on the target node. I'm sure I could create some code that determines if the repo was updated, touch a state/lock file, and then guard execution of the bash block by checking if that lockfile exists. To me, that would be a sub-optimal way to achieve my goal. What would be optimal is to use chef's state of the update as the method of setting the guard. Is that possible?? Read on...
In other words, when application app[:name] is hit during chef-client runtime, and a repo has been updated (and thus deployed on the node), chef-client reports the steps of application app[:name] deploying the new code. If the repo didn't need to be updated, chef-client happily skips the block with a "(up to date)" message. If the repo needed to be updated, chef-client shows the steps taken to deploy the code. So chef-client knows the state of the block of code it just ran.
Also, my observations of how chef-client runs in our environment has shown me that it doesn't matter if I put a notifies block in application app[:name] for bash[deploy-custom-docker-container] or use the subscribes method (pasted above); the bash block gets run irrespective of the state of the application app[:name]. I'd prefer that if the application app[:name] doesn't have an update to perform then the bash block doesn't run.
What I fear is that I will have to use a state file to determine the state of the update of the repo from the application app[:name] block. I'd rather just guard off the state of the run-time from chef's perspective of the application app[:name] block.
FIXED CODE
As pointed out by zts, my actions were wrong or missing. The following code is what I was able to come up with that resolved my issue.
application app[:name] do
...
notifies :run, 'link[%s]' % [filetolink], :immediately
end
link filetolink do
to file
notifies :run, 'bash[deploy-custom-docker-container]', :immediately
end
bash 'deploy-custom-docker-container' do
code <<-EO
#{CMD} restart
EO
action :nothing
end
This works for me now.
Notifications only fire if the notifying resource has changed (and the other way around, subscriptions only fire if the resource you're subscribing to has changed).
The reason the bash block runs irrespective of the notification is that, by default, bash blocks will run. If you only want a resource to run when notified, make sure to include action :nothing.
ie:
bash 'deploy-custom-docker-container' do
code <<-EO
#{CMD} restart
EO
action :nothing
subscribes :run, 'application[%s]' % [app[:name]]
end
Related
I need to take down Azure VMs in a controlled way, first stopping them and then removing them. The "stop" part needs to be executed only if the VM is existing, otherwise the task creates one in a stopped state, and then removes it. I tried variations on when: state == "present" but without success. Where can I find an example or documentation about how I can use that? Or maybe the solution to have a previous task retrieving the VM info, and act on that?
TIA!
If you haven't tried already, give it a try by having the first task using the module "azure_rm_virtualmachine_info" and make sure you save the result to a variable using "register" command. Then have the second task using "when" command to check if the value is "present" or not for the state object of the variable thats saved in the earlier task.
so I have my computer set up as an agent pool in azure-devops. I'm creating a test for latency so the developers can use it in their CI, the script runs in python and test various points in a system I have set up for the company which is connected to the cloud, it's mainly for informative purposes. When I run the script I have to wait some time, so the system I have connected goes through its normal network cycle inspecting all the devices in the local network, not very important for que question, however when I'm waiting I show in the terminal a message with "..." going from "." to ".." to "...", just to show the script didn't crash or anything.
the python code looks like this and works just fine when I run it locally:
sys.stdout.write("\rprocessing queue, timing varies depending on priority" + ("."*( i % 3 + 1))+ "\r")
sys.stdout.flush()
however the output shown in the azure pipeline shows all of the lines without replacing them. Is there a way to do what I want?
I am afraid showing progress is not supported in azure pipeline. Azure pipeline log console isnot user interactive. It just capture the agent machine terminal outputs.
You might have to use a simpler way to indicate that the script is now executing and not finished yet. For simple example:
sys.stdout.write("Waiting for processing queue ..." )
You can report this problem to microsoft development team. Hope they find a way to fix this in the future sprint.
I have seen it once but never actually used it myself, this can be done in both bash and PowerShell, not sure if this works inside a Python script, you might have to call bash/PowerShell from within your Python script.
It is possible to set a progress value in percent that is visible outside of the log, but as I understand it this value is step-spefific, meaning it only applies to the pipeline step you're currently in. You could drag the numeric value (however many percent) along into the next step, but the progress counter would then again show up in the next step. I believe it is not possible to have a pipeline global display of a progress.
If you export a progress value it will show up beside the step name in the left hand side step list.
This setting of a progress (also exporting one variable from one step to another, which is typically done that way) can be done by echoing special logging commands. There's a great description to be found here: Logging commands
What you want to do is something just as it is shown as an example on the linked page:
echo "Begin a lengthy process..."
for i in {0..100..10}
do
sleep 1
echo "##vso[task.setprogress value=$i;]Sample Progress Indicator"
done
echo "Lengthy process is complete."
All of these special logging commands start with ##vso[task... The VSO is a relict to the time when Azure DevOps was called Visual Studio Online.
There are a whole bunch of them, but most of the time what you really need is exporting variables from one build step context to another, which is done with ##vso[task.setvariable]value
In the official Puppet docs it says that there are two chaining arrows:
https://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html
-> (ordering arrow)
Causes the resource on the left to be applied before the resource on the right. Written with a hyphen and a greater-than sign.
~> (notification arrow)
Causes the resource on the left to be applied first, and sends a refresh event to the resource on the right if the left resource changes. Written with a tilde and a greater-than sign.
Can someone clarify the difference between these two?
The document you mentioned has given the best explanation. If you try to understand it by simple way, using the exist sample.
Package['ntp'] -> File['/etc/ntp.conf'] ~> Service['ntpd']
For File['/etc/ntp.conf'], puppet needs to make sure that the package ntp has been installed before it creates or updates the file ntp.conf. There is no restart request.
But for Service['ntpd'], ntp.conf needs to exist first - that's the same order as ->. * But if puppet finds the file ntp.conf has any changes (whether it is created or updated), service ntp needs to be restarted. That's the difference*.
For more reading about ordering in puppet, please see these documents:
Learning Puppet — Resource Ordering
And do some testing by yourself to understand how it works.
set Package['ntp'], File['/etc/ntp.conf'] ,Service['ntpd'] with the order.
run puppet apply to make sure, Package/File/Service are ready on the system.
make a change in the file ntp.conf.
enable the --debug option with the puppet apply command. The debug log will give you detail in the background - for example, you should see that the file gets updated and the ntpd service gets restarted.
I have a Vagrantfile that provisions a VM by running a chef recipe. The first resource in the chef recipe is:
directory "/downloads" do
owner "root"
group "root"
mode "0755"
action :create
end
# check that it worked:
raise "/downloads doesn't exist!" unless File.exists? "/downloads"
When I run this at work, it works fine.
When I run it at home, it fails, the exception is raised when I check to see if /downloads exists.
I'm not sure why this is happening. I would expect it to behave identically, since the underlying Vagrant box is the same both at work and at home. I am a chef newb so perhaps there is something I am not understanding about the order in which the resources are run within my recipe? I would expect them to run in sequential order...
I also tried putting a notifies call inside the directory block, where I call another execute block :immediately. That works, but inside the second execute block I test to see whether /downloads has been created and it hasn't.
Clearly I'm missing something very basic.
Chef has two phases of execution: a compile phase and a converge phase.
In the compile phase, any resource declarations you write (like directory) are compiled but not executed. Any bare Ruby code you write is also executed at this time. In the converge phase, any compiled resources are then converged in a test-and-set operation.
I'm going to assume that at work, you already have a /downloads directory pre-existing, so the resource is a no-op during converge, and the raise doesn't happen during compile.
If you want arbitrary Ruby code to execute at converge time, put it in a ruby_block resource.
My goal is to run a bash script automatically whenever any new file is added to a particular directory or any subdirectory of that particular directory.
Detail Scenario:
I am creating an automated process for file submission from teachers to students and vice versa. Sender will upload file and it will be stored inside the Uploads directory in the LAMP server in the format, ex. "name_course-name_filename.pdf". I want some method so that when any file stored inside the Uploads folder, the same time a script will be called and send that file to the list of receives.
From the database I can find the list of receiver for that particular course and student.
The only concern of mine is, how to call a script automatically and make it work on individual file whenever the content of the directory changes. Cron will do in intervals but not a real time work.
Linux provides a nice mechanism for that purpose which is called inotify. inotify is mostly available as a C API. But there have been developed shell utilities as well. You should use inotifywait from inotifytools (pkg name in debian) for this. Here comes a basic example:
#!/bin/bash
directory="/tmp" # or whatever you are interested in
inotifywait -m -e create "$directory" |
while read folder eventlist eventfile
do
echo "the following events happened in folder $folder:"
echo "$eventlist $eventfile"
done
Update:
If the problem goes complicated, for example you'll have to monitor recursive, dynamic directory structures, you should have a look at incron It's a cron like daemon which executes scripts on certain events. But the events are file system events rather than timer events.
There is another option to 'inotifywait':
-d --daemon
Same as --monitor, except run in the background logging events to a file
that must be specified by --outfile. Implies --syslog.
For completeness:
-m --monitor
Instead of exiting after receiving a single event, execute indefinitely.
The default behaviour is to exit after the first event occurs.
Within the do-done block of your 'while' statement, you might parse each event report for interesting details then use 'case-esac' to take action based on each event that you care about.
For something that you plan to rely on for your operations, you might also consider replacing the hard-coded '$directory' with some sort of configuration file. Such a file might include the path and filename, the interesting events for that path and file, and a script to run when those events happened.
The script might take the list of events as parameters and then 'case-esac' again.
Just one man's ramblins,
~~~ 8d;-Dan