I plan to use puppet via a cron job.
In my /etc/puppetlabs/puppet/puppet.conf I have:
splay = true
splaylimit = 30
In my cron job I run puppet agent -t which doesn't use splay.
Should the command be using splay here? Server url is set in the same puppet.conf file and that doesn't seem to have a problem.
I also tried splaylimit = 30m.
In short yes, but ...
My problem was due to not fully understanding the -t aka --test flag.
--test Enable the most common options used for testing. These are 'onetime', 'verbose', 'no-daemonize', 'no-usecacheonfailure',
'detailed-exitcodes', 'no-splay', 'show_diff', and
'no-use_cached_catalog'.
As --test specifically implies no-splay the values in puppet.conf are overridden.
Side note: Splay seems to be defined in seconds e.g. 300 = 5 minutes, 600 = 10 minutes. Minutes and hours can defined as well 5m, 10m, 1hr, 2hr, ect.
Related
till now i have been using puppet-enterprise 3.8.6 and i am finally allowed to upgrade to the latest version (2018.1.3).
in puppet 3.8.6 i was able to change the masterport by
setting the puppet_enterprise::puppet_master_port parameter,
execute puppet agent (which fails),
stopping all puppet services,
looking by hand for config files under /etc/puppetlabs and /opt/puppetlabs still containing 8140, and replace that value by the new port value in all places,
restarting all puppet services and
executing puppet agent once more
which is not nice nor easy nor fast, but it worked.
when i try to do that with the latest version i get on step 6 that there are still some few places where the value is reverted and the agent fails, in special:
puppet:~ # find /etc/puppetlabs/ -name "*.conf" -o -name "*.yaml" | xargs grep 8140
/etc/puppetlabs/console-services/conf.d/console.conf: "url": "https://puppet:8140"
/etc/puppetlabs/client-tools/services.conf: "port": 8140,
/etc/puppetlabs/client-tools/services.conf: "status_url": "https://puppet:8140/status",
/etc/puppetlabs/client-tools/services.conf: "url": "https://puppet:8140/"
and i figured out that those files are managed by the classes trapperkeeper/console_services.pp and cli_config.pp, respectively, and there the wrong port value comes from a "service_port" parameter, which i do not understand from which hiera value is read, but it does not default to the puppet_enterprise::puppet_master_port parameter.
Any hint how to get the service_port parameter set to a new value? Or, is not there a better way to change the master port?
many thanks
Michael
UPDATE:
i now understand that the parameter service_port a local variable is, as well as the status_port, that are read from class_parameters in an iteration.
and the relevant parameter is set a few lines above as Puppet_enterprise::Profile::Master::ssl_listen_port which defaults to puppet_enterprise::puppet_master_port. so i still do not understand why it does not work.
at the end i managed to run the puppet agent, shortly redirecting traffic from old port to the new one, with:
socat tcp-listen:8140,reuseaddr,fork tcp:localhost:<newport>
Is there a simple way to change the schedule of a kubernetes cronjob like kubectl change cronjob my-cronjob "10 10 * * *"? Or any other way without needing to do kubectl apply -f deployment.yml? The latter can be extremely cumbersome in a complex CI/CD setting because manually editing the deployment yaml is often not desired, especially not if the file is created from a template in the build process.
Alternatively, is there a way to start a cronjob manually? For instance, a job is scheduled to start in 22 hours, but I want to trigger it manually once now without changing the cron schedule for good (for testing or an initial run)?
You can update only the selected field of resourse by patching it
patch -h
Update field(s) of a resource using strategic merge patch, a JSON merge patch, or a JSON patch.
JSON and YAML formats are accepted.
Please refer to the models in
https://htmlpreview.github.io/?https://github.com/kubernetes/kubernetes/blob/HEAD/docs/api-reference/v1/definitions.html
to find if a field is mutable.
As provided in comment for ref :
kubectl patch cronjob my-cronjob -p '{"spec":{"schedule": "42 11 * * *"}}'
Also, in current kubectl versions, to launch a onetime execution of a declared cronjob, you can manualy create a job that adheres to the cronjob spec with
kubectl create job --from=cronjob/mycron
The more recent versions of k8s (from 1.10 on) support the following command:
$ kubectl create job my-one-time-job --from=cronjobs/my-cronjob
Source is this solved k8s github issue.
From #SmCaterpillar answer above kubectl patch my-cronjob -p '{"spec":{"schedule": "42 11 * * *"}}',
I was getting the error: unable to parse "'{spec:{schedule:": yaml: found unexpected end of stream
If someone else is facing a similar issue, replace the last part of the command with -
"{\"spec\":{\"schedule\": \"42 11 * * *\"}}"
I have a friend who developed a kubectl plugin that answers exactly that !
It takes an existing cronjob and just create a job out of it.
See https://github.com/vic3lord/cronjobjob
Look into the README for installation instructions.
And if you want to do patch a k8s cronjob schedule with the Python kubernetes library, you can do this like that:
from kubernetes import client, config
config.load_kube_config()
v1 = client.BatchV1beta1Api()
body = {"spec": {"schedule": "#daily"}}
ret = v1.patch_namespaced_cron_job(
namespace="default", name="my-cronjob", body=body
)
print(ret)
I'm deploying a 2-hosts service that also needs to setup a cron job. This job should only be run on one of the two machines (I dont care which). what's the easiest way to do so?
I know that the shell module in Ansible supports "run_once", but the cron module does not.
I could setup the cron job on both machines and then use the command "crontab -r" to remove all the jobs (provided no other jobs are needed there) on one machine. this is dirty, but very easy.
any better ideas?
I know that the shell module in Ansible supports "run_once", but the cron module does not.
Wrong. run_once is a property of a task, not of action modules.
Use cron module and set run_once for the task (mind the indentation level), for example:
- cron:
name: "check dirs"
minute: "0"
hour: "5,2"
job: "ls -alh > /dev/null"
run_once: true
I recently created a Grid Engine cluster on Compute Engine using Elasticluster (http://googlegenomics.readthedocs.org/en/latest/use_cases/setup_gridengine_cluster_on_compute_engine/index.html).
I was wondering what is the appropriate command to run shared-memory multithreaded batch jobs on a cluster of Compute Engine virtual machine running Grid Engine.
In other words, what is the name (i.e. pe_name) of the Grid Engine parallel environment.
Let's say I want to run a job requesting 4 cpus on 1 node, what would be the right qsub command.
So far I tried the following command:
qsub -cwd -l h_vmem=800G -pe smp 6 run.sh
Unable to run job: job rejected: the requested parallel environment "smp" does not exist.
qsub -cwd -l h_vmem=800G -pe omp 6 run.sh
Unable to run job: job rejected: the requested parallel environment "omp" does not exist.
Thank you for your help!
I don't believe that Elasticluster's Ansible playbook includes a parallel environment. You can see the main configuration run on the master here:
https://github.com/gc3-uzh-ch/elasticluster/blob/master/elasticluster/providers/ansible-playbooks/roles/gridengine/tasks/master.yml
I believe you can simply connect to the master and issue the "add parallele environment" command:
$ qconf -ap smp
and write a configuration file like:
pe_name smp
slots 9999
user_lists NONE
xuser_lists NONE
start_proc_args /bin/true
stop_proc_args /bin/true
allocation_rule $fill_up
control_slaves FALSE
job_is_first_task FALSE
urgency_slots min
accounting_summary FALSE
and then modify the queue configuration for all.q:
$ qconf -mq all.q
...
pe_list make smp
...
I would also suggest filing an issue with Elasticluster here:
https://github.com/gc3-uzh-ch/elasticluster/issues
I would expect that someone has already done this in a fork of Elasticluster and may be able to provide a pull request to the master fork.
Hope that helps.
-Matt
I want to print out messages and variables when Puppet runs.
I saw there are two functions that might help but couldn't really use them.
My site.pp file:
info "running site.pp info"
debug "running site.pp debug"
When I run on the client:
puppet -t
I don't get those prints.
Here is the puppet script with all the available puppet log functions.
log_levels.pp
node default {
notice("try to run this script with -v and -d to see difference between log levels")
notice("function documentation is available here: http://docs.puppetlabs.com/references/latest/function.html")
notice("--------------------------------------------------------------------------")
debug("this is debug. visible only with -d or --debug")
info("this is info. visible only with -v or --verbose or -d or --debug")
alert("this is alert. always visible")
crit("this is crit. always visible")
emerg("this is emerg. always visible")
err("this is err. always visible")
warning("and this is warning. always visible")
notice("this is notice. always visible")
#fail will break execution
fail("this is fail. always visible. fail will break execution process")
}
Script output (on puppet 2.7):
NB: puppet 3.x colours may alter (all the errors will be printed in red)!
from the Puppet function documentation
info: Log a message on the server at level info.
debug: Log a message on the server at level debug.
You have to look a your puppetmaster logfile to find your info/debug messages.
You may use
notify{"The value is: ${yourvar}": }
to produce some output to your puppet client
If you want to notify user with different type of messages like information, debug, error, warning, alerts, critical and emergency messages then use ‘loglevel’ metaparameter in puppet resources.
With the use of loglevel you can use the same resources for different type of error messages.
e.g for producing debug messages you can use it as,
notify {"debug message":
loglevel => debug,
}
Just as alternative you may consider using execs... (I wouldn't recommend it though)
exec { 'this will output stuff':
path => '/bin',
command => 'echo Hello World!',
logoutput => true,
}
So when you run puppet you should find some output like so:
notice: /Stage[main]//Exec[this will output stuff]/returns: Hello World!
notice: /Stage[main]//Exec[this will output stuff]/returns: executed successfully
notice: Finished catalog run in 0.08 seconds
The first line being logged output.
You can run the client like this ...
puppet agent --test --debug --noop
with that command you get all the output you can get.
excerpt puppet agent help
* --test:
Enable the most common options used for testing. These are 'onetime',
'verbose', 'ignorecache', 'no-daemonize', 'no-usecacheonfailure',
'detailed-exitcodes', 'no-splay', and 'show_diff'.
NOTE: No need to include --verbose when you use the --test|-t switch, it implies the --verbose as well.
That does the task for me. I use that to check vars and display notifications..
notify {"hello world $var1":}
Here's the documentation on Puppet website as well: http://docs.puppetlabs.com/learning/ordering.html#notify-and-subscribe
Easier way, use notice.
e.g
notice("foo.pp works")
or
notice($foo)
If, like me, you don't have access to the puppet master and need to print debug logs to inspect variables on your puppet client machine, you can try writing to a file from your puppet code itself:
file { '/tmp/puppet_debug.log':
content => inline_template('<%= #variable_x.to_s %>'),
}
Have you tried what is on the sample. I am new to this but here is the command: puppet --test --trace --debug. I hope this helps.
You could go a step further and break into the puppet code using a breakpoint.
http://logicminds.github.io/blog/2017/04/25/break-into-your-puppet-code/
This would only work with puppet apply or using a rspec test. Or you can manually type your code into the debugger console. Note: puppet still needs to know where your module code is at if you haven't set already.
gem install puppet puppet-debugger
puppet module install nwops/debug
cat > test.pp <<'EOF'
$var1 = 'test'
debug::break()
EOF
Should show something like.
puppet apply test.pp
From file: test.pp
1: $var1 = 'test'
2: # add 'debug::break()' where you want to stop in your code
=> 3: debug::break()
1:>> $var1
=> "test"
2:>>
https://www.puppet-debugger.com