Run section of puppet manifest once a day but with hourly poll - puppet

I have nodes checking into a puppet server every hour.
We have some tasks which we want to run on check-in but only once a day.
Would it be possible to make a function inside a puppet manifest that saves last run time and only runs if the last time was over 24 hours?
Update:
I did try one thing which semi-works. That is move the chunk of puppet code into a separate file, and have my main puppet ensure a cron job exists for it.
The complaint I go back from another department with this is that they can no longer see install errors on puppet board. This image shows 2 nodes on the old puppet branch and 1 on the new branch:
With having cron run puppet apply myFile.pp we no longer got the feedback from failures on Puppetboard, as the main script simply ensures that the cron job exists:

You have at least two options.
Assuming your unspecified task is handled by an exec resource, you could design this in such a way that Puppet only ever regards the exec as out of sync once per day. That could be achieved by having your exec write the calendar day into a file. Then you could add an unless attribute:
unless => "test $(</var/tmp/last_run) == $(date +%d)"
Obviously your exec would need to also keep track of updating that file.
A second option would be to use the schedule metaparameter:
schedule { 'everyday':
period => daily,
range => '1:00 - 1:59',
}
exec { 'do your thing':
schedule => 'everyday',
}
That assumes that Puppet really will run only once per hour. The risk of course is that Puppet runs more than once in that hour, e.g. a sysadmin might manually run it.

Related

Azure WebJob fails to run on its schedule but it ran when I uploaded the bat file and never ran again

I created a WebJob as a Triggered job to run on a Schedule. When I uploaded the file it was accepted by the form and I went ahead and clicked RUN because I figured you have to click RUN right after uploading it so that it knows it can go ahead and start running. (I am not sure if I actually have to click RUN, or if I should have just uploaded it and let it be so it should just run on its own according to the CRON Expression provided.)
Well, the job ran as soon as I clicked start and it succeeded which was good news. The issue is, it was supposed to run on its schedule every 4 hours, but never did. It only ran once, which was the time I clicked start.
The CRON Expression I created for it is **0 50 23/4 * * *** which translates to:
At 50 minutes past the hour, every 4 hours, starting at 11:00 PM.
Basically I need the job to run every 4 hours but most importantly at 11:50pm which is why I set that as the schedule. So it should run at 11:50pm, 3:50am, 7:50am, 11:50am, 3:50pm, 7:50pm 11:50pm everyday.
I uploaded the job at about 10pm and it ran at that time because I clicked on RUN but was still expecting it to do its REAL SCHEDULED RUN at 11:50 pm but it never did. The logs show success for that initial run as you can see below.
When I look at the WebJob area in Azure the next day, it shows completed 17 hours ago and only ran once at the time of writing this.
What could be my error here? Is it something wrong with the CRON Expression that I have provided for the job? Before this one I made one that would run every 2 minutes and that one worked perfectly fine, but this one with a more complex CRON Expression seems to give me issue.
What could be my problem here?
I was able to fix this issue scrapping everything and starting over. I was using the incorrect CRON Expression for the trigger times I needed. Also found out that I could just upload the file and not have to click the RUN button since it will just run on its own following the given expression.

Run triggered Gitlab jobs only on specific days

I have the following issue:
I have a scheduled git pipeline which runs every work day in the morning. This pipeline is triggering other pipelines of other projects. The jobs are defined in multiple gitlab.yml files in their corresponding projects. For better understanding, here is a minimal example with only one triggered job:
main job:
trigger: child job
child job:
// Do something here
Now the thing is that this triggered child job is only allowed to run on specific days. In our case, it is that the child job is not allowed to run on Mondays.
I already had the idea to distinguish in the main job on which day the child job should be executed, give the child job a variable and check the given variable with the only or except tags. But it seems like it is not that easy to get the current work day inside the gitlab.yml. Or am I wrong there?
Is there a way to achieve this?
UPDATE
#KamilCuk made me realize that I was missing an important aspect in the question. When the child job is executed on its own, it should run without any hinderance (also on monday) if possible. When triggered by the main job, the check should apply.
The simplest is to just check it in the job.
child job:
script:
- weekday=$(LC_ALL=C date +%a)
- case "$weekday" in
Mon) echo "Not running on monday"; exit 0; ;;
esac
- rest of the job
You can trigger the job via API. https://docs.gitlab.com/ee/api/pipeline_triggers.html https://docs.gitlab.com/ee/ci/triggers/index.html
main job trigger child job:
script:
- weekday=$(LC_ALL=C date +%a)
- case "$weekday" in
Mon) echo "Not running on monday"; exit 0; ;;
esac
- curl
-H "Authorization: Bearer somethingosmething"
$CI_GITALB_URL/.../api/4/....
trigger child job

How can I start periodical actions server side?

I'm trying to develop a react/redux application and I want to run some server code without needing to trigger an action client side. Right now my needs are to calculate and storage some info in the database and i don't want to run a client that triggers a server action, but in the future i want to make some automatic changes in the database everyday to update my database information based on my previous information. Is there a way to do it?
What you need is called a cronjob. The implementation of this depends mostly on your hosting, but the syntax is always pretty much the same. Let's say you have a file called cron.php which contains the logic for your automatic changes.
The cronjob would look something like this:
0 0 * * * location/to/cron.php
The first two numbers mean at 0 minutes and 0 hours (so 00:00), the following stars mean every day, every month and every weekday. More information about cronjobs here and here.
Start a cron-job script executing WGET command for open you web page, or directly call your script.
wget -O /dev/null http://www.example.com/youraction
in a cron job:
*/5 * * * * wget --quiet -O /dev/null http://www.example.com/youraction
here you can find a guide about edit your cron jobs
https://www.lifewire.com/crontab-linux-command-4095300

Change cron entry with Ansible without changing time

I have a bunch of systems, with a Cron-scheduled job on them, which was set up by Ansible. Checking the crontab, I can see
#Ansible: Job Name
0 22 * * * /path/to/script.sh >/var/log/folder/script.log
I need to change that entry to point to a different script. This is easy enough, except that some of the systems in question have had the timing changed, and I need to preserve that changed timing. I tried an ansible task without the timing:
- name: Update Cron
cron:
name: Job Name
job: /path/to/new/script.sh >/var/log/folder/script.log
But that of course just left the following, since each time option defaults to '*':
#Ansible: Job Name
* * * * * /path/to/new/script.sh >/var/log/folder/script.log
Is there any good way to get Ansible to update the cron entry without clobbering the current timing, or possibly a clean way to read it up and tell Ansible to write out the current time? Or am I going to have to fall back on something ugly and probably-not-recommended like using the lineinfile module to edit the appropriate /var/spool/cron/ files directly?
This will not be possible only with the cron module. But I see a way to do it cleanly and to get a chance to enhance your inventory at the same time.
use the command module with crontab -l.
parse the stdout_lines to find the cron name comment, the definition is on the next line
parse the cron definition to extract the schedule. Store this in var(s)
optionally update you host_vars in your inventory with those values so you have the schedule for a next run if needed
run the cron module with the parsed schedule.
I admit that's a lot of steps for something that is normally done with a single task but this is the only clean way I could think of.
Assuming some sort of logical grouping of your hosts, rather than parsing each host for the crontab entry, can you setup the times per-group then setup the groups in the inventory to set the times using the "cron:" module?
If you need to ensure that there is some randomness to the time entries (so you don't have 500 hosts all trying to upload a log file at the same second), you can use the "random" filter along with a seed as shown in this StackOverflow answer:
https://stackoverflow.com/a/45946949/187426

How can I make my PHP-based cron job execute more often?

I have a file "update.php" which does some MySQL operations. A cron job executes this file every 5 minutes. Unfortunately, I cannot execute the cron job more often.
So I had the idea that I could add ...
<html>
<head>
<meta http-equiv="refresh" content="2; URL=<?php echo $_SERVER['SCRIPT_NAME']; ?>" />
</head>
<body></body>
</html>
... to the page "update.php". Will cron execute the file in a way that the page will refresh automatically? Or will that not happen because there is no client with a browser?
If it the meta refresh has no effect, is there any other possibility to achieve the refreshing of the page?
Thanks in advance!
I'm afraid that won't work, because it's a browser feature to refresh the page.
Question: Why can't you set the cron job to run more frequently that every 5 minutes?
If there is no other option then you could create you're own daemon to do the job more frequently.
e.g.
Your php script could:
Run
Wait 60 seconds
Run
( Wait; Run; two more times)
exit
For example: (By variation of sshow's code)
<?php
$secs = 60;
ignore_user_abort(true);
set_time_limit(0);
dostuff();
sleep($secs);
dostuff();
sleep($secs);
dostuff();
sleep($secs);
dostuff();
sleep($secs);
dostuff();
?>
This version of the script will remain resident for four minutes, and execute the code 4 times which would be equivalent to running every minute, if this script is run by cron every 5 minutes.
There seems some confusion about what a cronjob is, and how it is run.
cron is a daemon, which sits in the background, and run tasks through the shell at a schedule specified in crontabs.
Each user has a crontab, and there is a system crontab.
Each user's crontab can specify jobs which are run as that user.
For example:
# run five minutes after midnight, every day
5 0 * * * $HOME/bin/daily.job >> $HOME/tmp/out 2>&1
# run at 2:15pm on the first of every month -- output mailed to paul
15 14 1 * * $HOME/bin/monthly
# run at 10 pm on weekdays, annoy Joe
0 22 * * 1-5 mail -s "It's 10pm" joe%Joe,%%Where are your kids?%
23 0-23/2 * * * echo "run 23 minutes after midn, 2am, 4am ..., everyday"
5 4 * * sun echo "run at 5 after 4 every sunday"
So to run every five minutes:
*/5 * * * * echo "This will be run every five minutes"
Or to run every minute:
* * * * * echo "This will be run every minute"
The output from the commands are emailed to the owner of the crontab (or as specified by MAILTO).
This means if you run something every minute it will email you every minute, unless you ensure all normal output is suppressed or redirected.
The commands are run as the user who owns the crontab, which contrasts with the scripts run by the web-server, which are run as the 'nobody' user (or similar - whatever the web-server is configured to run as).
This can make life more complicated if the cronjob is writing to files which are supposed to be accessed by the scripts run by the web-server. Basically you have to ensure that the permissions remain correct.
Now, I'm not sure that this is the system you are refering to. If you mean something else by cronjob then the above might not apply.
If you want to do something that your current host is not letting you do, then rather than hacking around the restriction, you might what to look at switching hosting provider?
An alternative is to put the script in you're normal scripts location, and have some external scheduler run wget against it at whatever frequency you like.
Another alternative is on-demand updating of the form of vartec's suggestion. However that may not solve your problems.
I'm pretty sure you can achieve it by doing this:
<?php
$secs = 120;
ignore_user_abort(true);
set_time_limit(0);
while (true)
{
// do something
// Sleep for some time
sleep($secs);
}
?>
Edit
You will have to execute it once after every server restart unless you do it like Douglas describes.
Update
Keep Douglas Leeder's answer in mind, and then take a look at this:
http://www.php.net/manual/en/function.ignore-user-abort.php
I'd say don't try to do this with php, change your crontab. If you need your application to do a cronjob every minute and your hosting doesn't provide this option, you have most likely outgrown your hosting. Get yourself a VPS hosting for 20$ a month (Slicehost, Servergrove).
Update: Editted based on new information.
Meta refresh won't work because cronjob.de will be using an automated system that doesn't actually read the contents of the page. No browser, so nothing to see the meta refresh.
You have a couple options. They vary in greater or lesser horribleness.
The best option is to change webhosts. A good webhost will have full support for cron. But if you need to touch cron, honestly, you should probably be on a VPS host anyways. A lot of hosts will object to a cron task running every minute unless the task is just updating something really quickly and exits. But VPS hosts won't usually care. Slicehost offers VPS servers for as little as $20/month. Not recommended for people who've never had root access before.
The only option you've got that will work with cronjob.de's 5 minute limitation is to build a loop that will run an iteration, sleep, run another iteration, and repeat however many times you need before the end of the 5 minutes. However, there are two major problems with this approach. First, if you have a request that lasts 4 minutes, there's a distinct possibility that your webhost might kill the request before it finishes. Second, if the webserver isn't configured just right, such a request might block other requests, preventing legitimate users from accessing your site — they would queue up, and be waiting for the cronjob.de request to finish before their requests could be completed. And since that request will take 4-5 minutes to finish, before being repeated a minute later, they might only be able to access your site once every 5 minutes. I'm guessing this is undesirable. Unfortunately, the only way to know if you'll run afoul of either of these problems is to ask your webhost. I don't recommend trying it before asking, because they may not appreciate it if it goes unexpectedly bad and starts affecting their other customers on the server.
If you're lucky, they may even be willing to set up a cron job for you.
You can use PHP to call the script...
<?php
$script='/path/to/my/php/script.php';
ignore_user_abort(1);
set_time_limit(0);
$php=exec('which php');
while (1) {
if (file_exists($script)) { exec($php.' '.$script); }
else { file_get_contents($script); }
sleep(2);
}
?>
The file you give $script needs to exist, otherwise it thinks it's a URL.
It should do the trick.
What you are trying to do it implies user interaction, so what if no client enters into your "page"?
For example Servlets and EJB containers can do it programmatically at container startup what you need, so I suppose that for PHP the only way to accomplish "automatically" cronlike jobs is to make some changes in your Apache source code, obviously only if you are using an housing server.
A more praticable option not contempling code changes could be some cron script calling directly your page (wget,netcat,etc...) launched during your web server startup

Resources