I'm trying to run some specific jobs at specific times during the day to capture particular data on my db during increments, and I currently have TONS of jobs that run perfectly fine, but these don't seem to run on there designated time, and I often times have to run them on my own.
Currently these are the scheduled times.
I PRESUME 9AM everyday. As I've checked everywhere and it appears correct syntax.
0 9 * * * bash /home/user/Desktop/CRON/OAK3/dw_3704255.sh
I PRESUME 1:30PM everyday. As I've checked everywhere and it appears correct syntax.
30 13 * * * bash /home/user/Desktop/CRON/OAK3/dw_3704278.sh
I PRESUME 6PM everyday. As I've checked everywhere and it appears correct syntax.
0 18 * * * bash /home/user/Desktop/CRON/OAK3/dw_3704286.sh
I PRESUME 10PM everyday. As I've checked everywhere and it appears correct syntax.
0 22 * * * bash /home/user/Desktop/CRON/OAK3/dw_3704294.sh
Now, I've tried changed changing the front zeros to 00, but the same result has occured. I recently changed to single zero, but I believe that's how I've originally had it.
I may just need a sanity check from outside perspective, because it appears right, but any insight would be appreciated. Thank you!
I'd assume the jobs will start running, but won't complete for some reason, making it look like they didn't start at all. This is often caused by an environment variable that's set in .profile - cron jobs won't execute .profile and won't have access to these variables.
I'd put a statement like
exec > /tmp/dw_3704255.log 2>&1
set -x
at the start of your dw_3704255.sh script; then you can check if the file appears at the time it should, and check its contents for a trace as well.
Also, i'd replace bash with /bin/bash to protect against weird PATH settings in the cron process, but i wouldn't assume this to be the cause of your current problem.
Please check the cron log
grep CRON /var/log/syslog
Related
For this specific project I am collecting data from the internet on an hourly basis and placing it into a directory within a hadoop framework. I need to run specific programs at specific times and crontab is the best way to do this. The problem is I don't know how to create the new files, put them in the proper directories and name them BASED on the time they were made. Crontab takes the commands exactly as they would appear in the command line. Do I have to write a separate program to modify the crontab file so its naming everything properly?
Basically what I want it to do is something like
1 * * * * python /location/of/pyfile/stream.py >
/home/hadoop/project/d2015/"currentmonth"/subfolder/"filenametimestamp".txt
Everything in quotes is what needs to change on an hourly/monthly basis.
I've setup a cron job by Ansible but when I use crontab -l it's said that my cron is empty.
Here's my script to set it up.
- name: Setup cron to run backup.sh every midnight
cron:
name="Backup S3 to GS"
minute="0"
hour="0"
job="sh ~/backup.sh"
cron_file=backup_s3
user=vagrant
But when I go inside the vagrant machine and ls /etc/cron.d/ I can see that backup_s3 file is there. But when I use command crontab -l it's said it's empty.
This is the content of backup_s3
#Ansible: Backup S3 to GS
0 0 * * * vagrant sh ~/backup.sh
I know that it's not running because I don't get any email saying that the backup is done and when I run the script manually it's working fine.
Okay. There are several layers of confusion here.
First, the crontab you see when you edit (crontab -e) or view (crontab -l) is a user cron. This sits in a magic spool directory/file. You can't edit it directly (approximately speaking), and it's not a good place to put any serious crons.
Since you are using cron_file=, Ansible is doing the appropriate thing by placing an entry in /etc/cron.d/. That means individual files can be placed there, which is much more sane than trying to edit a document. (look at all the people struggling with lineinfile here on stackoverflow)
This is why it isn't showing up in crontab -l, and it's a good thing.
As far as output from cron is concerned, does email even work for your Vagrant system? It likely doesn't. There are good ways around this. First, look at /var/log/cron. If you don't have one, look for CRON entries in /var/log/syslog. They may indicate if there are problems.
Next, crons typically don't have good access to a user shell. That means you should avoid ~. Further, if your permissions are wrong on backup.sh, it may not get executed. Finally, you can pipe output so you can see it. Here's what I'd recommend doing with your cron entry:
job="/bin/sh /home/vagrant/backup.sh >> /home/vagrant/backup.log"
You can also modify the minute/hour so it runs more frequently- so you don't have to wait overnight to see what is happening.
Once you've done that, you have plenty of places to look for information. There are two places in /var/log, there's a new backup.log which will give you information (if it exists, the cron has been run; if there is data in it, you should be able to figure out any problems).
TLDR
Change the job line. Look for execution in /var/log and /home/vagrant/backup.log.
I wrote a cron job in the cron tab recently that is supposed to be executed at 10:25 AM on every morning.
25 10 * * * /usr/bin/wget http://www.mysite.com/email.php
The file that's specified uses php to send me a practice email so that I know it works. It doesn't though. When I load the file in my own browser, it works. So, I know that it's not a php error.
What's strange is that when I specified a different file instead, that worked. But, this one doesn't.
Hum, looks like a problem with your script, not with your Cron Job.
This question may sound incredibly stupid, but if I don't ask I'll never know...
All the tasks setting I do is always by using "crontab". I heard the term "cronjob" somewhere. Is it another tool or just a name for something in "crontab"?
A cronjob is just a single entry in a crontab, that's all.
The cronjob is what you put into your crontab.
crontab is the actual command line entry that is used to kick off a cron job (not cronjob).
to create a cron job, do the following
crontab -e
And then fill in the cron syntax for the scripts/programs you want to run at a specific "cron'ed" interval.
Try playing in the cron sandbox - lets you try out combinations of crontab timing values and gives you a list of when the job would run. www.dataphyx.com
I have a crontab setup to run a perl script every hour, at 5 minutes past the hour (so 2:05, 3:05, 10:05, etc.):
5 * * * * perl /abs/path/to/my/script.pl >> /abs/path/two/my/script-log.txt 2>&1
As you can see, it's also redirecting both STDOUT and STDERR to the same log file.
I've been asked to refactor either the Perl script, the crontab entry, or to add any new code/scripts necessary so that every night, at midnight, the script-log.txt gets cleared/flushed/emptied.
That is, every night, at midnight, if script-log.txt has 20MB of text in it, to clean it out so that it now has nothing (0bytes) in it, and then at 12:05 AM the crontab would kick back in, run script.pl, and start adding more text to the same script-log.txt log file.
It would be enormously easier if there was a way to modify the crontab entry with some Linux/Perl magic to set up such a "daily rolling log file". In a worst-case scenario, we can always write a new script to purge script-log.txt and cron it to run at midnight. But my higher-ups would greatly prefer to not have yet-another cron job, and are looking for a way to do this from the entry itself (if possible).
In reality, we have dozens of entries that work like this, and so the problem with writing "purging script" and cronning it to run at midnight is that we'll constantly be updating the purging script as we add/delete other scripts that generate these kinds of log files. Thus, if we can set such purging up at the entry level, each cron entry cleans itself. Thanks for any insight/pointers/suggestions in advance.
You might want to look into logrotate.