I try to run the schedule.sh (shell script) in the crontab
This is the code in the script
#!/bin/sh -x
SHELL=/bin/bash
PATH=/home/peteryph/bin:/home/peteryph/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
python3 main_schedule.py
echo "Finish"
In the crontab terminal (crontab -e)
* * * * * /home/peteryph/Desktop/working/city_google/schedule.sh > /home/peteryph/Desktop/working/city_google/echo2.txt
* * * * * /usr/bin/python3 /home/peteryph/Desktop/working/city_google /schedule.py > /home/peteryph/Desktop/working/city_google/echo.txt
In the cron status
(peteryph) CMD (/home/peteryph/Desktop/working/city_google/schedule.sh > /home/peteryph/Desktop/working/city_google/echo2.txt)
(peteryph) CMD (/usr/bin/python3 /home/peteryph/Desktop/working/city_google /schedule.py > /home/peteryph/Desktop/working/city_google/echo.txt)
In the main_schedule, the code looks like this
(I did import the files. They are a lot of them, so I did not put them right here.)
print("Please wait.........\n")
infor = update()
infor.clear_previous_value() #update the data in the DB
start_time = time.time()
infor.select_update() #insert the data into DB
print("Running time --%s seonds--" %(time.time() - start_time) + "\n")
sys.exit()
I try to execute both python file and script at the same time, but I only receive the echo from the python file. But the data did not insert into the Database on both python file and script on the crontab.
However, the script works when I execute it manually.
(I do have non-English character in my file, would that effect the crontab ?)
(Or does the path of the database has to be absolute path ? )
I found where the PROBLEMS are.
First, you have to use absolute path for your database when you call the database for sqlite3.
Second, make sure your shell script is executable :P
Related
Good day everyone.
I have an issue, and Googling the issue has not helped me, basically I have the following requirement.
cronjob that runs 1st script, output is written to a file
file that is created, to have a date stamp
2nd script executes, mail the generated file as an attachment
The issue is with adding the timestamp, if I set the cron to run and just create a file with a generic filename the cronjob runs fine.
I have tried the following:
0 8-17/1 * * * python /usr/local/bin/script1.py >> /usr/local/bin/file_`date +\%Y-%m-%d`.txt 2>&1 && python /usr/local/bin/email_script.py
0 8-17/1 * * * python /usr/local/bin/acme_transcoding_check.py >> /usr/local/bin/file_$(date +"%Y-%m-%d").txt 2>&1 && python /usr/local/bin/email_script.py
Server is running Ubuntu 16.04
You need to escape the percent-sign (%) with a backslash as explained in this answer (not mine).
I tried to use crontab to execute my py file everyday, but it can only create empty log file
0 8 * * * /usr/local/bin/python3 /Users/UserName/Downloads/Crawling_1.py > /Users/UserName/Downloads/log.log
Then I tried to use SHELL file to execute a simple demand, if I put log file settings inside .sh file, no log file was created. Similarly, the crontab did not execute when I put python3 demand inside the SHELL file.
echo 1 > /Users/UserName/Downloads/new_log.log > /Users/UserName/Downloads/log.log
But if I directly run echo in Crontab, it can work out perfectly.
* * * * * echo 1 > /Users/UserName/Downloads/new_log.log
Does anyone know why this is happening? Thank you so much.
Try it with >>:
0 8 * * * /usr/local/bin/python3 /Users/UserName/Downloads/Crawling_1.py >> /Users/UserName/Downloads/log.log
then with >> it will create a file if it doesn't exist. If the file existe, it will be appended to the end of the file.
With > the whole file will replace it, if the file exist. If the file not exist*, nothing happens.
You can do the command also with >, but be sure, that the .log-file, where you will write inside, exist!
This is my cron script (a portion of):
CUSER=tim
APPDIR=/var/www/testing
APPVENV=/var/www/testing/ven
cat > $APPDIR/cronfile << EOF
PWD=$APPDIR/$CUSER
PATH=$APPVENV/bin:\$PATH
0 2 * * * testapp search newsite
EOF
crontab $APPDIR/cronfile
It seems to work but I'm really confused about how I would try to run this manually. What does this expand to if I wanted to run it from a command from shell?
I tried something like this but it didn't work :(
cd /var/www/testing/ven
testapp search newsite
This looks wrong:
PATH=$APPVENV/bin:\$PATH
There should not be a backslash in there.
Manually would be:
cd /var/www/testing/tim
PATH=/var/www/testing/ven/bin:$PATH
testapp search newsite
I have the following cron job command:
* * * * * /usr/bin/lynx -term=vt100 http://abc.com/dir1/di2/script.php?action=add&config=xyz >/dev/null 2>&1
My PHP script does not recognize _GET['config'] and I get a "Cron Daemon" email message which seems to alert me that the crontab instruction is not correct.
If I take out the 2nd _GET var I do not get the "Cron Daemon" email.
Any thoughts or suggestions on how to define multiple query string items in a crontab job?
BTW, I tried the URL Encode char for the ampersand and that did not work either.
Try putting your url in quotes :
* * * * * /usr/bin/lynx -term=vt100 "http://abc.com/dir1/di2/script.php?action=add&config=xyz" >/dev/null 2>&1
For the little explanation, & is a special character which put the process in the background, so you have to put the url in quotes, otherwise cron try to put the first part in the background and execute the second part.
How can I store variables in my crontab? I realize it's not shell but say I want to have some constants like a path to my app or something?
In Vixie cron, which is possibly the most common, you can do this almost exactly like a shell script.
VARIABLE=value
PATH=/bin:/path/to/doathing
0 0 * * * doathing.sh $VARIABLE
The man page says:
An active line in a crontab will be either an environment setting or a cron command. An environment setting is of the form,
name = value
where the spaces around the equal-sign (=) are optional, and any subsequent non-leading spaces in value will be part of the value assigned
to name. The value string may be placed in quotes (single or double, but matching) to preserve leading or trailing blanks. The name
string may also be placed in quote (single or double, but matching) to preserve leading, trailing or inner blanks.
You can tell if you have Vixie cron by checking the man page for crontab; the author will be Paul Vixie. Different crons may or may not support this (BusyBox's cron for example, does not), in which case your best option is to wrap your command in a shell script and run that script from cron instead. In fact, this is a good thing to do for anything complicated.
To keep my crontab clean, I would just call a shell script and do the fun stuff in the script.
I think the important fact to point out here is (as stated in an earlier comment by Pierre D Mar 25, 2015 at 18:58) that variable declarations are not expand/interpolated and so can not embed other variable values.
Variables are only expanded/interpolated in the commands themselves.
So:
var1 = bar
var2 = foo${var1}
42 17 * * * /path/to/command ${var2}
Results in: /path/to/command foo${var1}
While:
var1 = bar
var2 = foo
42 17 * * * /path/to/command ${var2}${var1}
Results in: /path/to/command foobar
So in my case the following works fine, no wrapping in shell scripts required:
SHELL=/bin/bash
timestamp=date +20%y_%m_%d_%H_%M_%S
logdir=/my/log/dir
0 2 * * * /my/command/path/mycmd param >> ${logdir}/myfile_$(${timestamp}).log
verses something like this which does not work:
logfile = /my/log/dir/myfile_${timestamp}.log
since the later is not expanded, but is rather interpreted as is including "${" and "}" as part of the string.
Just a working example of using variables in the crontab file and their substitution in the strings:
CURRENT_TIME=date +%Y.%m.%d_%H:%M:%S.%3N
CURRENT_DATE=date +%Y_%m_%d
SIMPLE_VAR=the_simple_var
LOG_DIR=/var/log/cron
* * * * * /bin/echo "simple variable test! ${SIMPLE_VAR}__test!" >> "${LOG_DIR}/test.log"
* * * * * /bin/echo "complex variable test! $(${CURRENT_TIME})__test!" >> "${LOG_DIR}/test.log"
Tested on this Docker image (paste the above crontab to the crontab.txt):
FROM debian:10-slim
# Install docker (Yep, this is a docker in docker):
RUN curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
# Install CRON:
RUN apt-get update && apt-get install -y --no-install-recommends cron
# Add a crontab_config.txt task:
COPY crontab.txt /var/crontab.txt
RUN crontab /var/crontab.txt
ENTRYPOINT ["cron", "-f"]
Add this to the crontab to run any commands inside another docker containers:
/usr/bin/docker exec container_name ls -la
If you have a few environment variables you want to set for a particular job, just inline those into the sh snippet.
42 17 * * * myvariable='value' path/to/command
In case it's not obvious, the sh syntax var=value command sets var to value for the duration of command. You can have several of these if you need more than one.
42 17 * * * firstname='Fred` lastname='Muggs' path/to/command
If you have nontrivial variables you want to access from several places, probably put them in a separate file, and source (.) that file from your shell startup script and your cron jobs.
Let's say you have a file $HOME/bin/myenv with settings like
myvar=$(output of some complex command)
another='another
complex
variable'
then you can add
. $HOME/bin/myenv
to your .profile (or .zshrc or .bash_profile or what have you; but .profileis portable, and also used bysh`) and
42 17 * * * . $HOME/bin/myenv; path/to/command
in your crontab.
Notice the lone dot before the space before the file name; this is the dot command (also known as source in e.g. Bash) which reads the file into the current shell instance as if you had typed in the things in the file here.
Tangentially, the $HOME/ part is strictly speaking redundant; cron jobs will always run in the invoking user's home directory.
Obviously, if you want a variable to be true in your entire crontab, set it at the top, before the scheduled jobs.