I've set crontab to execute scrapy script, but It is not work.
then I attempt same command in terminal. but It is work well.
rankAuction
crontab:
$ crontab -l
*/10 * * * * cd ~/PRG/tutorials/tutorials/spiders && scrapy crawl nodecrawler
How can i solve this issue? please answer this question
thank you
Check your Cron logs to see whats going on
grep CRON /var/log/syslog
You must be getting Unknown command Scrapy or similar error
I am sure its because Scrapy is not in PATH
To fix it, do this
In your Terminal type echo $PATH
Copy the output and then go into crontab -l
Now at the top of that file do this
PATH=<PASTE WHAT YOU COPIED IN LAST STEP>
Your crontab does not know scrapy location
which scrapy will tell you the location of scrapy e.g. "/usr/local/bin/scrapy"
Change to */10 * * * * cd ~/PRG/tutorials/tutorials/spiders && /usr/local/bin/scrapy crawl nodecrawler > /var/log/cronscrapy.log 2>&1
/usr/local/bin/scrapy will make scrapy work properly
> /var/log/cronscrapy.log will create a log at "val/log/cronscrapy.log" when your cron will run
2>&1 will output both standard output and error if any to the "cronscrapy.log"
Hope it helps someone
`
Related
Alright so I'm having an issue.
I've setup this correctly but something is out of order
I added this to my crontab
* * * * * /home/website/public_html/ && php artisan schedule:run >> /dev/null 2>&1
Now if I start this command from my terminal it will work, but on the crontab log I get this
/bin/sh: /home/website/public_html/ : is a directory
After the frequency of the execution (in your case five stars to run every minute) crontab syntax expects to find the command to run.
I guess you want change the working directory before running php, so you need to add cd to change directory:
cd /home/website/public_html/ && php artisan schedule:run
Anyway, there is plenty of examples and explanations about crontab in internet.
I am running a bash script that transfers files to my AWS bucket.If i run the bash script through my terminal it works fine (via ./myBash.sh).
However I put it in my crontab but there it doesn't work.This is my bash script
#!/bin/bash
s3cmd put /home/anonymous/commLogs.txt s3://myBucket/
echo transfer completed
echo now listing files in the s3 bucket
s3cmd ls s3://myBucket/
echo check
And this is my crontab-
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
46 13 * * * /bin/bash myBash.sh
And here is a list of things i have aready tried -
1)tried running the crontab with a node app to test whether crontab was working(the answer was yes)
2)tried running the crontab without the SHELL and PATH
3)Tried running the bash script from cron using sudo (46 13 * * * sudo myBash.sh)
4)tried running the bash without the /bin/bash
5) Searched many sites on the net for an answer without satisfactory results
Can anyone help me with what the problem may be?(I am running Ubuntu 14.04)
After a long time getting the same error, I just did this :
SHELL=/bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin
* * * * * /bin/bash /home/joaovitordeon/Documentos/test.sh
For anyone coming to this post.
I was having same problem and reason was crontab was running under root user and s3cmd was configured under ubuntu user.
so need to copy .s3cfg to root
cp -i /home/ubuntu/.s3cfg /root/.s3cfg
I'm trying to setup a Cron job on my system by adding the following line
17 12 * * * Rscript ~/path/to/file/script.R > ~/output_`date +\%d\%m\%y`.txt 2>&1
yet, I cannot see the file the output is being written to. I've consulted the following answers, but to no avail:
Why did my crontab not trigger
CronJob not running
When I run the following command on the terminal:
Rscript ~/path/to/file/script.R > ~/output_`date +\%d\%m\%y`.txt 2>&1
I get the output file as expected. I also added the following line to crontab:
* * * * * echo hi > ~/output.txt 2>&1
and it works just fine. I'm not sure what is wrong with the first command. Any help would be appreciated. Thanks.
Try Below trick, Create one script script.sh like below -
cat script.sh
Rscript ~/path/to/file/script.R > ~/output_$(date +\%d\%m\%y).txt 2>&1
And then create below entry in crontab.
17 12 * * * /bin/bash /path/to/script.sh
I fixed this by replacing Rscript in my crontab with /usr/local/bin/Rscript (or wherever your Rscript is located - do which Rscript to find out).
I am trying to make a cron job for the first time but i have some problems making it work.
Here is what i have done so far:
Linux commands:
crontab -e
My cronjob looks like this:
1 * * * * wget -qO /dev/null http://mySite/myController/myView
Now when i look in:
/var/spool/cron/crontabs/
I get the following output:
marc root
if i open the file root
i see my cronjob (the one above)
However it doesnt seem like it is running.
is there a way i can check if its running or make sure that it is running?
By default cron jobs do have a log file. It should be in /var/log/syslog (depends on your system). Vouch for it and you're done. Else you can simply append the output to a log file manually by
1 * * * * wget http://mySite/myController/myView >> ~/my_log_file.txt
and see what's your output. Notice I've changed removed the quiet parameter from wget command so that there is some output.
I'm trying to make a crontab with crontab -e, but it saves it in tmp/crontab.FTt6nI/crontab
the crons don't work so I guess that's the problem. But I don't understand why.
type:
crontab -l
to show list of crontab, your newly added crontab should be on the list. you could set the crontab to email the output to you by > youremail#aaa.com, in this way you can assure the cronjob is already run.
example:
* * * * * /usr/bin/php /home/username/public_html/cron.php > aaa#aaa.com
make sure the crond is running:
/etc/init.d/crond status
if it down, start it (centos/rhel):
/etc/init.d/crond start
debian/ubuntu:
/etc/init.d/cron start
hope that help.