I´m newbie with many sysadmin´s tasks, I want to create a crontab for firstly, scrap a web page and save info inside a file and then, send it daily to my personal mail.
I have created a spider with scrapy and it works fine
I have installed postfix and I can send the generated file inside the body of the mail
But when I mixed this two tasks in the cron, I have some troubles
I need sudo for execute scrapy
I don´t know how is to add the subject of the mail with the command I have used.
I received these mails from the system:
/bin/sh: 1: scrapy: not found
sudo: no tty present and no askpass program specified
And this is the cron (the time for execute it its simply for testing)
*/5 * * * * cd /var/www/wp-content/plugins/plugin-name/se_scrapy/ && sudo scrapy crawl quepeliponen -o cartelera.csv && cat cartelera.csv | sed -e 's/,,/, ,/g' | column -s, -t | less -#5 -N -S | /usr/lib/sendmail -v myemail#gmail.com
I would to like make it in crontab one line and not use a bash script due to my poor knowledge about that
Thanks
Edited
Ok... I have searched a litle more and I think I have found the solution...
I have added the route of the scrapy command
/usr/local/bin/scrapy
and I have created the cron job in the crontab root user with
sudo crontab -e
The only thing I need its send the email with a subject to avoid receive it in spam folder
This is the cron now:
*/5 * * * * cd /var/www/wp-content/plugins/se-plugin-name/se_scrapy/ && rm cartelera.csv && /usr/local/bin/scrapy crawl quepeliponen -o cartelera.csv && echo "Subject: Cartelera de mañana" | cat cartelera.csv | sed -e 's/,,/, ,/g' | column -s, -t | less -#5 -N -S | /usr/lib/sendmail -F sendfrom#gmail.com -t sendto#hotmail.com
Edit 2
I resolve it using mail and not sendmail
0 4 * * * cd /var/www/wp-content/plugins/se-plugin-name/se_scrapy/ && rm cartelera.csv && /usr/local/bin/scrapy crawl quepeliponen -o cartelera.csv && cat cartelera.csv | sed -e 's/,,/, ,/g' | column -s, -t | less -#5 -N -S | mail -aFrom:sender#gmail.com -s "La cartelera de mañana" email1#gmail.com email2#gmail.com
Related
I've made a little bash script to backup my nextcloud files including my database from my ubuntu 18.04 server. I want the backup to be executed every day. When the job is done I want to reseive one mail if the job was done (additional if it was sucessful or not). With the current script I reseive almost 20 mails and I can't figure out why. Any ideas?
My cronjob looks like this:
* 17 * * * "/root/backup/"backup.sh >/dev/null 2>&1
My bash script
#!/usr/bin/env bash
LOG="/user/backup/backup.log"
exec > >(tee -i ${LOG})
exec 2>&1
cd /var/www/nextcloud
sudo -u www-data php occ maintenance:mode --on
mysqldump --single-transaction -h localhost -u db_user --password='PASSWORD' nextcloud_db > /BACKUP/DB/NextcloudDB_`date +"%Y%m%d"`.sql
cd /BACKUP/DB
ls -t | tail -n +4 | xargs -r rm --
rsync -avx --delete /var/www/nextcloud/ /BACKUP/nextcloud_install/
rsync -avx --delete --exclude 'backup' /var/nextcloud_data/ /BACKUP/nextcloud_data/
cd /var/www/nextcloud
sudo -u www-data php occ maintenance:mode --off
echo "###### Finished backup on $(date) ######"
mail -s "BACKUP" name#domain.com < ${LOG}
Are you sure about the CRON string? For me this means "At every minute past hour 17".
Should be more like 0 17 * * *, right?
I am not a linux expert and I have a problem I do not manage to solve. I am sorry if it is obvious.
I am trying to execute a bash script in a cron table on a raspberry pi, and I don't manage to get it work.
Here is the example script I want to execute:
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
plouf=$( ps -aux | grep reviews | wc -l)
if [[ "$plouf" == 1 ]] ;
then
echo "plouf" >> /home/pi/Documents/french_pain/crontest.txt
fi
My script in the cron consist in starting a script if there is no progam with review in its name running. To test I am just appending "plouf" to a file. I count the number of line of ps -aux | grep reviews | wc -l , and if there in only one line I do append "plouf" in a file.
Here is my crontab:
crontab -l
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
* * * * * sudo /home/pi/Documents/french_pain/script2.sh
The script do work when I do ./ script2.sh or /home/pi/Documents/french_pain/script2.sh directly in terminl: it add a "plouf" to the file.
I came across this page and tried different possibilities, by setting my path as the path given by env, and by explicitly setting the shell in the cron. But still not working.
What I am I doing wrong ?
To answer Mark Setchell comment:
raspberrypi:~/Documents/french_pain $ sudo /home/pi/Documents/french_pain/script2.sh
raspberrypi:~/Documents/french_pain $ cat crontest.txt
plouf
and cron is running:
raspberrypi:~/Documents/french_pain $ pgrep cron
353
I manage to do simple jobs like
* * * * * /bin/echo "cron works" >> /tmp/file
I tried with the direct path to the commands:
plouf=$( /bin/ps -aux | /bin/grep 'R CMD.*reviews' | usr/bin/wc -l)
if [[ "$plouf" == 1 ]] ;
then
/bin/echo "plouf" >> /home/pi/Documents/french_pain/crontest.txt
fi
without any luck. The permission for the file:
-rw-rw-rw- 1 root root 6 juil. 3 23:30 crontest.txt
I tried deleting it, and did not work either.
help !
I guess you trying this as "pi" user then 'sudo' won't work unless you have allowed nopasswd:all or using a command that is able to handle the password that Sudo requires from stdin in this case. The example below is dangerous since it will not require any password for sudo command anymore but since you wanted to use sudo in cronie:
Example 1:
With default /etc/sudoers below example will create an empty file:
* * * * * sudo ls / > ~/cronietest.txt
Try add below in /etc/sudoers at bottom (obs: do not use pi as username on a rpi):
pi ALL=(ALL) NOPASSWD:ALL
Now try again to add below in crontab
* * * * * sudo ls / > ~/cronietest.txt
It works!
Example 2:
This is more safe, add this to sudoers file for allow 'command' for "pi" user without any password when sudo is executed:
pi ALL= NOPASSWD: /bin/<command>
Example 3:
Without editing sudoers file, this is another example that will work (this is dangerous since your password is stored in cron file as plaintext)
* * * * * echo "password" | sudo -S ls / > ~/cronietest.txt
I did not find the reason why the sript was not working, but I finally found a way to make it work thanks to this post: I used shell script instead of bash:
The file script3.sh
#!/bin/sh
if ps -ef | grep -v grep | grep 'reviews'; then
exit 0
else
echo "plouf" >> /home/pi/Documents/french_pain/crontest.txt
fi
together with
* * * * * /home/pi/Documents/french_pain/script3.sh
in my crontab did the work I wanted.
I am trying cron job for the first time. I am trying to generate file which will contain user installed application in Ubuntu and the same file needs to be uploaded to server.
I am unable to generate the text file with that information. Below is the command which i am trying.
Script file which needs to be run for the cron job /tmp/aptlist.sh
#!/bin/bash
comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) &> /tmp/$(hostname)-$(date +%d-%m-%Y-%S)
cron has following entry done using crontab -e
:~$ crontab -l
0 0 1 * * /tmp/aptlist.sh > /dev/null 2>&1
syslog has following entry however no file is generated
Oct 21 14:09:01 Astrome46 CRON[14592]: (user) CMD (/tmp/aptlist.sh > /dev/null 2>&1)
Oct 21 14:10:01 Astrome46 CRON[14600]: (user) CMD (/tmp/aptlist.sh > /dev/null 2>&1)
Kindly let me know how to fix the issue.
Thank You
Try this:
0 0 1 * * bash /tmp/aptlist.sh > /dev/null 2>&1
If this works then I suspect it is because the file doesn't have executable permissions.
You can find that out by typing in the terminal:
ls -l /tmp/aptlist.sh.
If that is really the case then you can also edit the file permissions to allow it to run by typing:
chmod u+x /tmp/aptlist.sh
This will enable the file owner to run it, but will not allow that to other users. If you need it to run for a different user do:
chmod a+x /tmp/aptlist.sh
Good luck!
I currently have a script running in a crontab every 4 hours. Normally it is running with the following setup:
python3 -u script.py -s -q --load=ims | tee -a script.log
Is it possible to say "hey crontab, next time you run use the --redo flag on the python script, then continue as normal".
So it would run that entry as python3 -u script.py -s -q --load=ims --redo | tee -a script.log and then continue on as normal
It does not look like this is currently supported in crontab.
Alternatively, you can use the at command to run a command at a specific time. For example:
at> python3 -u script.py -s -q --load=ims --redo | tee -a script.log
at> ^D
That will run with that parameter at that specific 12:00
Another option is to create a Bash file that will modify the crontab entry before the next crontab run and then fix it after the crontab starts the script:
crontab -l | grep -v 'python3 -u script.py -s -q --load=ims' | crontab -
(crontab -l ; echo "(crontab -l ; echo "* */4 * * * python3 -u script.py -s -q --load=ims --redo | tee -a script.log") | crontab -") | crontab -
sleep 5m
crontab -l | grep -v 'python3 -u script.py -s -q --load=ims --redo' | crontab -
(crontab -l ; echo "(crontab -l ; echo "* */4 * * * python3 -u script.py -s -q --load=ims | tee -a script.log") | crontab -") | crontab -
That will find and delete the old entry, add a new one with the flag, then sleep for 5 minutes.
Then it will find the new entry and delete it, then replace it with the old entry
My script is working manually but not working on crontab.I read all topics about this issue I tried so many things to execute via crontab but didnt work.
My script is below.
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
nodetool status > cqlsh_control.txt
cs1=`more cqlsh_control.txt | awk '{print $1}' | sed -ne 6p | cut -d"%" -f1`
SLACK_ICON=":red_circle:"
if [ "$cs1" != "UN" ]; then
curl -S -X POST --data "payload={\"text\": \"{Cqlsh is not responsing Cassandra2} \",\"username\":\"CQLSH\",\"icon_emoji\":\"${SLACK_ICON}\"
}" https://hooks.slack.com/services/T05xxxxW/B7xxxxxx09/QdotCzoxxxxxxxHxOsrnjS
fi
Edit the crontab and put shell /bin/sh before script like below.
*/1 * * * * /bin/sh /home/ec2-user/cqlsh_control.s