I have an AWS Amazon Linux 2 box. I am using a python script (Python 3.7) to send an email using Sendgrid as the SMTP service. I can send the email using $ python3 send_email.py but, when I use crontab ($ crontab -e then * * * * * python3 ~/apps/send_email.py), the error in the log file /var/log/cron is (CRON) EXEC FAILED (/usr/sbin/sendmail): No such file or directory. The crontab is working as expected (I've tested other cron commands and they work fine), but the email part is what's not working.
Here's what I've tried to fix it:
Run the command as a root crontab (ie $ sudo crontab -e)
Run the the crontab as a user crontab (ie $crontab -e)), but with sudo python3 ... in the crontab command.
Add the path directory at the top of the crontab file in case those directories couldn't be resolved
Installed Postfix to install something in the /usr/bin/sendmail directory but, since I don't need it to run the Sendgrid-powered email using $ python3 send_email.py, I'm not sure why I would need it through cron. I could be totally wrong on this, though. With Postfix installed, it resolves the (CRON) EXEC FAILED (/usr/sbin/sendmail): No such file or directory error in the cron log - the log entry in that case is (ec2-user) CMD (python3 ~/apps/send_email.py) - but I don't receive an email. Probably because Postfix isn't configured for the SMTP I'm using (Sendgrid).
(EDIT) I have configured Sendgrid to work with Postfix via the Sendgrid docs but it still won't send me an email although it looks like nothing is erroring out in the Postfix logs...
Jun 1 20:27:02 ip-[my-ip] postfix/pickup[25506]: 101769641F3: uid=1000 from=<ec2-user>
Jun 1 20:27:02 ip-[my-ip] postfix/cleanup[25508]: 101769641F3: message-id=<20210601202702.101769641F3#ip-[my-ip].us-east-2.compute.internal>
Jun 1 20:27:02 ip-[my-ip] postfix/qmgr[25507]: 101769641F3: from=<ec2-user#ip-[my-ip].us-east-2.compute.internal>, size=1165, nrcpt=1 (queue active)
Jun 1 20:27:02 ip-[my-ip] postfix/local[25510]: 101769641F3: to=<ec2-user#ip-[my-ip].us-east-2.compute.internal>, orig_to=<ec2-user>, relay=local, delay=0.01, delays=0/0/0/0, dsn=2.0.0, status=sent (delivered to mailbox)
Jun 1 20:27:02 ip-[my-ip] postfix/qmgr[25507]: 101769641F3: removed
My thought is that there is something in the cron call of the Sendgrid python lib (installed with sudo pip install sendgrid) that is failing out that doesn't happen when python calls it directly using the python3 interpreter from the CLI. I don't know why those would be different.
I was able to finally resolve this after much troubleshooting. Here's what worked.
I kept the original setup the same: Sendgrid for SMTP using their python lib (no Postfix or smtplib), using user crontab (using $ crontab -e not $ sudo crontab -e), and using Python 3.7.
Apparently the word 'email' in the python script name can cause interpreter issues so I renamed the file to remove the word 'email'. For example, send_noti.py instead of send_email.py
I used absolute paths for every call to a file in the python script. I was reading, writing, and executing files in the user directory in my python script as well as sending an email. Those R,W,X commands started erroring out in cron but not when called from outside of cron for some reason, even after renaming the file. Using absolute paths for all references to files fixed that.
I also used the absolute path for the python file in the crontab file. For example, * * * * * python3 /home/ec2-user/apps/send_noti.py instead of * * * * * python3 ~/apps/send_noti.py
I removed the unnecessary items in the crontab file I had put in before such as redefining the PATH directory.
I hope this helps someone because this took me about 3 weeks to troubleshoot and figure out.
Related
I run my Spring Boot/Batch Jar to process large number of files in my local environment like below:
java -jar /path/to_my_dir_for_jar/springbatch.jar --spring.profiles.active=qa
And logs get generated in the same directory: /path/to_my_dir_for_jar
No issues whatsoever till here.
Now I ran the same job using linux crontab using (11:30 am, daily) :
30 11 * * * java -jar /path/to_my_dir_for_jar/springbatch.jar --spring.profiles.active=qa
Please note: I have to rely on linux crontab to schedule and not spring cron-expression as Spring Batch wont read the new files in the staging directory after first run.
Now I can NOT find my logs at - /path/to_my_dir_for_jar. In the application there different types of logs are generated such as:
service_log.gnxError.log
service_log.gcnDone.log
service_log.error.log
service_log.done.log
service_log.log
I tried to follow this and some other similar help but none of them seem to be working as most of them say - No such file or directory
Where to look for them?
One possible way is to redirect the output of my crontab expression to a file like below:
30 11 * * * java -jar /path/to_my_dir_for_jar/springbatch.jar --spring.profiles.active=qa >> foobar.log
But this way I will loose my different type of logs as everything will be fused in just one file in this case.
Will I get the logs the way I get while running the jar manually ever using linux crontab? What are my options in this situation?
I have a simple bash script and it works fine when I execute it directly (bash testfile.sh). But when I use it in crontab, it is not working..
*/10 * * * * cd /var/trafficviewer/script; ./testfile.sh
I made the chmod +x testfile.sh as well.In error log, I could found below error..
Jul 12 16:45:01 XX crond[1439]: (cd) ERROR (getpwnam() failed - user unknown)
Can someone help me on this error?
Note: I dont have root access for this server. I installed these crons by using sudo commands.
Following these instructions I got R-Studio Server running on a Google Compute Instance: http://code.markedmondson.me/launch-rstudio-server-google-cloud-in-two-lines-r/
When I open a terminal from this RStudio-Server I note that man, ps, vi, cron are all absent.
bash: ps: command not found
My goal is to have a simple cron job periodically run an R-Script.
I manually installed cron with:
sudo apt-get update
sudo apt-get install cron
Still, I can't get cron to run this test:
cmd <- cron_rscript("/home/law9723/now_to_file.R")
cron_add(cmd, frequency = "*/1 * * * *", id = "now_to_file", description = "Write now to file every minute")
-Clearly Confused
I got things to work eventually by using these very helpful instructions: https://yuhuisdatascienceblog.blogspot.ca/2017/07/setting-up-r-studio-server-on-google.html
Using absolute path names with everything associated with cron is sage advice.
I think when I created a vm with this command the sandbox that Rstudio-server lives in is very minimal. Hence, no vi, man, cron, ps...
gce_vm(template = "rstudio",
name = "my-rstudio",
username = "mark", password = "mark1234",
predefined_type = "n1-highmem-2")
I have had success using the rstudio addin. A reference is
http://www.bnosac.be/index.php/blog/51-new-rstudio-add-in-to-schedule-r-scripts.
I am also under the impression that you have to start cron with
sudo cron start
As in https://cran.r-project.org/web/packages/cronR/README.html.
I install the packages shinyFiles, miniUI and cronR when I first get into the Rstudio on GCE after using googleComputeEngineR locally like you. Then "Schedule R scripts on Linux/Unix" will appear in the add-in list.
I am submitting a shell script on a remote host that in turn submits an R script, but the error R: command not found or Rscript: command not found (depending whether I tried R CMD BATCH or Rscript).
I have tried submitting in the following ways:
ssh <remote-host> exec $HOME/test_script.sh
ssh <remote-host> `sh $HOME/test_script.sh`
The script test_script.sh contains (have tried Rscript as well):
#!/bin/sh
Rscript --no-save --no-restore $HOME/greetme.R
exit 0
The script greetme.R contains only cat("Hello\n").
The reason I am getting flustered is that when I log into the remote-host and submit the original script with sh $HOME/test_script.sh, it runs as intended.
The system specs and R versions for both the local and remote hosts are identical:
> R.version
_
platform x86_64-unknown-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 3
minor 1.0
year 2014
month 04
day 10
svn rev 65387
language R
version.string R version 3.1.0 (2014-04-10)
nickname Spring Dance
Why is Linux refusing to recognize the commands?
I would prefer solutions using R CMD BATCH or Rscript but if there are known workarounds using littler or %R_TERM% I would like to hear them too.
I used this related question as reference, as well as the documents referenced in the comments: R.exe, Rcmd.exe, Rscript.exe and Rterm.exe: what's the difference?
EDIT for solution:
As #merlin2011 suggested, once I specified the full path in the test_script.sh, everything worked as intended:
#!/bin/sh
/opt/R/bin/Rscript --no-save --no-restore $HOME/greetme.R
exit 0
I got the path also by the provided suggestion:
$ which Rscript
/opt/R/bin/Rscript
It appears that you have a PATH issue, where R is not on your PATH when you try to run the command through ssh.
If you specify the full path to R and Rscript on the remote host, it should resolve the problem.
If you are not sure what the full path is, try logging into the server and running which R to get the path.
I'm trying to set up a node script to run as a cron job on Joyent. I can run arbitrary commands but node scripts to seem to execute. As an example:
# cron
# call a script every minute
# being specific about the location of node and the script to run
* * * * * /home/node/local/nodejs/bin/node /full/path/to/some-script.js
// node script at /full/path/to/some-script.js
var fs = require('fs');
fs.writeFile('/home/node/node-service/some-script.log', new Date.toString(), 'utf8');
What I expect to see after one minute is a file at /home/node/node-service/some-script.log with content like Mon Jan 21 2013 15:19:11 GMT-0600 but I see nothing. This is still the case even if the script is set to full read, write and execute permissions for all users and whether the crontab is set for the root or node users.
What am I missing?
Thanks
The fourth optional argument to writeFile is a callback to fire when the file system is done writing the file. You can use it to determine the error that is happening, as it's only argument is an error. Refer to the docs here.
It appears to be working now. I'm not sure what I changed that got it working. It may have been a permissions issue.