I want to know the use of . ./.profile whenever we execute cron jobs. I have seen many scripts having this included. The question is, what is the use and what if I don't add it?
Example:
00 1-22 * * 1-5 . ./.profile ; /global/u1/sie/rox/Scripts/Calls.ksh >/dev/null 2>&1
. somefile is the POSIX-compliant equivalent to the bash builtin source: Running source somefile in bash, or . somefile in any POSIX-compliant shell, executes every command inside that script in that existing shell.
In terms of why this is useful in a crontab: cron runs with a very minimal environment -- it may not even have a PATH set, and is unlikely to have many other facilities. If your scripts depend on environment variables being present, it can be necessary to either specify them in the crontab or to source in (that is, execute in the existing shell) a script which defines them.
That said, I advise against this idiom:
.profile is used by login sessions -- sessions with a user interacting with the shell in real-time -- and folks intending to customize their interactive session's behavior are liable to make modifications without keeping scheduled jobs in mind.
It's not obvious by reading your crontab which environment variables ~/.profile will or won't set, and thus difficult to reason about the state of the environment.
Instead, you should set environment variables at the top of your crontab:
PATH=/bin:/usr/bin:/usr/local/bin
VARNAME=VALUE
# ...etc...
0 1-22 * * 1-5 /global/u1/sie/rox/Scripts/Calls.ksh >/dev/null 2>&1
The profile files are the shell profiles, you can add code to it that will run as soon as the shell start up, ./profile is the profile file for Ksh and Bourne, /.bash_profile is for bash /.login is for Tcsh and Csh.
When a script calls the profile it's because it needs something from it, i.e $path variables or even specific commands that it might not have access to. In this case, since cron doesn't have access to much since it runs in a minimal enviroment that script will pull the .profile because it depends on something that's in there.
More info here
and here
Related
I need to change the greeting of user, which is logging in. So I modifyed file /etc/profile. In this greeting I need to know, which shell this user use and tell it to user. The problem is that then I change my shell on zsh or csh it doesnt work. Even if I just type in this file echo $SHELL it do nothing. As I think, when I use csh and zsh this file (/etc/profile) doesnt run at all. How can I fix this problem?
Thanks you, sorry for my English)
You should start by reading the manpage of every shell on your system.
There are different flavours of shells. Each flavours uses slightly different (per session and per shell, per site and per user) initialisation files. For example:
sh (and bash) use /etc/profile and ~/.profile
bash also uses ~/.bash_profile, ~/.bashrc, ~/.bash_logout
csh uses /etc/.login and ~/.cshrc
etc...
The above list is not meant to be exhaustive. It is to illustrate you will need to check the exact behaviour of each shell that is used on your system and configure it appropriately.
You also need to consider whether you want to change system-wide behaviour (corresponding to initialisation files under /etc) or user-specific behaviour (corresponding to initialisation files in the user's home directory).
For certain shells, there's also per-session (i.e. once per login) and per-shell settings (e.g. for every terminal window). A good example is ~/.bash_login (executed once per login) and ~/.bashrc (executed for every shell - e.g. terminal window).
They both execute different files:
From fro zsh http://zsh.sourceforge.net/Guide/zshguide02.html
Now here's a list of the startup files and when they're run. You'll
see they fall into two classes: those in the /etc directory, which are
put there by the system administrator and are run for all users, and
those in your home directory, which zsh, like many shells, allows you
to abbreviate to a `~'.
/etc/zshenv
Always run for every zsh.
~/.zshenv
Usually run for every zsh (see below).
/etc/zprofile
Run for login shells.
~/.zprofile
Run for login shells.
/etc/zshrc
Run for interactive shells.
~/.zshrc
Run for interactive shells.
/etc/zlogin
Run for login shells.
~/.zlogin
for csh http://unixhelp.ed.ac.uk/CGI/man-cgi?csh+1
A login shell begins by executing commands from the system files
/etc/csh.cshrc and /etc/csh.login.
You can make a soft link to point to the same file:
ln -s /etc/profile /etc/zshenv
ln -s /etc/profile /etc/csh.login
I have modified my etc/profile file to start a python script on startup. now my program is running but there is a black screen, because my program has a while True loop in it and now I am not able to stop it. Kindly tell me how to stop the program, I have tried ctrl+C but nothing happened.
I have a script used for zipping a database and site files, then dumps the output into a backup folder on the server. The script runs fine from the command line, but it will not work through cron.
After much research, I am thinking that cron cannot run it in its current form because it runs in a different environment.
Here is the script, saved as file_name.sh
#!/bin/bash
NOW=$(date +"%Y-%m-%d-%H%M")
FILE="website.com.$NOW.tar"
BACKUP_DIR="/backupfolder"
WWW_DIR="/var/www/website/"
DB_USER="dbuser"
DB_PASS="dbpw"
DB_NAME="dbname"
DB_FILE="website.com.$NOW.sql"
WWW_TRANSFORM='s,^var/www/website,www,'
DB_TRANSFORM='s,^backupfolder,database,'
tar -cvf $BACKUP_DIR/$FILE --transform $WWW_TRANSFORM $WWW_DIR
mysqldump -u$DB_USER -p$DB_PASS $DB_NAME > $BACKUP_DIR/$DB_FILE
tar --append --file=$BACKUP_DIR/$FILE --transform $DB_TRANSFORM $BACKUP_DIR/$DB_FILE
rm $BACKUP_DIR/$DB_FILE
gzip -9 $BACKUP_DIR/$FILE
I currently have the script stored in /usr/local/scripts/
Is there something wrong with the above code that does not allow it to run through cron?
Which crontab should it go in? crontab -e from terminal, or /etc/crontab? They are two different files.
Several things come to mind: first, one of the most common problems with cron jobs is that generally crond runs things with a very minimal PATH (usually just /usr/bin:/bin), so if the script uses any commands from some other binaries directory, it'll fail. Where is mysqldump on your system (run which mysqldump if you aren't sure)? If this is the problem, adding PATH=/usr/local/bin:/usr/bin:/bin (or whatever's appropriate in your case) at the beginning of your script should fix it. Alternately, you can set PATH in the crontab file (put this line before the entry that runs your script).
If that's not the problem, my next step would be to capture the script's output, with something like:
1 1 * * * /usr/local/scripts/file_name.sh >/tmp/file_name.log 2>&1
... and see if the output is informative. BTW, as #tripleee mentioned, the format of your cron entry is suitable for the files crontab -e edits, but not for /etc/crontab. The /etc version has an additional field specifying which user to run the job as, e.g.
1 1 * * * eric /usr/local/scripts/file_name.sh >/tmp/file_name.log 2>&1
Best practice is to always use crontab -e (the resultant files are usually in /var/spool/cron/) and this works on every unix and linux platform I ever worked on.
Other common issues with cron execution are missing environment variables. Any environment variables set in .bash_profile (or .profile if you use korn shell) will not necessarily be present in the cron environment. This can be overcome by including them in your script.
As Gordon said, paths are another suspect. You can always full path you executables in your script (eg /bin/mysqldump). Some of the more cynical of us do this anyway to make sure we are executing what we intended as apposed to some other file of the same name in the current path.
I can only guess at your specific problem since you fixed it by creating /scripts, that perhaps the permissions on /usr/local/scripts directory did not allow execution by the cron user?
I have had to remove the extension (.sh) for cron to run in some instances.
So I fixed it. Not sure what the problem was, but this worked for me.
I originally had the scripts located in /usr/local/scripts/
I created a new directory here - /scripts/ and moved the scripts there. The new crontab -e command looked like this:
1 1 * * * bash /scripts/file_name.sh
Works perfectly. Again, I am not sure what the issue was before, but it works now.
OS: Amazon Linux
I have a Ruby script that connects to a site, then it searches with an XPath request for a div block where is the stats counter I want to parse.
Then it compares the number from the site with the current value in the database, if the number has increased it sends me an email.
The problem is that, then I run the script from the current directory it works.
The script parses the block of text which contains a value.
I extract the value with Regex like this (/\d/)
...
But when it the script executes by crontab it gets some strange value like
...041704300440043504330438044104420440043804400430432043004304304304304304404370430432043004420435043043504390447043504400435043704320430044804430430...
I don't know how to debug it because, when I run the script manually it works, but fails with strange value when executed by crontab.
The text in the site is russian, encoded with Windows-1251.
Maybe there is something wrong with that.
I have set # encoding: utf-8, in the .rb file.
That could be an environment problem, which could include bad paths, etc. You can compare your ENV from the command-line to the environment when launched by crontab.
Try:
ruby -rpp -e 'pp ENV' > /tmp/crontab_env.out
from crontab, then:
ruby -rpp -e 'pp ENV' > /tmp/cmd_env.out
from the command-line, then:
vimdiff /tmp/*env.out
or use a regular editor.
If you're using RVM, note that it is typically only available to interactive shells. There's a whole section in the RVM manual dedicated to this topic: RVM: Ruby Version Manager - Using Cron with RVM
It could be that this is simply a problem of the wrong Ruby version, including its Gems, being used. Try removing the hashbang line in your script, and calling it like this in your crontab:
1 0 * * * /usr/local/rvm/bin/ruby-1.9.3-p362 /path/to/script.rb
This should make sure the proper environment is loaded with the Ruby binary.
If the actual problem is that RVM isn't even available for non-interactive scripts, you could also go one step further and do what your shell does when it's loading RVM—scroll to the right, this is a big line:
1 0 * * * /bin/bash -l -c 'source "$HOME/.rvm/scripts/rvm" && rvm use 1.9.3-p362 && ruby /path/to/script.rb
Problems with cron jobs are often caused by having the wrong environment. The script probably depends on an environment variable that's set when you start an interactive shell (through ~/.profile, ~/.bashrc or similar), but not when your program is started directly, by cron.
Get a list of environment variables and their current values by typing env. Add a cron job that simply runs env. Compare the outputs and chip away until you find the culprit.
I'd say LANG and friends are a good place to start. Get a list of language and encoding-related environment variables by typing locale.
Can I have certain settings that are universal for all my users?
As well as /etc/profile which others have mentioned, some Linux systems now use a directory /etc/profile.d/; any .sh files in there will be sourced by /etc/profile. It's slightly neater to keep your custom environment stuff in these files than to just edit /etc/profile.
If your LinuxOS has this file:
/etc/environment
You can use it to permanently set environmental variables for all users.
Extracted from: http://www.sysadmit.com/2016/04/linux-variables-de-entorno-permanentes.html
man 8 pam_env
man 5 pam_env.conf
If all login services use PAM, and all login services have session required pam_env.so in their respective /etc/pam.d/* configuration files, then all login sessions will have some environment variables set as specified in pam_env's configuration file.
On most modern Linux distributions, this is all there by default -- just add your desired global environment variables to /etc/security/pam_env.conf.
This works regardless of the user's shell, and works for graphical logins too (if xdm/kdm/gdm/entrance/… is set up like this).
Amazingly, Unix and Linux do not actually have a place to set global environment variables. The best you can do is arrange for any specific shell to have a site-specific initialization.
If you put it in /etc/profile, that will take care of things for most posix-compatible shell users. This is probably "good enough" for non-critical purposes.
But anyone with a csh or tcsh shell won't see it, and I don't believe csh has a global initialization file.
Some interesting excerpts from the bash manpage:
When bash is invoked as an interactive
login shell, or as a non-interactive
shell with the --login option, it
first reads and executes commands from
the file /etc/profile, if that file
exists. After reading that file, it
looks for ~/.bash_profile,
~/.bash_login, and ~/.profile, in that
order, and reads and executes commands
from the first one that exists and is
readable. The --noprofile option may
be used when the shell is started to
inhibit this behavior.
...
When an
interactive shell that is not a login
shell is started, bash reads and
executes commands from
/etc/bash.bashrc and ~/.bashrc, if
these files exist. This may be
inhibited by using the --norc option.
The --rcfile file option will force
bash to read and execute commands from
file instead of /etc/bash.bashrc and
~/.bashrc.
So have a look at /etc/profile or /etc/bash.bashrc, these files are the right places for global settings. Put something like this in them to set up an environement variable:
export MY_VAR=xxx
Every process running under the Linux kernel receives its own, unique environment that it inherits from its parent. In this case, the parent will be either a shell itself (spawning a sub shell), or the 'login' program (on a typical system).
As each process' environment is protected, there is no way to 'inject' an environmental variable to every running process, so even if you modify the default shell .rc / profile, it won't go into effect until each process exits and reloads its start up settings.
Look in /etc/ to modify the default start up variables for any particular shell. Just realize that users can (and often do) change them in their individual settings.
Unix is designed to obey the user, within limits.
NB: Bash is not the only shell on your system. Pay careful attention to what the /bin/sh symbolic link actually points to. On many systems, this could actually be dash which is (by default, with no special invocation) POSIXLY correct. Therefore, you should take care to modify both defaults, or scripts that start with /bin/sh will not inherit your global defaults. Similarly, take care to avoid syntax that only bash understands when editing both, aka avoiding bashisms.
Using PAM is execellent.
# modify the display PAM
$ cat /etc/security/pam_env.conf
# BEFORE: $ export DISPLAY=:0.0 && python /var/tmp/myproject/click.py &
# AFTER : $ python $abc/click.py &
DISPLAY DEFAULT=${REMOTEHOST}:0.0 OVERRIDE=${DISPLAY}
abc DEFAULT=/var/tmp/myproject
If I run a script from /home/<user>/<dir>/script.sh, as root, the cron works pretty well. But If I run the script from /root/<dir>/script.sh (as root, again), the cron does not seem to work.
Having run afoul of various default $PATHs in the past when using 'cron', I always spell in full the absolute $PATH for each executable file and each target file. I always assume that 'cron' has NO $PATH set and has NO current-working-directory.
In other words don't use a command like
"myprocess abc*.txt"
but do it in full like
"/usr/localbin/myprocess /home/jvs/abc*.txt".
Alternatively, create a bash script which does the job, and call that bash script with a full absolute path, such as
"/usr/local/bin/myprocess_abc_txts".
If you need to have some flexibility in the script, use environment variables which are set specifically within the bash script you call with 'cron'.
I think you need to add a little more information. I'd guess it is a permissions thing though. Add the permissions of the file, the directories, and the line in your crontab so we can help. Also, if you are putting this in /root, are you running this in root's crontab?
Remember the environment - especially when run by cron rather than by root. When cron runs something, you probably don't have anything much set of your environment, unlike when you run a command via at. It is also not clear what your current directory will be. So, for commands that will be run by cron, use a script (as you're already doing) and make sure it sets enough of the environment for it to run. And make sure your environment setting code is not interactive!
On my machines, I have a mechanism such that the cron entry reads (for example):
23 1 * * 1-5 /usr/bin/ksh /work1/jleffler/bin/Cron/weekday
The weekday script in the Cron directory is a link to a standard script that first sets the environment and then runs the command /work1/jleffler/bin/weekday (in this case - it uses the name of the command to determine what to run).
The actual script in the Cron directory is:
: "$Id: runcron.sh,v 2.1 2001/02/27 00:53:22 jleffler Exp $"
#
# Commands to be performed by Cron (no debugging options)
# Set environment -- not done by cron (usually switches HOME)
. $HOME/.cronfile
base=`basename $0`
cmd=${REAL_HOME:-/real/home}/bin/$base
if [ ! -x $cmd ]
then cmd=${HOME}/bin/$base
fi
exec $cmd ${#:+"$#"}
I've been using it a while now - this version since 2001 - and it works a treat for me. I'm using a basic (Sun Solaris 10) implementation of cron; there may be new features in new versions of cron on other platforms to make some of this unnecessary. (The $REAL_HOME stuff is a weirdness of mine; pretend it says $HOME - though that makes some of the script unnecessary for you.) The .cronfile is responsible for the environment setting - it does quite a lot, but that's my problem, not yours.
It could be because you're looking for relative directories/files in the script which are located when running it from /home/ but not from /root, because /root is not in /home/root nor would it look like a users homefolder in /home/
Can you check and see if it is looking for relative files, or post the script?
On another note, why don't you just set it to run from a user's homefolder then?
Another way to run sh script is place your bash script in /usr/bin directory and simply run command bash yourscript.sh without adding /usr/bin/ directory