I need a scheduler (for one time only actions) for a site I'm coding (in php), and I had two ideas:
1- Run a php script with crontab and verify against a database of scheduled actions and execute ones that are older than current time.
2- Schedule various tasks with the "at" command.
The second option seems much better and simpler, so that's what I'm trying to do. However, I haven't found a way to tell "at" to run a command using the PHP interpreter, and so far I've been creating a .sh script, which contains a single command, which is to run a file through the php interpreter. That is far from the optimal setting, and I wish I could just execute the php code directly through "at", something like:
at -e php -f /path/to/phpscript time
Is it possible? I haven't found anything about using environments other than bash in either the man or online.
You can prepend phpscript with a #!/usr/bin/php (or wherever your php script is stored) and make /path/to/phpscript executable. This is exactly what the #! syntax is for.
Just so it's clear, your phpscript would look like this:
#!/usr/bin/php
...your code goes here
The command you specify to at is executed by /bin/sh, but sh can invoke any command, executed directly or by any specified interpreter.
The following works on my Ubuntu 12.04 system with the bash shell:
$ cat hello.php
#!/usr/bin/php
<?php
echo "Hello, PHP\n";
?>
$ echo "$PWD/hello.php > hello.php.out" | at 16:11
warning: commands will be executed using /bin/sh
job 4 at Sat Aug 25 16:11:00 2012
$ date
Sat Aug 25 16:11:05 PDT 2012
$ cat hello.php.out
Hello, PHP
$
In some cases, you'll have to do some extra work to set environment variables correctly (it's not necessary for this simple case). Quoting the man page:
For both at and batch, commands are read from standard input or the
file specified with the -f option and executed. The working directory,
the environment (except for the variables BASH_VERSINFO, DISPLAY,
EUID, GROUPS, SHELLOPTS, TERM, UID, and _) and the umask are retained
from the time of invocation.
As at is currently implemented as a setuid program, other environment
variables (e.g. LD_LIBRARY_PATH or LD_PRELOAD) are also not exported.
This may change in the future. As a workaround, set these variables
explicitly in your job.
Related
I want to know the use of . ./.profile whenever we execute cron jobs. I have seen many scripts having this included. The question is, what is the use and what if I don't add it?
Example:
00 1-22 * * 1-5 . ./.profile ; /global/u1/sie/rox/Scripts/Calls.ksh >/dev/null 2>&1
. somefile is the POSIX-compliant equivalent to the bash builtin source: Running source somefile in bash, or . somefile in any POSIX-compliant shell, executes every command inside that script in that existing shell.
In terms of why this is useful in a crontab: cron runs with a very minimal environment -- it may not even have a PATH set, and is unlikely to have many other facilities. If your scripts depend on environment variables being present, it can be necessary to either specify them in the crontab or to source in (that is, execute in the existing shell) a script which defines them.
That said, I advise against this idiom:
.profile is used by login sessions -- sessions with a user interacting with the shell in real-time -- and folks intending to customize their interactive session's behavior are liable to make modifications without keeping scheduled jobs in mind.
It's not obvious by reading your crontab which environment variables ~/.profile will or won't set, and thus difficult to reason about the state of the environment.
Instead, you should set environment variables at the top of your crontab:
PATH=/bin:/usr/bin:/usr/local/bin
VARNAME=VALUE
# ...etc...
0 1-22 * * 1-5 /global/u1/sie/rox/Scripts/Calls.ksh >/dev/null 2>&1
The profile files are the shell profiles, you can add code to it that will run as soon as the shell start up, ./profile is the profile file for Ksh and Bourne, /.bash_profile is for bash /.login is for Tcsh and Csh.
When a script calls the profile it's because it needs something from it, i.e $path variables or even specific commands that it might not have access to. In this case, since cron doesn't have access to much since it runs in a minimal enviroment that script will pull the .profile because it depends on something that's in there.
More info here
and here
I infrequently have to write bash scripts for various unrelated purposes and while I usually have a good idea what commands I want in the script, I often have no idea what header to use or why I'm using one when I do find it. For example(s):
Standard shell script:
#!/bin/bash
Python:
#!/usr/bin/env python
Scripts seem to work fine without headers but if headers are the standard, there's a reason for them and they shouldn't be ignored. If it has an effect, then it's a valuable tool that could be used to accomplish more.
Minimally, I'd like to know what headers to use with MySQL scripts and what the headers do on Standard, Python, and MySQL scripts. Ideally, I'd like a generic list of headers or an understanding of how to create a header based on what program is being used.
How the Kernel Executes Things
Simplified (a bit), there are two ways the kernel in a POSIX system knows how to execute a program. One, if the program is in a binary format the kernel understands (such as ELF), the kernel can execute it "directly" (more detail out of scope). If the program is a text file starting with a shebang, such as
#!/usr/bin/somebinary -arg
or what-have-you, the kernel actually executes the command as if it had been directed to execute:
/usr/bin/somebinary -arg "$0"
where $0 here is the name of the script file you just tried to execute. (So you can immediately tell why so many scripting languages use # as a comment-starter – it means they don't have to treat the shebang as special.)
PATH and the env command
The kernel does not look at the PATH environment variable to determine which executable you're talking about, so if you are distributing a python script to systems that may have multiple versions of python installed, you can't guarantee that there will be a
#!/usr/bin/python
env, however, is POSIX, so you can count on it existing, and it will look up python in PATH. Thus,
#!/usr/bin/env python
will execute the script with the first python found in your PATH.
BASH, SH and Special Meanings for Invocation
Some programs have special semantics for how they're invoked. In particular, on many systems /bin/sh is a symlink to another shell, such as /bin/bash. While bash does not contain a perfectly POSIXLY_STRICT implementation of sh, when it is invoked as /bin/sh it is stricter than it would be if invoked as plain-old-bash.
MySQL and arg limitations
The shebang line can be length limited and technically, it can only support one argument, so mysql is a bit tricky – you can't expect to pass a username and database name to a mysql script.
#!/usr/bin/env mysql
use mydb;
select * from mytbl;
Will fail because the kernel will try mysql "$0". Even if you have your credentials in a .my.cnf file, mysql itself will try to treat "$0" as a database name. Likewise:
#!/usr/bin/mysql -e
use mydb;
select * from mytbl;
will fail because again, "$0" is not a table name (you hope).
There does not seem to be an appropriate syntax for directly executing a mysql script this way. Your best bet is to pipe the sql commands to mysql directly:
mysql < my_sql_commands
http://mywiki.wooledge.org/BashGuide/Practices#Choose_Your_Shell
When the first line of a script starts with #!, that's what's called a "shebang". When that script is run as an executable, the operating system uses that line to determine how to run the script -- that is to say, to find the program with which the script should be executed.
It's incorrect that "scripts work fine without headers" -- if you don't have a shebang line, you can't be invoked using the execve() call, which means that many (most?) programs won't be able to execute your script. Sometimes invocation from a shell will try to use that shell itself in the absence of a shebang, but you can't trust that to be the case.
(There's an exception to that -- if someone starts your script by running sh yourscript or bash yourscript, the shebang line isn't read at all, and the script they chose is used; however, running scripts this way is a bad practice, as the author typically knows better than the user what the correct interpreter is).
In short:
If you want to use modern features, and you want the user to be able to override the shell version in use by putting a different release of bash earlier in their path, use #!/usr/bin/env bash
If you want to use modern features and ensure that you always run with the system shell, use #!/bin/bash
If you're going to write your script to strictly conform with POSIX sh, use #!/bin/sh
There's not a limited list of shebang lines we can give you, since any native executable (non-script program) can be used as a script interpreter, and thus be placed in a shebang. If you created a file called myscript with #!/usr/bin/env yourprogram, gave it executable permissions, and ran ./myscript foo bar, this would result in /usr/bin/env yourprogram myscript foo bar being invoked; yourprogram would be run by /usr/bin/env (after a PATH lookup), and would be responsible for knowing what to do with myscript and its arguments.
For an extremely detailed history of shebang lines and how they work across systems both modern and ancient, see http://www.in-ulm.de/~mascheck/various/shebang/
I have a simple script cmakeclean to clean cmake temp files:
#!/bin/bash -f
rm CMakeCache.txt
rm *.cmake
which I call like
$ cmakeclean
And it does remove CMakeCache.txt, but it doesn't remove cmake_install.cmake:
rm: *.cmake: No such file or directory
When I run it like:
$ . cmakeclean
it does remove both.
What is the difference and can I make this script work like an usual linux command (without . in front)?
P.S.
I am sure the both times is same script is executed. To check this I added echo meme in the script and rerun it in both ways.
Remove the -f from your #!/bin/bash -f line.
-f prevents pathname expansion, which means that *.cmake will not match anything. When you run your script as a script, it interprets the shebang line, and in effect runs /bin/bash -f scriptname. When you run it as . scriptname, the shebang is just seen as a comment line and ignored, so the fact that you do not have -f set in your current environment allows it to work as expected.
. script is short for source script which means the current shell executes the commands in the script. If there's an exit in there, the current shell will exit (and e. g. the terminal window will close).
This is typically used to modify the environment of the current shell (set variables etc.).
script asks the shell to fork itself, then exec the given script in the child process, and then wait in the father for the termination of the child. If there's an exit in the script, this will be executed by the child shell and thus only terminate this. The father shell stays intact and unaltered by this call.
This is typically used to start other programs from the current shell.
Is this about ClearCase? What did you do in your poor life where you've been assigned to work in the deepest bowels of hell?
For years, I was a senior ClearCase Administer. I haven't touched it in over a decade. My life is way better now. The sky is bluer, bird songs are more melodious, and my dread over coming to work every day is now a bit less.
Getting back to your issue: It's hard to say exactly what's going on. ClearCase does some wacky things. In a dynamic view, the ClearCase repository on Unix systems is hidden in the shell's environment. Now you see it, now you don't.
When you run a shell script, it starts up a new environment. If a particular shell variable is not imported, it is invisible that shell script. When you merely run cmakeclean from the command line, you are spawning a new shell -- one that does not contain your ClearCase environment.
When you run a shell script with a dot prefix like . cmakeclean, you are running that shell script in the current shell which contains your ClearCase environment. Thus, it can see your ClearCase view.
If you're using a snapshot view, it is possible that you have a $HOME/.bashrc that's changing directories on you. When a new shell environment runs in BASH (the default shell in MacOS X and Linux), it first runs $HOME/.bashrc. If this sets a particular directory, then you end up in that directory and not in the directory where you ran your shell script. I use to see this when I too was involved in ClearCase hell. People setup their .kshrc script (it was the days before BASH and most people used Kornshell) to setup their views. Unfortunately, this made running any other shell script almost impossible to do.
OS: Amazon Linux
I have a Ruby script that connects to a site, then it searches with an XPath request for a div block where is the stats counter I want to parse.
Then it compares the number from the site with the current value in the database, if the number has increased it sends me an email.
The problem is that, then I run the script from the current directory it works.
The script parses the block of text which contains a value.
I extract the value with Regex like this (/\d/)
...
But when it the script executes by crontab it gets some strange value like
...041704300440043504330438044104420440043804400430432043004304304304304304404370430432043004420435043043504390447043504400435043704320430044804430430...
I don't know how to debug it because, when I run the script manually it works, but fails with strange value when executed by crontab.
The text in the site is russian, encoded with Windows-1251.
Maybe there is something wrong with that.
I have set # encoding: utf-8, in the .rb file.
That could be an environment problem, which could include bad paths, etc. You can compare your ENV from the command-line to the environment when launched by crontab.
Try:
ruby -rpp -e 'pp ENV' > /tmp/crontab_env.out
from crontab, then:
ruby -rpp -e 'pp ENV' > /tmp/cmd_env.out
from the command-line, then:
vimdiff /tmp/*env.out
or use a regular editor.
If you're using RVM, note that it is typically only available to interactive shells. There's a whole section in the RVM manual dedicated to this topic: RVM: Ruby Version Manager - Using Cron with RVM
It could be that this is simply a problem of the wrong Ruby version, including its Gems, being used. Try removing the hashbang line in your script, and calling it like this in your crontab:
1 0 * * * /usr/local/rvm/bin/ruby-1.9.3-p362 /path/to/script.rb
This should make sure the proper environment is loaded with the Ruby binary.
If the actual problem is that RVM isn't even available for non-interactive scripts, you could also go one step further and do what your shell does when it's loading RVM—scroll to the right, this is a big line:
1 0 * * * /bin/bash -l -c 'source "$HOME/.rvm/scripts/rvm" && rvm use 1.9.3-p362 && ruby /path/to/script.rb
Problems with cron jobs are often caused by having the wrong environment. The script probably depends on an environment variable that's set when you start an interactive shell (through ~/.profile, ~/.bashrc or similar), but not when your program is started directly, by cron.
Get a list of environment variables and their current values by typing env. Add a cron job that simply runs env. Compare the outputs and chip away until you find the culprit.
I'd say LANG and friends are a good place to start. Get a list of language and encoding-related environment variables by typing locale.
If I run a script from /home/<user>/<dir>/script.sh, as root, the cron works pretty well. But If I run the script from /root/<dir>/script.sh (as root, again), the cron does not seem to work.
Having run afoul of various default $PATHs in the past when using 'cron', I always spell in full the absolute $PATH for each executable file and each target file. I always assume that 'cron' has NO $PATH set and has NO current-working-directory.
In other words don't use a command like
"myprocess abc*.txt"
but do it in full like
"/usr/localbin/myprocess /home/jvs/abc*.txt".
Alternatively, create a bash script which does the job, and call that bash script with a full absolute path, such as
"/usr/local/bin/myprocess_abc_txts".
If you need to have some flexibility in the script, use environment variables which are set specifically within the bash script you call with 'cron'.
I think you need to add a little more information. I'd guess it is a permissions thing though. Add the permissions of the file, the directories, and the line in your crontab so we can help. Also, if you are putting this in /root, are you running this in root's crontab?
Remember the environment - especially when run by cron rather than by root. When cron runs something, you probably don't have anything much set of your environment, unlike when you run a command via at. It is also not clear what your current directory will be. So, for commands that will be run by cron, use a script (as you're already doing) and make sure it sets enough of the environment for it to run. And make sure your environment setting code is not interactive!
On my machines, I have a mechanism such that the cron entry reads (for example):
23 1 * * 1-5 /usr/bin/ksh /work1/jleffler/bin/Cron/weekday
The weekday script in the Cron directory is a link to a standard script that first sets the environment and then runs the command /work1/jleffler/bin/weekday (in this case - it uses the name of the command to determine what to run).
The actual script in the Cron directory is:
: "$Id: runcron.sh,v 2.1 2001/02/27 00:53:22 jleffler Exp $"
#
# Commands to be performed by Cron (no debugging options)
# Set environment -- not done by cron (usually switches HOME)
. $HOME/.cronfile
base=`basename $0`
cmd=${REAL_HOME:-/real/home}/bin/$base
if [ ! -x $cmd ]
then cmd=${HOME}/bin/$base
fi
exec $cmd ${#:+"$#"}
I've been using it a while now - this version since 2001 - and it works a treat for me. I'm using a basic (Sun Solaris 10) implementation of cron; there may be new features in new versions of cron on other platforms to make some of this unnecessary. (The $REAL_HOME stuff is a weirdness of mine; pretend it says $HOME - though that makes some of the script unnecessary for you.) The .cronfile is responsible for the environment setting - it does quite a lot, but that's my problem, not yours.
It could be because you're looking for relative directories/files in the script which are located when running it from /home/ but not from /root, because /root is not in /home/root nor would it look like a users homefolder in /home/
Can you check and see if it is looking for relative files, or post the script?
On another note, why don't you just set it to run from a user's homefolder then?
Another way to run sh script is place your bash script in /usr/bin directory and simply run command bash yourscript.sh without adding /usr/bin/ directory