When running a script with cron, any executable called inside must have the full path. I discovered this trying to run wondershaper, when many errors showed when it tried to call tc. So my question is, what's the proper way to overcome this problem?
Possible solutions:
cd to the executable folder and prepare symbolic links to any other called executable there (not sure if it works - low portability)
use full paths in the script (it works - low portability across different distros)
exporting a path variable with the needed paths inside the script (not sure if it works)
Well, thanks in advance for anyone helping.
If you're on linux/bsd/mac you can set some environment variables like PATH right in the crontab, and with that you're generally good to go.
If you're on Solaris, well, I pray for you. But, I do have an answer too: I generally source .profile before running anything:
0 0 * * 0 . /home/myuser/.profile && cd /path && ./script
Mind you, my .profile loads .bash_profile and .bashrc. Just be sure whatever file you source has what you need.
Declaring variables inside your cron job is more explicit and easier to maintain : all you have to modify is contained in your cron job, and you don't need to transfer multiple files should you move it to another system.
PATH=/usr/bin:/your/fancy/dir
MYAPPROOT=/var/lib/myapp
*/2 * * * * myappinpath
*/3 * * * * $MYAPPROOT/mylocalapp
Since cron does not run login, .profile and /etc/profile are not sourced. Therefore PATH may not be set to a value you expect. I would either
set and export PATH to an appropriate value
use full paths in the script
Your trick with symlinks assumes . is in the PATH and just does not seem nice
My recomendation:
Set all variables in a external file. I use 'process_name.env' file located in /etc/process_name or similar. Imagine you have a backup script. Then you:
Create /etc/backup.env and put all environment variables needed for do the "backup" task.
Modify your backup script and add this line after Shebang:
. /etc/backup.env #There is a dot and a space before full path to backup environment.
IMO this approach is better than declaring variables at CRON definitions because:
Easy to maintain. Just edit a file.
Easy to switch configuration/centralized configuration:
You can have multiple .env for using your script in different situations (for example, consider you have backup locations on your .env, you can pass .env location as an argument and run your cron job daily providing an .env with few locations and weekly with different locations by providing another .env, just a example).
You can keep your .env files in a VCS like SVN or Git.
Much easy to test your scripts (there is no need to execute it from CRON).
Regards
Related
I want to add a small script to the linux PATH so I don't have to actually run it where it's physically placed on disk.
The script is quite simple is about giving apt-get access through a proxy I made it like this:
#!/bin/bash
array=( $# )
len=${#array[#]}
_args=${array[#]:1:$len}
sudo http_proxy="http://user:password#server:port" apt-get $_args
Then I saved this as apt-proxy.sh, set it to +x (chmod) and everything is working fine when I am in the directory where this file is placed.
My question is : how to add this apt-proxy to PATH so I can actually call it as if it where the real apt-get ? [from anywhere]
Looking for command line only solutions, if you know how to do by GUI its nice, but not what I am looking for.
Try this:
Save the script as apt-proxy (without the .sh extension) in some directory, like ~/bin.
Add ~/bin to your PATH, typing export PATH=$PATH:~/bin
If you need it permanently, add that last line in your ~/.bashrc. If you're using zsh, then add it to ~/.zshrc instead.
Then you can just run apt-proxy with your arguments and it will run anywhere.
Note that if you export the PATH variable in a specific window it won't update in other bash instances.
You want to define that directory to the path variable, not the actual binary e.g.
PATH=$MYDIR:$PATH
where MYDIR is defined as the directory containing your binary e.g.
PATH=/Users/username/bin:$PATH
You should put this in your startup script e.g. .bashrc such that it runs each time a shell process is invoked.
Note that order is important, and the PATH is evaluated such that if a script matching your name is found in an earlier entry in the path variable, then that's the one you'll execute. So you could name your script as apt-get and put it earlier in the path. I wouldn't do that since it's confusing. You may want to investigate shell aliases instead.
I note also that you say it works fine from your current directory. If by that you mean you have the current directory in your path (.) then that's a potential security risk. Someone could put some trojan variant of a common utility (e.g. ls) in a directory, then get you to cd to that directory and run it inadvertently.
As a final step, after following the solution form proposed by #jlhonora (https://stackoverflow.com/a/20054809/6311511), change the permissions of the files in the folder "~/bin". You can use this:
chmod -R 755 ~/bin
make an alias to the executable into the ~/.bash_profile file and then use it from anywhere or you can source the directory containing the executables you need run from anywhere and that will do the trick for you.
adding to #jlhonora
your changes in ~./bashrc or ~./zshrc won't reflect until you do
source ~./zshrc or source ./bashrc , or restart your pc
Maybe the title is a bit "stupid" but I do not know how to express my question and how to search for the question also, even if it is something very simple.
I have a set of scripts that produce a set of reports in the folder they are executed by. For example I have the script "my_script.sh" in the folder /a/folder/ and in this folder a set of output is stored. Since I have a lot of experiments that I want to let them run for the whole week I was thinking of creating a bash script that will call all the other scripts.
But the output will be stored in the folder that the global script is present.
For example:
/global/folder/global_script.sh
---> All the output is stored in this folder.
The global_script.sh may contain something like this:
/experiments/exp1/script1.sh >report1.txt
/experiments/exp1/script2.sh >report2.txt
/experiments/exp1/script2.sh >report3.txt
And I want the output of the bash scripts to be in their folder and not in the global folder.
Currently I am doing this manually navigating to the folder and executing the script.
(Ok I can change the code and use absolute paths! but is any better way to do that? )
you could change the working directory before you execute each script, or redirect the output to the directory you want:
cd /experiments/exp1/
sh /experiments/exp1/script1.sh >report1.txt
or
sh /experiments/exp1/script1.sh > /experiments/exp1/report1.txt
What's wrong with simply changing directory?
cd /experiments/exp1
./script1.sh >report1.txt
./script2.sh >report2.txt
./script2.sh >report3.txt
In my workplace, there's one Perl script that runs on a Unix machine every time someone tries to check-in a file to the SVN repo for any of the 10-20 projects.
The way it works is that each project has its own "Hooks" folder with a file called "pre-commit" which SVN automatically executes when someone check-in something. Except the "pre-commit" file is actually a symbolic link to the one central Perl script common to all projects just so that if a change needs to be made to the Perl script it doesn't need to be done for every project.
So my problem is this: I need to put a text file in each of these projects' "hooks" directory, each one containing some settings specific to that project. So there will be 10-20 settings files (one per project) each in their respective "hooks" directory.
The problem is that I need to open these text files in the Perl script and read from them but I'm having issues letting Perl know where to find it. I tried using the $0 parameter which is supposed to tell me where the script is being executed from but because it's a symbolic link it just says "Not a directory" and the script terminates. I need to get the path of the "hooks" directory so that I can find the text file.
The SVN pre-commit script is supposed to be invoked with the path to the repository as its first argument. Inside a Perl script, that argument should be available as $ARGV[0]. You should be able to build the path to the corresponding hooks directory or to a file inside that directory by simply appending to the repository path, like this:
$repopath = $ARGV[0];
$hookspath = $repopath . "/hooks";
$myfilepath = $hookspath . "/myfile";
although for maximum portability it would be cleaner to use the pathname-manipulation functions in the File::Spec module to do this.
If this approach doesn't work then you'll have to explain more about how your Perl script gets invoked. For instance, if your pre-commit script is really a shell script wrapper that eventually invokes perl then perhaps it's not passing the pre-commit arguments along properly.
Showing us your current code that's failing would be a good thing too.
I am doing the following in a shell script:
tar cvzf mytar.tgz *
It works fine when I run the shell script from a terminal. When it runs the shell script from a cron job using crontab it looks like it is archived because the tgz file is there but the filesize is nothing and when I untar it there is nothing there. However, when I run the shell script via terminal the tgz has a larger filesize and I can untar them.
Anyone know why it won't work via the cronjob?
Try specifying the complete path to the files you want to archive:
tar cvzf mytar.tgz /path/to/your/files/*
Cron runs from a different directory from your $HOME.
What's the working directory of the cronjob process? If there's nothing in it, then the command will archive all of the nothing.
First, no need to be verbose in a cron.
Second, it looks like you are using relative pathing there. Consider using absolute paths, even for the tar command itself.
Last, which user is running the cron? Is there a potential for a permissions issue or a quota issue?
The other answers so far give good advice. Cron has a lot of special rules wrt what is allowed in the command. I have he most success when I make a simple shell script, and put it in $HOME/cron, chmox 755 it and put the full path to it in cron. Making sure to test the script - ensuring to cd'ing as necessary. Be aware that cron not only won't necessarily run the command from your home, but it will also likely have a different PATH and other environment settings will be missing.
I want to make the following kind of reference:
"ls" command, for example, is universally available in most *nix environments. User can type in from anywhere to execute the scripts.
So, I write script "x". I want to make sure that from wherever the user type in x, the actual script "x" is referenced.
Thus, if I have script "x" stored in home/user/Desktop directory, I should not have to reference the script as follow:
home/user/Desktop/x
I should be able to do:
x
Thanks!
You want to add the directory to your PATH. E.g.
PATH="$PATH:/home/user/someDirectory"
You can add this line to .bash_profile to do it on startup. However, you probably shouldn't add Desktop to the path because some browsers download to there by default (though it shouldn't be executable by default).
You can also put your script in an existing directory that's already in your path such as /usr/local/bin or create a symlink there to your script's location.
cp /home/user/Desktop/x /usr/local/bin
or
mv /home/user/Desktop/x /usr/local/bin
or
ln -s /home/user/Desktop/x /usr/local/bin
Don't mean to be obnoxiously repetitive, but this is my first time answering a question, I can't reply to someone's already-good answer, and I think they are missing some important bits.
First, if you want to make sure everyone can access the script, you'll need to be sure everyone has execute permissions:
chmod a+x /path/to/script.sh
You'll also want to make sure it's in somewhere $PATH references (as the other answers mentioned):
echo $PATH # place the script in one of these directories
I would personally prefer /usr/local/bin, since that's considered the place for custom global scripts. Something the other answers didn't mention is that, if you do want to use a directory besides one in $PATH (say, /opt/myscriptfolder/) you'll want to add another PATH entry at the end of /etc/profile:
PATH="$PATH:/opt/myscriptfolder/"
By putting this in the end of /etc/profile, all users will receive this modified PATH variable on their next login.