I have a crontab set up that errors out every time I attempt to do it. It works fine in the shell. It's the format I'm using when I attempt to automatically insert the date into the filename of the database backup. Does anyone know the syntax I need to use to get cron to let me insert the date into the filename?
mysqldump -hServer -uUser -pPassword Table | gzip >
/home/directory/backups/table.$(date +"%Y-%m-%d").gz
Thanks in advance!
What about something like this for the "command" part of the crontab :
mysqldump --host=HOST --user=USER --password=PASSWORD DATABASE TABLE | gzip > /tmp/table.`date +"\%Y-\%m-\%d"`.gz
What has changed from OP is the escaping of the date format :
date +"\%Y-\%m-\%d"
(And I used backticks -- but that should do much of a difference)
(Other solution would be to put your original command in a shell-script, and execute this one from the crontab, instead of the command -- would probably be easier to read/write ^^)
The most typical reason for "works in shell but not in cron" is that commands you try to execute are not in PATH.. Reason is that shell invoked from cron aint loading same files as your login shell.
Fix: add absolute path to each command you try to execute.
Second thing i notice in your command. Syntax for running your date command looks like its not very portable. Change that to be in backticks, or run put your whole command to shellscript (also, you can use it to set your path too) and execute that script from cron..
EDIT:
During the writing my original reply my keyboard layout didnt have backticks so check what Pascal wrote.
And example of what you could do with a shellscript:
Copy following to /usr/local/bin/dumptable.sh
#!/bin/sh
/usr/bin/mysqldump --host=HOST --user=USER --password=PASSWORD DATABASE TABLE | /bin/gzip > /tmp/table.`/bin/date +"\%Y-\%m-\%d"`.gz
and then put the the /usr/local/bin/dumptable.sh into cron..
Related
I want to print the date after every bash command I run.
This could help me understand how much a command took to execute when I am away from keyboard.
I know I could do
`DATE=`date +%d/%m/%Y\ %H:%M:%S` && echo $DATE`
to get the date but I don't know how or even if it could be possible to run this command after every command I execute on bash.
I would also be interested in running the same command before every command so I could know how long a command took.
Is it possible?
What file should I edit?
For example:
$ wget google.com
15/07/2017 23:40:05
I would be happy, if I could also introduce this following feauture:
$ wget google.com
15/07/2017 23:40:05 15/07/2017 23:40:11
Program run for 00:00:06
where the first date is when I ran the program, second is when program terminated the third is self-explanatonary.
As you understood, I don't want to type every time
$ wget google.com && `DATE=`date +%d/%m/%Y\ %H:%M:%S` && echo $DATE`
To execute a cmd before every command entered, set a trap on DEBUG. Eg.
trap date DEBUG
To execute that command before emitting a prompt, set PROMPT_COMMAND:
PROMPT_COMMAND=date
This does exactly that:
PROMPT_COMMAND+=$'\n'"date +%d/%m/%Y\ %H:%M:%S"
The string in PROMPT_COMMAND gets evaluated after every command. You just need to add the date command to whatever you already had in it. ($'\n' (newline) is a somewhat more robust joiner than ; as two consecutive ; would give you a syntax error)
You can add date/time to your prompt, via PS1 variable. You could use date command, but it's more efficient to use the supported special characters, like \d for date, or \D{strftime-fmt}.
For example:
PS1='\u#\h[\D{%F} \D{%T}]\w\$ '
or, with color:
PS1='\[\033[01;32m\]\u#\h\[\033[00m\][\[\033[02;33m\]\D{%F}\[\033[08m\]T\[\033[00m\]\[\033[02;33m\]\D{%T}\[\033[00m\]]\[\033[01;34m\]\w\[\033[00m\]\$ '
will show:
user#host[2017-07-16 00:01:17]~/somedir$
Note that in the second case (with color) we have a valid ISO8601 timestamp, with a "hidden" date/time separator T in the middle. If you select it with a mouse, T is visible and can be copied. (Also double-click will select the complete timestamp, not only date or time.)
To print timestamp after every command just modify your PS1 prompt and add date to it. The only catch here is that it will tell you time when command ended and new prompt showed. So in case you have your prompt open for long time just hit enter to capture start time before running your command.
PS1="\D{%F %T} \$ "
See this arch wiki page or just google bash prompt customization.
To add time spent executing program just add time before the command
$ time wget google.com
It will give you output like this
real 0m0.177s
user 0m0.156s
sys 0m0.020s
And you can get even more lazy and for commands that you dont't feel like typing time every time you run it, just create alias.
alias wget="time wget"
Because in bash aliases are run before other commands you can do it this way even if it looks like recursion. Then you will call it as you are used to.
And of course, aliases and prompt settings can be put in your .bashrc file, so you don't have to type them every time you open terminal.
The problem...
I use trick77's IP blacklist script to configure the firewall of my apache server and am able to run his script in terminal.
However, when assigning the bash script in ipset-blacklist to crontab, it will not run no matter what I do.
Code written in crontab file for root:
#daily /var/bash/update-blacklist.sh
What I think is the culprit...
Since I haven't done this sort of thing before, I believe that the PATH of the bash script isn't set correctly... but again, I'm not sure.
I have seen others using a line such as PATH=/usr/bin:/bin:/usr/sbin:/sbin to resolve problems involving the script's location, but, I don't exactly know what this does.
I set the location of the bash file to /var/bash instead of /usr/bin and I believe that this is throwing things off.
Pardon my lack of understanding. I really am a beginner when it comes to bash.
Any help is greatly appreciated.
What I have done...
Per #EtanReisner:
Added echo here >> /tmp/update-blacklist.out to top of update-blacklist.sh and set cron to run it every minute (* * * * *).
The file was successfully created.
Added type -p curl grep egrep ipset >> /tmp/update-blacklist.out to top of update-blacklist.sh and returned:
-p: not found
curl is /usr/bin/curl
grep is /bin/grep
egrep is /bin/egrep
ipset: not found
The output from type ipset indicated that ipset was not in the cron script PATH which isn't surprising.
The default PATH for cron jobs is fairly limited.
With ipset located in /usr/sbin that is the path that must be added to the cron script's PATH variable.
You talked about this in your question
I have seen others using a line such as PATH=/usr/bin:/bin:/usr/sbin:/sbin to resolve problems involving the script's location, but, I don't exactly know what this does.
What that does is set the PATH variable to those paths (from whatever the default value was).
The PATH variable contains the paths where the shell looks for binaries/scripts/etc. to run when you try to run them as commands.
I am writing a shell script to sync to a github repo, kick off the build, then take the output file, rename it, and move it to a location where it can be seen by Apache.
It's the renaming of the file that I've got not the faintest how to do within a shell script (I have virtually no experience with shell scripts - my understanding
Compiler will create /var/espbuild/firstpart_1vXX_secondpart.bin
I need to move this file to:
/var/www/html/builds/espbuild/firstpart_1vXX_DATE_secondpart_postfix.bin
1vXX is the version number
DATE is the output of date +%m-%d
postfix is just a string.
I'm not really certain where to start for something like this - I'm sure there's a graceful way, since this is the kind of thing shell scripts are made for, but I know just about nothing about shell scripts.
Thanks in advance
You can get the result of a command into a variable by using $():
DATE=$(date +%m-%d)
Then just use it in the new filename:
INPUT=/var/espbuild/firstpart_1vXX_secondpart.bin
OUTPUT=/var/www/html/builds/espbuild/firstpart_1vXX_${DATE}_secondpart_postfix.bin
mv ${INPUT} ${OUTPUT}
Edit: To get out the version part, here's a quick example:
VERSION=$(grep -o 1v.. <<< ${INPUT})
Then OUTPUT should be set like:
OUTPUT=/var/www/html/builds/espbuild/firstpart_${VERSION}_${DATE}_secondpart_postfix.bin
You can use this in BASH:
f='/var/espbuild/firstpart_1vXX_secondpart.bin'
s="${f##*/}"
s2=${s##*_}
dest="/var/www/html/builds/espbuild/${s%_*}_$(date '+%m-%d')_${s2%.*}_postfix.bin"
echo "$dest"
/var/www/html/builds/espbuild/firstpart_1vXX_07-14_secondpart_postfix.bin
cp "$f" "$dest"
I have a script used for zipping a database and site files, then dumps the output into a backup folder on the server. The script runs fine from the command line, but it will not work through cron.
After much research, I am thinking that cron cannot run it in its current form because it runs in a different environment.
Here is the script, saved as file_name.sh
#!/bin/bash
NOW=$(date +"%Y-%m-%d-%H%M")
FILE="website.com.$NOW.tar"
BACKUP_DIR="/backupfolder"
WWW_DIR="/var/www/website/"
DB_USER="dbuser"
DB_PASS="dbpw"
DB_NAME="dbname"
DB_FILE="website.com.$NOW.sql"
WWW_TRANSFORM='s,^var/www/website,www,'
DB_TRANSFORM='s,^backupfolder,database,'
tar -cvf $BACKUP_DIR/$FILE --transform $WWW_TRANSFORM $WWW_DIR
mysqldump -u$DB_USER -p$DB_PASS $DB_NAME > $BACKUP_DIR/$DB_FILE
tar --append --file=$BACKUP_DIR/$FILE --transform $DB_TRANSFORM $BACKUP_DIR/$DB_FILE
rm $BACKUP_DIR/$DB_FILE
gzip -9 $BACKUP_DIR/$FILE
I currently have the script stored in /usr/local/scripts/
Is there something wrong with the above code that does not allow it to run through cron?
Which crontab should it go in? crontab -e from terminal, or /etc/crontab? They are two different files.
Several things come to mind: first, one of the most common problems with cron jobs is that generally crond runs things with a very minimal PATH (usually just /usr/bin:/bin), so if the script uses any commands from some other binaries directory, it'll fail. Where is mysqldump on your system (run which mysqldump if you aren't sure)? If this is the problem, adding PATH=/usr/local/bin:/usr/bin:/bin (or whatever's appropriate in your case) at the beginning of your script should fix it. Alternately, you can set PATH in the crontab file (put this line before the entry that runs your script).
If that's not the problem, my next step would be to capture the script's output, with something like:
1 1 * * * /usr/local/scripts/file_name.sh >/tmp/file_name.log 2>&1
... and see if the output is informative. BTW, as #tripleee mentioned, the format of your cron entry is suitable for the files crontab -e edits, but not for /etc/crontab. The /etc version has an additional field specifying which user to run the job as, e.g.
1 1 * * * eric /usr/local/scripts/file_name.sh >/tmp/file_name.log 2>&1
Best practice is to always use crontab -e (the resultant files are usually in /var/spool/cron/) and this works on every unix and linux platform I ever worked on.
Other common issues with cron execution are missing environment variables. Any environment variables set in .bash_profile (or .profile if you use korn shell) will not necessarily be present in the cron environment. This can be overcome by including them in your script.
As Gordon said, paths are another suspect. You can always full path you executables in your script (eg /bin/mysqldump). Some of the more cynical of us do this anyway to make sure we are executing what we intended as apposed to some other file of the same name in the current path.
I can only guess at your specific problem since you fixed it by creating /scripts, that perhaps the permissions on /usr/local/scripts directory did not allow execution by the cron user?
I have had to remove the extension (.sh) for cron to run in some instances.
So I fixed it. Not sure what the problem was, but this worked for me.
I originally had the scripts located in /usr/local/scripts/
I created a new directory here - /scripts/ and moved the scripts there. The new crontab -e command looked like this:
1 1 * * * bash /scripts/file_name.sh
Works perfectly. Again, I am not sure what the issue was before, but it works now.
I know history will capture commands that I run, but it is shell specific. I work with multiple shells and multiple hosts and would like to write a small script which, after every command I run, dumps that command to some file along with the host name. This way, i can implement my own history command which reads from that file, and can take a host as an argument which would be handy for me. I'm not sure how to get the first part though..i.e., get every shell command I type to trigger a "dump that command into a file" part. Any ideas?
Thanks
In bash, the PROMPT_COMMAND environment variable contains a command that will be executed before the PS1 prompt is displayed. So yours could be something like history | tail -n1 | perl -npe 's/^\s+\d+\s+//' | yourcommand HOST
The script utility should solve your problem. It records everything you type and all that is printed on the terminal in a file (even including terminal control codes, so if you cat that file on the console, you even reproduce the original text colors).