I'm new around the neighborhood and stuck with a syntax error. Please take a look and maybe someone can assist. I'm trying to run the following script:
#!/bin/bash
main () {
dpkg -query -s $1 &> /tmp/pkg_verify
if grep -q 'not installed' /tmp/verify
then
echo -e "\e[31m$1 is not installed. installing..\e[0m"
apt-get install $1
echo -e "\e[31m$1 is not installed and ready to use\e[0m"
else
echo -e "\e[31m$1 is already installed\e[0m"
fi
rm -f /tmp/pkg_verify
for test in $#; do main $test; shift; done
echo -e "\e[31mDone\e[0m"
}
for test in $#; do main $test; shift; done
echo -e "\e[31mDone\e[0m"
But when I try to execute it I'm facing with endless loop:
grep: /tmp/verify: No such file or directory
16 is already installed
I truly tried to find the answer, tried to change the if to couple of different forms but with out any success. Does any one have an idea why that is? What should I change so that the script can run?
Thanks in advance to all the helpers.
You have two else following each other. That can't work. It's either elif condition or just a single else.
The infinite loop is caused by main calling itself recursively.
And third, it's probably a bug to shift when iterating with for i in "$#".
To debug a script (free of syntax errors) use set -x near the beginning.
Replace this line:
if [[ -z `grep 'not installed' /tmp/pkg_verify` ]]
with this if condition:
if grep -q 'not installed' /tmp/pkg_verify
Full Script:
main () {
dpkg-query -s "$1" > /tmp/pkg_verify
if grep -q 'not installed' /tmp/pkg_verify
then
echo -e "\e[31m$1 is not installed. installing..\e[0m"
apt-get install "$1"
echo -e "\e[31m$1 is not installed and ready to use\e[0m"
else
echo -e "\e[31m$1 is already installed\e[0m"
fi
}
rm -f /tmp/pkg_verify
for test in $#; do main "$test"; done
echo -e "\e[31mDone\e[0m"
Related
I want to run a shell script when a specific file or directory changes.
How can I easily do that?
You may try entr tool to run arbitrary commands when files change. Example for files:
$ ls -d * | entr sh -c 'make && make test'
or:
$ ls *.css *.html | entr reload-browser Firefox
or print Changed! when file file.txt is saved:
$ echo file.txt | entr echo Changed!
For directories use -d, but you've to use it in the loop, e.g.:
while true; do find path/ | entr -d echo Changed; done
or:
while true; do ls path/* | entr -pd echo Changed; done
I use this script to run a build script on changes in a directory tree:
#!/bin/bash -eu
DIRECTORY_TO_OBSERVE="js" # might want to change this
function block_for_change {
inotifywait --recursive \
--event modify,move,create,delete \
$DIRECTORY_TO_OBSERVE
}
BUILD_SCRIPT=build.sh # might want to change this too
function build {
bash $BUILD_SCRIPT
}
build
while block_for_change; do
build
done
Uses inotify-tools. Check inotifywait man page for how to customize what triggers the build.
Use inotify-tools.
The linked Github page has a number of examples; here is one of them.
#!/bin/sh
cwd=$(pwd)
inotifywait -mr \
--timefmt '%d/%m/%y %H:%M' --format '%T %w %f' \
-e close_write /tmp/test |
while read -r date time dir file; do
changed_abs=${dir}${file}
changed_rel=${changed_abs#"$cwd"/}
rsync --progress --relative -vrae 'ssh -p 22' "$changed_rel" \
usernam#example.com:/backup/root/dir && \
echo "At ${time} on ${date}, file $changed_abs was backed up via rsync" >&2
done
How about this script? Uses the 'stat' command to get the access time of a file and runs a command whenever there is a change in the access time (whenever file is accessed).
#!/bin/bash
while true
do
ATIME=`stat -c %Z /path/to/the/file.txt`
if [[ "$ATIME" != "$LTIME" ]]
then
echo "RUN COMMNAD"
LTIME=$ATIME
fi
sleep 5
done
Check out the kernel filesystem monitor daemon
http://freshmeat.net/projects/kfsmd/
Here's a how-to:
http://www.linux.com/archive/feature/124903
As mentioned, inotify-tools is probably the best idea. However, if you're programming for fun, you can try and earn hacker XPs by judicious application of tail -f .
Just for debugging purposes, when I write a shell script and want it to run on save, I use this:
#!/bin/bash
file="$1" # Name of file
command="${*:2}" # Command to run on change (takes rest of line)
t1="$(ls --full-time $file | awk '{ print $7 }')" # Get latest save time
while true
do
t2="$(ls --full-time $file | awk '{ print $7 }')" # Compare to new save time
if [ "$t1" != "$t2" ];then t1="$t2"; $command; fi # If different, run command
sleep 0.5
done
Run it as
run_on_save.sh myfile.sh ./myfile.sh arg1 arg2 arg3
Edit: Above tested on Ubuntu 12.04, for Mac OS, change the ls lines to:
"$(ls -lT $file | awk '{ print $8 }')"
Add the following to ~/.bashrc:
function react() {
if [ -z "$1" -o -z "$2" ]; then
echo "Usage: react <[./]file-to-watch> <[./]action> <to> <take>"
elif ! [ -r "$1" ]; then
echo "Can't react to $1, permission denied"
else
TARGET="$1"; shift
ACTION="$#"
while sleep 1; do
ATIME=$(stat -c %Z "$TARGET")
if [[ "$ATIME" != "${LTIME:-}" ]]; then
LTIME=$ATIME
$ACTION
fi
done
fi
}
Quick solution for fish shell users who wanna track a single file:
while true
set old_hash $hash
set hash (md5sum file_to_watch)
if [ $hash != $old_hash ]
command_to_execute
end
sleep 1
end
replace md5sum with md5 if on macos.
Here's another option: http://fileschanged.sourceforge.net/
See especially "example 4", which "monitors a directory and archives any new or changed files".
inotifywait can satisfy you.
Here is a common sample for it:
inotifywait -m /path -e create -e moved_to -e close_write | # -m is --monitor, -e is --event
while read path action file; do
if [[ "$file" =~ .*rst$ ]]; then # if suffix is '.rst'
echo ${path}${file} ': '${action} # execute your command
echo 'make html'
make html
fi
done
Suppose you want to run rake test every time you modify any ruby file ("*.rb") in app/ and test/ directories.
Just get the most recent modified time of the watched files and check every second if that time has changed.
Script code
t_ref=0; while true; do t_curr=$(find app/ test/ -type f -name "*.rb" -printf "%T+\n" | sort -r | head -n1); if [ $t_ref != $t_curr ]; then t_ref=$t_curr; rake test; fi; sleep 1; done
Benefits
You can run any command or script when the file changes.
It works between any filesystem and virtual machines (shared folders on VirtualBox using Vagrant); so you can use a text editor on your Macbook and run the tests on Ubuntu (virtual box), for example.
Warning
The -printf option works well on Ubuntu, but do not work in MacOS.
I'm looking to understand what is going on in a simple script, that seems to produce random results.
What I am trying to do:
Replace variables in pre-existing files from the values defined in the environment.
This is done inside a Docker container with a bash script, that runs the command:
envsubst '$VAR1 $VAR2' < $FILE | tee $FILE
What happens:
Sometimes a $FILE in question has contents before the command, but contains nothing after the command.
How to reproduce the issue:
Dockerfile:
FROM debian:stretch
RUN apt-get update -qy
RUN apt-get install -qy gettext
COPY main-script /main-script
RUN chmod +x /main-script
ENTRYPOINT [ "/main-script" ]
Bash script:
#!/bin/bash
mkdir -p /test
export TEST1=1
export TEST2=2
export TEST3=3
for I in {1..300} ; do
echo '$TEST1 $TEST2 $TEST3' > /test/file-$I
done
for FILE in /test/file-* ; do
envsubst < $FILE | tee $FILE
done
for FILE in /test/file-* ; do
if [[ -z "$(cat $FILE)" ]]; then
echo "$FILE is empty!"
FAIL=1
fi
done
if [[ -n "$FAIL" ]]; then
exit 2
fi
Output looks something like this:
...
/test/file-11 is empty!
/test/file-180 is empty!
/test/file-183 is empty!
/test/file-295 is empty!
Pipes are asynchronous, and you've introduced a race condition. You can't predict if envsubst reads from $FILE before or after tee truncates it.
The correct approach is to write the changes to a temporary file, then replace the original with the temporary file after that has succeeded.
tmp=$(mktemp)
envsubst < "$FILE" > "$tmp" && mv "$tmp" "$FILE"
I want to run a shell script when a specific file or directory changes.
How can I easily do that?
You may try entr tool to run arbitrary commands when files change. Example for files:
$ ls -d * | entr sh -c 'make && make test'
or:
$ ls *.css *.html | entr reload-browser Firefox
or print Changed! when file file.txt is saved:
$ echo file.txt | entr echo Changed!
For directories use -d, but you've to use it in the loop, e.g.:
while true; do find path/ | entr -d echo Changed; done
or:
while true; do ls path/* | entr -pd echo Changed; done
I use this script to run a build script on changes in a directory tree:
#!/bin/bash -eu
DIRECTORY_TO_OBSERVE="js" # might want to change this
function block_for_change {
inotifywait --recursive \
--event modify,move,create,delete \
$DIRECTORY_TO_OBSERVE
}
BUILD_SCRIPT=build.sh # might want to change this too
function build {
bash $BUILD_SCRIPT
}
build
while block_for_change; do
build
done
Uses inotify-tools. Check inotifywait man page for how to customize what triggers the build.
Use inotify-tools.
The linked Github page has a number of examples; here is one of them.
#!/bin/sh
cwd=$(pwd)
inotifywait -mr \
--timefmt '%d/%m/%y %H:%M' --format '%T %w %f' \
-e close_write /tmp/test |
while read -r date time dir file; do
changed_abs=${dir}${file}
changed_rel=${changed_abs#"$cwd"/}
rsync --progress --relative -vrae 'ssh -p 22' "$changed_rel" \
usernam#example.com:/backup/root/dir && \
echo "At ${time} on ${date}, file $changed_abs was backed up via rsync" >&2
done
How about this script? Uses the 'stat' command to get the access time of a file and runs a command whenever there is a change in the access time (whenever file is accessed).
#!/bin/bash
while true
do
ATIME=`stat -c %Z /path/to/the/file.txt`
if [[ "$ATIME" != "$LTIME" ]]
then
echo "RUN COMMNAD"
LTIME=$ATIME
fi
sleep 5
done
Check out the kernel filesystem monitor daemon
http://freshmeat.net/projects/kfsmd/
Here's a how-to:
http://www.linux.com/archive/feature/124903
As mentioned, inotify-tools is probably the best idea. However, if you're programming for fun, you can try and earn hacker XPs by judicious application of tail -f .
Just for debugging purposes, when I write a shell script and want it to run on save, I use this:
#!/bin/bash
file="$1" # Name of file
command="${*:2}" # Command to run on change (takes rest of line)
t1="$(ls --full-time $file | awk '{ print $7 }')" # Get latest save time
while true
do
t2="$(ls --full-time $file | awk '{ print $7 }')" # Compare to new save time
if [ "$t1" != "$t2" ];then t1="$t2"; $command; fi # If different, run command
sleep 0.5
done
Run it as
run_on_save.sh myfile.sh ./myfile.sh arg1 arg2 arg3
Edit: Above tested on Ubuntu 12.04, for Mac OS, change the ls lines to:
"$(ls -lT $file | awk '{ print $8 }')"
Add the following to ~/.bashrc:
function react() {
if [ -z "$1" -o -z "$2" ]; then
echo "Usage: react <[./]file-to-watch> <[./]action> <to> <take>"
elif ! [ -r "$1" ]; then
echo "Can't react to $1, permission denied"
else
TARGET="$1"; shift
ACTION="$#"
while sleep 1; do
ATIME=$(stat -c %Z "$TARGET")
if [[ "$ATIME" != "${LTIME:-}" ]]; then
LTIME=$ATIME
$ACTION
fi
done
fi
}
Quick solution for fish shell users who wanna track a single file:
while true
set old_hash $hash
set hash (md5sum file_to_watch)
if [ $hash != $old_hash ]
command_to_execute
end
sleep 1
end
replace md5sum with md5 if on macos.
Here's another option: http://fileschanged.sourceforge.net/
See especially "example 4", which "monitors a directory and archives any new or changed files".
inotifywait can satisfy you.
Here is a common sample for it:
inotifywait -m /path -e create -e moved_to -e close_write | # -m is --monitor, -e is --event
while read path action file; do
if [[ "$file" =~ .*rst$ ]]; then # if suffix is '.rst'
echo ${path}${file} ': '${action} # execute your command
echo 'make html'
make html
fi
done
Suppose you want to run rake test every time you modify any ruby file ("*.rb") in app/ and test/ directories.
Just get the most recent modified time of the watched files and check every second if that time has changed.
Script code
t_ref=0; while true; do t_curr=$(find app/ test/ -type f -name "*.rb" -printf "%T+\n" | sort -r | head -n1); if [ $t_ref != $t_curr ]; then t_ref=$t_curr; rake test; fi; sleep 1; done
Benefits
You can run any command or script when the file changes.
It works between any filesystem and virtual machines (shared folders on VirtualBox using Vagrant); so you can use a text editor on your Macbook and run the tests on Ubuntu (virtual box), for example.
Warning
The -printf option works well on Ubuntu, but do not work in MacOS.
I have a build process, kicked off by Make, that executes a lot of child scripts.
A couple of these child scripts require root privileges, so instead of running everything as root, or everything as sudo, I'm trying to only execute the scripts that need to be as root, as root.
I'm accomplishing this like so:
execute_as_user() {
su "$1" -s /bin/bash -c "$2;exit \$?"
}
Arg $1 is the user to run the script as, arg $2 is the script.
Arg $1 is either root (gotten with: $(whoami) since everything is under sudo), or the current user's account (gotten with: $(logname))
The entire build is kicked off as:
sudo make all
Sample from the Makefile:
LOG="runtime.log"
ROTATE_LOG:=$(shell bash ./scripts/utils/rotate_log.sh)
system:
/bin/bash -c "time ./scripts/system.sh 2>&1 | tee ${LOG}"
My problem is... none of the child scripts are printing output to stdout. I believe it to be some sort of issue with an almost recursive call of su root... but I'm unsure. From my understanding, these scripts should already be outputting to stdout, so perhaps I'm mistaken where the output is going?
To be clear, I'm seeing no output in either the logfile nor displaying to the terminal (stdout).
Updating for clarity:
Previously, I just ran all the scripts either with sudo or just as the logged in user... which with my makefile above, would print to the terminal (stdout) and logfile. Adding the execute_as_user() function is where the issue cropped up. The scripts execute and build the project... just no display "that it's working" and no logs.
UPDATE
Here is some snippets:
system.sh snippet:
execute_script() {
echo "Executing as user $3: $2"
RETURN=$(execute_as_user $3 ${SYSTEM_SCRIPTS}/$2)
if [ ${RETURN} -ne ${OK} ]
then
error $1 $2 ${RETURN}
fi
}
build_package() {
local RETURN=0
case "$1" in
system)
declare -a scripts=(\
"rootfs.sh" \
"base_files.sh" \
"busybox.sh" \
"iana-etc.sh" \
"kernel.sh" \
"firmware.sh" \
"bootscripts.sh" \
"network.sh" \
"dropbear.sh" \
"wireless_tools.sh" \
"e2fsprogs.sh" \
"shared_libs.sh"
)
for SCRIPT_NAME in "${scripts[#]}"; do
execute_script $1 ${SCRIPT_NAME} $(logname)
echo ""
echo -n "${SCRIPT_NAME}"
show_status ${OK}
echo ""
done
# finalize base system
echo ""
echo "Finalizing base system"
execute_script $1 "finalize.sh" $(whoami)
echo ""
echo -n "finalize.sh"
show_status ${OK}
echo ""
# package into tarball
echo ""
echo "Packing base system"
execute_script $1 "archive.sh" $(whoami)
echo ""
echo -n "archive.sh"
show_status ${OK}
echo ""
echo ""
echo -n "Build System: "
show_status ${OK}
;;
*)
echo "$1 is not supported!"
exit 1
esac
}
sample child script executed by system.sh
cd ${CLFS_SOURCES}/
tar -xvjf ${PKG_NAME}-${PKG_VERSION}.tar.bz2
cd ${CLFS_SOURCES}/${PKG_NAME}-${PKG_VERSION}/
make distclean
RESPONSE=$?
if [ ${RESPONSE} -ne 0 ]
then
pkg_error ${RESPONSE}
exit ${RESPONSE}
fi
ARCH="${CLFS_ARCH}" make defconfig
RESPONSE=$?
if [ ${RESPONSE} -ne 0 ]
then
pkg_error ${RESPONSE}
exit ${RESPONSE}
fi
# fixup some bugs with musl-libc
sed -i 's/\(CONFIG_\)\(.*\)\(INETD\)\(.*\)=y/# \1\2\3\4 is not set/g' .config
sed -i 's/\(CONFIG_IFPLUGD\)=y/# \1 is not set/' .config
etc...
Here's the entire system.sh script:
https://github.com/SnakeDoc/LiLi/blob/master/scripts/system.sh
(i know the project is messy... it's a learn-as-you-go style project)
Previously, I just ran all the scripts either with sudo or just as the
logged in user... which with my makefile above, would print to the
terminal (stdout) and logfile. Adding the execute_as_user() function
is where the issue cropped up. The scripts execute and build the
project... just no display "that it's working" and no logs.
Just a guess, but you're probably not calling your function or not calling it properly:
execute_as_user() {
su "$1" -s /bin/bash -c "$2;exit \$?"
}
execute_as_user "$#"
I also noticed that you're not passing any argument to the script at all. Is this meant?
./scripts/system.sh ???
I would like to check if a certain file exists on the remote host.
I tried this:
$ if [ ssh user#localhost -p 19999 -e /home/user/Dropbox/path/Research_and_Development/Puffer_and_Traps/Repeaters_Network/UBC_LOGS/log1349544129.tar.bz2 ] then echo "okidoke"; else "not okay!" fi
-sh: syntax error: unexpected "else" (expecting "then")
In addition to the answers above, there's the shorthand way to do it:
ssh -q $HOST [[ -f $FILE_PATH ]] && echo "File exists" || echo "File does not exist";
-q is quiet mode, it will suppress warnings and messages.
As #Mat mentioned, one advantage of testing like this is that you can easily swap out the -f for any test operator you like: -nt, -d, -s etc...
Test Operators: http://tldp.org/LDP/abs/html/fto.html
Here is a simple approach:
#!/bin/bash
USE_IP='-o StrictHostKeyChecking=no username#192.168.1.2'
FILE_NAME=/home/user/file.txt
SSH_PASS='sshpass -p password-for-remote-machine'
if $SSH_PASS ssh $USE_IP stat $FILE_NAME \> /dev/null 2\>\&1
then
echo "File exists"
else
echo "File does not exist"
fi
You need to install sshpass on your machine to work it.
Can't get much simpler than this :)
ssh host "test -e /path/to/file"
if [ $? -eq 0 ]; then
# your file exists
fi
As suggested by dimo414, this can be collapsed to:
if ssh host "test -e /path/to/file"; then
# your file exists
fi
one line, proper quoting
ssh remote_host test -f "/path/to/file" && echo found || echo not found
You're missing ;s. The general syntax if you put it all in one line would be:
if thing ; then ... ; else ... ; fi
The thing can be pretty much anything that returns an exit code. The then branch is taken if that thing returns 0, the else branch otherwise.
[ isn't syntax, it's the test program (check out ls /bin/[, it actually exists, man test for the docs – although can also have a built-in version with different/additional features.) which is used to test various common conditions on files and variables. (Note that [[ on the other hand is syntax and is handled by your shell, if it supports it).
For your case, you don't want to use test directly, you want to test something on the remote host. So try something like:
if ssh user#host test -e "$file" ; then ... ; else ... ; fi
Test if a file exists:
HOST="example.com"
FILE="/path/to/file"
if ssh $HOST "test -e $FILE"; then
echo "File exists."
else
echo "File does not exist."
fi
And the opposite, test if a file does not exist:
HOST="example.com"
FILE="/path/to/file"
if ! ssh $HOST "test -e $FILE"; then
echo "File does not exist."
else
echo "File exists."
fi
ssh -q $HOST [[ -f $FILE_PATH ]] && echo "File exists"
The above will run the echo command on the machine you're running the ssh command from. To get the remote server to run the command:
ssh -q $HOST "[[ ! -f $FILE_PATH ]] && touch $FILE_PATH"
Silent check if file exist and perform if not
if ! ssh $USER#$HOST "test -e file.txt" 2> /dev/null; then
echo "File not exist"
fi
You can specify the shell to be used by the remote host locally.
echo 'echo "Bash version: ${BASH_VERSION}"' | ssh -q localhost bash
And be careful to (single-)quote the variables you wish to be expanded by the remote host; otherwise variable expansion will be done by your local shell!
# example for local / remote variable expansion
{
echo "[[ $- == *i* ]] && echo 'Interactive' || echo 'Not interactive'" |
ssh -q localhost bash
echo '[[ $- == *i* ]] && echo "Interactive" || echo "Not interactive"' |
ssh -q localhost bash
}
So, to check if a certain file exists on the remote host you can do the following:
host='localhost' # localhost as test case
file='~/.bash_history'
if `echo 'test -f '"${file}"' && exit 0 || exit 1' | ssh -q "${host}" sh`; then
#if `echo '[[ -f '"${file}"' ]] && exit 0 || exit 1' | ssh -q "${host}" bash`; then
echo exists
else
echo does not exist
fi
I wanted also to check if a remote file exist but with RSH. I have tried the previous solutions but they didn't work with RSH.
Finally, I did I short function which works fine:
function existRemoteFile ()
{
REMOTE=$1
FILE=$2
RESULT=$(rsh -l user $REMOTE "test -e $FILE && echo \"0\" || echo \"1\"")
if [ $RESULT -eq 0 ]
then
return 0
else
return 1
fi
}
On CentOS machine, the oneliner bash that worked for me was:
if ssh <servername> "stat <filename> > /dev/null 2>&1"; then echo "file exists"; else echo "file doesnt exits"; fi
It needed I/O redirection (as the top answer) as well as quotes around the command to be run on remote.
This also works :
if ssh user#ip "[ -s /path/file_name ]" ;then
status=RECEIVED ;
else
status=MISSING ;
fi
#its simple
if [[ "`ssh -q user#hostname ls /dir/filename.abc 2>dev/null`" == "/dir/filename.abc" ]]
then
echo "file exists"
else
echo "file not exists"
fi