I have a bash script which I would like to handle spaces. I know there a ton of questions on here about this, but I was unable to resolve my problem.
According to what I've read, the following should work. The space in
../tool_profile/OS\ Firmware/updater is being escaped. In the script, the $2 variable is being enclosed in quotes when being assigned to DEST.
If I pass this path in to ls enclosed in quotes or with escaped spaces on the command line, it works.
example script command:
./make_initramfs.sh initramfs_root/ ../tool_profile/OS\ Firmware/updater/ initramfs
error from ls in script:
ls: cannot access ../tool_profile/OS Firmware/updater/: No such file or directory
make_initramfs.sh:
#!/bin/bash
if [ $# -ne 3 ]; then
echo "Usage: `basename $0` <root> <dest> <archive_name>"
exit 1
fi
ROOT=$1
DEST="$2"
NAME=$3
echo "[$DEST]"
# cd and hide output
cd $ROOT 2&>/dev/null
if [ $? -eq 1 ]; then
echo "invalid root: $ROOT"
exit 1
fi
ls "$2" # doesn't work
ls "$DEST" # doesn't work
# check for 'ls' errors
#if [ $? -eq 1 ]; then
# echo "invalid dest: $DEST"
# exit 1
#fi
#sudo find . | sudo cpio -H newc -o | gzip --best > "$DEST"/"$NAME"
Thank you for any clues to what I am doing wrong! ^_^
Okay... so right as I submitted this I realized what I was doing wrong.
I was passing two relative paths in and changing to the first one before verifying the second one. So once I changed directory, the second relative path was no longer valid. I will post an updated script once I get it finished.
Edit: I finished my script. See below.
Edit2: I updated this based on everyone's comments. Thanks everyone!
make_initramfs.sh:
#!/bin/bash
if (( $# != 2 )); then
echo "Usage: `basename $0` <root> <dest>"
exit 1
fi
root="$1"
archive="${2##*/}"
dest="$PWD/${2%/*}"
# cd and hide errors
cd "$root" &>/dev/null
if (( $? != 0 )); then
echo "invalid path: $root"
exit 1
fi
if [ ! -d "$dest" ]; then
echo "invalid path: $dest"
exit 1
fi
if [ "$archive" = "" ]; then
echo "no archive file specified"
exit 1
fi
if [ `whoami` != root ]; then
echo "please run this script as root or using sudo"
exit 1
fi
find . | cpio -H newc -o | gzip --best > "$dest"/"$archive"
Related
Just looking for some help with my mounting shell script, wondering if anyone could advice me on how to make it check for the directory at the mount point exists and is empty, or is created by the script if it does not exist
#!/bin/bash
MOUNTPOINT="/myfilesystem"
if grep -qs "$MOUNTPOINT" /proc/mounts; then
echo "It's mounted."
else
echo "It's not mounted."
mount "$MOUNTPOINT"
if [ $? -eq 0 ]; then
echo "Mount success!"
else
echo "Something went wrong with the mount..."
fi
fi
Your use of grep will return any mountpoint that contains the string /myfilesystem in... e.g: both of these:
/myfilesystem
/home/james/myfilesystem
Prefer to use something more prescriptive like the following:
mountpoint -q "${MOUNTPOINT}"
You can use [ to test if a path is a directory:
if [ ! -d "${MOUNTPOINT}" ]; then
if [ -e "${MOUNTPOINT}" ]; then
echo "Mountpoint exists, but isn't a directory..."
else
echo "Mountpoint doesn't exist..."
fi
fi
mkdir -p will create all parent directories, as necessary:
mkdir -p "${MOUNTPOINT}"
Finally, test if a directory is empty by exploiting bash's variable expansion:
[ "$(echo ${MOUNTPOINT}/*)" != "${MOUNTPOINT}/*" ]
It's also a good idea to run scripts with some level of 'safety'. See the set built-in command: https://linux.die.net/man/1/bash
-e Exit immediately if a pipeline (which may consist of a single simple command), a
list, or a compound command (see SHELL GRAMMAR above), exits with a non-zero
status.
-u Treat unset variables and parameters other than the special parameters "#" and "*"
as an error when performing parameter expansion.
In full: (note bash -eu)
#!/bin/bash -eu
MOUNTPOINT="/myfilesystem"
if [ ! -d "${MOUNTPOINT}" ]; then
if [ -e "${MOUNTPOINT}" ]; then
echo "Mountpoint exists, but isn't a directory..."
exit 1
fi
mkdir -p "${MOUNTPOINT}"
fi
if [ "$(echo ${MOUNTPOINT}/*)" != "${MOUNTPOINT}/*" ]; then
echo "Mountpoint is not empty!"
exit 1
fi
if mountpoint -q "${MOUNTPOINT}"; then
echo "Already mounted..."
exit 0
fi
mount "${MOUNTPOINT}"
RET=$?
if [ ${RET} -ne 0 ]; then
echo "Mount failed... ${RET}"
exit 1
fi
echo "Mounted successfully!"
exit 0
Here is how can you check directory exist and it is empty:
if [ -d /myfilesystem ] && [ ! "$(ls -A /myfilesystem/)" ]; then
echo "Directory exist and it is empty"
else
echo "Directory doesnt exist or not empty"
fi
have some problem with shell script.
In our office we set up only few commands, that available for devs when they are trying ssh to server. It is configured with help of .ssh/authorized_keys file and available command for user there is bash script:
#!/bin/sh
if [[ $1 == "--help" ]]; then
cat <<"EOF"
This script has the purpose to let people remote execute certain commands without logging into the system.
For this they NEED to have a homedir on this system and uploaded their RSA public key to .ssh/authorized_keys (via ssh-copy-id)
Then you can alter that file and add some commands in front of their key eg :
command="/usr/bin/dev.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
The user will do the following : ssh testuser#server tail testserver.example.com/2017/01/01/user.log
EOF
exit 0;
fi
# set global variable
set $SSH_ORIGINAL_COMMAND
# set the syslog path where the files can be found
PATH="/opt/syslog/logs"
# strip ; or any other unwanted signs out of the command, this prevents them from breaking out of the setup command
if [[ $1 != "" ]]; then
COMMAND=$1
COMMAND=${COMMAND//[;\`]/}
fi
if [[ $2 != "" ]]; then
ARGU1=$2
ARGU1=${ARGU1//[;\`]/}
fi
if [[ $3 != "" ]]; then
ARGU2=$3
ARGU2=${ARGU2//[;\`]/}
fi
if [[ $4 != "" ]]; then
ARGU3=$4
ARGU3=${ARGU3//[;\`]/}
fi
# checking for the commands
case "$COMMAND" in
less)
ARGU2=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
exit 1;
fi
#echo " --------------------------------- LESS $FILE"
/usr/bin/less $FILE
;;
grep)
if [[ $ARGU2 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
if [[ $ARGU1 == "" ]]; then
echo "Pls give a string to search for"
exit 1
fi
ARGU2=${ARGU2//\.\./}
FILE=$PATH/$ARGU2
/usr/bin/logger -t restricted-command -- "------- $USER Executing grep $ARGU1 \"$ARGU2\" $FILE"
if [ ! -f $FILE ]; then
echo "File doesn't exist"
/usr/bin/logger -t restricted-command -- "$USER Executing $#"
exit 1;
fi
/bin/grep $ARGU1 $FILE
;;
tail)
if [[ $ARGU1 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
ARGU1=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
/usr/bin/logger -t restricted-command -- "$USER Executing $# ($FILE)"
exit 1;
fi
/usr/bin/tail -f $FILE
;;
cat)
ARGU2=${ARGU1//\.\./}
FILE=$PATH/$ARGU1
if [ ! -f $FILE ]; then
echo "File doesn't exist"
exit 1;
fi
/bin/cat $FILE
;;
help)
/bin/cat <<"EOF"
# less LOGNAME (eg less testserver.example.com/YYYY/MM/DD/logfile.log)
# grep [ARGUMENT] LOGNAME
# tail LOGNAME (eg tail testserver.example.com/YYYY/MM/DD/logfile.log)
# cat LOGNAME (eg cat testserver.example.com/YYYY/MM/DD/logfile.log)
In total the command looks like this : ssh user#testserver.example.com COMMAND [ARGUMENT] LOGFILE
EOF
/usr/bin/logger -t restricted-command -- "$USER HELP requested $#"
exit 1
;;
*)
/usr/bin/logger -s -t restricted-command -- "$USER Invalid command $#"
exit 1
;;
esac
/usr/bin/logger -t restricted-command -- "$USER Executing $#"
The problem is next:
when i try to exec some command, it takes only first argument, if i do recursion in files by using {n,n1,n2} - it doesn't work:
[testuser#local ~]$ ssh testuser#syslog.server less srv1838.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
0
[testuser#local ~]$ ssh testuser#syslog.server less srv2010.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
11591
[testuser#local ~]$ ssh testuser#syslog.server less srv{1838,2010}.example.com/2017/02/10/local1.log |grep 'srv2010' | wc -l
0
[testuser#local ~]$ ssh testuser#syslog.server less srv{2010,1838}.example.com/2017/02/21/local1.log |grep 'srv2010' | wc -l
11591
Could someone help me, how can i parse\count command arguments to make it work?
Thank you and have a nice day!
The number of arguments for a bash script would be $#. As a quick example:
#!/bin/bash
narg=$#
typeset -i i
i=1
while [ $i -le $narg ] ; do
echo " $# $i: $1"
shift
i=$i+1
done
gives, for bash tst.sh a b {c,d}
4 1: a
3 2: b
2 3: c
1 4: d
In your script, the command to execute (cat, less, ...) gets explicitly only the second argument to the script. If you want to read all arguments, you should do something like this (note: only a hint, removed all sorts of checks etc..)
command="$1"
shift
case $command in
(grep) pattern="$1"
shift
while [ $# -gt 0 ] ; do
grep "$pattern" "$1"
shift
done
;;
esac
note: added some quotes as comment suggested, but, being only a hint, you should carefully look at quoting and your checks in your own script.
Less command working now:
case "$COMMAND" in
less)
if [[ $ARGU1 == "" ]]; then
echo "Pls give a filename"
exit 1
fi
FILES_LIST=${#:2}
FILE=(${FILES_LIST//\.\./})
for v in "${FILE[#]}";do
v=${v//[;\']/}
if [ ! -f $v ]; then
echo "File doesn't exist"
fi
/usr/bin/less $PATH/$v
done;;
tail command works too with 2 and more files, but i can't execute tail -f command on two files unfortunately.
I have a function I wrote in bash that copies files.
It was written so it would be less painful for us to turn our batch scripts that use xcopy to bash scripts. This is because the copy commands in Linux work a little bit different.
The function does several things:
It creates a path to the target directory if it doesn't exist yet.
It uses cp to copy files
it uses cp -r to copy directories.
it uses rsync -arv --exclude-from=<FILE> to copy all the files and folders in a gives directory except the files/folders listed in FILE
The problem is, that when I try to copy files with * it gives me an error:
cp: cannot stat 'some dir with * in it': No such file or directory.
I found out that I can instead write something like that: cp "<dir>/"*".<extension>" "<targetDir>" and the command itself works. But when I try to pass that to my function, it gets 3 arguments instead of 2.
How can I use the cp command in my function while being able to pass a path with wildcard in it? meaning the argument will have double quotes in the beginning of the path and in the end of them, for example: Copy "<somePath>/*.zip" "<targetDir>"
function Copy {
echo "number of args is: $#"
LastStringInPath=$(basename "$2")
if [[ "$LastStringInPath" != *.* ]]; then
mkdir -p "$2"
else
newDir=$(dirname "$2")
mkdir -p "newDir"
fi
if [ "$#" == "2" ]; then
echo "Copying $1 to $2"
if [[ -d $1 ]]; then
cp -r "$1" "$2"
else
cp "$1" "$2"
fi
if [ $? -ne 0 ]; then
echo "Error $? while trying to copy $1 to $2"
exit 1
fi
else
rsync -arv --exclude-from="$3" "$1" "$2"
if [ $? -ne 0 ]; then
echo "Error $? while trying to copy $1 to $2"
exit 1
fi
fi
}
Okay, so I couldn't solve this with the suggestions I was given. What was happening is either the * was expanding before it was sent to function or it wouldn't expand at all inside the function. I tried different methods and eventually I decided to rewrite the function so it would instead support multiple arguments.
The expansion of the wild card happens before it sent to my function, and the copy function does all the actions it was doing before while supporting more than one file/dir to copy.
function Copy {
argumentsArray=( "$#" )
#Check if last argument has the word exclude, in this case we must use rsync command
if [[ ${argumentsArray[$#-1],,} == exclude:* ]]; then
mkdir -p "$2"
#get file name from the argument
excludeFile=${3#*:}
rsync -arv --exclude-from="$excludeFile" "$1" "$2"
if [ $? -ne 0 ]; then
echo "Error while to copy $1 to $2"
exit 1
fi
else
mkdir -p "${argumentsArray[$#-1]}"
if [[ -d $1 ]]; then
cp -r "${argumentsArray[#]}"
if [ $? -ne 0 ]; then
exit 1
fi
else
cp "${argumentsArray[#]}"
if [ $? -ne 0 ]; then
exit 1
fi
fi
fi
}
If a file called “output” already exists, rather than a directory, the script
should display an error and quit.
here is my code so far
for file in *
do
if [ ! -f output ]
then echo "error"
exit 1
fi
done
for file in *; do
if [ "$file" = "output" -a -f "$file" ]; then
echo "error"
exit 1
fi
done
Or
for file in *; do
if [ "$file" = "output" ] && [ -f "$file" ]; then
echo "error"
exit 1
fi
done
And with bash, this one's preferred:
for file in *; do
if [[ $file == output && -f $file ]]; then
echo "error"
exit 1
fi
done
If you want to check if the filename contains the word, not just exactly matches it:
for file in *; do
if [[ $file == *output* && -f $file ]]; then
echo "error"
exit 1
fi
done
Why are we processing every file in the subdirectory? Very Odd.
if [ -f output ]; then
echo "'output exists and is a file"
exit 1
fi
The test command (which is also [) (and is also built-in to most shells (see bash man page too) ), responds with a TRUE response for -f output only when output is a file. You can check if it's a directory with -d.
touch something
if [ -f something ]; then echo "something is a file"; fi
if [ -d something ]; then echo "something is not a file"; fi
rm something
mkdir something
if [ -f something ]; then echo "something is not a subdir"; fi
if [ -d something ]; then echo "something is a subdir"; fi
rmdir something
If you try those commands, you'll get:
something is a file
something is a subdir
No point in iterating through the entire directory contents if you're just looking if a specific file/dir exists.
I am working on a shell script which I need to run on machineX. It will check for a certain folder which is in this format YYYYMMDD inside this folder MAPPED_LOCATION in other two machines - machineP and machineQ. So the path will be like this in both machineP and machineQ-
/bat/testdata/t1_snapshot/20140311
And inside the above folder path, there will be some files inside in it. Below is my shell script -
#!/bin/bash
readonly MACHINES=(machineP machineQ)
readonly MAPPED_LOCATION=/bat/testdata/t1_snapshot
readonly FILE_TIMESTAMP=20140311
# old code which I was using to get the latest folder inside each machine (P and Q)
dir1=$(ssh -o "StrictHostKeyChecking no" david#${MACHINES[0]} ls -dt1 "$MAPPED_LOCATION"/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | head -n1)
dir2=$(ssh -o "StrictHostKeyChecking no" david#${MACHINES[1]} ls -dt1 "$MAPPED_LOCATION"/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | head -n1)
dir3=$MAPPED_LOCATION/$FILE_TIMESTAMP # /bat/testdata/t1_snapshot/20140311
echo $dir1
echo $dir2
echo $dir3
if dir3 path exists in both the machines (P and Q) and number of files is greater than zero in each machine
then
# then do something here
echo "Hello World"
else
# log an error - folder is missing or number of files is zero in which servers or both servers
fi
Noow what I am supposed to do is - If this path exists /bat/testdata/t1_snapshot/20140311 in both of the machines and number of files are greater than zero in both of the machines, then do somethting. Else if the folder is missing in any of the servers or number of files is zero in any of ther servers, I will exit out of the shell script with non zero status and a message with an actual error.
How can I do this in shell script?
Update:-
for machine in $MACHINES; do
dircheck=($(ssh -o "StrictHostKeyChecking no" david#${machine} [[ ! -d "$dir3" ]] \&\& exit 1 \; ls -t1 "$dir3"))
#On the ssh command, we exit 1 if the folder doesn't exist. We check the return code with `$?`
if [[ $? != 0 ]] ;then
echo "Folder doesn't exist on $machine";
exit 1
fi
# check number of files retrieved
if [[ "${dircheck[#]}" = 0 ]] ;then
echo "0 Files on server $machine";
exit 1
fi
#all good for $machine here
done
echo "Everything is Correct"
If I am adding a new empty folder 20140411 inside machineP and then execute the above script, it always prints out -
echo "Everything is Correct"
Infact, I didn't added any folder in machineQ. Not sure what is the problem?
Another Update-
I have created an empty folder 20140411 in machineP only. And then I ran the script in debug mode -
david#machineX:~$ ./test_file_check_1.sh
+ FILERS_LOCATION=(machineP machineQ)
+ readonly FILERS_LOCATION
+ readonly MEMORY_MAPPED_LOCATION=/bexbat/data/be_t1_snapshot
+ MEMORY_MAPPED_LOCATION=/bexbat/data/be_t1_snapshot
+ readonly FILE_TIMESTAMP=20140411
+ FILE_TIMESTAMP=20140411
+ dir3=/bexbat/data/be_t1_snapshot/20140411
+ echo /bexbat/data/be_t1_snapshot/20140411
/bexbat/data/be_t1_snapshot/20140411
+ for machine in '$FILERS_LOCATION'
+ dircheck=($(ssh -o "StrictHostKeyChecking no" david#${machine} [[ ! -d "$dir3" ]] \&\& exit 1 \; ls -t1 "$dir3"))
++ ssh -o 'StrictHostKeyChecking no' david#machineP '[[' '!' -d /bexbat/data/be_t1_snapshot/20140411 ']]' '&&' exit 1 ';' ls -t1 /bexbat/data/be_t1_snapshot/20140411
+ [[ 0 != 0 ]]
+ [[ '' = 0 ]]
+ echo 'Everything is Correct'
Everything is Correct
What you want to do is, ls the remote directory (remove the -d flag to ls (which lists only folders), and the head -n1 command as it only prints the first file) and retrieve the data in an array variable.
I also added a check for directory existance [[ -d "$dir3" ]] before executing the ls and escaped the && to not be interpreted on the current bash script.
[[ -d "$dir3" ]] \&\& ls -t1 "$dir3"
To define a bash array, add extra ( ) arround the command., then compare the array size.
dir3="$MAPPED_LOCATION/$FILE_TIMESTAMP" # /bat/testdata/t1_snapshot/20140311
for machine in ${MACHINES[*]}; do
dir3check=($(ssh -o "StrictHostKeyChecking no" david#${machine} [[ -d "$dir3" ]] \&\& ls -t1 "$dir3"))
if [[ "${#dir3check[#]}" -gt 0 ]] ;then
# then do something here
echo "Hello World"
else
# log an error - folder is missing or number of files is zero in server $machine
fi
done
UPDATE:
for machine in ${MACHINES[*]}; do
dircheck=($(ssh -o "StrictHostKeyChecking no" david#${machine} [[ ! -d "$dir3" ]] \&\& exit 1 \; ls -t1 "$dir3"))
#On the ssh command, we exit 1 if the folder doesn't exist. We check the return code with `$?`
if [[ $? != 0 ]] ;then
echo "Folder doesn't exist on $machine";
exit 1
fi
# check number of files retrieved
if [[ "${#dircheck[#]}" = 0 ]] ;then
echo "0 Files on server $machine";
exit 1
fi
#all good for $machine here
done
#all good for all machines here