Prevent ssh from breaking up shell script parameters - linux

I have a script, which is essentially a wrapper around an executable by the same name on a different machine. For the sake of example, i'll wrap printf here. My current script looks like this:
#!/bin/bash
ssh user#hostname.tld. printf "$#"
Unfortunately, this breaks when one of the arguments contains a space, e.g. i'd expect the following commands to give identical outputs.:
~$ ./wrap_printf "%s_%s" "hello world" "1"
hello_world1_
~$ printf "%s_%s" "hello world" "1"
hello world_1
The problem gets even worse when (escaped) newlines are involved. How would I properly escape my arguments here?

Based on the answer from Peter Lyons, but also allow quotes inside arguments:
#!/bin/bash
QUOTE_ARGS=''
for ARG in "$#"
do
ARG=$(printf "%q" "$ARG")
QUOTE_ARGS="${QUOTE_ARGS} $ARG"
done
ssh user#hostname.tld. "printf ${QUOTE_ARGS}"
This works for everything i've tested so far, except newlines:
$ /tmp/wrap_printf "[-%s-]" "hello'\$t\""
[-hello'$t"-]

#!/bin/sh
QUOTE_ARGS=''
for ARG in "$#"
do
QUOTE_ARGS="${QUOTE_ARGS} '${ARG}'"
done
ssh user#hostname.tld. "${QUOTE_ARGS}"
This works for spaces. It doesn't work if the argument has an embedded single quote.

Getting quoting right is pretty hard and doing it in bash (in a general and robust way) almost impossible.
Use Perl:
#!/usr/bin/perl
use Net::OpenSSH;
my $ssh = Net::OpenSSH->new('user#hostname');
$ssh->system('printf', #ARGV);

Based on the answers from Koert and Peter Lyons, here a wrapper for ssh; i call it "sshsystem". (also available at https://gist.github.com/4672115)
#!/bin/bash
# quote command in ssh call to prevent remote side from expanding any arguments
# uses bash printf %q for quoting - no idea how compatible this is with other shells.
# http://stackoverflow.com/questions/6592376/prevent-ssh-from-breaking-up-shell-script-parameters
sshargs=()
while (( $# > 0 )); do
case "$1" in
-[1246AaCfgKkMNnqsTtVvXxYy])
# simple argument
sshargs+=("$1")
shift
;;
-[bcDeFIiLlmOopRSWw])
# argument with parameter
sshargs+=("$1")
shift
if (( $# == 0 )); then
echo "missing second part of long argument" >&2
exit 99
fi
sshargs+=("$1")
shift
;;
-[bcDeFIiLlmOopRSWw]*)
# argument with parameter appended without space
sshargs+=("$1")
shift
;;
--)
# end of arguments
sshargs+=("$1")
shift
break
;;
-*)
echo "unrecognized argument: '$1'" >&2
exit 99
;;
*)
# end of arguments
break
;;
esac
done
# user#host
sshargs+=("$1")
shift
# command - quote
if (( $# > 0 )); then
# no need to make COMMAND an array - ssh will merge it anyway
COMMAND=
while (( $# > 0 )); do
arg=$(printf "%q" "$1")
COMMAND="${COMMAND} ${arg}"
shift
done
sshargs+=("${COMMAND}")
fi
exec ssh "${sshargs[#]}"

The easiest and quickest is to just use Bash's Quoting Parameter Transformation: ${parameter#Q}. This can automatically applied during array expansion with ${array[#]#Q}, but when using the builtin argument array, the name and the brackets are dropped, so it becomes ${##Q}. Therefore the original script only needs 4 characters added to it to work.
#!/bin/bash
ssh user#hostname.tld. printf "${##Q}"
Now any escaping will work, even terminal colors like this:
./wrap_printf "%s\e[39m\e[49m\n" $'\e[30m\e[42mBlack on Green' "Just Normal Text"

Related

linux terminal execute echo function

when I read the book linux shell scripting cookbook
they say when you wanna print !,you shouldn't put it in double quote,or you can add \ before ! to escape it.
e.g.
$echo "Hello,world!"
bash: !:event not found error
$echo "Hello,world\\!"
Hello,world!
but in my situation(ubuntu14.04), I get the answer like that:
$echo "Hello,world!"
Hello,world!
$echo "Hello,world\\!"
Hello,world\!
So, why in my machine can't get the same answer?
Why the escape symbol \ was printed as a normal symbol?
When you're typing interactively to the shell, ! has special meaning, it's the history expansion character. To prevent this special meaning, you need to put it in single quotes or escape it.
echo 'Hello, world!'
echo "Hello, world\!'
The reason it's not happening on Ubuntu may be because it's running a newer version of bash, which is apparently more selective about when history expansion occurs. It seems to require ! to be followed by alphanumerics, not punctuation.
You don't need to do this in scripts, because history is not normally enabled there. It's just for interactive shells.
Create a shell script called file.sh:
#!/bin/bash
# file.sh: a sample shell script to demonstrate the concept of Bash shell functions
# define usage function
usage(){
echo "Usage: $0 filename"
exit 1
}
# define is_file_exits function
# $f -> store argument passed to the script
is_file_exits(){
local f="$1"
[[ -f "$f" ]] && return 0 || return 1
}
# invoke usage
# call usage() function if filename not supplied
[[ $# -eq 0 ]] && usage
# Invoke is_file_exits
if ( is_file_exits "$1" )
then
echo "File found"
else
echo "File not found"
fi
Run it as follows:
chmod +x file.sh
./file.sh
./file.sh /etc/resolv.conf

Bash how check if an argument is a sed patterns?

I'm working on a bash script that should receive many arguments. those arguments could be simple like:
Myscript [-r] [-l|-u] <dir/file names...>
or it could be just a sed patterns given instead of a simple arguments like:
Myscript <sed pattern> <dir/file names...>
So my question is how to check if the arguments given is a sed patterns and not directory path or filename?
I have done something but it doesn't work for a long path (dir1/dir2/dir)
while [[ $# > 0 ]]
do
key="$1"
case $key in
-ru|-ur)
config_recusive
config_upper
shift
;;
-rl|-lr)
config_recusive
config_lower
shift
;;
-r)
config_recusive
shift
;;
-l)
config_lower
shift
;;
-u)
config_upper
shift
;;
*)
check="^.*/.*/.*$"
if [[ $key =~ $check ]] ; then
config_sed_pattern
else
TARGET=$key
fi
shift
;;
esac
done
To be more clear, here is an example of my problem when I'm trying to run the script like that:
./myscript -u tmp/users/regions
its being confused taking the path (tmp/users/regions) as a sed patterns.
hope that I was clear enough.
waiting for your help :)
thanks
i think you could use try run that script on something
echo a | sed "$pattern" >/dev/null 2>/dev/null
and then check the $?
if [ $? != 0 ]; then # means sed failed
#...
but maybe sed will think it will be its argument too.
btw, handle arguments using getopt will make things easier.
i don't think the way you design argument format is good. maybe you think making arguments in different forms could make your program looks dynamic and awesome. but now you are trying to solve a problem have nothing to do with your serious business and wasting your own time and ours. that's not good. it's better to make things stupid and clear.
in your case, maybe you can add another argument to show it's a sed expression that follows.
YourScript -e <sed expr> <others>
and in such a way, you will have idea about what you have done two weeks later.

How get next to next argument in linux shell script?

I have wrote simple shell script test.sh as follows:
while getopts ":A:" OPTION
do
case $OPTION in
A)
echo $OPTARG
?)
echo "no option"
esac
done
And executed the scripts as follows
$ ./test.sh -A 1 2
Now if got argument 1 by $OPTARG but how can i access the second argument ( 2 in this case)?
Regards
Jayesh
There are several options.
(1) You can use shift and take $1
while -n "$1"
do
# do something with $1
shift
done
(2) You can iterate through the args:
for i
do
# do something with $i
done
There are other alternatives also.

How can I preserve quotes in printing a bash script's arguments

I am making a bash script that will print and pass complex arguments to another external program.
./script -m root#hostname,root#hostname -o -q -- 'uptime ; uname -a'
How do I print the raw arguments as such:
-m root#hostname,root#hostname -o -q -- 'uptime ; uname -a'
Using $# and $* removes the single quotes around uptime ; uname -a which could cause undesired results. My script does not need to parse each argument. I just need to print / log the argument string and pass them to another program exactly how they are given.
I know I can escape the quotes with something like "'uptime ; uname -a'" but I cannot guarantee the user will do that.
The quotes are removed before the arguments are passed to your script, so it's too late to preserve them. What you can do is preserve their effect when passing the arguments to the inner command, and reconstruct an equivalent quoted/escaped version of the arguments for printing.
For passing the arguments to the inner command "$#" -- with the double-quotes, $# preserves the original word breaks, meaning that the inner command receives exactly the same argument list that your script did.
For printing, you can use the %q format in bash's printf command to reconstruct the quoting. Note that this won't always reconstruct the original quoting, but will construct an equivalent quoted/escaped string. For example, if you passed the argument 'uptime ; uname -a' it might print uptime\ \;\ uname\ -a or "uptime ; uname -a" or any other equivalent (see #William Pursell's answer for similar examples).
Here's an example of using these:
printf "Running command:"
printf " %q" innercmd "$#" # note the space before %q -- this inserts spaces between arguments
printf "\n"
innercmd "$#"
If you have bash version 4.4 or later, you can use the #Q modifier on parameter expansions to add quoting. This tends to prefer using single-quotes (as opposed to printf %q's preference for escapes). You can combine this with $* to get a reasonable result:
echo "Running command: innercmd ${*#Q}"
innercmd "$#"
Note that $* mashes all arguments together into a single string with whitespace between them, which is normally not useful, but in this case each argument is individually quoted so the result is actually what you (probably) want. (Well, unless you changed IFS, in which case the "whitespace" between arguments will be the first character of $IFS, which may not be what you want.)
Use ${##Q} for a simple solution. To test put the lines below in a script bigQ.
#!/bin/bash
line="${##Q}"
echo $line
./bigQ 1 a "4 5" b="6 7 8"
'1' 'a' '4 5' 'b=6 7 8'
If the user invokes your command as:
./script 'foo'
the first argument given to the script is the string foo without the quotes. There is no way for your script to differentiate between that and any of the other methods by which it could get foo as an argument (eg ./script $(echo foo) or ./script foo or ./script "foo" or ./script \f\o""''""o).
If you want to print the argument list as close as possible to what the user probably entered:
#!/bin/bash
chars='[ !"#$&()*,;<>?\^`{|}]'
for arg
do
if [[ $arg == *"'"* ]]
then
arg=\""$arg"\"
elif [[ $arg == *$chars* ]]
then
arg="'$arg'"
fi
allargs+=("$arg") # ${allargs[#]} is to be used only for printing
done
printf '%s\n' "${allargs[*]}"
It's not perfect. An argument like ''\''"' is more difficult to accommodate than is justified.
As someone else already mentioned, when you access the arguments inside of your script, it's too late to know which arguments were quote when it was called. However, you can re-quote the arguments that contain spaces or other special characters that would need to be quoted to be passed as parameters.
Here is a Bash implementation based on Python's shlex.quote(s) that does just that:
function quote() {
declare -a params
for param; do
if [[ -z "${param}" || "${param}" =~ [^A-Za-z0-9_#%+=:,./-] ]]; then
params+=("'${param//\'/\'\"\'\"\'}'")
else
params+=("${param}")
fi
done
echo "${params[*]}"
}
Your example slightly changed to show empty arguments:
$ quote -m root#hostname,root#hostname -o -q -- 'uptime ; uname -a' ''
-m root#hostname,root#hostname -o -q -- 'uptime ; uname -a' ''
In my case, I have tried to call the bash like script --argument="--arg-inner=1 --arg-inner2".
Unfortunately any solution upper don't help in my case.
Definitive solution was
#!/bin/bash
# Fix given array argument quotation
function quote() {
local QUOTED_ARRAY=()
for ARGUMENT; do
case ${ARGUMENT} in
--*=*)
QUOTED_ARRAY+=( "${ARGUMENT%%=*}=$(printf "%q" "${ARGUMENT#*=}")" )
shift
;;
*)
QUOTED_ARRAY+=( "$(printf " %q" "${ARGUMENT}")" )
;;
esac
done
echo ${QUOTED_ARRAY[#]}
}
ARGUMENTS="$(quote "${#}")"
echo "${ARGUMENTS}"
The result in the case of MacOS is --argument=--arg-inner=1\ --arg-inner2 which is logically the same.
Just separate each argument using quotes, and the nul character:
#! /bin/bash
sender () {
printf '"%s"\0' "$#"
}
receiver () {
readarray -d '' args < <(function "$#")
}
receiver "$#"
As commented by Charles Duffy.

How to properly handle wildcard expansion in a bash shell script?

#!/bin/bash
hello()
{
SRC=$1
DEST=$2
for IP in `cat /opt/ankit/configs/machine.configs` ; do
echo $SRC | grep '*' > /dev/null
if test `echo $?` -eq 0 ; then
for STAR in $SRC ; do
echo -en "$IP"
echo -en "\n\t ARG1=$STAR ARG2=$2\n\n"
done
else
echo -en "$IP"
echo -en "\n\t ARG1=$SRC ARG2=$DEST\n\n"
fi
done
}
hello $1 $2
The above is the shell script which I provide source (SRC) & desitnation (DEST) path. It worked fine when I did not put in a SRC path with wild card ''. When I run this shell script and give ''.pdf or '*'as follows:
root#ankit1:~/as_prac# ./test.sh /home/dev/Examples/*.pdf /ankit_test/as
I get the following output:
192.168.1.6
ARG1=/home/dev/Examples/case_Contact.pdf ARG2=/home/dev/Examples/case_howard_county_library.pdf
The DEST is /ankit_test/as but DEST also get manupulated due to '*'. The expected answer is
ARG1=/home/dev/Examples/case_Contact.pdf ARG2=/ankit_test/as
So, if you understand what I am trying to do, please help me out to solve this BUG.
I'll be grateful to you.
Thanks in advance!!!
I need to know exactly how I use '*.pdf' in my program one by one without disturbing DEST.
Your script needs more work.
Even after escaping the wildcard, you won't get your expected answer. You will get:
ARG1=/home/dev/Examples/*.pdf ARG2=/ankit__test/as
Try the following instead:
for IP in `cat /opt/ankit/configs/machine.configs`
do
for i in $SRC
do
echo -en "$IP"
echo -en "\n\t ARG1=$i ARG2=$DEST\n\n"
done
done
Run it like this:
root#ankit1:~/as_prac# ./test.sh "/home/dev/Examples/*.pdf" /ankit__test/as
The shell will expand wildcards unless you escape them, so for example if you have
$ ls
one.pdf two.pdf three.pdf
and run your script as
./test.sh *.pdf /ankit__test/as
it will be the same as
./test.sh one.pdf two.pdf three.pdf /ankit__test/as
which is not what you expect. Doing
./test.sh \*.pdf /ankit__test/as
should work.
If you can, change the order of the parameters passed to your shell script as follows:
./test.sh /ankit_test/as /home/dev/Examples/*.pdf
That would make your life a lot easier since the variable part moves to the end of the line. Then, the following script will do what you want:
#!/bin/bash
hello()
{
SRC=$1
DEST=$2
for IP in `cat /opt/ankit/configs/machine.configs` ; do
echo -en "$IP"
echo -en "\n\t ARG1=$SRC ARG2=$DEST\n\n"
done
}
arg2=$1
shift
while [[ "$1" != "" ]] ; do
hello $1 $arg2
shift
done
You are also missing a final "done" to close your outer for loop.
OK, this appears to do what you want:
#!/bin/bash
hello() {
SRC=$1
DEST=$2
while read IP ; do
for FILE in $SRC; do
echo -e "$IP"
echo -e "\tARG1=$FILE ARG2=$DEST\n"
done
done < /tmp/machine.configs
}
hello "$1" $2
You still need to escape any wildcard characters when you invoke the script
The double quotes are necessary when you invoke the hello function, otherwise the mere fact of evaluating $1 causes the wildcard to be expanded, but we don't want that to happen until $SRC is assigned in the function
Here's what I came up with:
#!/bin/bash
hello()
{
# DEST will contain the last argument
eval DEST=\$$#
while [ $1 != $DEST ]; do
SRC=$1
for IP in `cat /opt/ankit/configs/machine.configs`; do
echo -en "$IP"
echo -en "\n\t ARG1=$SRC ARG2=$DEST\n\n"
done
shift || break
done
}
hello $*
Instead of passing only two parameters to the hello() function, we'll pass in all the arguments that the script got.
Inside the hello() function, we first assign the final argument to the DEST var. Then we loop through all of the arguments, assigning each one to SRC, and run whatever commands we want using the SRC and DEST arguments. Note that you may want to put quotation marks around $SRC and $DEST in case they contain spaces. We stop looping when SRC is the same as DEST because that means we've hit the final argument (the destination).
For multiple input files using a wildcard such as *.txt, I found this to work perfectly, no escaping required. It should work just like a native bash app like "ls" or "rm." This was not documented just about anywhere so since I spent a better part of 3 days trying to figure it out I decided I should post it for future readers.
Directory contains the following files (output of ls)
file1.txt file2.txt file3.txt
Run script like
$ ./script.sh *.txt
Or even like
$ ./script.sh file{1..3}.txt
The script
#!/bin/bash
# store default IFS, we need to temporarily change this
sfi=$IFS
#set IFS to $'\n\' - new line
IFS=$'\n'
if [[ -z $# ]]
then
echo "Error: Missing required argument"
echo
exit 1
fi
# Put the file glob into an array
file=("$#")
# Now loop through them
for (( i=0 ; i < ${#file[*]} ; i++ ));
do
if [ -w ${file[$i]} ]; then
echo ${file[$i]} " writable"
else
echo ${file[$i]} " NOT writable"
fi
done
# Reset IFS to its default value
IFS=$sfi
The output
file1.txt writable
file2.txt writable
file3.txt writable
The key was switching the IFS (Internal Field Separator) temporarily. You have to be sure to store this before switching and then switch it back when you are done with it as demonstrated above.
Now you have a list of expanded files (with spaces escaped) in the file[] array which you can then loop through. I like this solution the best, easiest to program for and easiest for the users.
There's no need to spawn a shell to look at the $? variable, you can evaluate it directly.
It should just be:
if [ $? -eq 0 ]; then
You're running
./test.sh /home/dev/Examples/*.pdf /ankit_test/as
and your interactive shell is expanding the wildcard before the script gets it. You just need to quote the first argument when you launch it, as in
./test.sh "/home/dev/Examples/*.pdf" /ankit_test/as
and then, in your script, quote "$SRC" anywhere where you literally want the things with wildcards (ie, when you do echo $SRC, instead use echo "$SRC") and leave it unquoted when you want the wildcards expanded. Basically, always put quotes around things which might contain shell metacharacters unless you want the metacharacters interpreted. :)

Resources