so I have this bash function:
function xyz(){
echo $#
}
now when I run
xyz whaat "loool * hahhaa"
instead of echoing whaat "loool * hahhaa"
it insteads echoes:
whaat loool all default hahhaa
First of all the quotation marks got stripped off.
Secondly the * got replaced by "all default" . This is because the current directory has 2 folders called "all" and "default" hence it think * refers to all directory in the current directory
Is there a way to modify my function so that the output becomes the intended whaat "loool * hahhaa" accordingly (including the quotation marks and *)
I tried doing ${#} "$#" and "${#}" to no avail
The quotation marks are really gone, so don't be expecting anything to put them back. Bash removed them as part of parsing the command line (search for "quote removal" in man bash).
However, either echo "$#" or echo "${#}" will avoid the glob expansion (replacing * with the directory listing).
You practically never want $#. Best to get in the habit of writing "$#".
If you want to see a quoted version of the arguments, you can a bash-extension to printf:
xyz() {
printf "%q " "$#"
printf "\n"
}
Related
Hi… Need a little help here…
I tried to emulate the DOS' dir command in Linux using bash script. Basically it's just a wrapped ls command with some parameters plus summary info. Here's the script:
#!/bin/bash
# default to current folder
if [ -z "$1" ]; then var=.;
else var="$1"; fi
# check file existence
if [ -a "$var" ]; then
# list contents with color, folder first
CMD="ls -lgG $var --color --group-directories-first"; $CMD;
# sum all files size
size=$(ls -lgGp "$var" | grep -v / | awk '{ sum += $3 }; END { print sum }')
if [ "$size" == "" ]; then size="0"; fi
# create summary
if [ -d "$var" ]; then
folder=$(find $var/* -maxdepth 0 -type d | wc -l)
file=$(find $var/* -maxdepth 0 -type f | wc -l)
echo "Found: $folder folders "
echo " $file files $size bytes"
fi
# error message
else
echo "dir: Error \"$var\": No such file or directory"
fi
The problem is when the argument contains an asterisk (*), the ls within the script acts differently compare to the direct ls command given at the prompt. Instead of return the whole files list, the script only returns the first file. See the video below to see the comparation in action. I don't know why it behaves like that.
Anyone knows how to fix it? Thank you.
Video: problem in action
UPDATE:
The problem has been solved. Thank you all for the answers. Now my script works as expected. See the video here: http://i.giphy.com/3o8dp1YLz4fIyCbOAU.gif
The asterisk * is expanded by the shell when it parses the command line. In other words, your script doesn't get a parameter containing an asterisk, it gets a list of files as arguments. Your script only works with $1, the first argument. It should work with "$#" instead.
This is because when you retrieve $1 you assume the shell does NOT expand *.
In fact, when * (or other glob) matches, it is expanded, and broken into segments by $IFS, and then passed as $1, $2, etc.
You're lucky if you simply retrieved the first file. When your first file's path contains spaces, you'll get an error because you only get the first segment before the space.
Seriously, read this and especially this. Really.
And please don't do things like
CMD=whatever you get from user input; $CMD;
You are begging for trouble. Don't execute arbitrary string from the user.
Both above answers already answered your question. So, i'm going a bit more verbose.
In your terminal is running the bash interpreter (probably). This is the program which parses your input line(s) and doing "things" based on your input.
When you enter some line the bash start doing the following workflow:
parsing and lexical analysis
expansion
brace expansion
tidle expansion
variable expansion
artithmetic and other substitutions
command substitution
word splitting
filename generation (globbing)
removing quotes
Only after all above the bash
will execute some external commands, like ls or dir.sh... etc.,
or will do so some "internal" actions for the known keywords and builtins like echo, for, if etc...
As you can see, the second last is the filename generation (globbing). So, in your case - if the test* matches some files, your bash expands the willcard characters (aka does the globbing).
So,
when you enter dir.sh test*,
and the test* matches some files
the bash does the expansion first
and after will execute the command dir.sh with already expanded filenames
e.g. the script get executed (in your case) as: dir.sh test.pas test.swift
BTW, it acts exactly with the same way for your ls example:
the bash expands the ls test* to ls test.pas test.swift
then executes the ls with the above two arguments
and the ls will print the result for the got two arguments.
with other words, the ls don't even see the test* argument - if it is possible - the bash expands the wilcard characters. (* and ?).
Now back to your script: add after the shebang the following line:
echo "the $0 got this arguments: $#"
and you will immediatelly see, the real argumemts how your script got executed.
also, in such cases is a good practice trying to execute the script in debug-mode, e.g.
bash -x dir.sh test*
and you will see, what the script does exactly.
Also, you can do the same for your current interpreter, e.g. just enter into the terminal
set -x
and try run the dir.sh test* = and you will see, how the bash will execute the dir.sh command. (to stop the debug mode, just enter set +x)
Everbody is giving you valuable advice which you should definitely should follow!
But here is the real answer to your question.
To pass unexpanded arguments to any executable you need to single quote them:
./your_script '*'
The best solution I have is to use the eval command, in this way:
#!/bin/bash
cmd="some command \"with_quetes_and_asterisk_in_it*\""
echo "$cmd"
eval $cmd
The eval command takes its arguments and evaluates them into the command as the shell does.
This solves my problem when I need to call a command with asterisk '*' in it from a script.
This question already has an answer here:
Bash: Confused by expanding asterisk
(1 answer)
Closed 7 years ago.
My problem can be reproduced with the following script:
#!/bin/bash
echo ".*" > foo.txt
echo $(cat foo.txt)
When I run this script I get a list of folders/files in my current directory:
$ ./test.sh
. .. .testfile
$ cat foo.txt
.*
My question is really a two parter:
1) Why does this happen?
2) Is there any way to get the literal string ".*" rather than a file list returned from $(cat [args])?
I originally ran into this problem working on a more complex script. Fixing this with an additional option passed into cat and/or alternative syntax would be ideal.
echo $(cat foo.txt)
Capturing cat's output and then echoing it right back out is not only redundant, it's error prone, as you've discovered. Just write:
cat foo.txt
Simpler, faster, it's the bees knees!
Or if you really, really want to capture it and then print it back out, use quotes. Quotes will prevent the .* from being interpreted as wildcards.
echo "$(cat foo.txt)"
There are still subtle problems with this command. If foo.txt contains -n, for instance, echo won't print -n, it'll print nothing. It turns out that echo simply isn't usable if you're the extra paranoid type. The super safe option is to eschew echo in favor of printf.
printf '%s\n' "$(cat foo.txt)"
This is as safe as one can get. It prints the contents of foo.txt and won't get tripped up by any special characters.
Although, you know, this is an awfully long winded way of writing:
cat foo.txt
Because the result returned by $() is un-quoted. It is equivalent to:
echo .*
which is subject to shell expansion.
You should double quote it:
echo "$(cat foo.txt)"
It will give correct output .*
Use More Quotes !
echo ".*" > foo.txt
echo "$(cat foo.txt)"
.*
The double quotes are mandatory, to avoid shell expansion:
Without the double quotes, all the characters are expanded from the shell, so the * becomes files in current directory
I have a script that needs to dynamically assemble another script for later execution, however I'm experiencing problems with characters being substituted, specifically any that are escaped such as \\, \t, \n and so-on. While I can work around this issue with variables, it's extremely annoying, especially as the code is provided in segments wrapped as here documents in quoted form, i.e - such that they should not be processed at all.
Even more annoying, on some platforms this substitution seems to extend to other characters as well, such as \1, which has been causing havoc with my testing of regular expressions.
Here's a simple example:
#!/bin/sh
script=$(cat << 'SCRIPT'
#!/bin/sh
printf '\t%s' "$1"
SCRIPT
)
DIR=$(dirname "$PWD")
echo "$script" > "$DIR/test_script.sh"
I would expect this to produce a simple script with the line printf '\t%s' "$1", however instead the line is produced as printf ' %s' "$1", when no substitution is expected.
Can anyone explain why this is happening, and ideally, how to prevent this from happening? Like I say, I can work around this with variables for substituted characters, but it's destroying the readability of my script (and is hell to debug).
It would appear your version of echo processes some escape characters when you dump the script to a file. echo is not well-standardized, since any standard would have conflicted with some previous implementation. Use printf instead.
#!/bin/sh
script=$(cat << 'SCRIPT'
#!/bin/sh
printf '\t%s' "$1"
SCRIPT
)
DIR=$(dirname "$PWD")
printf "%s\n" "$script" > "$DIR/test_script.sh"
In my script, how can I distinguish when the asterisk wildcard character was used instead of strongly typed parameters?
This
# myscript *
from this
# myscript p1 p2 p3 ... (where parameters are unknown number)
The shell expands the wildcard. By the time a script is run, the wildcard has been expanded, and there is no way a script can tell whether the arguments were a wildcard or an explicit list.
Which means that your script will need help from something else which is not a script. Specifically, something which is run before command-line processing. That something is an alias. This is your alias
alias myscript='set -f; globstopper /usr/bin/myscript'
What this does is set up an alias called 'myscript', so when someone types 'myscript', this is what gets run. The alias does two things: firstly, it turns off wildcard expansion with set -f, then it runs a function called globstopper, passing in the path to your script, and the rest of the command-line arguments.
So what's the globstopper function? This:
globstopper() {
if [[ "$2" == "*" ]]
then echo "You cannot use a wildcard"
return
fi
set +f
"$#";
}
This function does three things. Firstly, it checks to see if the argument to the script is a wildcard (caveat: it only checks the first argument, and it only checks to see if it's a simple star; extending this to cover more cases is left as an exercise to the reader). Secondly, it switches wildcard expansion back on. Lastly, it runs the original command.
For this to work, you do need to be able to set up the alias and the shell function in the user's shell, and require your users to use the alias, not the script. But if you can do that, it ought to work.
I should add that i am leaning heavily on the resplendent Simon Tatham's essay 'Magic Aliases: A Layering Loophole in the Bourne Shell' here.
I had a similar question, but rather than detecting when the user called the script using a wildcard, I simply wanted to prevent the use of the wildcard, and pass the string pre-expansion.
Tom's solution is great if you want to detect, but I'd rather prevent. In other words, if I had a script called findin that looked like
#!/bin/bash
echo "[${1}]"
and ran it using:
$ findin *
I would expect the output to be simply
[*]
To do this, you could just alias findin by
alias findin='set -f; /path/to/findin'
But then you would have the shell option set for the rest of your session. This will likely break many programs that don't expect this (e.g. ls -lh *.py). You could verify this by typing
echo $-
in console. If you see an f, that option is set.
You could manually clear the option by typing
set +f
after every instance of findin, but that would get tedious and annoying.
Since shell scripts spawn subshells and you cannot clear the flag from within the script (set +f), the solution I came up with was the following:
g(){ /usr/local/bin/findin "$#"; set +f; }
alias findin='set -f; g'
Note: 'g' might not be the best name for the function, so you'd be encouraged to change it.
Finally, you could generalize this by doing something like:
reset_expansion(){ CMD="$1"; shift; $CMD "$#"; set +f; }
alias findin='set -f; reset_expansion /usr/local/bin/findin'
That way another script where you would want expansion disabled would only require an additional alias, e.g.
alias newscript='set -f; reset_expansion /usr/local/bin/newscript'
and not an additional wrapper function.
For a much longer than necessary writeup, see my post here.
You can't.
It is one of the strengths (or, in some eyes, weaknesses) of Unix.
See the diatribe(s) in "The UNIX-HATERS Handbook".
$arg_num == ***; // detects *(literally anything since it's a global wildcard)
$arg_num == *_*; // detects _
here is an example of it working with _
for i in $*
do
if [[ "$i" == *_* ]];
then echo $i;
fi
done
output of ./bash test * test2 _
_
output of ./bash test * test2 with ********* rather then ****
test
bash
pass.rtf
test2
_
NOTE: the * is so global in bash that it printed out files matching that description or in my case of the files on my oh-so-unused desktop. I wish I could give you a better answer but the best choice it to use something other then * or another scripting language.
Addendum
I found this post while looking for a workaround for my command line calculator:
alias c='set -f; call_clc'
where "call_clc" is the function: "function call_clc { clc "$*"; set +f; }"
and "clc" is the script
#!/bin/bash
echo "$*" | sed -e 's/ //g' >&1 | tee /dev/tty | bc
I need 'set -f' and 'set +f' in order to make inputs such as 'c 4 * 3' to work,
therefore an asterix with white space before and after,
in order to prevent globbing of the bash.
Update: the previous variant 'alias c='set -f; clc "$*"; set+f;'' did not work
because for some reason the correct result was given after invoking the command "c 4 * 4' twice.
Anyone an idea why this is so?
If this is something you feel you must do, perhaps:
# if the number of parms is not the same as the number of files in cwd
# then user did not use *
dir_contents=(*)
if [[ "${##}" -ne "${#dir_contents[#]}" ]]; then
used_star=false
else
# if one of the params is not a file in cwd
# then user did not use *
used_star=true
for f; do [[ ! -a "$f" ]] && { used_star=false; break; }; done
fi
unset dir_contents
$used_star && echo "used star" || echo "did not use star"
Pedantically, this will echo "used star" if the user actually used an asterisk or if the user manually entered the directory contents in any order.
In a BASH script, I'm trying to detect whether a file exists. The filename is in a variable but the -e command seems to be unable to detect the file. The following code always outputs "~/misc/tasks/drupal_backup.sh does not exist"
filename="~/misc/tasks/drupal_backup.sh"
if [ -e "$filename" ]; then
echo "$filename exists"
else
echo "$filename does not exist"
fi
On the other hand, the following code detects the file correctly:
if [ -e ~/misc/tasks/drupal_backup.sh ]; then
echo "$filename exists"
else
echo "$filename does not exist"
fi
Why would this be? How can I get it to detect the file when the filename is in a variable?
That's an interesting one. Substituting $HOME for ~ works as does removing the quotes from the assignment.
If you put a set -x at the top of that script, you'll see that the version with quotes sets filename to ~/... which is what's given to -e. If you remove the quotes, filename is set to the expanded /home/somebody/.... So in the first case, you see:
+ [ -e ~/... ]
and it doesn't like it. In the second case, you see:
+ [ -e /home/somebody/... ]
and it does work.
If you do it without the variable, you see:
+ [ -e /home/somebody/... ]
and, yes, it works.
After a bit of investigation, I've found that it's actually the order in which bash performs its expansions. From the bash man page:
The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion.
That's why it's not working, the variable is substituted after the tilde expansion. In other words, at the point where bash wants to expand ~, there isn't one. It's only after variable expansion does the word get changed into ~/... and there will be no tilde expansion after that.
One thing you could do is to change your if statement to:
if [[ -e $(eval echo $filename) ]]; then
This will evaluate the $filename argument twice. The first time (with eval), there will be no ~ during the tilde expansion phase but $filename will be changed to ~/... during the variable expansion phase.
Then, on the second evaluation (the one being done as part of the if itself), the ~ will be there during the tilde expansion phase.
I've tested that on my .profile file and it seems to work, I suggest you confirm in your particular case.