While in the course of learning bash, I often tweak an existing thing and see it's output.
~$ for i in {1..19}; do echo "Everything in UNIX is a file."; sleep 1; done
I had this, and out of curiosity I tweaked the above one into the following:-
~$ for i in {1..19 * 2}; do echo "Everything in UNIX is a file."; echo "The value of i is ${i}"; sleep 1; done
Now to my surprise I started getting the following output :-
Everything in UNIX is a file.
The value of i is OneDrive
Everything in UNIX is a file.
The value of i is opera autoupdate
Everything in UNIX is a file.
The value of i is Personal_Workspace
Everything in UNIX is a file.
The value of i is Pictures
Everything in UNIX is a file.
The value of i is PrintHood
Everything in UNIX is a file.
The value of i is Recent
Everything in UNIX is a file.
The value of i is Roaming
Everything in UNIX is a file.
The value of i is Saved Games
Everything in UNIX is a file.
The value of i is Searches
Some of the values of i are the names of files and directories in my home directory, I am in home directory, while executing this script.
What I was expecting that the i values would range from 1 to 19*2 = 38, so i would take values from 1,2,3...30...38.
But obviously it did n't Why?
Yes in bash, range expansion happens before everything else. You were expecting arithmetic expansion to happen which did not happen as expected because of the order of expansion bash shell. Your code ended up interpreting {1..19, * and 2} as literal strings.
Since * has a special meaning in shell which is a glob expansion listing all files/directories in current folder. Also you could see one entry stating the other two strings interpreted literally.
From the man bash(1) page under section Expansion
The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion.
You are much better off using a for loop with a ((..)) construct if you are targeting scripts for bourne again shell
for ((i=1; i<=38; i++)); do
The * symbol in unix shell translates to a wildcard, so basically what {1..19 * 2} means is 1 through 19, all files in current dir (that's the *), and than 2. these will be the values of i in your loop
Related
I'm experimenting with bash scripting and noticed the following behavior:
file="test.jar"
echo "${file: -4}" #prints .jar
echo "${file:-4}" #prints test.jar
Very confusing behavior actually. Can someone explain why the second case prints the whole test.jar?
This is due to inconsistent syntax. {"string":-} means default (all the string), whatever follows -. So you need either a space or parenthesis:
{"string": -4}
{"string":(-4)}
Read bash string manipulation.
This is a compromise due to the timeline of features being added to Bash.
The ${parameter:-word} or ${parameter-word} syntax for "replace parameter with word if parameter is null or unset (:-) / null (-)" was around for pretty much always; the - version was already in the Version 7 Bourne Shell.
The ${parameter:offset:length} and ${parameter:offset} syntax for "substring of parameter starting at offset (with optional length length)" was introduced in Bash 2.0 (no conflict so far).
Negative offsets and length specifications for the substring construct were introduced in Bash 4.2. This leads to a problem:
$ string=01234567890abcdefgh
$ echo ${string:7:-2} # Not ambiguous
7890abcdef
$ echo ${string:-7} # Interpreted as "undefined/null, or..."
01234567890abcdefgh
$ echo ${string: -7} # Interpreted as offset from the end
bcdefgh
$ echo ${string:(-7)} # Interpreted as offset from the end
bcdefgh
The space before - or the parentheses around the negative offset are there to tell the expansion apart from the :- (default value) expansion.
If the space is not there, the expansion ${file:-4} is interpreted as "print 4 if the parameter file is null or unset, and the expansion of file otherwise".
References:
BashFAQ/061: Is there a list of which features were added to specific releases (versions) of Bash?
Bash hackers wiki: Bash changes
Shell parameter expansion in the bash manual
Bash NEWS file describing feature added per version
I have a BASH script that has a long set of arguments and two ways of calling it:
my_script --option1 value --option2 value ... etc
or
my_script val1 val2 val3 ..... valn
This script in turn compiles and runs a large FORTRAN code suite that eventually produces a netcdf file as output. I already have all the metadata in the netcdf output global attributes, but it would be really nice to also include the full run command one used to create that experiment. Thus another user who receives the netcdf file could simply reenter the run command to rerun the experiment, without having to piece together all the options.
So that is a long way of saying, in my BASH script, how do I get the last command entered from the parent shell and put it in a variable? i.e. the script is asking "how was I called?"
I could try to piece it together from the option list, but the very long option list and two interface methods would make this long and arduous, and I am sure there is a simple way.
I found this helpful page:
BASH: echoing the last command run
but this only seems to work to get the last command executed within the script itself. The asker also refers to use of history, but the answers seem to imply that the history will only contain the command after the programme has completed.
Many thanks if any of you have any idea.
You can try the following:
myInvocation="$(printf %q "$BASH_SOURCE")$((($#)) && printf ' %q' "$#")"
$BASH_SOURCE refers to the running script (as invoked), and $# is the array of arguments; (($#)) && ensures that the following printf command is only executed if at least 1 argument was passed; printf %q is explained below.
While this won't always be a verbatim copy of your command line, it'll be equivalent - the string you get is reusable as a shell command.
chepner points out in a comment that this approach will only capture what the original arguments were ultimately expanded to:
For instance, if the original command was my_script $USER "$(date +%s)", $myInvocation will not reflect these arguments as-is, but will rather contain what the shell expanded them to; e.g., my_script jdoe 1460644812
chepner also points that out that getting the actual raw command line as received by the parent process will be (next to) impossible. Do tell me if you know of a way.
However, if you're prepared to ask users to do extra work when invoking your script or you can get them to invoke your script through an alias you define - which is obviously tricky - there is a solution; see bottom.
Note that use of printf %q is crucial to preserving the boundaries between arguments - if your original arguments had embedded spaces, something like $0 $* would result in a different command.
printf %q also protects against other shell metacharacters (e.g., |) embedded in arguments.
printf %q quotes the given argument for reuse as a single argument in a shell command, applying the necessary quoting; e.g.:
$ printf %q 'a |b'
a\ \|b
a\ \|b is equivalent to single-quoted string 'a |b' from the shell's perspective, but this example shows how the resulting representation is not necessarily the same as the input representation.
Incidentally, ksh and zsh also support printf %q, and ksh actually outputs 'a |b' in this case.
If you're prepared to modify how your script is invoked, you can pass $BASH_COMMANDas an extra argument: $BASH_COMMAND contains the raw[1]
command line of the currently executing command.
For simplicity of processing inside the script, pass it as the first argument (note that the double quotes are required to preserve the value as a single argument):
my_script "$BASH_COMMAND" --option1 value --option2
Inside your script:
# The *first* argument is what "$BASH_COMMAND" expanded to,
# i.e., the entire (alias-expanded) command line.
myInvocation=$1 # Save the command line in a variable...
shift # ... and remove it from "$#".
# Now process "$#", as you normally would.
Unfortunately, there are only two options when it comes to ensuring that your script is invoked this way, and they're both suboptimal:
The end user has to invoke the script this way - which is obviously tricky and fragile (you could however, check in your script whether the first argument contains the script name and error out, if not).
Alternatively, provide an alias that wraps the passing of $BASH_COMMAND as follows:
alias my_script='/path/to/my_script "$BASH_COMMAND"'
The tricky part is that this alias must be defined in all end users' shell initialization files to ensure that it's available.
Also, inside your script, you'd have to do extra work to re-transform the alias-expanded version of the command line into its aliased form:
# The *first* argument is what "$BASH_COMMAND" expanded to,
# i.e., the entire (alias-expanded) command line.
# Here we also re-transform the alias-expanded command line to
# its original aliased form, by replacing everything up to and including
# "$BASH_COMMMAND" with the alias name.
myInvocation=$(sed 's/^.* "\$BASH_COMMAND"/my_script/' <<<"$1")
shift # Remove the first argument from "$#".
# Now process "$#", as you normally would.
Sadly, wrapping the invocation via a script or function is not an option, because the $BASH_COMMAND truly only ever reports the current command's command line, which in the case of a script or function wrapper would be the line inside that wrapper.
[1] The only thing that gets expanded are aliases, so if you invoked your script via an alias, you'll still see the underlying script in $BASH_COMMAND, but that's generally desirable, given that aliases are user-specific.
All other arguments and even input/output redirections, including process substitutiions <(...) are reflected as-is.
"$0" contains the script's name, "$#" contains the parameters.
Do you mean something like echo $0 $*?
Using ksh. Trying to reuse a current script without modifying it, which basically boils down to something like this:
`expr 5 $1 $2`
How do i pass in a a multiplication command (*) as parameter $1 ?
I first attempted using "*" and even \* but that isn't working.
I've tried multiple escape backslash and quote combinations but i think im doing it wrong.
Without modifying the script, I don't think this can be done:
On calling, you can pass a literal * as '*', "*" or \* (any will do): this will initially protect the * from shell expansions (interpretation by the shell).
The callee (the script) will then receive literal * (as $1), but due to unquoted use of $1 inside the script, * will invariably be subject to filename expansion (globbing), and will expand to all (non-hidden) filenames in the current folder, breaking the expr command.
Trying to add an extra layer of escaping - such as "'*'" or \\\* - will NOT work, because the extra escaping will become an embedded, literal part of the argument - the target script will see literal '*' or \* and pass it as-is to expr, which will fail, because neither is a valid operator.
Here's a workaround:
Change to an empty directory.
By default, ksh will return any glob (pattern) as-is if there are no matching filenames. Thus, * (or any other glob) will be left unmodified in an empty directory, because there's nothing to match (thanks, #Peter Cordes).
For the calling script / interactive shell, you could disable globbing altogether by running set -f, but note that this will not affect the called script.
It's now safe to invoke your script with '*' (or any other glob), because it will simply be passed through; e.g., script '*' 2, will now yield 10, as expected
If both the shell you invoke from and the script's shell are ksh (or bash) with their default configuration, you can even get away with script * 2; i.e., you can make do without quoting * altogether.
Glob expansion happens very late, after parameter expansion, and word-splitting (in that order). Quote-removal doesn't happen on the results of earlier expansions, just what was on the command line to start with. This rules out passing in a quoted \* or similar (see mklement0's answer), by using an extra layer of quoting.
It also rules out passing in space-padded *: Word-splitting removes the spaces before pathname (glob) expansion, so it still ends up expanding * to all the filenames in the directory.
foo(){ printf '"%s"\n' "$#"; set -x; expr 5 $1 $2; set +x; }
$ foo ' * ' 4
" * "
"4"
+ expr 5 ...contents of my directory... 4
expr: syntax error
+ set +x
You should fix this buggy script before someone runs it with an arg that breaks it in a dangerous way, rather than just inconvenient.
If you don't need to support exactly the same operators as expr, you might want to use arithmetic expansion to do it without running an external command:
result=$((5 $1 $2)) # arithmetic expansion for the right-hand side
# or
((result=5 "$1" "$2")) # whole command is an arithmetic expression.
Double-quotes around parameters are optional inside an arithmetic expression, but you need to not use them in an arithmetic expansion (in bash. Apparently this works in ksh).
Normally it's not a bad habit to just always quote unless you specifically want word-splitting and glob expansion.
I'am new in Linux and I want to write a bash script that can read in a file name of a directory that starts with LED + some numbers.(Ex.: LED5.5.002)
In that directory there is only one file that will starts with LED. The problem is that this file will every time be updated, so the next time it will be for example LED6.5.012 and counting.
I searched and tried a little bit and came to this solution:
export fspec=/home/led/LED*
LedV=`basename $fspec`
echo $LedV
If I give in those commands one by one in my terminal it works fine, LedV= LED5.5.002 but if i run it in a bash scripts it gives the result: LedV = LED*
I search after another solution:
a=/home/led/LED*
LedV=$(basename $a)
echo $LedV
but here again the same, if i give it in one by one it's ok but in a script: LedV = LED*.
It's probably something small but because of my lack of knowledge over Linux I cannot find it. So can someone tell what is wrong?
Thanks! Jan
Shell expansions don't happen on scalar assignments, so in
varname=foo*
the expansion of "$varname" will literally be "foo*". It's more confusing when you consider that echo $varname (or in your case basename $varname; either way without the double quotes) will cause the expansion itself to be treated as a glob, so you may well think the variable contains all those filenames.
Array expansions are another story. You might just want
fspec=( /path/LED* )
echo "${fspec[0]##*/}" # A parameter expansion to strip off the dirname
That will work fine for bash. Since POSIX sh doesn't have arrays like this, I like to give an alternative approach:
for fspec in /path/LED*; do
break
done
echo "${fspec##*/}"
pwd
/usr/local/src
ls -1 /usr/local/src/mysql*
/usr/local/src/mysql-cluster-gpl-7.3.4-linux-glibc2.5-x86_64.tar.gz
/usr/local/src/mysql-dump_test_all_dbs.sql
if you only have 1 file, you will only get 1 result
MyFile=`ls -1 /home/led/LED*`
Today I tried a script in linux to get all files in one dir. It was pretty straightforward, but I found something interesting.
#!/bin/bash
InputDir=/home/XXX/
for file in $InputDir'*'
do
echo $file
done
The output is:
/home/XXX/fileA /home/XXX/fileB
But when I just input the dir directly, like:
#!/bin/bash
InputDir=/home/XXX/
for file in /home/XXX/*
do
echo $file
done
The output is:
/home/XXX/fileA
/home/XXX/fileB
It seems, in the first script, there was only one loop and all the file names were stored in the variable $file in the FIRST loop, separated by space. But in the second script, one file name was stored in $file just in one loop, and there were more than one loop. What is exactly the difference between these two scripts?
Thanks very much, maybe my question is a little bit naive..
The behavior is correct and "as expected".
for file in $InputDir'*' means assign "/home/XXX/*" to $file (note the quotes). Since you quoted the asterisk, it will not be executed at this time. When the shell sees echo $file, it first expands the variables and then it does glob expansion. So after the first step, it sees
echo /home/XXX/*
and after glob expansion, it sees:
echo /home/XXX/fileA /home/XXX/fileB
Only now, it will execute the command.
In the second case, the pattern /home/XXX/* is expanded before the for is executed and thus, each file in the directory is assigned to file and then the body of the loop is executed.
This will work:
for file in "$InputDir"*
but it's brittle; it will fail, for example, when you forget to add a / to the end of the variable $InputDir.
for file in "$InputDir"/*
is a little bit better (Unix will ignore double slashes in a path) but it can cause trouble when $InputDir is not set or empty: You'll suddenly list files in the / (root) folder. This can happen, for example, because of a typo:
inputDir=...
for file in "$InputDir"/*
Case matters on Unix :-)
To help you understand code like this, use set -x ("enable tracing") in a line before the code you want to debug.
The difference is the quoting of '*'. In the first case the loop only executes once, with $file equal to /home/XXX/* which then expands to all the files in the directory when passed to echo. In the second case it executes once per file, with $file equal to each file name in turn.
Bottom line - change:
for file in $InputDir'*'
to:
for file in $InputDir*
or, better, and to make it more readable - change:
InputDir=/home/XXX/
for file in $InputDir'*'
to:
InputDir=/home/XXX
for file in $InputDir/*