Why the Linux command CP behave differently in CLI and in script? - linux

Question

I want to copy a bunch of Verilog/systemverilog sources, so I use CP with wildcard expression:
cp <some_dir>/*.{v,sv,svh} .
It works. But when I put it to a script with exactly the same line, the CP command fails with the log:
cp: cannot stat `../../mytest/spiTest/*.{v,sv,svh}': No such file or directory
How is that happening?
PS: I use bash as the shell.
And here is my script:
#!/bin/bash
rdir=../../mytest/spiTest
f1="$rdir/bench.lst"
f2="$rdir/cphex" #the script to copy rom data
f3="$rdir/make*" #makefile scripts
f4="$rdir/*.hex" #rom files
f5="$rdir/*.{v,sv,svh}" #testbench files
echo 'Copying files...'
cp $f1 $f2 $f3 $f4 .
cp $f5 .
I do changed the first line to
#!/bin/bash -vx
and run this script again, and I get:
#!/bin/bash -vx
rdir=../../mytest/spiTest
+ rdir=../../mytest/spiTest
f1="$rdir/bench.lst"
+ f1=../../mytest/spiTest/bench.lst
f2="$rdir/cphex" #the script to copy rom data
+ f2=../../mytest/spiTest/cphex
f3="$rdir/make*" #makefile scripts
+ f3='../../mytest/spiTest/make*'
f4="$rdir/*.hex" #rom files
+ f4='../../mytest/spiTest/*.hex'
f5="$rdir/*.{v,sv,svh}" #testbench files
+ f5='../../mytest/spiTest/*.{v,sv,svh}'
echo 'Copying files...'
+ echo 'Copying files...'
Copying files...
cp $f1 $f2 $f3 $f4 .
+ cp ../../mytest/spiTest/bench.lst ../../mytest/spiTest/cphex ../../mytest/spiTest/makefile ../../mytest/spiTest/makefile.defines ../../mytest/spiTest/rom.hex ../../mytest/spiTest/rom_if.hex .
cp $f5 .
+ cp '../../mytest/spiTest/*.{v,sv,svh}' .
cp: cannot stat `../../mytest/spiTest/*.{v,sv,svh}': No such file or directory

Answer 1

Check the first line of the script. It probably reads:
#!/bin/sh
which switches the shell from BASH to Bourne Shell. Use
#!/bin/bash
instead.
[EDIT] You're running into problems with expansion. BASH has a certain order in which it expands patterns and variables. That means:
f5="$rdir/*.{v,sv,svh}" #testbench files
is quoted, so no file name expansion happens at this time. Only the variable $rdir is expanded. When
cp $f5 .
is executed, BASH first looks for file names to expand and there are none. Then it expands variables (f5) and then calls cp with two arguments: ../../mytest/spiTest/*.{v,sv,svh} and .. Since cp expects the shell to have performed the file name expansion already, you get an error.
To fix this, you have to use arrays:
f5=($rdir/*.{v,sv,svh})
This replaces the variable and then expands the file names and puts everything into the array f5. You can then call cp with this array while preserving whitespaces:
cp "${f5[#]}" .
Every single character here is important. [#] tells BASH to expand the whole array here. The quotes say: Preserve whitespace. {} is necessary to tell BASH that [#] is part of the variable "name" to expand.

Answer 2

Here's the problem: the order of substitutions. Bash performs brace expansion before variable expansion. In the line cp $f5 ., bash will do:
brace expansion: n/a
this is the key point: the variable contains a brace expression, but the shell does not see it now when it needs to.
tilde expansion: n/a
parameter expansion: yes -- cp ../../mytest/spiTest/*.{v,sv,svh} .
command substitution: n/a
arithmetic expansion: n/a
process substitution: n/a
word splitting: n/a
filename expansion: yes, bash looks for files in that directory ending with the string
.{v,sv,svh}. It finds none, nullglob is not set, thus the pattern is not removed from the command
quote removal: n/a
Now the command is executed and fails with the error you see.
https://www.gnu.org/software/bash/manual/bashref.html#Shell-Expansions
Solutions:
use Aaron's idea of an array
(not recommended) force a 2nd round of expansions: eval cp $f5 .

Answer 3

The line
f5="$rdir/*.{v,sv,svh}" #testbench files
is probably wrong. First, avoid comments at end of line, they should be (at least for readability) in a separate line. Then, avoid using globbing in variable assignment. So remove that line, and code later (that is, replace the old cp $f5 . line with)
cp "$rdir"/*.{v,sv,svh} .
BTW, I would test that "$rdir" is indeed a directory with
if [ ! -d "$rdir" ] ; then
echo invalid directory $rdir > /dev/stderr
exit 1
fi
You should read the Advanced Bash Scripting Guide