I am a beginner in bash scripting and I am trying to write a script which has as variables directory names, and who uses those variable values to run simple bash commands such as "ls" and "cd". It works perfectly fine when the directory has a "normal" name, for example
testfolder/folder01
But fails miserably when the directory has spaces and parenthesis in their names, which happens for example when you do a copy a subdirectory and paste in the same directory containing the subdirectory. The problem can be seen in this script:
[boblacerda#localhost MyScripts]$ cat test.sh
#!/bin/bash
VARDIR="testfolder/folder01"
ls $VARDIR
VARDIR="testfolder/folder01\ \(copy\)"
ls $VARDIR
[boblacerda#localhost MyScripts]$
This is the output of the script in debugging mode:
[boblacerda#localhost MyScripts]$ bash -x test.sh
+ VARDIR=testfolder/folder01
+ ls testfolder/folder01
testefile01 testefile02
+ VARDIR='testfolder/folder01\ \(copy\)'
+ ls 'testfolder/folder01\' '\(copy\)'
ls: cannot access testfolder/folder01\: No such file or directory
ls: cannot access \(copy\): No such file or directory
+ exit
[boblacerda#localhost MyScripts]$
As you see the first part, who uses a directory with a "normal" name works, but the second part, who uses a directory with spaces and parenthesis in its name, fails. The problem persists if I quote VARDIR in the ls command, i.e., if I use ls like this
ls "$VARDIR"
The output in this case is like this:
[boblacerda#localhost MyScripts]$ bash -x test.sh
+ VARDIR=testfolder/folder01
+ ls testfolder/folder01
testefile01 testefile02
+ VARDIR='testfolder/folder01\ \(copy\)'
+ ls 'testfolder/folder01\ \(copy\)'
ls: cannot access testfolder/folder01\ \(copy\): No such file or directory
+ exit
[boblacerda#localhost MyScripts]$
A final remark to add that the command
ls testfolder/folder01\ \(copy\)
works fine in the cmd as shown below:
[boblacerda#localhost MyScripts]$ls testfolder/folder01\ \(copy\)
testefile01 testefile02
[boblacerda#localhost MyScripts]$
Thank you all for the attention.
There are two problems with your script. First, you are not setting VARDIR correctly as you have too many backslashes. Second, you should put quotes around any use of any variable.
$ cat test.sh
#!/bin/bash
VARDIR="testfolder/folder01"
ls "$VARDIR"
VARDIR="testfolder/folder01 (copy)"
ls "$VARDIR"
When setting VARDIR, you can either use backslashes, or quotes, but not both:
VARDIR="testfolder/folder01 (copy)"
or
VARDIR=testfolder/folder01\ \(copy\)
Try:
ls "$VARDIR"
The double quotes will preserve the space, no need for backslashes.
Related
I'm trying to redirect ls command's errors. But I found my redirection is wrong. For example, if I wrote this ls commands,
$ ls ;;;
Terminal says,
bash: syntax error near unexpected token `;;'
But, my redirected file wrote this,
ls: cannot access ;;;: No such file or directory
How can I catch differences between redirected file and terminal?
Put the ;;; in quotes, bash will then always pass that argument to the ls command. Without quotes bash is trying to parse the ;;;, hence the error.
ls ';;;' 2> stderr.txt
< no output >
cat stderr.txt
ls: ;;;: No such file or directory
I want to copy a bunch of Verilog/systemverilog sources, so I use CP with wildcard expression:
cp <some_dir>/*.{v,sv,svh} .
It works. But when I put it to a script with exactly the same line, the CP command fails with the log:
cp: cannot stat `../../mytest/spiTest/*.{v,sv,svh}': No such file or directory
How is that happening?
PS: I use bash as the shell.
And here is my script:
#!/bin/bash
rdir=../../mytest/spiTest
f1="$rdir/bench.lst"
f2="$rdir/cphex" #the script to copy rom data
f3="$rdir/make*" #makefile scripts
f4="$rdir/*.hex" #rom files
f5="$rdir/*.{v,sv,svh}" #testbench files
echo 'Copying files...'
cp $f1 $f2 $f3 $f4 .
cp $f5 .
I do changed the first line to
#!/bin/bash -vx
and run this script again, and I get:
#!/bin/bash -vx
rdir=../../mytest/spiTest
+ rdir=../../mytest/spiTest
f1="$rdir/bench.lst"
+ f1=../../mytest/spiTest/bench.lst
f2="$rdir/cphex" #the script to copy rom data
+ f2=../../mytest/spiTest/cphex
f3="$rdir/make*" #makefile scripts
+ f3='../../mytest/spiTest/make*'
f4="$rdir/*.hex" #rom files
+ f4='../../mytest/spiTest/*.hex'
f5="$rdir/*.{v,sv,svh}" #testbench files
+ f5='../../mytest/spiTest/*.{v,sv,svh}'
echo 'Copying files...'
+ echo 'Copying files...'
Copying files...
cp $f1 $f2 $f3 $f4 .
+ cp ../../mytest/spiTest/bench.lst ../../mytest/spiTest/cphex ../../mytest/spiTest/makefile ../../mytest/spiTest/makefile.defines ../../mytest/spiTest/rom.hex ../../mytest/spiTest/rom_if.hex .
cp $f5 .
+ cp '../../mytest/spiTest/*.{v,sv,svh}' .
cp: cannot stat `../../mytest/spiTest/*.{v,sv,svh}': No such file or directory
Check the first line of the script. It probably reads:
#!/bin/sh
which switches the shell from BASH to Bourne Shell. Use
#!/bin/bash
instead.
[EDIT] You're running into problems with expansion. BASH has a certain order in which it expands patterns and variables. That means:
f5="$rdir/*.{v,sv,svh}" #testbench files
is quoted, so no file name expansion happens at this time. Only the variable $rdir is expanded. When
cp $f5 .
is executed, BASH first looks for file names to expand and there are none. Then it expands variables (f5) and then calls cp with two arguments: ../../mytest/spiTest/*.{v,sv,svh} and .. Since cp expects the shell to have performed the file name expansion already, you get an error.
To fix this, you have to use arrays:
f5=($rdir/*.{v,sv,svh})
This replaces the variable and then expands the file names and puts everything into the array f5. You can then call cp with this array while preserving whitespaces:
cp "${f5[#]}" .
Every single character here is important. [#] tells BASH to expand the whole array here. The quotes say: Preserve whitespace. {} is necessary to tell BASH that [#] is part of the variable "name" to expand.
Here's the problem: the order of substitutions. Bash performs brace expansion before variable expansion. In the line cp $f5 ., bash will do:
brace expansion: n/a
this is the key point: the variable contains a brace expression, but the shell does not see it now when it needs to.
tilde expansion: n/a
parameter expansion: yes -- cp ../../mytest/spiTest/*.{v,sv,svh} .
command substitution: n/a
arithmetic expansion: n/a
process substitution: n/a
word splitting: n/a
filename expansion: yes, bash looks for files in that directory ending with the string
.{v,sv,svh}. It finds none, nullglob is not set, thus the pattern is not removed from the command
quote removal: n/a
Now the command is executed and fails with the error you see.
https://www.gnu.org/software/bash/manual/bashref.html#Shell-Expansions
Solutions:
use Aaron's idea of an array
(not recommended) force a 2nd round of expansions: eval cp $f5 .
The line
f5="$rdir/*.{v,sv,svh}" #testbench files
is probably wrong. First, avoid comments at end of line, they should be (at least for readability) in a separate line. Then, avoid using globbing in variable assignment. So remove that line, and code later (that is, replace the old cp $f5 . line with)
cp "$rdir"/*.{v,sv,svh} .
BTW, I would test that "$rdir" is indeed a directory with
if [ ! -d "$rdir" ] ; then
echo invalid directory $rdir > /dev/stderr
exit 1
fi
You should read the Advanced Bash Scripting Guide
This probably quite basic but I have spent whole day finding an answer without much success.
I have an executable script that resides in ~/Desktop/shell/myScript.sh
I want a single line command to run this script from my terminal that outputs to a new directory in same directory where the script is located no matter what my present working directory is.
I was using:
mkdir -p tmp &&
./Desktop/shell/myScript.sh|grep '18x18'|cut -d":" -f1 > tmp/myList.txt
But it creates new directory in present working directory and not on the target location.
Any help would be appreciated.
Thanks!
You could solve it in one line if you pre-define a variable:
export LOC=$HOME/Desktop/shell
Then you can say
mkdir -p $LOC/tmp && $LOC/myScript.sh | grep '18x18' | cut -d":" -f1 > $LOC/tmp/myList.txt
But if you're doing this repeatedly it might be better long-term to wrap myScript.sh so that it creates the directory, and redirects the output, for you. The grep and cut parameters, as well as the output file name, would be passed as command-line arguments and options to the wrapper.
How about this:
SCRIPTDIR="./Desktop/shell/" ; mkdir "$SCRIPTDIR/tmp" ; "$SCRIPTDIR/myScript.sh" | grep '18x18' | cut -d ":" -f 1 > "$SCRIPTDIR/tmp/myList.txt"
In your case you have to give the path to the script anyway. If you put the script in the path where it is automatically searched, e.g. $HOME/bin, and you can just type myScript.sh without the directory prefix, you can use SCRIPTDIR=$( dirname $( which myScript.sh ) ).
Mixing directories with binaries and data files is usually a bad idea. For temporary files /tmp is the place to go. Consider that your script might become famous and get installed by the administrator in /usr/bin and run by several people at the same time. For this reason, try to think mktemp.
YOUR SCRIPT CAN DO THIS FOR YOU WITH SOME CODES
Instead of doing this manually from the command line and who knows where you will move your script and put it. add the following codes
[1] Find your script directory location using dirname
script_directory=`dirname $0`
The above code will find your script directory and save it in a variable.
[2] Create your "tmp" folder in your script directory
mkdir "$script_directory/tmp 2> /dev/null"
The above code will make a directory called "tmp" in your script directory. If the directory exist, mkdir will not overwrite any existing directory using this command line and gave an error. I hide all errors by "2> /dev/null"
[3] Open your script and modify it using "cut" and then redirect the output to a new file
cat "$0"|grep '18x18'|cut -d":" -f1 > "$script_directory"/tmp/myList.txt
I would like to create a bash script to list all the directories in a directory provided by the user via input, or all the directories in the current directory (given no input).
Here's what I have thus far, but when I execute it I encounter two problems.
1) The script completely ignores my input. The file is located on my desktop but when I type in "home" as the input, the script simply prints the directories of the Desktop (current directory).
2) The directories are printed on their own lines (intended) but it treats each word in a folder name as its own folder. i.e. is printed as:
this
folder
Here's the code I have so far:
#!/bin/bash
echo -n "Enter a directory to load files: "
read d
if [ $d="" ]; #if input is blank, assume d = current directory
then d=${PWD##*/}
for i in $(ls -d */);
do echo ${i%%/};
done
else #otherwise, print sub-directories of given directory
for i in $(ls -d */);
do echo ${i%%/};
done
fi
Also in your response please explain your answer as I'm very new to bash.
Thanks for looking, I appreciate your time.
EDIT: Thanks to John1024's answer, I came up with the following:
#!/bin/bash
echo -n "Enter a directory to load files: "
IFS= read d
ls -1 -d "${d:-.}"/*/
And it does everything I need. Much appreciated!
I believe that this script accomplishes what you want:
#!/bin/sh
ls -1 -d "${1:-.}"/*/
Usage example:
$ bash ./script.sh /usr/X11R6
/usr/X11R6/bin
/usr/X11R6/man
Explanation:
-1 tells ls to print each file/directory on a separate line
-d tells ls to list directories by name instead of their contents
The shell will ${1:-.} to be the first argument to the script if there is one or . (which means the current directory) if there isn't.
Enhancement
The above script displays a / at the end of each directory name. If you don't want that, we can use sed to remove trailing slashes from the output:
#!/bin/sh
ls -1d ${1:-.}/*/ | sed 's|/$||'
Revised Version of Your Script
Starting with your script, some simplifications can be made:
#!/bin/bash
echo -n "Enter a directory to load files: "
IFS= read d
d=${d:-$PWD}
for i in "$d"/*/
do
echo ${i%%/}
done
Notes:
IFS= read d
Normally leading and trailing white space are stripped before the input is assigned to d. By setting IFS to an empty value, however, leading and trailing white space will be preserved. Thus this will work even if the pathologically strange case where the user specifies a directory whose name begins or ends with white space.
If the user enters a backslash, the shell will try to process it as an escape. If you don't like that, use IFS= read -r d and backslashes will be treated as normal characters, not escapes.
d=${d:-$PWD}
If the user supplied a value for d, this leaves it unchanged. If he didn't, this assigns it to $PWD.
for i in "$d"/*/
This will loop over every subdirectory of $d and will correctly handle subdirectory names with spaces, tabs, or any other odd character.
By contrast, consider:
for i in $(ls -d */)
After ls executes here, the shell will split up the output into individual words. This is called "word splitting" and is why this form of the for loop should be avoided.
Notice the double-quotes in for i in "$d"/*/. They are there to prevent word splitting on $d.
I know this question has been asked numerous times, but I still could not find any good solution. Hence, asking it again if anyone can help !!
I am trying to change a my working directory inside a shell script with help of a variable. But I get " No such file or directory" everytime.
#!/bin/bash
echo ${RED_INSTANCE_NAME} <-- This correctly displays the directory name
cd $RED_INSTANCE_NAME <-- This line gives the error
Now, when I try to give the actually directory name instead of using the variable, shell changes the directory without issues
cd test <-- No error
Does anyone knows what can be the issue here ? Please help !!
You variable contains a carriage return. Try saying:
cd $(echo $RED_INSTANCE_NAME | tr -d '\r')
and it should work. In order to remove the CR from the variable you can say:
RED_INSTANCE_NAME=$(echo $RED_INSTANCE_NAME | tr -d '\r')
The following would illustrate the issue:
$ mkdir abc
$ foo=abc$'\r'
$ echo "${foo}"
abc
$ cd "${foo}"
: No such file or directory
$ echo $foo | od -x
0000000 6261 0d63 000a
0000005
$ echo $foo | tr -d '\r' | od -x
0000000 6261 0a63
0000004
$ echo $'\r' | od -x
0000000 0a0d
0000002
One way to encounter your described problem is to have a tilde (~) in the variable name. Use the absolute path or $HOME variable instead. Note that using $HOME will require double quotations.
# doesn't work
$ vartilde='~/'
$ cd $vartilde
-bash: cd: ~: No such file or directory
# works
$ varfullpath='/Users/recurvirostridae'
$ cd $varfullpath
# works
$ varwithhome="$HOME"
$ cd $varwithhome
Try
cd "$RED_INSTANCE_NAME"
Also, make sure the path makes sense to the current directory where cd command is executed.
I ran into a different issue. My "cd $newDir" was failing because I added logging into my script. Apparently if you add a pipe to any cd command it does nothing or gets gobbled up.
Wasted 3 hours figuring that out.
newDir=$oldDir/more/dirs/
cd $newDir #works
cd $newDir | tee -a log #does nothing
cd $newdir | echo hi #does nothing
So cd with any pipe does nothing. No idea why cd fails. The pipe means finish what command you doing then feed any output to next command. This is on RHEL 7.
I was trying to log all my commands and hit this nice error. Figured I'd post it in case anyone else hits it.
You can check for carriage returns, ANSI escapes and other special characters with
cat -v <<< "$RED_INSTANCE_NAME"
This will show all the characters that echo $RED_INSTANCE_NAME would just hide or ignore.
In particular, if your error message is : No such file or directory as opposed to bash: cd: yourdir: No such file or directory, it means you have a carriage return at the end of your variable, probably from reading it from a DOS formatted file.
I don't know what is going wrong for you, but I can offer one piece of general advice:
cd "$RED_INSTANCE_NAME" # Quote the string in case it has spaces.error
You should nearly always put the "$VARIABLE" in quotes. This will protect from surprises when the value of the variable contains funny stuff (like spaces).