Listing directories with spaces using Bash in linux - linux

I would like to create a bash script to list all the directories in a directory provided by the user via input, or all the directories in the current directory (given no input).
Here's what I have thus far, but when I execute it I encounter two problems.
1) The script completely ignores my input. The file is located on my desktop but when I type in "home" as the input, the script simply prints the directories of the Desktop (current directory).
2) The directories are printed on their own lines (intended) but it treats each word in a folder name as its own folder. i.e. is printed as:
this
folder
Here's the code I have so far:
#!/bin/bash
echo -n "Enter a directory to load files: "
read d
if [ $d="" ]; #if input is blank, assume d = current directory
then d=${PWD##*/}
for i in $(ls -d */);
do echo ${i%%/};
done
else #otherwise, print sub-directories of given directory
for i in $(ls -d */);
do echo ${i%%/};
done
fi
Also in your response please explain your answer as I'm very new to bash.
Thanks for looking, I appreciate your time.
EDIT: Thanks to John1024's answer, I came up with the following:
#!/bin/bash
echo -n "Enter a directory to load files: "
IFS= read d
ls -1 -d "${d:-.}"/*/
And it does everything I need. Much appreciated!

I believe that this script accomplishes what you want:
#!/bin/sh
ls -1 -d "${1:-.}"/*/
Usage example:
$ bash ./script.sh /usr/X11R6
/usr/X11R6/bin
/usr/X11R6/man
Explanation:
-1 tells ls to print each file/directory on a separate line
-d tells ls to list directories by name instead of their contents
The shell will ${1:-.} to be the first argument to the script if there is one or . (which means the current directory) if there isn't.
Enhancement
The above script displays a / at the end of each directory name. If you don't want that, we can use sed to remove trailing slashes from the output:
#!/bin/sh
ls -1d ${1:-.}/*/ | sed 's|/$||'
Revised Version of Your Script
Starting with your script, some simplifications can be made:
#!/bin/bash
echo -n "Enter a directory to load files: "
IFS= read d
d=${d:-$PWD}
for i in "$d"/*/
do
echo ${i%%/}
done
Notes:
IFS= read d
Normally leading and trailing white space are stripped before the input is assigned to d. By setting IFS to an empty value, however, leading and trailing white space will be preserved. Thus this will work even if the pathologically strange case where the user specifies a directory whose name begins or ends with white space.
If the user enters a backslash, the shell will try to process it as an escape. If you don't like that, use IFS= read -r d and backslashes will be treated as normal characters, not escapes.
d=${d:-$PWD}
If the user supplied a value for d, this leaves it unchanged. If he didn't, this assigns it to $PWD.
for i in "$d"/*/
This will loop over every subdirectory of $d and will correctly handle subdirectory names with spaces, tabs, or any other odd character.
By contrast, consider:
for i in $(ls -d */)
After ls executes here, the shell will split up the output into individual words. This is called "word splitting" and is why this form of the for loop should be avoided.
Notice the double-quotes in for i in "$d"/*/. They are there to prevent word splitting on $d.

Related

How to list all the folder in a folder and exclude a specific one

Let's say I have a folder like this:
my_folder
====my_sub_folder_1
====my_sub_folder_2
====my_sub_folder_3
====exclude
I would like a command that return a string like this :
["my_sub_folder_1", "my_dub_folder_2", "my_dub_folder_3"]
(Notice the exclusion of the excude folder)
The best I could is :
ls -dxm */
That return the following.
my_sub_folder_1/, my_dub_folder_2/, my_dub_folder_3/
So I'm still trying to remove the / at the end of each folder, add the [] and the "".
If it's possible I would like to do that in one line so I could diretly put in a shell variable, other wise I will put it in .sh file that will return the string I'm trying to build.
(I don't know if the last part is really possible).
Assuming you are executing the script in the directory where my_folder
belongs, how about:
while IFS= read -r -d "" f; do
ary+=("$f")
done < <(find "my_folder" -maxdepth 1 -mindepth 1 -type d -not -name "exclude" -printf "\"%f\"\0")
(IFS=","; echo "[${ary[*]}]")
[Explanations]
-printf option to find command specifies the output format. The format "\"%f\"\0"
prints the filename (excluding leading directory name) wrapped by
double quotes and followed by a NUL character \0.
The NUL character is used as a filename delimiter and the filenames
are split again in the read builtin by specifying the delimiter
to the NUL character with -d "".
Then the filenames (with double quotes) are stored in the array ary
one by one.
Finally echo "[${ary[*]}]" command prints out the elements of ary
separated by IFS. The whole output are surrounded by the square brackets [].
The last line is surrounded by parens () to be executed in the subprocess.
The purpose is just not to overwrite the current IFS.
If you save the script in my answer as my_script.sh, then you can assign
a variable MY_VAR to the output by saying:
MY_VAR=$(./my_script.sh)
echo "$MY_VAR"
# or another_command "$MY_VAR" or whatever
Alternatively you can assign the variable within the script by modifying
the last line as:
MY_VAR=$(IFS=","; echo "[${ary[*]}]")
echo "$MY_VAR"
Hope this helps.
In bash this can be done as follows, it's close but it doesn't work in one line.
Change the Internal Field Separator to be a new line rather than a space. This allows spaces in directory names to be ignored.
Then perform the following:
List the directories, one per line
Use grep to remove the directory to be excluded
Iterate over the results:
Output the directory name with the last character removed
Pipe everything to xargs to recombine into a single line and store in $var
Trim the last , from ${var} and wrap in '[]'
IFS=$'\n'
var=`for d in \`ls -d1 */ | grep -v exclude_dir \`; do echo '\"'${d::-1}'\",' ; done | xargs`
echo '['${var::-1}']'

How to create directories automatically in linux?

I am having a file named temp.txt where inside this file it contains the following content
https://abcdef/12345-xyz
https://ghifdfg/5426525-abc
I need to create a directories automatically in linux by using only th number part from each line in the file.
So the output should be something like 12345 and 5426525 directories created.
Any approach on how to do this could be helpful.
This is the code that i searched and got from internet,wherein this code, new directories will be created by the file name that starts with BR and W0 .
for file in {BR,W0}*.*; do
dir=${file%%.*}
mkdir -p "$dir"
mv "$file" "$dir"
done
Assuming each URL is of the form
http[s]://any/symbols/some_digits-some_letters
Then you indeed could use the simple prefix and suffix modifiers in shell variable expansion.
${x##*/} expands to the suffix part of x that starts after the last slash /.
${y%%-*} expands to the prefix part of y before the first -.
while read x ; do
y=${x##*/}
z=${y%%-*}
mkdir $z
done < temp.txt

how to pass asterisk into ls command inside bash script

Hi… Need a little help here…
I tried to emulate the DOS' dir command in Linux using bash script. Basically it's just a wrapped ls command with some parameters plus summary info. Here's the script:
#!/bin/bash
# default to current folder
if [ -z "$1" ]; then var=.;
else var="$1"; fi
# check file existence
if [ -a "$var" ]; then
# list contents with color, folder first
CMD="ls -lgG $var --color --group-directories-first"; $CMD;
# sum all files size
size=$(ls -lgGp "$var" | grep -v / | awk '{ sum += $3 }; END { print sum }')
if [ "$size" == "" ]; then size="0"; fi
# create summary
if [ -d "$var" ]; then
folder=$(find $var/* -maxdepth 0 -type d | wc -l)
file=$(find $var/* -maxdepth 0 -type f | wc -l)
echo "Found: $folder folders "
echo " $file files $size bytes"
fi
# error message
else
echo "dir: Error \"$var\": No such file or directory"
fi
The problem is when the argument contains an asterisk (*), the ls within the script acts differently compare to the direct ls command given at the prompt. Instead of return the whole files list, the script only returns the first file. See the video below to see the comparation in action. I don't know why it behaves like that.
Anyone knows how to fix it? Thank you.
Video: problem in action
UPDATE:
The problem has been solved. Thank you all for the answers. Now my script works as expected. See the video here: http://i.giphy.com/3o8dp1YLz4fIyCbOAU.gif
The asterisk * is expanded by the shell when it parses the command line. In other words, your script doesn't get a parameter containing an asterisk, it gets a list of files as arguments. Your script only works with $1, the first argument. It should work with "$#" instead.
This is because when you retrieve $1 you assume the shell does NOT expand *.
In fact, when * (or other glob) matches, it is expanded, and broken into segments by $IFS, and then passed as $1, $2, etc.
You're lucky if you simply retrieved the first file. When your first file's path contains spaces, you'll get an error because you only get the first segment before the space.
Seriously, read this and especially this. Really.
And please don't do things like
CMD=whatever you get from user input; $CMD;
You are begging for trouble. Don't execute arbitrary string from the user.
Both above answers already answered your question. So, i'm going a bit more verbose.
In your terminal is running the bash interpreter (probably). This is the program which parses your input line(s) and doing "things" based on your input.
When you enter some line the bash start doing the following workflow:
parsing and lexical analysis
expansion
brace expansion
tidle expansion
variable expansion
artithmetic and other substitutions
command substitution
word splitting
filename generation (globbing)
removing quotes
Only after all above the bash
will execute some external commands, like ls or dir.sh... etc.,
or will do so some "internal" actions for the known keywords and builtins like echo, for, if etc...
As you can see, the second last is the filename generation (globbing). So, in your case - if the test* matches some files, your bash expands the willcard characters (aka does the globbing).
So,
when you enter dir.sh test*,
and the test* matches some files
the bash does the expansion first
and after will execute the command dir.sh with already expanded filenames
e.g. the script get executed (in your case) as: dir.sh test.pas test.swift
BTW, it acts exactly with the same way for your ls example:
the bash expands the ls test* to ls test.pas test.swift
then executes the ls with the above two arguments
and the ls will print the result for the got two arguments.
with other words, the ls don't even see the test* argument - if it is possible - the bash expands the wilcard characters. (* and ?).
Now back to your script: add after the shebang the following line:
echo "the $0 got this arguments: $#"
and you will immediatelly see, the real argumemts how your script got executed.
also, in such cases is a good practice trying to execute the script in debug-mode, e.g.
bash -x dir.sh test*
and you will see, what the script does exactly.
Also, you can do the same for your current interpreter, e.g. just enter into the terminal
set -x
and try run the dir.sh test* = and you will see, how the bash will execute the dir.sh command. (to stop the debug mode, just enter set +x)
Everbody is giving you valuable advice which you should definitely should follow!
But here is the real answer to your question.
To pass unexpanded arguments to any executable you need to single quote them:
./your_script '*'
The best solution I have is to use the eval command, in this way:
#!/bin/bash
cmd="some command \"with_quetes_and_asterisk_in_it*\""
echo "$cmd"
eval $cmd
The eval command takes its arguments and evaluates them into the command as the shell does.
This solves my problem when I need to call a command with asterisk '*' in it from a script.

Linux bash shell scripts - spaces in file names

It has been a long time since I did much bash script writing.
This is a bash script to copy and rename files by deleting all before the first period delimiter:
#!/bin/bash
mkdir fullname
mv *.audio fullname
cd fullname
for x in * ;
do
cp $x ../`echo $x | cut -d "." -f 2-`
done
cd ..
ls
It works well for file names with no embedded spaces but not for those with spaces.
How can I change the code to fix this simple Linux bash script? Any suggestions for improving the code for other reasons would also be welcome.
Example filenames, some with embedded spaces and some not (from link)
http://www.homenetvideo.com/demo/index.php?/Radio%20%28VLC%29
Ambient.A6.SOMA Space Station.audio
Blues.B9.Blues Radio U.K.audio
Classical.K3.Radio Stephansdom - Vienna.audio
College.CI.KDVS U of California, Davis.audio
Country.Q1.K-FROG.audio
Easy.G4.WNYU.audio
Eclectic.M2.XPN.audio
Electronica.E2.Rinse.audio
Folk.F1.Radionomy.audio
Hiphop.H1.NPR.audio
Indie.I4.WAUG.audio
Jazz.J6.KCSM.audio
Latin.L3.Mega.audio
Misc.X7.Gaydio.audio
News.N9.KQED.audio
Oldies.O1.Lonestar.audio
OldTime.Y1.Roswell.audio
Progressive.P1.Aural Moon.audio
Rock.R8.WXRT.audio
Scanner.Z3.Montreal.audio
Soul.S1.181.FM.audio
Talk.T2.TWiT.audio
World.W3.Persian.audio
http://lh5.googleusercontent.com/-QjLEiAtT4cw/U98_UFcWvvI/AAAAAAAABv8/gyPhbg8s7Bw/w681-h373-no/homenet-radio.png
Whenever you deal with file names that might have spaces in them, you must reference them as "$x" rather than just $x. That's what's causing your cp command to fail.
Your echo command is also problematic. Although echo does the right thing for simple spaces - it echoes a file named A B C as A B C - it will still fail if you have more than one consecutive space in the name, or whitespace that isn't a simple space character.
Instead of passing the file names to external programs for processing, which always requires getting them through the whitespace-hostile command line, you should use bash built-in functions for string manipulations wherever possible, e.g. ${x%%foo}, ${x#bar} and similar functions. The man page describes them under "Parameter expansion".
Here's my suggestion:
#!/bin/bash
shopt -s nullglob
mkdir fullname
mv *.audio fullname
(
cd fullname || exit
for x in *; do
cp "$x" "../${x#*.}"
done
)
ls
nullglob prevents * from presenting itself if no file matches it. Just optional.
() summons a subshell and saves you from changing back to another directory.
|| exit terminates the subshell if cd fails to change directory.
${x#*.} removes the <first>. from $x and expands it.

How to remove the extension of a file?

I have a folder that is full of .bak files and some other files also. I need to remove the extension of all .bak files in that folder. How do I make a command which will accept a folder name and then remove the extension of all .bak files in that folder ?
Thanks.
To remove a string from the end of a BASH variable, use the ${var%ending} syntax. It's one of a number of string manipulations available to you in BASH.
Use it like this:
# Run in the same directory as the files
for FILENAME in *.bak; do mv "$FILENAME" "${FILENAME%.bak}"; done
That works nicely as a one-liner, but you could also wrap it as a script to work in an arbitrary directory:
# If we're passed a parameter, cd into that directory. Otherwise, do nothing.
if [ -n "$1" ]; then
cd "$1"
fi
for FILENAME in *.bak; do mv "$FILENAME" "${FILENAME%.bak}"; done
Note that while quoting your variables is almost always a good practice, the for FILENAME in *.bak is still dangerous if any of your filenames might contain spaces. Read David W.'s answer for a more-robust solution, and this document for alternative solutions.
There are several ways to remove file suffixes:
In BASH and Kornshell, you can use the environment variable filtering. Search for ${parameter%word} in the BASH manpage for complete information. Basically, # is a left filter and % is a right filter. You can remember this because # is to the left of %.
If you use a double filter (i.e. ## or %%, you are trying to filter on the biggest match. If you have a single filter (i.e. # or %, you are trying to filter on the smallest match.
What matches is filtered out and you get the rest of the string:
file="this/is/my/file/name.txt"
echo ${file#*/} #Matches is "this/` and will print out "is/my/file/name.txt"
echo ${file##*/} #Matches "this/is/my/file/" and will print out "name.txt"
echo ${file%/*} #Matches "/name.txt" and will print out "/this/is/my/file"
echo ${file%%/*} #Matches "/is/my/file/name.txt" and will print out "this"
Notice this is a glob match and not a regular expression match!. If you want to remove a file suffix:
file_sans_ext=${file%.*}
The .* will match on the period and all characters after it. Since it is a single %, it will match on the smallest glob on the right side of the string. If the filter can't match anything, it the same as your original string.
You can verify a file suffix with something like this:
if [ "${file}" != "${file%.bak}" ]
then
echo "$file is a type '.bak' file"
else
echo "$file is not a type '.bak' file"
fi
Or you could do this:
file_suffix=$(file##*.}
echo "My file is a file '.$file_suffix'"
Note that this will remove the period of the file extension.
Next, we will loop:
find . -name "*.bak" -print0 | while read -d $'\0' file
do
echo "mv '$file' '${file%.bak}'"
done | tee find.out
The find command finds the files you specify. The -print0 separates out the names of the files with a NUL symbol -- which is one of the few characters not allowed in a file name. The -d $\0means that your input separators are NUL symbols. See how nicely thefind -print0andread -d $'\0'` together?
You should almost never use the for file in $(*.bak) method. This will fail if the files have any white space in the name.
Notice that this command doesn't actually move any files. Instead, it produces a find.out file with a list of all the file renames. You should always do something like this when you do commands that operate on massive amounts of files just to be sure everything is fine.
Once you've determined that all the commands in find.out are correct, you can run it like a shell script:
$ bash find.out
rename .bak '' *.bak
(rename is in the util-linux package)
Caveat: there is no error checking:
#!/bin/bash
cd "$1"
for i in *.bak ; do mv -f "$i" "${i%%.bak}" ; done
You can always use the find command to get all the subdirectories
for FILENAME in `find . -name "*.bak"`; do mv --force "$FILENAME" "${FILENAME%.bak}"; done

Resources