parse result of `ls -l` with bash script [duplicate] - linux

Question

This question already has answers here:
How to loop over files in directory and change path and add suffix to filename
(6 answers)
Closed 5 years ago.
I need to store the name of every file contained in a directory with a bash script and processes it in some way:
drwxrwxr-x 5 matteorr matteorr 4096 Jan 10 17:37 Cluster
drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 10:43 Desktop
drwxrwxr-x 9 matteorr matteorr 4096 Jan 20 10:01 Developer
drwxr-xr-x 11 matteorr matteorr 4096 Dec 20 13:55 Documents
drwxr-xr-x 2 matteorr matteorr 12288 Jan 20 13:44 Downloads
drwx------ 11 matteorr matteorr 4096 Jan 20 14:01 Dropbox
drwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Music
drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 22:12 Pictures
drwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Public
drwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Templates
drwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Videos
with the following command I'm able to split the result of ls -l in between all the spaces and then access the last element, which contains the name:
ls -l | awk '{split($0,array," ")} END{print array[9]}'
However it returns only the last line (i.e. Videos) so I need to iterate it over all the lines returned by the ls -l command.
how can I do this?
Is there a better way to approach this whole problem?
ADDED PART
To be a little more specific on what I need to do:
For all the files contained in a directory if it is a file I won't do anything, if it is a directory I should append the name of the directory to all the files it contains.
So supposing the directory Videos has the files:
-rwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 video1.mpeg
-rwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Video2.wmv
I need to rename them as follows:
-rwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 video1_Videos.mpeg
-rwxr-xr-x 2 matteorr matteorr 4096 Oct 18 18:43 Video2_Videos.wmv

Answer 1

A better way would be to use bash globbing
Just listing all files
echo *
Or doing something with them
for file in *; do
echo "$file" # or do something else
done
Or recursively with bash 4+
shopt -s globstar
for file in **/*; do
echo "$file" # or do something else
done
Update to get directory name and append it to all files within it
Replace mv with an echo to test what it does. Also note ${file##*.} assumes the extension is everything after the last period, so if you had a file like file.tar.gz in directory on, below would turn it into file.tar_on.gz. As far as I know there is no easy way to handle this problem, though you could skip files with multiple . if you want)
#!/bin/bash
d="/some/dir/to/do/this/on"
name=${d##*/} #name=on
for file in "$d"/*; do
extension=${file##*.}
filename=${file%.*}
filename=${filename##*/}
[[ -f $file ]] && mv "$file" "$d/${filename}_${name}.$extension"
done
e.g.
> ls /some/dir/to/do/this/on
video1.mpeg Video2.wmv
> ./abovescript
> ls /some/dir/to/do/this/on
video1_on.mpeg Video2_on.wmv
Explanation
In bash you can do this
${parameter#word} Removes shortest matching prefix
${parameter##word} Removes longest matching prefix
${parameter%word} Removes shortest matching suffix
${parameter%%word} Removes longest matching suffix
To remove everything anything (*) before and including the last period, I did below
extension=${file##*.}
To remove everything including and from the last period, I did below (think about shortest match here as going from right to left, e.g. * looks for any non-period text right to left, then when it finds a period it removes that whole section)
filename=${file%.*}
To remove everything up to and including the last /, I did below.
filename=${filename##*/}
Some other notes:
"$d/${filename}_${name}.$extension" Variables can have _ so I switched syntax for a couple of variables here for it to work
"$d"/* Expands to every file of any type (regular, dir, symlink etc...) directly in "$d"

Answer 2

What is wrong with
ls > myfile.txt
This will only list the file names (nothing else) and send them to myfile.txt
If you want to go the awk route, just do
ls -l | awk '{print $9}'
The default action of awk is to split fields on space - and this prints the 9th field for every line…
If you want to do other things with the file names, you can just extend your awk script. For example, an array with these file names could be created with
ls -l | awk '{a[NR]=$9}'
and you can use this array (called a) in further processing. If the processing requires something other than awk (from the comments I think it does), you would be better off with something that looks like
#!/bin/bash
for f in $1"/"*
do
if [ -d "$f" ] ; then
./listdir $f
else
echo $f
fi
done
Save this as listdir in your current directory, and you're good to go.
./listdir .
Will list the entire directory, recursing down (with full relative path appended) as needed.
If you want this to be available "from anywhere" (it is a pretty useful command after all) you would put it somewhere in your path (and do a "rehash" command so it will be "known"); then you don't need the ./ at the start of the command.

Answer 3

Good question! Glad you asked. Parsing ls's output is rarely the right thing to do. There are myriad ways to process a list of files. It depends what you want to do with them.
Here are some examples of things you can do. I've used touch as an example command. Replace that with whatever command or commands you want to do.
To run a command over multiple files, often you can simply pass all the files on the command-line.
touch /var/myapp/*
To loop over the files in the current directory:
for file in *; do
touch "$file"
done
To loop over files in another directory:
for file in /some/dir/*; do
touch "$file"
done
To rename files named *.txt to '*.bak', both here and in sub-directories:
find . -name '*.txt' -exec mv {} {}.bak \;
To delete JPEGs in Bob's home directory (damn you Bob and your wandering eyes):
find ~bob/ -name '*.jpg' -delete
To loop over files recursively and do complicated things to them:
find /dir/to/search -print0 | while read -d $'\0' file; do
echo "$file"
touch "$file"
if [[ -L $file ]]; then
# $file is a symlink, do something special
fi
done

Answer 4

ls -l | awk '{split($0,array," ")} {print array[9]}'
or
ls -l | awk '{print $9}'
but why not just ls?