This is my hierarchy:
aaaaaaaa
|_q
|_a.txt
|_w
|_l1
|_l2
l1 and l2 are symlinks to a.txt.
I run this code to find all symlinks to a.txt in the /aaaaaaaa:
find ~/aaaaaaaa/ -exec ls -a {} ';' | grep '/home/khodor/aaaaaaaa/q/a.txt'
And it obviously doesn't work, cause I must compare realpath of file with path of a.txt. In what way I should do this?
If you have GNU/BSD find just use -samefile primary.
$ find -L ~/aaaaaaaa/ -samefile ~/aaaaaaaa/q/a.txt
/home/oguz/aaaaaaaa/q/a.txt
/home/oguz/aaaaaaaa/w/l2
/home/oguz/aaaaaaaa/w/l1
referenceid=$(stat -Lc '%d-%i' /home/khodor/aaaaaaaa/q/a.txt)
find ~/aaaaaaaa/ -type l -print0 | while IFS= read -r -d '' filename
do
if [ "$(stat -Lc '%d-%i' "$filename")" = "$referenceid" ]
then
printf -- '%s\n' "$filename"
fi
done
This initially gets a unique ID for a base file, e.g. /home/khodor/aaaaaaaa/q/a.txt. The ID is computed from the device ID and the inode, using stat.
Then it parses your folder using file, limited to symbolic links (thanks to -type l), and for each filename it gets the device ID and inode using stat again, using its -L option that dereferences the link before fetching the ID.
For each device ID and inode that matches the reference ID, it prints the filename.
Using find with -type l to search for symbolic links:
find aaaaaaaa -type l -exec sh -c '
for i; do
printf "%s -> %s\n" "$i" "$(readlink -f "$i")"
done
' sh {} +
The shell script prints link path and canonical path of the symlink using readlink -f (but you could also use realpath instead):
Example output:
aaaaaaaa/w/l1 -> /home/khodor/aaaaaaaa/q/a.txt
aaaaaaaa/w/l2 -> /home/khodor/aaaaaaaa/q/a.txt
Use grep to filter the result using the absolute path, e.g.
find aaaaaaaa -type l -exec sh -c '
for i; do
printf "%s -> %s\n" "$i" "$(readlink -f "$i")"
done
' sh {} + | grep '/home/khodor/aaaaaaaa/q/a.txt$'
Note the $ at the end of the pattern to match the end of the line.
Related
I have multiple files in multiple directories and i have to rename these files from lowercase to uppercase; the file extension may vary and needs to be in lowercase (should be renamed too for files with extensions in uppercase).
NB: I have rename version from util-linux on CentOS Linux7.
i tried this :
find /mydir -depth | xargs -n 1 rename -v 's/(.*)\/([^\/]*)/$1\/\U$2/' {} \;
find /mydir -depth | xargs -n 1 rename -v 's/(.*)\/([^\/]*)/$2\/\L$2/' {} \;
but it's not working it changes nothing and i have no output.
Itried another solution :
for SRC in `find my_root_dir -depth`
do
DST=`dirname "${SRC}"`/`basename "${SRC}" | tr '[A-Z]' '[a-z]'`
if [ "${SRC}" != "${DST}" ]
then
[ ! -e "${DST}" ] && mv -T "${SRC}" "${DST}" || echo "${SRC} was not renamed"
fi
done
this one partially works but transforms the files extensions to uppercase too.
Any suggestions on how to keep/transform the extensions to lowercase ?
Thank you!
Possible solution with Perl rename:
find /mydir -depth -type f -exec rename -v 's/(.*\/)?([^.]*)/$1\U$2/' {} +
The commands in the question have several problems.
You seem to confuse the syntax of find's -exec action and xargs.
find /mydir -depth -type f -exec rename -v 'substitution_command' {} \;
find /mydir -depth -type f| xargs -n 1 rename -v 'substitution_command'
The xargs version has problems in case a file name contains a space.
If you replace \; with +, multiple file names are passed to one invocation of rename.
The substitution command is only supported by the Perl version of the rename command. You might have to install this version. See Get the Perl rename utility instead of the built-in rename
The substitution did not work in my test. I successfully used
rename -v 's/(.*\/)?([^.]*)/$1\U$2/' file ...
The first group (.*\/)? optionally matches a sequence of characters with a trailing /. This is used to copy the directory unchanged.
The second group ([^.]*) matches a sequence of characters except ..
This is the file name part before the first dot (if any) which will be converted to uppercase. In case the file name has more than one extension, all will remain unchanged, e.g.
Path/To/Foo.Bar.Baz -> Path/To/FOO.Bar.Baz
rename-independent solution (using find together with mv)
You can rename all files in a directory with a following command:
for i in $( ls | grep [A-Z] ); do mv -i $i `echo $i | tr 'A-Z' 'a-z'`; done
First part (for i in $( ls | grep [A-Z] );) looks for all uppercase characters and executes until all files are "scanned".
Second part (``) turns all uppercase characters into lowercase ones.
Perl-based rename dependent solution
rename -f 'y/A-Z/a-z/' *
This command changes uppercase characters to the lowercase ones. -f option allows overwriting of existing files, but it is not necessary.
suggesting a trick with awk that will generate all required mv commands:
awk '{f=$0;split($NF,a,".");$NF=tolower(a[1])"."toupper(a[2]);print "mv "f" "$0}' FS=/ OFS=/ <<< $(find . -type f)
Inspect the result, and run all mv commands together:
bash <<< $(awk '{f=$0;split($NF,a,".");$NF=tolower(a[1])"."toupper(a[2]);print "mv "f" "$0}' FS=/ OFS=/ <<< $(find . -type f))
awk script script.awk explanation
BEGIN { # preprocessing configuration
FS="/"; # set awk field separtor to /
OFS="/"; # set awk output field separtor to /
}
{ # for each line in input list
filePath = $0; # save the whole filePath in variable
# fileName is contained in last field $NF
# split fileName by "." to head: splitedFileNameArr[1] and tail: splitedFileNameArr[2]
split($NF,splitedFileNameArr,".");
# recreate fileName from lowercase(head) "." uppercase(tail)
$NF = tolower(splitedFileNameArr[1]) "." toupper(splitedFileNameArr[2]);
# generate a "mv" command from original filePath and regenerated fileName
print "mv "filePath" "$0;
}
Testing:
mkdir {a1,B2}/{A1,b2} -p; touch {a1,B2}/{A1,b2}/{A,b}{b,C}.{c,D}{d,C}
find . -type f
./a1/A1/Ab.cC
./a1/A1/Ab.cd
./a1/A1/Ab.DC
./a1/A1/Ab.Dd
./B2/b2/AC.DC
./B2/b2/AC.Dd
.....
./B2/b2/bC.cd
./B2/b2/bC.DC
./B2/b2/bC.Dd
awk -f script.awk <<< $(find . -type f)
.....
mv ./a1/b2/Ab.cd ./a1/b2/ab.CD
mv ./a1/b2/Ab.DC ./a1/b2/ab.DC
mv ./a1/b2/Ab.Dd ./a1/b2/ab.DD
mv ./B2/A1/bC.Dd ./B2/A1/bc.DD
.....
mv ./B2/b2/bC.DC ./B2/b2/bc.DC
mv ./B2/b2/bC.Dd ./B2/b2/bc.DD
bash <<< $(awk -f script.awk <<< $(find . -type f))
find . -type f
I am very new to bash commands. Could someone please help me on this issue?
I have to find all .txt files in the current directory and add a text in the beginning of those files. I have written below command-
find . -name *.txt | xargs sed -i '1iadd text here'
This command works fine for all the non-empty files. But it's not working for those files which are empty. I figured out that it's because the sed command fails to find 1st line in empty files and hence the command is not executed.
Is there any other way to prepend the text for empty files as well?
The ed Unix text editor can do that.
for f in *.txt; do
printf '%s\n' '0a' 'insert some text here' . w | ed -s "$f"
done
find . -type f -name '*.txt' -exec sh -c 'for f; do printf "%s\n" 0a "insert some text here" . w | ed -s "$f"; done' {} +
find . -type f -name '*.txt' -print0 | while IFS= read -rd '' file; do ed -s "$file" <<< $'0a\ninsert some text here\n.\nw\nq'; done
Can be done using an ed script.
cat script.ed
Out put
0a
insert some text here
.
w
q
Now a for loop
for f in *.txt; do ed -s "$f" < ./script.ed; done
Using find.
find . -type f -name '*.txt' -exec sh -c 'ed -s "$1" < ./script.ed' sh {} \;
A combination of both.
find . -type f -name '*.txt' -exec sh -c 'for f; do ed -s "$f" < ./script.ed; done' _ {} +
The first line is 1 and the action is i (which means insert) in your example which is also true with ed, meaning it will not work also with ed because the file is empty and contains no lines, but here I'm using the address as 0 and the action is a which means append, works.
The script.ed is an arbitrary name and the ./ in ./script.ed means the current working directory if your ed script is somewhere else add/change that to the absolute path of your ed script.
A word of caution, ed edit's the files in-place so make sure make a backup of what you're editing just in case...
It is hard to improve short oneliners. Sometimes it is good to prepare a typical, self-explaining piece of code, less compact, with some assumptions (here: temporary file), but working in 100% of the cases, e.g:
for file in `ls *.txt`; do awk 'BEGIN {print "add text here"}{print$0}' $file > tmp.tmp | mv tmp.tmp $file; done
or rather (edited):
for file in ./*.txt; do awk 'BEGIN {print "add text here"}{print$0}' "$file" > tmp.tmp | mv tmp.tmp "$file"; done
and then to try to mix the solutions.
EDIT:
If you have to use find, xargs and sed, and sed does not work properly with empty files, you can append an empty line to the file, insert the text, and then delete the appended line:
find . -type f -name '*.txt' | xargs -I "%" sh -c 'echo "" >> "%"; sed -i -e "1iadd text here" -e "$ d" "%"'
Prepend in-place with ex:
ex -s '+0s/^/add text here/' '+:wq' my_file
From stdin to stdout:
ex -s '+0s/^/add text here/' '+%print' '+:wq' /dev/stdin
Note that this only works with a single file, unlike sed.
So for your situation:
$ ls
empty not_empty
$ stat --format '%n: %s' *
empty: 0
not_empty: 6
$ cat empty
$ cat not_empty
a
b
c
$ find . -type f | xargs -I '{}' ex -s '+0s/^/add text here/' '+:wq' '{}'
$ cat empty
add text here
$ cat not_empty
add text herea
b
c
Note that -I is used to force xargs to execute ex once per file rather than try to aggregate arguments.
And for completeness, a filter from stdin to stdout example:
$ printf "%s\n" hello world | ex -s '+0s/^/add text here/' '+%print' '+:wq' /dev/stdin
add text herehello
world
$ cat /dev/null | ex -s '+0s/^/add text here/' '+%print' '+:wq' /dev/stdin
add text here
I am searching for the longest filename from my root directory to the very bottom.
I have coded a C program that will calculate the longest file name's length and its name.
However, I cannot get the shell to redirect the long list of file names to standard input for my program to receive it.
Here is what I did:
ls -Rp | grep -v / | grep -v "Permission denied" | ./home/user/findlongest
findlongest has been compiled and I check it on one of my IDE's to make sure it's working correctly. No run time errors were detected so far.
How do I get the list of file names into my 'findlongest' code by redirecting stdin?
Try this:
find / -type f -printf '%f\n' 2>/dev/null | /home/user/findlongest
The 2>/dev/null will discard all data written to stderr (which is where you're seeing the 'Permission denied' messages from).
Or the following to remove the dependancy on your application (from here):
find / -type f -printf '%f\n' 2>/dev/null | \
awk 'length > max_length {
max_length = length; longest_line = $0
}
END {
print length(longest_line) " " longest_line
}'
What about
find / -type f | /home/user/findlongest
It will list all files from root with absolute path and print only those files you have permissions to list.
Based on the command:
find -exec basename '{}' ';'
which prints recursively only the filenames of all the files starting from the directory you are: all the filenames.
This bash line will provide the file with longest name and the its number of characters:
Note that the loop involved will make the process slow.
for i in $(find -exec basename '{}' ';'); do printf $i" " && echo -e -n $i | wc -c; done | sort -nk 2 | tail -1
By parts:
Prints the name of the file followed by a single space:
printf $i" "
Prints the number of characters of such file:
echo -e -n $i | wc -c
Sorts the output by number of characters and takes the longest one (the very latest):
sort -nk 2 | tail -1
All this inside a for loop to handle line by line.
The for sentence can be also changed by:
for i in $(find -type f -printf '%f\n');
As stated in #Attie's answer
Is there some way to make this working?
pFile=find ${destpath} (( -iname "${mFile##*/}" )) -o (( -iname "${mFile##*/}" -a -name "*[],&<>*?|\":'()[]*" )) -exec printf '.' \;| wc -c
i need pFile return the number of file with the same filename, or if there aren't, return 0.
I have to do this, because if i only use:
pFile=find ${destpath} -iname "${mFile##*/}" -exec printf '.' \;| wc -c
It doesn't return if there are same filename with metacharacter.
Thanks
EDIT:
"${mFile##*/}" have as output file name in start folder without path.
echo "${mFile##*/}" -> goofy.mp3
Exmple
in start folder i have:
goofy.mp3 - mickey[1].avi - donald(2).mkv - scrooge.3gp
In destination folder i have:
goofy.mp3 - mickey[1].avi -donald(2).mkv -donald(1).mkv -donald(3).mkv -minnie.iso
i want this:
echo pFile -> 3
With:
pFile=find ${destpath} -iname "${mFile##*/}" -exec printf '.' \;| wc -c
echo pFile -> 2
With:
pFile=find ${destpath} -name "*[],&<>*?|\":'()[]*" -exec printf '.' \;| wc -c
echo pFile -> 4
With Same file name i mean:
/path1/mickey[1].avi = /path2/mickey[1].avi
I am not sure I understood your intended semantics of ${mFile##*/}, however looking at your start/destination folder example, I have created the following use case directory structure and the script below to solve your issue:
$ find root -type f | sort -t'/' -k3
root/dir2/donald(1).mkv
root/dir1/donald(2).mkv
root/dir2/donald(2).mkv
root/dir2/donald(3).mkv
root/dir1/goofy.mp3
root/dir2/goofy.mp3
root/dir1/mickey[1].avi
root/dir2/mickey[1].avi
root/dir2/minnie.iso
root/dir1/scrooge.3gp
Now, the following script (I've used gfind to indicated that you need GNU find for this to work, but if you're on Linux, just use find):
$ pFile=$(($(gfind root -type f -printf "%f\n" | wc -l) - $(gfind root -type f -printf "%f\n" | sort -u | wc -l)))
$ echo $pFile
3
I'm not sure this solves your issue, however it does print the number you expected in your provided example.
I'm trying to to wc -l an entire directory and then display the filename in an echo with the number of lines.
To add to my frustration, the directory has to come from a passed argument. So without looking stupid, can someone first tell me why a simple wc -l $1 doesn't give me the line count for the directory I type in the argument? I know i'm not understanding it completely.
On top of that I need validation too, if the argument given is not a directory or there is more than one argument.
wc works on files rather than directories so, if you want the word count on all files in the directory, you would start with:
wc -l $1/*
With various gyrations to get rid of the total, sort it and extract only the largest, you could end up with something like (split across multiple lines for readability but should be entered on a single line):
pax> wc -l $1/* 2>/dev/null
| grep -v ' total$'
| sort -n -k1
| tail -1l
2892 target_dir/big_honkin_file.txt
As to the validation, you can check the number of parameters passed to your script with something like:
if [[ $# -ne 1 ]] ; then
echo 'Whoa! Wrong parameteer count'
exit 1
fi
and you can check if it's a directory with:
if [[ ! -d $1 ]] ; then
echo 'Whoa!' "[$1]" 'is not a directory'
exit 1
fi
Is this what you want?
> find ./test1/ -type f|xargs wc -l
1 ./test1/firstSession_cnaiErrorFile.txt
77 ./test1/firstSession_cnaiReportFile.txt
14950 ./test1/exp.txt
1 ./test1/test1_cnaExitValue.txt
15029 total
so your directory which is the argument should go here:
find $your_complete_directory_path/ -type f|xargs wc -l
I'm trying to to wc -l an entire directory and then display the
filename in an echo with the number of lines.
You can do a find on the directory and use -exec option to trigger wc -l. Something like this:
$ find ~/Temp/perl/temp/ -exec wc -l '{}' \;
wc: /Volumes/Data/jaypalsingh/Temp/perl/temp/: read: Is a directory
11 /Volumes/Data/jaypalsingh/Temp/perl/temp//accessor1.plx
25 /Volumes/Data/jaypalsingh/Temp/perl/temp//autoincrement.pm
12 /Volumes/Data/jaypalsingh/Temp/perl/temp//bless1.plx
14 /Volumes/Data/jaypalsingh/Temp/perl/temp//bless2.plx
22 /Volumes/Data/jaypalsingh/Temp/perl/temp//classatr1.plx
27 /Volumes/Data/jaypalsingh/Temp/perl/temp//classatr2.plx
7 /Volumes/Data/jaypalsingh/Temp/perl/temp//employee1.pm
18 /Volumes/Data/jaypalsingh/Temp/perl/temp//employee2.pm
26 /Volumes/Data/jaypalsingh/Temp/perl/temp//employee3.pm
12 /Volumes/Data/jaypalsingh/Temp/perl/temp//ftp.plx
14 /Volumes/Data/jaypalsingh/Temp/perl/temp//inherit1.plx
16 /Volumes/Data/jaypalsingh/Temp/perl/temp//inherit2.plx
24 /Volumes/Data/jaypalsingh/Temp/perl/temp//inherit3.plx
33 /Volumes/Data/jaypalsingh/Temp/perl/temp//persisthash.pm
Nice question!
I saw the answers. Some are pretty good. The find ...|xrags is my most preferred. It could be simplified anyway using find ... -exec wc -l {} + syntax. But there is a problem. When the command line buffer is full a wc -l ... is called and every time a <number> total line is printer. As wc has no arg to disable this feature wc has to be reimplemented. To filter out these lines with grep is not nice:
So my complete answer is
#!/usr/bin/bash
[ $# -ne 1 ] && echo "Bad number of args">&2 && exit 1
[ ! -d "$1" ] && echo "Not dir">&2 && exit 1
find "$1" -type f -exec awk '{++n[FILENAME]}END{for(i in n) printf "%8d %s\n",n[i],i}' {} +
Or using less temporary space, but a little bit larger code in awk:
find "$1" -type f -exec awk 'function pr(){printf "%8d %s\n",n,f}FNR==1{f&&pr();n=0;f=FILENAME}{++n}END{pr()}' {} +
Misc
If it should not be called for subdirectories then add -maxdepth 1 before -type to find.
It is pretty fast. I was afraid that it would be much slower then the find ... wc + version, but for a directory containing 14770 files (in several subdirs) the wc version run 3.8 sec and awk version run 5.2 sec.
awk and wc consider the not \n ended lines differently. The last line ended with no \n is not counted by wc. I prefer to count it as awk does.
It does not print the empty files
To find the file with most lines in the current directory and its subdirectories, with zsh:
lines() REPLY=$(wc -l < "$REPLY")
wc -l -- **/*(D.nO+lined[1])
That defines a lines function which is going to be used as a glob sorting function that returns in $REPLY the number of lines of the file whose path is given in $REPLY.
Then we use zsh's recursive globbing **/* to find regular files (.), numerically (n) reverse sorted (O) with the lines function (+lines), and select the first one [1]. (D to include dotfiles and traverse dotdirs).
Doing it with standard utilities is a bit tricky if you don't want to make assumptions on what characters file names may contain (like newline, space...). With GNU tools as found on most Linux distributions, it's a bit easier as they can deal with NUL terminated lines:
find . -type f -exec sh -c '
for file do
size=$(wc -c < "$file") &&
printf "%s\0" "$size:$file"
done' sh {} + |
tr '\n\0' '\0\n' |
sort -rn |
head -n1 |
tr '\0' '\n'
Or with zsh or GNU bash syntax:
biggest= max=-1
find . -type f -print0 |
{
while IFS= read -rd '' file; do
size=$(wc -l < "$file") &&
((size > max)) &&
max=$size biggest=$file
done
[[ -n $biggest ]] && printf '%s\n' "$max: $biggest"
}
Here's one that works for me with the git bash (mingw32) under windows:
find . -type f -print0| xargs -0 wc -l
This will list the files and line counts in the current directory and sub dirs. You can also direct the output to a text file and import it into Excel if needed:
find . -type f -print0| xargs -0 wc -l > fileListingWithLineCount.txt