I wrote small shell script, to identify the PDF file associate pages in my website.
It’s take the pdf source list url one by one, as an input and finding recursive in website content.
Problem is when I run the script find result not appending to the output file,
But when I take the find command and run in terminal/putty manually can see the result.
Script:
#!/bin/bash
filename="PDF_Search_File.txt"
while read -r line
do
name="$line"
echo "*******pdf******** - $name\n" >>output_pdf_new.txt
find . -type f -exec grep -l "$name" '{}' \; >>output_pdf_new.txt
echo "*******pdf******** - $name\n" >>output_pdf_new.txt
done < "$filename"
source list url input file (PDF_Search_File.txt)
/static/pdf/pdf1.pdf
/static/pdf/pdf2.pdf
/static/pdf/pdf3.pdf
--------------------
out put result file (output_pdf_new.txt)
./Search_pdf.sh
*******pdf******** - /static/pdf/pdf1.pdf\n
*******pdf******** - /static/pdf/pdf1.pdf\n
./Search_pdf.sh
*******pdf******** - /static/pdf/pdf2.pdf\n
*******pdf******** - /static/pdf/pdf2.pdf\n
./Search_pdf.sh
*******pdf******** - /static/pdf/pdf3.pdf\n
*******pdf******** - /static/pdf/pdf3.pdf\n
------------------------------------------
terminal/putty can see the result for below, when manually run the find.
find . -type f -exec grep -l "/static/pdf/pdf1.pdf" '{}' \;
./en/toyes/zzz/index.xhtml
./en/toyes/kkk/index.xhtml
--------------
but having issue with script , only out put the echo result as above output result .
Update
when i execute the script with bash -x , it's giving below result
[user#server1 generated_content]# bash -x Search_pdf.sh
+ filename=PDF_Search_File.txt
+ read -r line
+ name=$'/static/pdf/pdf1.pdf\r'
\n'cho '*******pdf******** - /static/pdf/pdf1.pdf
+ find . -type f -exec grep -l $'/static/pdf/pdf1.pdf\r' '{}' ';'
\n'cho '*******pdf******** - /static/pdf/pdf1.pdf
+ read -r line
+ name=$'/static/pdf/pdf2.pdf\r'
\n'cho '*******pdf******** - /static/pdf/pdf2.pdf
+ find . -type f -exec grep -l $'/static/pdf/pdf2.pdf\r' '{}' ';'
is something wrong here
+ find . -type f -exec grep -l $'/static/pdf/pdf2.pdf\r' '{}' ';'
find command should be like below , but it's taking as above when executing
find . -type f -exec grep -l "/static/pdf/pdf1.pdf" '{}' \;
Have you tried -e option in echo to enable interpretation of backslash escapes?
Also why don't you simply do find | grep?
find ./ -type f | grep "$name" >> output_pdf_new.txt
Try following (./ instead of .) in find
find ./ -type f -exec grep -l "$name" '{}' \; >>output_pdf_new.txt
grep -rl for the file inside of your for loop:
cd /www/webroot/
grep -rl "${name}" * | while read file_path; do
# I need to do something with each file
echo $file_path
done
OR I just need to run the output to file
cd /www/webroot/
grep -rl "${name}" * >> output_pdf_new.txt
Related
Using terminal I need to write a command with mv/tr/find to:
Find files with names that start with a, o or e: [a,o,e]*.
Change those filenames first letters from lower to capital using tr or mv.
Everything has to be in one command not script. I've tried something like this:
find -name "[a,o,e]*" -exec {} mv [a,o,e]* [A,O,E]* \;
but it doesn't work.
This is a sample with some of my files starting with letters p and g:
$ find . -name '[pg]*.py'
./pycurl-example.py
./pymusic.py
./gtkmenu.py
You could use something like bellow, but unfortunatelly will not stop on the first occurence - it will translate all letters found:
$ find . -name '[pg]*.py' |tr '[pg]' '[PG]'
./Pycurl-examPle.Py
./Pymusic.py
./Gtkmenu.Py
Sinc tr has not an option to stop at first occurence like sed, you can make a trick like this:
$ find . -name '[pg]*.py' -exec bash -c 'echo -n "${0:2:1}"|tr 'pg' 'PG' && echo "${0:3}"' {} \;
Pycurl-example.py
Pymusic.py
Gtkmenu.py
Or even simplier without tr but with pure bash (idea from #Nominal Animal comment)
$ find . -name '[pg]*.py' -exec bash -c 'fn="${0:2}";echo "${fn^}"' {} \;
Pycurl-example.py
Pymusic.py
Gtkmenu.py
Or if your purpose is to do a real rename:
$ find . -name '[p]*c.py' -exec bash -c 'fn="${0:2}";mv -v "${fn}" "${fn^}"' {} \;
renamed 'pymusic.py' -> 'Pymusic.py'
And a more sophisticated rename that will work with any path:
$ find . -name 'g*.sh' -exec bash -c 'pn="${0%/*}";fn="${0##*/}";mv -v $0 ${pn}/${fn^}' {} \;
renamed './kalitools/Bluelog/scripts/gen_oui.sh' -> './kalitools/Bluelog/scripts/Gen_oui.sh'
renamed './greptest.sh' -> './Greptest.sh'
find . -name '[aoe]*' -exec sh -c 'echo mv "$1" "$(echo "$1"|sed -E "s,(.*/)(.),\1\u\2,")"' _ {} ';'
Remove the first "echo" if it looks like it's going to do what you want.
In my tmp dir with all kinds of crap in it, some of the output is:
mv ./a b c ./A b c
mv ./a.awk ./A.awk
mv ./a.prop ./A.prop
mv ./a1234 ./A1234
mv ./event.logger ./Event.logger
mv ./example.txt ./Example.txt
mv ./link/a ./link/A
mv ./link/a~ ./link/A~
mv ./old.csv ./Old.csv
mv ./rename/one-dash ./rename/One-dash
Try this:
for f in `find . -name "*" -type f|sed "s/^.\///"`; do f1stc=$(echo ${f::1}|tr [a-z] [A-Z]); fallbut1stc=$(echo ${f:1}); nf="${f1stc}${fallbut1stc}"; mv $f ${nf} 2>/dev/null || true; done
I am looking to combine the output of the Linux find and head commands (to derive a list of filenames) with output of another Linux/bash command and save the result in a file such that each filename from the "find" occurs with the other command output on a separate line.
So for example,
- if a dir testdir contains files a.txt, b.txt and c.txt,
- and the output of the other command is some number say 10, the desired output I'm looking for is
10 a.txt
10 b.txt
10 c.txt
On searching here, I saw folks recommending paste for doing similar merging but I couldn't figure out how to do it in this scenario as paste seems to be expecting files . I tried
paste $(find testdir -maxdepth 1 -type f -name "*.text" | head -2) $(echo "10") > output.txt
paste: 10: No such file or directory
Would appreciate any pointers as to what I'm doing wrong. Any other ways of achieving the same thing are also welcome.
Note that if I wanted to make everything appear on the same line, I could use xargs and that does the job.
$find testdir -maxdepth 1 -type f -name "*.text" | head -2 |xargs echo "10" > output.txt
$cat output.txt
10 a.txt b.txt
But my requirement is to merge the two command outputs as shown earlier.
Thanks in advance for any help!
find can handle both the -exec and -print directives, you just need to merge the output:
$ find . -maxdepth 1 -type f -name \*.txt -exec echo hello \; -print | paste - -
hello ./b.txt
hello ./a.txt
hello ./all.txt
Assuming your "command" requires the filename (here's a very contrived example):
$ find . -maxdepth 1 -type f -name \*.txt -exec sh -c 'wc -l <"$1"' _ {} \; -print | paste - -
4 ./b.txt
4 ./a.txt
7 ./all.txt
Of course, that's executing the command for each file. To restrict myself to your question:
cmd_out=$(echo 10)
for file in *.txt; do
echo "$cmd_out $file"
done
Try this,
$find testdir -maxdepth 1 -type f -name "*.text" | head -2 |tr ' ' '\n'|sed -i 's/^/10/' > output.txt
You can make xargs operate on one line at a time using -L1:
find testdir -maxdepth 1 -type f -name "*.text" | xargs -L1 echo "10" > output.txt
I'm having an adventure in the world of bash scripting with find today.
Say I'm looking to copy any png file in any subdirectory of /home/mine/Pictures to /home/mine/pngcoppies and rename it "copy[basename]"using find and -exec. This would require me to use both the full path name and the basename in the same exec command. My problem is that I don't know how to get the basename. (See below)
find /home/mine -iname "*.png" -exec cp {} /home/mine/pngcoppies/copy{what_do_I_enter_here?} \;
Note: The above isn't actually what I'm doing, but it's a fundamental example of the issue, so a workaround using some other method to achieve the same ends wouldn't really apply here. The question is fundamentally about find -exec and its use of basenames.
Thanks in advance!
To see what is going on when you execute the find, just type set -xv
-x : Prints commands and their arguments as they are executed.
-v : Prints shell input lines as they are read.
Here is what I have :
find . -name "*.xml" -exec echo {} \;
Gives the output:
./log.xml
./svnLog.xml
And when I try :
set -xv
find . -name "*.xml" -exec echo {} \;
I get :
find . -name "*.xml" -exec echo {} \;
+ find . -name '*.xml' -exec echo '{}' ';'
./log.xml
./svnLog.xml
And then find execute echo passing the found filename instead of the litteral : '{}'
but when you add something to the {} like below :
find . -name "*.xml" -exec echo something{} \;
+ find . -name '*.xml' -exec echo 'something{}' ';'
something{}
something{}
Here the echo is executed twice for the 2 xml files that I have and since there is no more '{}' is the parameter list of the exec, it is not going to be replaced. so we got the echo 'something{}' for each file found.
To deal with this, you can think about executing echo passing to it the filename as parameter like for example :
sh -xvc 'echo sothing/$0' filename
We already know what is -x and -v. -c is to get the command from the string after it (man sh)
so the result is :
sh -xvc 'echo somthing/$0' filename
+ sh -xvc 'echo somthing/$0' filename
echo somthing/$0
+ echo somthing/filename
sothing/filename
I used 'echo somthing/$0' between ' ' so that $0 don't get expanded by the current shell. try it with " " and you will see the expantion of $0 ;)
So to get back to your 'problem', the find should be formatted as below:
find . -name "*.xml" -exec sh -xvc 'echo sothing/$0' {} \;
And we will get :
find . -name "*.xml" -exec sh -xvc 'echo sothing/$0' {} \;
+ find . -name '*.xml' -exec sh -xvc 'echo sothing/$0' '{}' ';'
echo sothing/$0
+ echo sothing/./log.xml
sothing/./log.xml
echo sothing/$0
+ echo sothing/./svnLog.xml
sothing/./svnLog.xml
As we can see know, the find is going to execute the shell cammand echo sothing/$0 passing to it '{}' (replaced by the filename found by find) so we get the desired echo sothing/./log.xml
set +xv to remove the verbose mode
and we can get :
find . -name "*.xml" -exec sh -c 'echo "cp $0 someWhereElse/$0"' {} \;
cp ./log.xml someWhereElse/./log.xml
cp ./svnLog.xml someWhereElse/./svnLog.xml
so in your case , you have just to execute the copy in a sub shell (add sh or bash or you favorit shell after the exec) and let find pass the filename as parapeter to the it ;)
find /home/mine -iname "*.png" -exec sh -c 'cp $0 /home/mine/pngcoppies/copy/$0' {} \;
Hope this can help, and execuse me for my English.
From man find:
"The -execdir primary is identical to the -exec primary with the exception that utility will be executed from the directory that holds the current file. The filename substituted for the string ``{}'' is not qualified."
find /home/mine -iname "*.png" -execdir cp {} /home/mine/pngcoppies/copy{} \;
try something like this :
find /home/mine -iname "*.png" -printf "%P\n " | xargs -I % -n1 cp % /home/mine/pngcoppies/copy%
To get basename you use
basename $your_full_path
To get that path before the basename
dirname $your_full_path
You need to combine the first two answers
I was tailing everything in one directory into another directory with
find . -type f -exec sh -c 'tail -n 1000 $0 >../tail1000/$0.tail' {} \;
The first Answer gives
cp ./log.xml someWhereElse/./log.xml
or for my tail command
find . -type f -exec sh -c 'tail -n 1000 $0 >../tail1000/$0.tail' {} \;
tail -n 1000 ./filenane > ../tail1000/./filenane.tail
which surprisingly works but does not look like a nice path and I expect there are cases where a path/./morepath does something unexpected with some command.
Combining the answers gives
find . -type f -execdir sh -c 'tail -n 1000 $0 >../tail1000/$0.tail' {} \;
which executes
tail -n 1000 filenane > ../tail1000/filenane.tail
And looks much more likely to give the expected results.
I have lots of .ini files which configures certain properties for various projects.
I want to filter for PROJECT_A in the ini file first and if that matches, then i want to filter the second pattern CONFIG_B. Since CONFIG_B is a property of PROJECT_A...X, i want to grep only the files which contains the PROJECT_A settings and CONFIG_B is also present. I know it is bit challenging, but if i can narrow down ini files with both PROJECT_A and CONFIG_A is present, i can manually inspect them to minimum list. I have 1000 files like this :-(
Typical Config is like this
[F-Project:PROJECT_A]
stream-window-start=0
stream-window-end=0
network-feed=LIVE:
test-config=pdl tf_dms_hiab
Expected out:-
file1.ini
proj:PROJECT_A
cfg1:CONFIG_A
cfg1:CONFIG_B
cfg1:CONFIG_C
proj:PROJECT_B
cfg1:CONFIG_A
cfg1:CONFIG_C
file2.ini
proj:PROJECT_X
cfg1:CONFIG_A
cfg1:CONFIG_B
cfg1:CONFIG_C
proj:PROJECT_Y
cfg1:CONFIG_B
cfg1:CONFIG_C
file3.ini
proj:PROJECT_A
cfg1:CONFIG_B
cfg1:CONFIG_C
proj:PROJECT_B
cfg1:CONFIG_A
Results : file1.ini, file3.ini
find . -name *.ini -exec grep -w PROJECT_A {} \; -print | grep ini -exec grep CONFIG_A {} \;
[proj:PROJECT_A]
./PLATFORM/build/integration/suites/System_Maintenance_Suite/ini/Test_0621_1.ini
Since i get the output like above, im filtering only the lines containing .ini
find . -name *.ini -exec grep -w PROJECT_A {} \; -print | grep ini
./PLATFORM/build/integration/suites/System_Maintenance_Suite/ini/Test_0722_1.ini
./PLATFORM/build/integration/suites/System_Maintenance_Suite/ini/Test_0579_15.ini
./PLATFORM/build/integration/suites/System_Maintenance_Suite/ini/Test_0460_1.ini
how can i grep one line at a time for pattern CONFIG_A now
I understand i can write this to a file and read a line at a time, but i want a efficient way to do this.
Please help with your suggestions.
Saying:
find . -name *.ini -exec sh -c "grep -q PROJECT_A {} && grep -q CONFIG_A {} && echo {}" \;
would list files that contain both PROJECT_A and CONFIG_A.
Using the -q option for grep would evaluate to true only if the specified pattern existed in the file.
If you are looking for files where CONFIG_B occurs only in the stanza for proj:PROJECT_A, something like this?
find . -type f -name '*.ini' -exec awk '
/^proj:/ { prj=$1; next }
/CONFIG_B/ && prj="proj:PROJECT_A" {
print FILENAME; exit 0 }' {} \;
... or with the "real" values from the comment below,
find . -type f -name '*.ini' -exec awk '
/^F-Project:/ { prj=$1; next }
/LIVE:/ && prj="F-Project:PROJECT_A" {
print FILENAME; exit 0 }' {} \;
find . -name *.ini -exec sh -c "grep -q PROJECT_A {} && grep -q CONFIG_A {} && echo {}" \;
How it works ?
find . -name *.ini <= filters the .ini files
-exec <= executes the following command using the output from find one at a time
sh -c <= accepts string input for the shell, we need this for executing multiple commands which follws that
grep -q PROJECT_A {} <= quietly grep for PROJECT_A
grep -q PROJECT_A {} && grep -q CONFIG_A {} && echo {} <= prints the filename if both the strings are matches, simple logical and on three commands.
Hope it helps !!!
What about this neat awk solution:
awk -vPR="PROJECT_A" -vCF="CONFIG_A" 'BEGIN{R="(" CF "|" PR ")"}
{if($0 ~ R)d[FILENAME]+=1}
END{for(i in d)if(d[i]>=2)print i}' file*
file3.ini
file1.ini
Is there some way to make this working?
pFile=find ${destpath} (( -iname "${mFile##*/}" )) -o (( -iname "${mFile##*/}" -a -name "*[],&<>*?|\":'()[]*" )) -exec printf '.' \;| wc -c
i need pFile return the number of file with the same filename, or if there aren't, return 0.
I have to do this, because if i only use:
pFile=find ${destpath} -iname "${mFile##*/}" -exec printf '.' \;| wc -c
It doesn't return if there are same filename with metacharacter.
Thanks
EDIT:
"${mFile##*/}" have as output file name in start folder without path.
echo "${mFile##*/}" -> goofy.mp3
Exmple
in start folder i have:
goofy.mp3 - mickey[1].avi - donald(2).mkv - scrooge.3gp
In destination folder i have:
goofy.mp3 - mickey[1].avi -donald(2).mkv -donald(1).mkv -donald(3).mkv -minnie.iso
i want this:
echo pFile -> 3
With:
pFile=find ${destpath} -iname "${mFile##*/}" -exec printf '.' \;| wc -c
echo pFile -> 2
With:
pFile=find ${destpath} -name "*[],&<>*?|\":'()[]*" -exec printf '.' \;| wc -c
echo pFile -> 4
With Same file name i mean:
/path1/mickey[1].avi = /path2/mickey[1].avi
I am not sure I understood your intended semantics of ${mFile##*/}, however looking at your start/destination folder example, I have created the following use case directory structure and the script below to solve your issue:
$ find root -type f | sort -t'/' -k3
root/dir2/donald(1).mkv
root/dir1/donald(2).mkv
root/dir2/donald(2).mkv
root/dir2/donald(3).mkv
root/dir1/goofy.mp3
root/dir2/goofy.mp3
root/dir1/mickey[1].avi
root/dir2/mickey[1].avi
root/dir2/minnie.iso
root/dir1/scrooge.3gp
Now, the following script (I've used gfind to indicated that you need GNU find for this to work, but if you're on Linux, just use find):
$ pFile=$(($(gfind root -type f -printf "%f\n" | wc -l) - $(gfind root -type f -printf "%f\n" | sort -u | wc -l)))
$ echo $pFile
3
I'm not sure this solves your issue, however it does print the number you expected in your provided example.