I have two folders which have same file names, but different contents. So, I am trying to generate a script to get the difference and to see what is being changed. I wrote a script below :
folder1="/opt/dir1"
folder2=`ls/opt/dir2`
find "$folder1/" /opt/dir2/ -printf '%P\n' | sort | uniq -d
for item in `ls $folder1`
do
if [[ $item == $folder2 ]]; then
diff -r $item $folder2 >> output.txt
fi
done
I believe this script has to work, but it is not giving any output on output folder.
So the desired output should be in one file . Ex:
cat output.txt
diff -r /opt/folder1/file1 /opt/folder2/file1
1387c1387
< ALL X'25' BY SPACE
---
> ALL X'0A' BY SPACE
diff -r /opt/folder1/file2 /opt/folder2/file2
2591c2591
< ALL X'25' BY SPACE
---
> ALL X'0A' BY SPACE
Any help is appreciated!
Ok. So twofold:
First get the files in one folder. Never use ls. Forget it exists. ls is for nice printing in our console. In scripts, use find.
Then for each file do some command. A simple while read loop.
So:
{
# make find print relative to `/opr/dir1` director
cd /opt/dir1 &&
# Use `%P` so that print without leading `./`
find . -mindepth 1 -type f -print "%P\n"
} |
while IFS= read -r file; do
diff /opt/dir1/"$file" /opt/dir2/"$file" >> output/"$file"
done
Notes:
always quote your variable
Why you shouldn't parse the output of ls(1)
Related
I need to implement a script (duplq.sh) that would rename all the text files existing in the current directory using the command line arguments. So if the command duplq.sh pic 0 3 was executed, it would do the following transformation:
pic0.txt will have to be renamed pic3.txt
pic1.txt to pic4.txt
pic2.txt to pic5.txt
pic3.txt to pic6.txt
etc…
So the first argument is always the name of a file the second and the third always a positive digit.
I also need to make sure that when I execute my script, the first renaming (pic0.txt to pic3.txt), does not erase the existing pic3.txt file in the current directory.
Here's what i did so far :
#!/bin/bash
name="$1"
i="$2"
j="$3"
for file in $name*
do
echo $file
find /var/log -name 'name[$i]' | sed -e 's/$i/$j/g'
i=$(($i+1))
j=$(($j+1))
done
But the find command does not seem to work. Do you have other solutions ?
A possible solution...
#!/bin/sh
NUMBERS=$(ls $1|sed -e 's/pic//g' -e 's/\.txt//g'|sort -n -r)
for N in $NUMBERS
do
NEW=$(($N + 3))
echo "pic$N.txt -> pic$NEW.txt"
mv "pic$N.txt" "pic$NEW.txt"
done
find possible files, sort the names by number, use sed to remove files with lower numbers than $1, reverse sort by number, and start at the top renaming from the highest number down to the lowest:
#!/bin/sh
find . -maxdepth 1 -type f -name "$1"'[0-9]*' |
sort -g | sed -n "/^$1$2."'/,$p' | sort -gr |
while read x ; do
p="${x%%[0-9]*.*}"; s="${x##*[0-9]}" ; i="${x%$s}" i="${i#$p}"
mv "$x" "$p$((i+$3))$s"
done
I am finding some files in a directory using this command:
find /Users/myname -type f
output is:
/Users/myname/test01/logs1/err.log
/Users/myname/test01/logs1/std
/Users/myname/test01/logs2/std
/Users/myname/test02/logs2/velocity.log
/Users/myname/test03/logs3/err.log
/Users/myname/test03/logs3/invalid-arg
I need to move this files to a different directory by appending the test directory name to the end of the files. Like below:
err.log-test01
std-test01
std-test01
velocity.log-test02
err.log-test03
invalid-arg-test03
I am trying with the cut command but not getting the desired output.
find /Users/myname -type f | cut -d'/' -f6,4
plus, I also need to move the files to a different directory. I guess a suitable way could be there using sed command, but I am not proficient with sed. How this can be achieved in an efficient way?
You can let find create the mv command, use sed to modify it and then have it run by the shell:
find /Users/myname -type f -printf "mv %p /other/dir/%f\n" |
sed 's,/\(test[0-9]*\)/\(.*\),/\1/\2-\1,' | sh
This assumes there are no spaces in any argument, otherwise liberally add ' or ". Also run it without the final | sh to see what it actually wants to do. If you need to anchor the test[0-9]* pattern better you can include part of the left or right string to match:
's,myname/\(test[0-9]*\)/\(.*\),myname/\1/\2-\1,'
You can move it from the dst to the dst_dir appending the directory, using awk, and the target name would be awk -F/ '{print $5 "-" $4}'. The full command could be as simple as:
for i in `find . -type f`
do mv $i /dst_dir/`echo $i| awk -F/ '{print $5 "-" $4}' `
done
There are a number of things going on that you may want to use a helper script with find to insure you can validate the existence of the directory to move the files to, etc.. A script might take the form of:
#!/bin/bash
[ -z $1 -o -z $2 ] && { # validate at least 2 arguments
printf "error: insufficient input\n"
exit 1
}
ffn="$1" # full file name provided by find
newdir="$2" # the target directory
# validate existence of 'newdir' or create/exit on failure
[ -d "$newdir" ] || mkdir -p "$newdir"
[ -d "$newdir" ] || { printf "error: uname to create '$newdir'\n"; exit 1; }
tmp="${ffn##*test}" # get the test## number
num="${tmp%%/*}"
fn="${ffn##*/}" # remove existing path from ffn
mv "$ffn" "${newdir}/${fn}-test${num}" # move to new location
exit 0
Save it in a location where it is accessible under a name like myscript and make it executable (e.g. chmod 0755 myscript) You may also choose to put it in a directory within your path. You can then call the script for every file returned by find with:
find /Users/myname -type f -exec ./path/to/myscript '{}' somedir \;
Where somedir is the target directory for the renamed file. Helper scripts generally provide the ability to do required validation that would otherwise not be done in one-liners.
I want to remove all the logs except the current log and the log before that.
These log files are created after 20 minutes.So the files names are like
abc_23_19_10_3341.log
abc_23_19_30_3342.log
abc_23_19_50_3241.log
abc_23_20_10_3421.log
where 23 is today's date(might include yesterday's date also)
19 is the hour(7 o clock),10,30,50,10 are the minutes.
In this case i want i want to keep abc_23_20_10_3421.log which is the current log(which is currently being writen) and abc_23_19_50_3241.log(the previous one)
and remove the rest.
I got it to work by creating a folder,putting the first files in that folder and removing the files and then deleting it.But that's too long...
I also tried this
files_nodelete=`ls -t | head -n 2 | tr '\n' '|'`
rm *.txt | egrep -v "$files_nodelete"
but it didnt work.But if i put ls instead of rm it works.
I am an amateur in linux.So please suggest a simple idea..or a logic..xargs rm i tried but it didnt work.
Also read about mtime,but seems abit complicated since I am new to linux
Working on a solaris system
Try the logadm tool in Solaris, it might be the simplest way to rotate logs. If you just want to get things done, it will do it.
http://docs.oracle.com/cd/E23823_01/html/816-5166/logadm-1m.html
If you want a solution similar (but working) to your try this:
ls abc*.log | sort | head -n-2 | xargs rm
ls abc*.log: list all files, matching the pattern abc*.log
sort: sorts this list lexicographical (by name) from oldes to to newest logfile
head -n-2: return all but the last two entry in the list (you can give -n a negativ count too)
xargs rm: compose the rm command with the entries from stdin
If there are two or less files in the directory, this command will return an error like
rm: missing operand
and will not delete any files.
It is usually not a good idea to use ls to point to files. Some files may cause havoc (files which have a [Newline] or a weird character in their name are the usual exemples ....).
Using shell globs : Here is an interresting way : we count the files newer than the one we are about to remove!
pattern='abc*.log'
for i in $pattern ; do
[ -f "$i" ] || break ;
#determine if this is the most recent file, in the current directory
# [I add -maxdepth 1 to limit the find to only that directory, no subdirs]
if [ $(find . -maxdepth 1 -name "$pattern" -type f -newer "$i" -print0 | tr -cd '\000' | tr '\000' '+' | wc -c) -gt 1 ];
then
#there are 2 files more recent than $i that match the pattern
#we can delete $i
echo rm "$i" # remove the echo only when you are 100% sure that you want to delete all those files !
else
echo "$i is one of the 2 most recent files matching '${pattern}', I keep it"
fi
done
I only use the globbing mechanism to feed filenames to "find", and just use the terminating "0" of the -printf0 to count the outputed filenames (thus I have no problems with any special characters in those filenames, I just need to know how many files were outputted)
tr -cd "\000" will keep only the \000, ie the terminating NUL character outputed by print0. Then I translate each \000 to a single + character, and I count them with the wc -c. If I see 0, "$i" was the most recent file. If I see 1, "$i" was the one just a bit older (so the find sees only the most recent one). And if I see more than 1, it means the 2 files (mathching the pattern) that we want to keep are newer than "$i", so we can delete "$i"
I'm sure someone will step in with a better one, but the idea could be reused, I guess...
Thanks guyz for all the answers.
I found my answer
files=`ls -t *.txt | head -n 2 | tr '\n' '|' | rev |cut -c 2- |rev`
rm `ls -t | egrep -v "$files"`
Thank you for the help
I'm writing a simple Bash script that needs to find out how many files are in a directory. Basically, if the directory contains no files, the script will exit. If it has any, it will do other things.
How can I find out if there are files in a directory using Bash?
Thanks!
List almost (no . or ..) all files and directories and count the lines: ls -A1 /mydir | wc -l
If you'd like only files I think you can use find instead: find /mydir -maxdepth 1 -type f | wc -l
shopt -s nullglob
cd /path/to/dir/
arr=( * )
echo "${#arr[#]}"
for i in "${!arr[#]}"; do echo "${arr[i]}"; done
The other answers have shown you ways to get the number of files. Here is how to use the number in your script.
(This is presented in the context of a function, with $1 being the directory name specified when you run it. To use it on the current directory, just omit that, or you can hardwire a directory name in that location.)
checkdir(){
numfiles=$(ls -A "$1" | wc -l)
if [ $numfiles -gt 0 ]
then
echo YES;
else
echo NO;
fi
}
Note: this will count directories as files. If you would like only files, then replace the ls...wc portion with this line:
ls -AF "$1" | grep -v "/$" | wc -l
I needed the same thing yesterday, and found this (it's near the bottom of the page):
# From http://www.etalabs.net/sh_tricks.html
is_empty () (
cd "$1"
set -- .[!.]* ; test -f "$1" && return 1
set -- ..?* ; test -f "$1" && return 1
set -- * ; test -f "$1" && return 1
return 0 )
It's POSIX-compatible, and uses three patterns to match any file name other than . and .. which are guaranteed to exist in an otherwise empty directory.
The first two lines match all files starting with a single . (of length at least 2) or one or more . (of length at least 3), which covers all hidden files that aren't . or ... The third pattern matches all non-hidden files.
Notice that the body of the function is a (...) expression, not {...}. This forces a subshell, in which it is safe to change the working directory to simplify the patterns.
Someone has very kindly help get me started on a mass rename script for renaming PDF files.
As you can see I need to add a bit of logical to stop the below happening - so something like add a unique number to a duplicate file name?
rename 's/^(.{5}).*(\..*)$/$1$2/' *
rename -n 's/^(.{5}).*(\..*)$/$1$2/' *
Annexes 123114345234525.pdf renamed as Annex.pdf
Annexes 123114432452352.pdf renamed as Annex.pdf
Hope this makes sense?
Thanks
for i in *
do
x='' # counter
j="${i:0:2}" # new name
e="${i##*.}" # ext
while [ -e "$j$x" ] # try to find other name
do
((x++)) # inc counter
done
mv "$i" "$j$x" # rename
done
before
$ ls
he.pdf hejjj.pdf hello.pdf wo.pdf workd.pdf world.pdf
after
$ ls
he.pdf he1.pdf he2.pdf wo.pdf wo1.pdf wo2.pdf
This should check whether there will be any duplicates:
rename -n [...] | grep -o ' renamed as .*' | sort | uniq -d
If you get any output of the form renamed as [...], then you have a collision.
Of course, this won't work in a couple corner cases - If your files contain newlines or the literal string renamed as, for example.
As noted in my answer on your previous question:
for f in *.pdf; do
tmp=`echo $f | sed -r 's/^(.{5}).*(\..*)$/$1$2/'`
mv -b ./"$f" ./"$tmp"
done
That will make backups of deleted or overwritten files. A better alternative would be this script:
#!/bin/bash
for f in $*; do
tar -rvf /tmp/backup.tar $f
tmp=`echo $f | sed -r 's/^(.{5}).*(\..*)$/$1$2/'`
i=1
while [ -e tmp ]; do
tmp=`echo $tmp | sed "s/\./-$i/"`
i+=1
done
mv -b ./"$f" ./"$tmp"
done
Run the script like this:
find . -exec thescript '{}' \;
The find command gives you lots of options for specifing which files to run on, works recursively, and passes all the filenames in to the script. The script backs all file up with tar (uncompressed) and then renames them.
This isn't the best script, since it isn't smart enough to avoid the manual loop and check for identical file names.