I have a directory which has many folder and each folder contains a list of XML files. I am writing a bash script that traverses through the files and renames the extension of the file to "manual" if the size of the file is greater than 65Mb. This is my first writing a shell script and I was able to write the code for traversing the files but I am having difficulty in the renaming part.
for file in $dir
do
size=$(stat -c%s "$file")
if test "$size" -gt "68157440"; then
echo "Before Renaming...."
echo $file
echo "After renaming"
mv *.manual `basename $file`.xml
echo $file
else
echo $file >> outlog.log
fi
done
an example of $file is,
/apps/jAS/dev/products-app/BConverter/data/supplier-data/TF/output/Fiber Optics and Fiber Management Solutions/Fiber Optic Cable Assemblies.xml
mv *.manual `basename $file`.xml
If you want to change the extension of $file from xml to manual, do instead
mv "$file" "${file%.xml}".manual
What exactly is the difficulty you're having?
If it's white space in file names, try
mv *.manual `basename "$file"`.xml
Note that your script will not work if *.manual expands to more than one file name.
No need for a script on this, combination of find and xargs should do the trick:
find . -size +65M | xargs -IQ mv Q Q.manual
The little-used -I option to Xargs:
runs each input as a separate command, and
lets you replace the filename, so you can use it multiple time, ideal for a mv
Related
I am trying to prepare a bash script for sync 2 directories. But I am not able to file name return from diff. everytime it converts to array.
Here is my code :
#!/bin/bash
DIRS1=`diff -r /opt/lampp/htdocs/scripts/dev/ /opt/lampp/htdocs/scripts/www/ `
for DIR in $DIRS1
do
echo $DIR
done
And if I run this script I get out put something like this :
Only
in
/opt/lampp/htdocs/scripts/www/:
file1
diff
-r
"/opt/lampp/htdocs/scripts/dev/File
1.txt"
"/opt/lampp/htdocs/scripts/www/File
1.txt"
0a1
>
sa
das
Only
in
/opt/lampp/htdocs/scripts/www/:
File
1.txt~
Only
in
/opt/lampp/htdocs/scripts/www/:
file
2
-
second
Actually I just want to file name where I find the diffrence so I can take perticular action either copy/delete.
Thanks
I don't think diff produces output which can be parsed easily for your purposes. It's possible to solve your problem by iterating over the files in the two directories and running diff on them, using the return value from diff instead (and throwing the diff output away).
The code to do this is a bit long, but here it is:
DIR1=./one # set as required
DIR2=./two # set as required
# Process any files in $DIR1 only, or in both $DIR1 and $DIR2
find $DIR1 -type f -print0 | while read -d $'\0' -r file1; do
relative_path=${file1#${DIR1}/};
file2="$DIR2/$relative_path"
if [[ ! -f "$file2" ]]; then
echo "'$relative_path' in '$DIR1' only"
# Do more stuff here
elif diff -q "$file1" "$file2" >/dev/null; then
echo "'$relative_path' same in '$DIR1' and '$DIR2'"
# Do more stuff here
else
echo "'$relative_path' different between '$DIR1' and '$DIR2'"
# Do more stuff here
fi
done
# Process files in $DIR2 only
find $DIR2 -type f -print0 | while read -d $'\0' -r file2; do
relative_path=${file2#${DIR2}/};
file1="$DIR1/$relative_path"
if [[ ! -f "$file2" ]]; then
echo "'$relative_path' in '$DIR2 only'"
# Do more stuff here
fi
done
This code leverages some tricks to safely handle files which contain spaces, which would be very difficult to get working by parsing diff output. You can find more details on that topic here.
Of course this doesn't do anything regarding files which have the same contents but different names or are located in different directories.
I tested by populating two test directories as follows:
echo "dir one only" > "$DIR1/dir one only.txt"
echo "dir two only" > "$DIR2/dir two only.txt"
echo "in both, same" > $DIR1/"in both, same.txt"
echo "in both, same" > $DIR2/"in both, same.txt"
echo "in both, and different" > $DIR1/"in both, different.txt"
echo "in both, but different" > $DIR2/"in both, different.txt"
My output was:
'dir one only.txt' in './one' only
'in both, different.txt' different between './one' and './two'
'in both, same.txt' same in './one' and './two'
Use -q flag and avoid the for loop:
diff -rq /opt/lampp/htdocs/scripts/dev/ /opt/lampp/htdocs/scripts/www/
If you only want the files that differs:
diff -rq /opt/lampp/htdocs/scripts/dev/ /opt/lampp/htdocs/scripts/www/ |grep -Po '(?<=Files )\w+'|while read file; do
echo $file
done
-q --brief
Output only whether files differ.
But defitnitely you should check rsync: http://linux.die.net/man/1/rsync
I'm writing a script that takes an argument which is a directory .
i want to be able to construct list/array with all the files that have a certain extension in that directory and cut their extension .
For example if i have directory containing :
aaa.xx
bbb.yy
ccc.xx
and im searching for *.xx .
my list/array would be : aaa ccc.
I'm trying to use the code in this thread example the accepted answer .
set tests_list=[]
for f in $1/*.bpt
do
echo $f
if [[ ! -f "$f" ]]
then
continue
fi
set tmp=echo $f | cut -d"." -f1
#echo $tmp
tests_list+=$tmp
done
echo ${tests_list[#]}
if i run this script i get that the loop only executes once with $f is tests_list=[]/*.bpt which is weird since $f should be a file name in that directory , and echo empty string.
i validated that i'm in the correct directory and that the argument directory have files with .bpt extensions .
This should work for you:
for file in *.xx ; do echo "${file%.*}" ; done
To expand this to a script that takes an argument as a directory:
#!/bin/bash
dir="$1"
ext='xx'
for file in "$dir"/*."$ext"
do
echo "${file%.*}"
done
edit: switched ls with for - thanks #tripleee for the correction.
filear=($(find path/ -name "*\.xx"))
filears=()
for f in ${filear[#]}; do filears[${#filears[#]}]=${f%\.*}; done
I apologize if this is a trivial question. I am learning how to use linux bash and this little task is giving me a headache...
So I need to write a script, let's call it count.sh. I want that: for each file in the working directory, prints the filename, the number of lines, and the number of words to the console:
test.txt 100 1023
someOtherfiles 10 233
So far, I know that the following gives me all the files names in the directory. And thanks for all who helped me, I get this working version:
for f in *; do
echo -n "$f"
cat "$f" | wc -wl
done
I would really appreciate your help! Thanks ahead!
P.s. If you know great resources (links for tutorials) for learning about script and you are willing to share it with me. I think I really need to know these basics. Thanks again!
If you must have the file name as the first field in your output, try this:
for f in *; do
if [ -f "$f" ]; then
echo -n "$f"
cat "$f" | wc -wl
fi
done
for f in *; do
if [[ -f $f ]]; then
echo "$f $(wc -wl < "$f")"
fi
done
[[ -f $f ]] processes only files (excludes subdirectories) and also handles the case where the directory is empty (in which case * is (by default) left unexpanded, i.e. assigned to $f as is).
echo "$f $(wc -wl < "$f")" uses command substitution ($( ... )) to directly include the output from the enclosed command in the output string passed to echo.
Note that the reason that < is used to direct the content of file $f to wc via stdin is that wc would otherwise append the name of the input file to its output (thanks, #R Sahu).
I would like to write a linux script that will move or copy all files with the same filename (but different extensions) to a new filename for all those files, while maintaining their different extensions. In other words:
if I start with a directory listing:
file1.txt, file1.jpg, file1.doc, file12.txt, file12.jpg, file12.doc
I would like to write a script to change all the filenames without changing the extensions. For the same example, choosing file2 as the new filename the result would be:
file2.txt, file2.jpg and file2.doc, file12.txt, file12.jpg, file12.doc
So the files whose filename do not match the current criteria will not be changed.
Best wishes,
George
Note: If there's file1.doc in variable i, expression ${i##*.} extracts extension i.e. doc in this case.
One line solution:
for i in file1.*; do mv "$i" "file2.${i##*.}"; done
Script:
#!/bin/sh
# first argument - basename of files to be moved
# second arguments - basename of destination files
if [ $# -ne 2 ]; then
echo "Two arguments required."
exit;
fi
for i in $1.*; do
if [ -e "$i" ]; then
mv "$i" "$2.${i##*.}"
echo "$i to $2.${i##*.}";
fi
done
The util-linux-ng package (most of linux flavours have it installed by default) has the command 'rename'. See man rename for use instructions. Using it your task can be done simply as that rename file1 file2 file1.*
To handle input files whose basenames contain special characters, I would modify plesiv's script to the following:
if [ $# -ne 2 ]; then
echo "Two arguments required."
exit;
fi
for i in "$1".*; do
if [ -e "$i" ]; then
mv "$i" "$2.${i##*.}"
echo "$i to $2.${i##*.}";
fi
done
Note the extra quotes around $1.
I originally had a set of images of the form image_001.jpg, image_002.jpg, ...
I went through them and removed several. Now I'd like to rename the leftover files back to image_001.jpg, image_002.jpg, ...
Is there a Linux command that will do this neatly? I'm familiar with rename but can't see anything to order file names like this. I'm thinking that since ls *.jpg lists the files in order (with gaps), the solution would be to pass the output of that into a bash loop or something?
If I understand right, you have e.g. image_001.jpg, image_003.jpg, image_005.jpg, and you want to rename to image_001.jpg, image_002.jpg, image_003.jpg.
EDIT: This is modified to put the temp file in the current directory. As Stephan202 noted, this can make a significant difference if temp is on a different filesystem. To avoid hitting the temp file in the loop, it now goes through image*
i=1; temp=$(mktemp -p .); for file in image*
do
mv "$file" $temp;
mv $temp $(printf "image_%0.3d.jpg" $i)
i=$((i + 1))
done
A simple loop (test with echo, execute with mv):
I=1
for F in *; do
echo "$F" `printf image_%03d.jpg $I`
#mv "$F" `printf image_%03d.jpg $I` 2>/dev/null || true
I=$((I + 1))
done
(I added 2>/dev/null || true to suppress warnings about identical source and target files. If this is not to your liking, go with Matthew Flaschen's answer.)
Some good answers here already; but some rely on hiding errors which is not a good idea (that assumes mv will only error because of a condition that is expected - what about all the other reaons mv might error?).
Moreover, it can be done a little shorter and should be better quoted:
for file in *; do
printf -vsequenceImage 'image_%03d.jpg' "$((++i))"
[[ -e $sequenceImage ]] || \
mv "$file" "$sequenceImage"
done
Also note that you shouldn't capitalize your variables in bash scripts.
Try the following script:
numerate.sh
This code snipped should do the job:
./numerate.sh -d <your image folder> -b <start number> -L 3 -p image_ -s .jpg -o numerically -r
This does the reverse of what you are asking (taking files of the form *.jpg.001 and converting them to *.001.jpg), but can easily be modified for your purpose:
for file in *
do
if [[ "$file" =~ "(.*)\.([[:alpha:]]+)\.([[:digit:]]{3,})$" ]]
then
mv "${BASH_REMATCH[0]}" "${BASH_REMATCH[1]}.${BASH_REMATCH[3]}.${BASH_REMATCH[2]}"
fi
done
I was going to suggest something like the above using a for loop, an iterator, cut -f1 -d "_", then mv i i.iterator. It looks like it's already covered other ways, though.