I wanted to uncompress a tar.gz, with appending a suffix to each of the file name. So, for example abc.tar.gz contains files 'first' and 'second', so, after extracting, if I want to append suffix '.append'the file name of each files should be 'first.append' and 'second.append'. Is there a command or way to this?
Note: Files with name 'first' and 'second' are already there, and I wanted to decompress without affecting already available files.
One thing I can think of is uncompress in a temp dir and then, copy all files one by one. But, I wanted to do it in a one shot, if possible, so that, it will save time.
Thanks in advance.
try this:
#!/bin/bash
COMPRESS_F=abc.tar.gz
if [ $# -ne 0 ]; then
COMPRESS_F=$1
fi
for i in `tar -tf "$COMPRESS_F"`; do
if [ -f $i ]; then
echo "mv ${i} ${i}.orig"
mv ${i} ${i}.orig
fi
done
for i in `tar -xvzf "$COMPRESS_F"`; do
echo "mv ${i} ${i}.append"
mv ${i} ${i}.append
if [ -f ${i}.orig ]; then
echo "mv ${i}.orig ${i};"
mv ${i}.orig ${i};
fi
done
Got the solution, we can do this using '--transform' option of tar command, as shown below:
tar -(x/c)vf archive.tar --transform 's,/abc$,/abc.append,'
this will convert all abc files to abc.append.
Related
I have a bunch of .txt files in a directory.
I m looking for a command to copy all .txt files and save it with <filename>_2.txt.
Eg: abc.txt -> abc_2.txt (After copy)
Thanks in tons in advance
EDIT: As per OP's extension request adding following code now.
for file in *.txt
do
if [[ ! -f "${file%.*}_MED_2.txt" ]]
then
cp "$file" "${file%.*}_MED_2.txt"
fi
done
Try following.
for file in *.txt
do
echo "cp $file ${file%.*}_2"
done
Above will print cp commands, if you are ok with them then run following.
for file in *.txt
do
if [[ ! -f "${file%.*}_2" ]]
then
cp "$file" "${file%.*}_2"
fi
done
My pictures are kept in the folder with the picture-date for folder name, for example the original path and file names:
.../Pics/2016_11_13/wedding/DSC0215.jpg
.../Pics/2016_11_13/afterparty/DSC0234.jpg
.../Pics/2016_11_13/afterparty/DSC0322.jpg
How do I rename the pictures into the format below, with continuous sequences and 4-digit padding?
.../Pics/2016_11_13_wedding.0001.jpg
.../Pics/2016_11_13_afterparty.0002.jpg
.../Pics/2016_11_13_afterparty.0003.jpg
I'm using Bash 4.1, so only mv command is available. Here is what I have now but it's not working
#!/bin/bash
p=0
for i in *.jpg;
do
mv "$i" "$dirname.%03d$p.JPG"
((p++))
done
exit 0
Let say you have something like .../Pics/2016_11_13/wedding/XXXXXX.jpg; then go in directory .../Pics/2016_11_13; from there, you should have a bunch of subdirectories like wedding, afterparty, and so on. Launch this script (disclaimer: I didn't test it):
#!/bin/sh
for subdir in *; do # scan directory
[ ! -d "$subdir" ] && continue; # skip non-directory
prognum=0; # progressive number
for file in $(ls "$dir"); do # scan subdirectory
(( prognum=$prognum+1 )) # increment progressive
newname=$(printf %4.4d $prognum) # format it
newname="$subdir.$newname.jpg" # compose the new name
if [ -f "$newname" ]; then # check to not overwrite anything
echo "error: $newname already exist."
exit
fi
# do the job, move or copy
cp "$subdir/$file" "$newname"
done
done
Please note that I skipped the "date" (2016_11_13) part - I am not sure about it. If you have a single date, then it is easy to add these digits in # compose the new name. If you have several dates, then you can add a nested for for scanning the "date" directories. One more reason I skipped this, is to let you develop something by yourself, something you can be proud of...
Using only mv and bash builtins:
#! /bin/bash
shopt -s globstar
cd Pics
p=1
# recursive glob for .jpg files
for i in **/*.jpg
do
# (date)/(event)/(filename).jpg
if [[ $i =~ (.*)/(.*)/(.*).jpg ]]
then
newname=$(printf "%s_%s.%04d.jpg" "${BASH_REMATCH[#]:1:2}" "$p")
echo mv "$i" "$newname"
((p++))
fi
done
globstar is a bash 4.0 feature, and regex matching is available even in OSX's anitque bash.
I am trying to iterate in a loop, tar a couple of directories with each iteration and then compare the md5 sums of both of them. I notice that my first tar statement produces the tar files one level above the actual path of the directory. i.e. the statement:
tar -czvf ${folder_name}.tar.gz /tmp/psk1/hadoop_validation$ENV/${folder_name}
produces the ${folder_name}.tar.gz in /tmp/psk1/ rather than /tmp/psk1/hadoop_validation$ENV/
and the second tar statement:
tar -czvf ${folder_name}.tar.gz ${edge_base_dir}/wlossf$ENV/app/${folder_name}
doesn't produce the tar file at all. I can't find it even on one level above the actual path.
hdfs dfs -ls /haas/wlf/wlossf$ENV/app | while read rec; do
echo $rec
folder_path=`echo ${rec} | awk -F ' ' '{print $8}'`
folder_name=`echo ${folder_path} | awk -F '/' '{print $6}'`
if [ ! -z ${folder_name} ] && [ ! -z ${folder_path} ]; then
hdfs dfs -get ${folder_path} /tmp/psk1/hadoop_validation$ENV/
if [ $? -eq 0 ]; then
echo "Hadoop to local copy job Successful"
else
echo "Hadoop to local copy job Failed"
fi
tar -czvf ${folder_name}.tar.gz /tmp/psk1/hadoop_validation$ENV/${folder_name}
hadoop_md5=$(md5sum /tmp/psk1/hadoop_validation$ENV/${folder_name}.tar.gz)
tar -czvf ${folder_name}.tar.gz ${edge_base_dir}/wlossf$ENV/app/${folder_name}
edge_md5=$(md5sum ${edge_base_dir}/wlossf$ENV/app/${folder_name}.tar.gz)
if [ ${hadoop_md5} == ${edge_md5} ]; then
echo "${folder_name} is good"
else
echo "${folder_name} is bad"
fi
fi
echo ${folder_name}
echo ${folder_path}
done
What am I missing here? Any help would be appreciated.
Thank you.
As mouviciel said in the comments, tar by default creates the file in the current working directory.
Simply prefix the tar.gz file with the folder and it will create it where you want it:
tar -czvf /tmp/psk1/hadoop_validation$ENV/${folder_name}.tar.gz /tmp/psk1/hadoop_validation$ENV/${folder_name}
Note that as you will be creating the tar inside the same folder that you are archiving, you'll get a file changed as we read it warning as part of the output. Nothing to worry about.
Good Morning,
I am trying to write a korn shell script to look inside a directory that contains loads of files and check that each file also exists with .orig on the end.
For example if a file inside the directory is called 'mercury_1' there must also be a file called 'mercury_1.orig'
If there isn't, it needs to move the mercury_1 file to another location. However if the .orig file exists do nothing and move onto the next file.
I am sure it is really simple but I am not that experienced in writing Linux scripts and help would be greatly appreciated!!
Here's a small ksh snippet to check if a file exists in the current directory
fname=mercury_1
if [ -f $fname ]
then
echo "file exists"
else
echo "file doesn't exit"
fi
Edit:
The updated script that does the said functionality
#/usr/bin/ksh
if [ ! $# -eq 1 ]
then
echo "provide dir"
exit
fi
dir=$1
cd $dir
#process file names not ending with orig
for fname in `ls | grep -v ".orig$"`
do
echo processing file $fname
if [ -d $fname ] #skip directory
then
continue
fi
if [ -f "$fname.orig" ] #if equiv. orig file present
then
echo "file exist"
continue
else
echo "moving"
mv $fname /tmp
fi
done
Hope its of help!
You can use the below script
script.sh :
#!/bin/sh
if [ ! $# -eq 2 ]; then
echo "error";
exit;
fi
for File in $1/*
do
Tfile=${File%%.*}
if [ ! -f $Tfile.orig ]; then
echo "$File"
mv $File $2/
fi
done
Usage:
./script.sh <search directory> <destination dir if file not present>
Here, for each file with extension stripped check if "*.orig" is present, if not then move file to different directory, else do nothing.
Extension is stripped because you don't want to repeat the same steps for *.orig files.
I tested this on OSX (basically mv should not differ to much from linux). My test directory is zbar and destination is /tmp directory
#!/bin/bash
FILES=zbar
cd $FILES
array=$(ls -p |grep -v "/") # we search for file without extension so put them in array and ignore directory
echo $array
for f in $array #loop in array and find .orig file
do
#echo $f
if [ -e "$f.orig" ]
then
echo "found $f.orig"
else
mv -f "$f" "/tmp"
fi
done
I have a directory which has many folder and each folder contains a list of XML files. I am writing a bash script that traverses through the files and renames the extension of the file to "manual" if the size of the file is greater than 65Mb. This is my first writing a shell script and I was able to write the code for traversing the files but I am having difficulty in the renaming part.
for file in $dir
do
size=$(stat -c%s "$file")
if test "$size" -gt "68157440"; then
echo "Before Renaming...."
echo $file
echo "After renaming"
mv *.manual `basename $file`.xml
echo $file
else
echo $file >> outlog.log
fi
done
an example of $file is,
/apps/jAS/dev/products-app/BConverter/data/supplier-data/TF/output/Fiber Optics and Fiber Management Solutions/Fiber Optic Cable Assemblies.xml
mv *.manual `basename $file`.xml
If you want to change the extension of $file from xml to manual, do instead
mv "$file" "${file%.xml}".manual
What exactly is the difficulty you're having?
If it's white space in file names, try
mv *.manual `basename "$file"`.xml
Note that your script will not work if *.manual expands to more than one file name.
No need for a script on this, combination of find and xargs should do the trick:
find . -size +65M | xargs -IQ mv Q Q.manual
The little-used -I option to Xargs:
runs each input as a separate command, and
lets you replace the filename, so you can use it multiple time, ideal for a mv