i have a directory with a lot of sub-directories including files.
For each WAV file i would like to rename WAV file by adding creation date (date when file WAV has been firstly created) at the beginning of the file (without changing timestamps of file itself).
Next step would be to convert the WAV file to MP3 file, so i will save hard drive space.
for that purpose, i'm trying to create a bash script but i'm having some issues.
I want to keep the same structure as original directory and therefore i was thinking of something like:
for file in `ls -1 *.wav`
do name=`stat -c %y $file | awk -F"." '{ print $1 }' | sed -e "s/\-//g" -e "s/\://g" -e "s/[ ]/_/g"`.wav
cp -r --preserve=timestampcp $dir_original/$file $dir_converted/$name
done
Don't use ls to generate a list of file names, just let the shell glob them (that's what ls *.wav does anyway):
for file in ./*.wav ; do
I think you want the timestamp in the format YYYYMMDD_HHMMSS ?
You could use GNU date with stat to have a somewhat neater control of the output format:
epochtime=$(stat -c %Y "$file" )
name=$(date -d "#$epochtime" +%Y%m%d_%H%M%S).wav
stat -c %Y (or %y) gives the last modification date, but you can't really get the date of the file creation on Linux systems.
That cp looks ok, except for the stray cp at the end of timestampcp, but that must be a typo. If you do *.wav, the file names will be relative to current directory anyway, so no need to prefix with $dir_original/.
If you want to walk through a whole subdirectory, use Bash's globstar feature, or find. Something like this:
shopt -s globstar
cd "$sourcedir"
for file in ./**/*.wav ; do
epochtime=$(stat -c %Y "$file" )
name=$(date -d "#$epochtime" +%Y%m%d_%H%M%S).wav
dir=$(dirname "$file")
mkdir -p "$target/$dir"
cp -r --preserve=timestamp "$file" "$target/$dir/$name"
done
The slight inconvenience here is that cp can't create the directories in the path, so we need to use mkdir there. Also, I'm not sure if you wanted to keep the original filename as part of the resulting one, this would remove it and just replace the file names with the timestamp.
I did some experimenting with the calculation of name to see if I could get it more succinctly, and came up with this:
name=$(date "+%Y%m%d_%H%M%S" -r "$file")
I wanted to append all file names in that folder with the date they were created , and below works perfectly.
#############################
#!/bin/sh
for file in `ls *.JPG`;
do
mv -f "$file" "$(date -r "$file" +"%Y%m%d_%H_%M_%S")_"$file".jpg"
done
##############################
Related
I have a lot of daily files that are sort by hours which comes from a data-logger (waveform). I downloaded inside a USB stick, now I need to save them inside folders named with the first 8 characters of waveform.
Those files have the following pattern:
Year-Month-Day-hourMinute-##.Code_Station_location_Channel
for example, inside the USB I have:
2020-10-01-0000-03.AM_REDDE_00_EHE; 2020-10-01-0100-03.AM_REDDE_00_EHE; 2020-10-02-0300-03.AM_REDDE_00_EHE; 2020-10-20-0000-03.AM_REDDE_00_EHE; 2020-10-20-0100-03.AM_REDDE_00_EHE; 2020-11-15-2000-03.AM_REDDE_00_EHE; 2020-11-15-2100-03.AM_REDDE_00_EHE; 2020-11-19-0400-03.AM_REDDE_00_EHE; 2020-11-19-0900-03.AM_REDDE_00_EHE;
I modified a little a code from #user3360767 (shell script to create folder daily with time-stamp and push time-stamp generated logs) to speed up the procedure of creating a folder and moving the files to them
for filename in 2020-10-01*EHE; do
foldername=$(echo "$filename" | awk '{print (201001)}');
mkdir -p "$foldername"
mv "$filename" "$foldername"
echo "$filename $foldername" ;
done
2020-10-01*EHE
Here I list all hours from 2020-10-01-0000-03.AM_REDDE_00_EHE
foldername=$(echo "$filename" | awk '{print (201001)}');
Here I create the folder that belongs to 2020-10-01 and with the following lines create the folder and then move all files to created folder.
mkdir -p "$foldername"
mv "$filename" "$foldername"
echo "$filename $foldername" ;
As you may notice, I will always need to modify the line for filename in 2020-10-01*EHE each time the file changes the date.
Is there a way to try to create folders with the first 8 number of the file?
Tonino
Use date
And since the foldername doesn't change, you don't need to keep creating one inside the loop.
files="$(date +%Y-%m-%d)*EHE"
foldername=$(date +%Y%m%d)
mkdir -p "$foldername"
for filename in $files; do
mv "$filename" "$foldername"
echo "$filename $foldername"
done
Edit:
If you want to specify the folder each time, you can pass it as an argument and use sed to get the filename pattern
foldername=$1
files=$(echo $1 | sed 's/\(....\)\(..\)\(..\)/\1-\2-\3/')
filepattern="$files*EHE"
mkdir -p "$foldername"
for filename in $filepattern; do
mv "$filename" "$foldername"
echo "$filename $foldername"
done
You call it with
./<yourscriptname>.sh 20101001
I think you want to move all files whose names end in *EHE into subdirectories. The subdirectories will be created as necessary and will be named according to the date at the start of each filename without the dashes/hyphens.
Please test the following on a copy of your files in a temporary directory somewhere.
#!/bin/bash
for filename in *EHE ; do
# Derive folder by deleting all dashes from filename, then taking first 8 characters
folder=${filename//-/}
folder=${folder:0:8}
echo "Would move $filename to $folder"
# Uncomment next 2 lines to actually move file
# mkdir -p "$folder"
# mv "$filename" "$folder"
done
Sample Output
Would move 2020-10-01-0000-03.AM_REDDE_00_EHE to 20201001
Would move 2020-10-01-0100-03.AM_REDDE_00_EHE to 20201001
Note that the 2 lines:
folder=${filename//-/}
folder=${folder:0:8}
use "bash parameter substitution", which is described here if you want to learn about it, and obviate the need to create whole new processes to run awk, sed or cut to extract the fields.
I'd love to have a more elegant solution for a mass rename of files, as shown below. Files were of format DEV_XYZ_TIMESTAMP.dat and we needed them as T-XYZ-TIMESTAMP.dat.
In the end, I copied them all (to be on the same side) into renamed folder:
ls -l *dat|awk '{system("cp " $10 " renamed/T-" substr($10, index($10, "_")+1))}'
So, first I listed all dat files, then picked up 10th column (file name) and executed a command using awk's system function.
The command was essentially copying of original filename into renamed folder with new file name.
New file name was created by removing (awk substring function) prefix before (including) _ and adding "T-" prefix.
Effectively:
cp DEV_file.dat renamed/T-file.dat
Is there a way to use cp or mv together with some regex rules to achieve the same in a bit more elegant way?
Thx
You may use this script:
for file in *.dat; do
f="${file//_/-}"
mv "$file" renamed/T-"${f#*-}"
done
You must avoid parsing output of ls command.
If you have rename utilitity
rename -E "s/[^_]*/T/" -e "s/_/-/g" *dat
Demo
$ls -1
ABC_DEF_TIMESTAMP.dat
DEV_XYZ_TIMESTAMP.dat
$rename -E "s/[^_]*/T/" -e "s/_/-/g" *
$ls -1
T-DEF-TIMESTAMP.dat
T-XYZ-TIMESTAMP.dat
$
This is how I would do it:
cpdir=renamed
for file in *dat; do
newfile=$(echo "$file" | sed -e "s/[^_]*/T/" -e "y/_/-/")
cp "$file" "$cpdir/$newfile"
done
The sed scripts transforms every non-underscore leading characters in a single T and then replaces every _ with -. If cpdir is not sure to exist before execution, you can simply add mkdir "$cpdir" after first line.
I searched a while and tried it by myself but unable to get this sorted so far. My folder looks below, 4 files
1.txt, 2.txt, 3.txt, 4.txt, 5.txt, 6.txt
I want to print file modified time and echo the time stamp in it
#!/bin/bash
thedate= `ls | xargs stat -s | grep -o "st_mtime=[0-9]*" | sed "s/st_mtime=//g"` #get file modified time
files= $(ls | grep -Ev '(5.txt|6.txt)$') #exclud 5 and 6 text file
for i in $thedate; do
echo $i >> $files
done
I want to insert each timestamp to each file. but having "ambiguous redirect" error. am I doing it incorrectly? Thanks
In this case, files is a "list" of files, so you probably want to add another loop to handle them one by one.
Your description is slightly confusing but, if your intent is to append the last modification date of each file to that file, you can do something like:
for fspec in [1-4].txt ; do
stat -c %y ${fspec} >>${fspec}
done
Note I've used stat -c %y to get the modification time such as 2017-02-09 12:21:22.848349503 +0800 - I'm not sure what variant of stat you're using but mine doesn't have a -s option. You can still use your option, you just have to ensure it's done on each file in turn, probably something like (in the for loop above):
stat -s ${fspec} | grep -o "st_mtime=[0-9]*" | sed "s/st_mtime=//g" >>${fspec}
You can not redirect the output to several files as in > $files.
To process several files you need something like:
#!/bin/bash
for f in ./[0-4].txt ; do
# get file modified time (in seconds)
thedate="$(stat --printf='%Y\n' "$f")"
echo "$thedate" >> "$f"
done
If you want a human readable time format change %Y by %y:
thedate="$(stat --printf='%y\n' "$f")"
I'm encountering many files with the same content and the same name on some of my servers. I need to quarantine these files for analysis so I can't just remove the duplicates. The OS is Linux (centos and ubuntu).
I enumerate the file names and locations and put them into a text file.
Then I do a for statement to move the files to quarantine.
for file in $(cat bad-stuff.txt); do mv $file /quarantine ;done
The problem is that they have the same file name and I just need to add something unique to the filename to get it to save properly. I'm sure it's something simple but I'm not good with regex. Thanks for the help.
Since you're using Linux, you can take advantage of GNU mv's --backup.
while read -r file
do
mv --backup=numbered "$file" "/quarantine"
done < "bad-stuff.txt"
Here's an example that shows how it works:
$ cat bad-stuff.txt
./c/foo
./d/foo
./a/foo
./b/foo
$ while read -r file; do mv --backup=numbered "$file" "./quarantine"; done < "bad-stuff.txt"
$ ls quarantine/
foo foo.~1~ foo.~2~ foo.~3~
$
I'd use this
for file in $(cat bad-stuff.txt); do mv $file /quarantine/$file.`date -u +%s%N`; done
You'll get everyfile with a timestamp appended (in nanoseconds).
You can create a new file name composed by the directory and the filename. Thus you can add one more argument in your original code:
for ...; do mv $file /quarantine/$(echo $file | sed 's:/:_:g') ; done
Please note that you should replace the _ with a proper character which is special enough.
Someone has very kindly help get me started on a mass rename script for renaming PDF files.
As you can see I need to add a bit of logical to stop the below happening - so something like add a unique number to a duplicate file name?
rename 's/^(.{5}).*(\..*)$/$1$2/' *
rename -n 's/^(.{5}).*(\..*)$/$1$2/' *
Annexes 123114345234525.pdf renamed as Annex.pdf
Annexes 123114432452352.pdf renamed as Annex.pdf
Hope this makes sense?
Thanks
for i in *
do
x='' # counter
j="${i:0:2}" # new name
e="${i##*.}" # ext
while [ -e "$j$x" ] # try to find other name
do
((x++)) # inc counter
done
mv "$i" "$j$x" # rename
done
before
$ ls
he.pdf hejjj.pdf hello.pdf wo.pdf workd.pdf world.pdf
after
$ ls
he.pdf he1.pdf he2.pdf wo.pdf wo1.pdf wo2.pdf
This should check whether there will be any duplicates:
rename -n [...] | grep -o ' renamed as .*' | sort | uniq -d
If you get any output of the form renamed as [...], then you have a collision.
Of course, this won't work in a couple corner cases - If your files contain newlines or the literal string renamed as, for example.
As noted in my answer on your previous question:
for f in *.pdf; do
tmp=`echo $f | sed -r 's/^(.{5}).*(\..*)$/$1$2/'`
mv -b ./"$f" ./"$tmp"
done
That will make backups of deleted or overwritten files. A better alternative would be this script:
#!/bin/bash
for f in $*; do
tar -rvf /tmp/backup.tar $f
tmp=`echo $f | sed -r 's/^(.{5}).*(\..*)$/$1$2/'`
i=1
while [ -e tmp ]; do
tmp=`echo $tmp | sed "s/\./-$i/"`
i+=1
done
mv -b ./"$f" ./"$tmp"
done
Run the script like this:
find . -exec thescript '{}' \;
The find command gives you lots of options for specifing which files to run on, works recursively, and passes all the filenames in to the script. The script backs all file up with tar (uncompressed) and then renames them.
This isn't the best script, since it isn't smart enough to avoid the manual loop and check for identical file names.