I have files in a directory as shown in the below format.
$today and $yesterday are two variables holding today's date and yesterday's date , both will hold date as shown in below structure .
today=$(date +"%Y-%m-%d")
yesterday=$(date -d "yesterday 13:00" '+%Y-%m-%d')
example-$today.txt
polar-$today.txt
example-$yesteday.txt
polar-$yesteday.txt
Example yesterday : example-2020-09-24.txt
Example today: example-2020-09-25.txt
Files are created on a daily basis using cronjob , so there will be files in below structure with tomorrow's date.
example-$tomorrow.txt
polar-$tomorrow.txt
I want to compare files starting with same name on different dates and if there is difference execute a python script.The python script takes today's file as first argument if there is a difference.
if diff example-$today.txt example-$yesteday.txt
then
echo "No difference"
else
python script.py example-$today.txt
fi
If I have only 2 or 3 files I can use if else code for each file using diff as mentioned above, but the list will be populated with more unique names in future , and writing if command is tedious.
Requirement :
Compare all the txt files in the directory with same named file names on yesterday and today , if there is a difference execute the python script.
Seems like it's probably not too terrible to do:
for base in example polar; do
if ! diff ${base}-$today.txt ${base}-$yesteday.txt; then
python script.py ${base}-$today.txt
fi
done
That should be fairly maintainable, and you can write list='example polar ...' ... for base in $list, or list=$( cmd to dynamically generate names), or use an array. There's a lot of flexibility. For example, if you don't want to maintain the list of files, you could do:
for file in *-${today}.txt; do
base="${file%-${today}.txt}"
if ! diff "${base}-$today.txt" "${base}-$yesteday.txt"; then
python script.py "${base}-$today.txt"
fi
done
Note that I've removed the excess verbosity. Succeed quietly, fail loudly.
Related
I am looking to adapt a shell script to find a way to cycle through files that have different table names as well as different dates between files that have the same table name, and return the highest suffix file.
An example of my files in a given directory:
firsttable_20230112093000_1
firsttable_20230112093000_2
firsttable_20230112093000_3
firsttable_20230112093000_4
firsttable_19990202090000_1
firsttable_19990202090000_2
secondtable_20220112090000_1
secondtable_20220112090000_2
secondtable_20220112090000_3
Desired Result:
firsttable_20230112093000_4
firsttable_19990202090000_2
secondtable_20220112090000_3
What's been done
Originally I only needed to find the highest suffix as the dates would be the same for all tables, and what I had worked:
allTables=(
'firsttable'
'secondtable'
'thirdtable'
...
)
for table in ${allTables[#]}; do
substring="_2"
searchstring="$table$substring"
# Check if the file for a given table exists:
if ls $Path/$searchString* 1> /dev/null 2>&1; then
echo "$searchString* files exist. Proceeding..."
lastFile=$(ls "$Path/searchString"* | sort -rV | head -n1)
echo "Highest suffix file: $lastFile"
else
echo "File searchstring not found: '$Path/$searchString' "
fi
done
If I was to apply that to my new directory shown above, it would only be able to find:
Highest suffix file: firsttable_20230112093000_4
Highest suffix file: secondtable_20220112090000_3
I need to find a way to make the script also look at the dates and see if they are different, and if they are, treat them as such. Would this require a regex to assess the filename? The filename format stays the same: "tablename_$$$$$$$$$$$$$$_nn" (underscore placing after table and date, suffix can go above single figures, date is always 14 characters)
Thanks in advance for any help!
I have some files in Linux directory like below.
email_Tracking_export_2018_08_26.zip
email_Tracking_export_2018_08_27.zip
email_Tracking_export_2018_08_28.zip
email_Tracking_export_2018_08_29.zip
email_Tracking_export_2018_09_03.zip
email_Tracking_export_history_Novemeber.zip
email_Tracking_export_history_December.zip
email_Tracking_export_history_january.zip
email_Tracking_export_history_february.zip
email_Tracking_export_history_march.zip
email_Tracking_export_history_April.zip
Now I want to change the files names to be like below.
email_Tracking_export_2018_08_26.zip
email_Tracking_export_2018_08_27.zip
email_Tracking_export_2018_08_28.zip
email_Tracking_export_2018_08_29.zip
email_Tracking_export_2018_09_03.zip
email_Tracking_export_2017_11_01.zip
email_Tracking_export_2017_12_01.zip
email_Tracking_export_2018_01_01.zip
email_Tracking_export_2018_02_01.zip
email_Tracking_export_2018_03_01.zip
email_Tracking_export_2018_04_01.zip
Conditions:
If the file names are in yyyy-mm-dd format then leave them as is
if the file names are in Alphabetical form convert to yyyy-mm-dd
if month has passed in that particular year than leave as is if not then year should be previous year.
How can I achive that in bash/Linux
for f in email_Tracking_export_*.zip; do
case "$f" in
email_Tracking_export_????_??_??.zip) : ignore ;;
*) date=$(stat -c %Y "$f") # mod time in seconds
fmtdate=$(date --date="#$date" +%Y_%m_%d) # formatted
mv "$f" email_Tracking_export_$fmtdate.zip
;;
esac
done
Here are the steps in small parts which you can find answers of and execute as a bash script,
Make a key value pair of mapping, which maps a month to its numerical value. (Take care of lowercase/uppercase)
For each file, check their format.
Generate the new name for each file.
Then, use the mv command to rename the file with the new name.
I have an actioncam that saves my video in a folder into the SD Card.
Using linux, here is the path:
/media/mattiapdo/EOS_DIGITAL/_REC/100MEDIA
Files are saved in the REC_0001.AVI format
I would write a script that renames each file using the writing date.
Furthermore I notice that for some strange reason, the date and the hour are different from the effective: for example, 12/07/2017 10:30 is written as 09/02/2011 07:55
As the camera is very old and minimal, I can't reset the correct date and the correct hour so I would prefer to manipulate them in aftermath.
The goal would be to rename REC_0001.AVI in 2017_07_12__10_30.AVI
Does anyone have any ideas?
You can use the date command to print the elapsed seconds since, Unix lingo, the Epoch, aka 1970-01-01 UTC. Assuming that the camera date is in Anglo format, and that by default date likes the Anglo format, you have to swap the month and day in your date
$ date --date='09/02/2011 07:55' +%s
1314942900
$ date --date='07/12/2017 10:30' +%s
1499848200
$
so that you can compute a Delta between the real date and the camera idea of time
$ Delta=$(($(date --date='07/12/2017 10:30' +%s)-$(date --date='02/09/2011 07:55' +%s)))
$ echo $Delta
184905300
$
You haven't (yet?) told us how you fetch the date from the camera, but let's
say that
$ camera=$(fetch_date $current_file_name)
and assuming that $camera is in a format that date likes,
$ fromEpoch=$(($(date --date="$camera" +%s)+$Delta))
the last step is to get back the date in a format that you like , I suggest
the ISO 8601 format, so that your files are correctly sorted by ls
$ corrected_date=$(date --date="#$fromEpoch" +%Y-%m-%dT%H:%M)
$ cp $current_file_name other_directory/$corrected_date.AVI
The boring details about the date command, that is indeed VERY flexible and useful, are available using
$ man date
I hope that you can write your script with the info that I gave you, thank you for the question.
Addendum
Caveat emptor: totally untested
$ cat script
Delta=$(($(date --date='07/12/2017 10:30' +%s)-$(date --date='02/09/2011 07:55' +%s)))
mkdir -p ATTIC
mv *AVI ATTIC
for file in ./ATTIC/*.AVI ; do
########## fetch_date command is a placeholder for the real command
cam_date=$(fetch_date "$file")
cam_fromEpoch=$(date --date="$cam_date" +%s)
correct_fromEpoch=$(($cam_fromEpoch+$Delta))
ISO_8601=$(date --date="#$correct_fromEpoch" +%Y-%m-%dT%H:%M)
cp $file $ISO_8601.AVI
done
# cleanup, e.g. list current directory and ATTIC and ask if ATTIC is to be removed
$
Writing a bash script that will copy a file into a directory where the new copy has the same name, but with the timestamp appended to the filename (prior to the extension).
How can I achieve this?
to insert the time stamp of the file itself into the original file name, as well as preserving that timestamp in the target file, the following works in GNU environments:
file="/some/dir/path-to-file.xxx";
cp -p "$file" "${file%.*}-$(date -r"$file" '+%Y%m%d-%H%M%S').${file##*.}"
Adding proper use of the basename(1) command into the mix would allow you to copy the file into a different directory.
It's more challenging to do this outside of GNU/Linux environments and you have to start visiting languages like awk, perl, python, even php, to replace the date -r command.
file="file_to_copy"
cp $file "/path/to/dest/$file"`stat --printf "%X" $file`
You can look at the manual page of stat (man 1 stat) to choose the appropriate timestamp for your needs (creation, last access etc.)
In this example, I chose %X which means time of last access, seconds since Epoch
Suppose
var="/path/to/filename.ext" #path is optional
Do
var1="${var##*/}
cp "$var" "/path/to/new/directory/${var1%.*}$(date +%s).${var1##*.}"
For more on ${var%.*} & ${var##*.} , see [ shell parameter expansion ].
date manpage says :
%s seconds since 1970-01-01 00:00:00 UTC
I'm finding it difficult to word my question in a way I can search for the answer, my problem is as follows.....
I have a webcam that takes a photo every 2mins and saves as a numbered file, the first photo is taken at 0000hrs and is named image001.jpg, at 0002hrs image002.jpg and so on. At 2359hrs all the photos are turned in to 24hr time lapse video and saved as daily_video.mov. At 0000hrs (of the next day) the old image001.jpg is over written and the whole process repeated including generation of a new daily_video.mov.
This is all working fine with the webcam doing the file naming and overwriting, and a cron job running fffmpeg once a day to make the video.
What I want to do now is make a time lapse video over say a month by copying every 30th file from the days images to a new folder and naming in a sequential order. ie.
Day 1; image030.jpg, image060.jpg, etc... are renamed to Archive001.jpg, Archive002.jpg,etc...
But on day 2; image030.jpg, image060.jpg etc... Will need to be named to Archive025.jpg, Archive026.jpg etc.. and repeat untill the end of the month copying files from the day to a sequentially increasing in name list of files to use at the end of month, where the process can be repeated.
Does that make sense?!!
You could use a bash script like the following. Just call it at 2359hrs.
Remeber to make it executable using chmod +x myScript
I did not rename to Archive00X.jpg, but by adding the current date, they will be in proper alphabetical order.
example output:
cp files/image000.jpg >> archive/image_2012-08-29_000.jpg
cp files/image030.jpg >> archive/image_2012-08-29_030.jpg
....
adapt pSource and pDest to your paths (preferrably absolute paths)
adapt offset and maxnum to your needs. If maxnum is too big it will tell you some files are missing, but otherwise work properly.
Remove the echo lines if they disturb you ;)
Code:
#!/bin/bash
pSource="files"
pDest="archive"
offset=30
maxnum=721
curdate=`date "+%F"`
function rename_stuff()
{
myvar=0
while [ $myvar -lt $maxnum ]
do
forg=`printf image%03d.jpg ${myvar}`
fnew=`printf image_%s_%03d.jpg ${curdate} ${myvar}`
forg="$pSource/$forg"
fnew="$pDest/$fnew"
if [ -f "$forg" ]; then
echo "cp $forg >> $fnew"
cp "$forg" "$fnew"
else
echo "missing file $forg"
fi
myvar=$(( $myvar + $offset ))
done
}
rename_stuff