Image format calculator and converter with GNU/Linux - linux

Wouldn't it be great if there with a command for GNU/Linux that would do the following:
Open -Recursive *.png -Not-Case-Sensitive if exported-to-jpg#100%quality=less bytes than the original png then write jpg and delete the png
It would also be able to do the inverse of that command:
if png=less bytes than jpg then delete jpg

Looking for the One True Command is not going to help: if it existed, it would only be useful for you and the (presumably) small set of people who had exactly your needs in the future.
The UNIX Way is to link together several commands to do what you want. For example:
"open-recursive": feed files into the hopper using "find", eg find /path -type f -name '*.png' -print and then send the list out through a pipe.
"not case-sensitive": either increase the scope of the find (-o) or get find to dump out all the files and then use grep to look for what you want, eg find . -print | grep -i '.png'
"if-exported-to-jpg": this is slightly tricky because I believe that the only way to check if the conversion saves bytes is to actually convert it and see. You can use the convert tool from the ImageMagick package to do this. ImageMagick has been standard in the big name distros for years so should be easy to find.
"if less bytes than": straightforward to do in the shell or your favorite scripting language - Perl, python etc.
The net is that you build up what you want from these smaller pieces and you should be able to do what you want now and have something that you can modify in the future or share with others for their unique needs. That is the UNIX Way. Ommmm :)

Some time ago, I wrote a script to convert my photos. The script reduces the dimensions of all JPG file in current folder if any width or height is greater than MAX (default = 1024), keeping aspect ratio, and put them in a different folder (created). I hope this help you.
#!/bin/bash
if [ ! -d reduced ]
then
mkdir reduced
fi
if [ $# -lt 1 ]
then
MAX=1024
else
MAX=$1
fi
for obj in *.jpg
do
echo "------> File: $obj"
tam=$(expr `identify -format "%b" "$obj" | tr -d "B"` / 1024)
width=$(identify -format "%w" "$obj")
height=$(identify -format "%h" "$obj")
echo -e "\tDimensions: $width x $height px"
echo -e "\tFile size: $tam kB"
if [ $width -gt $height ] && [ $width -gt $MAX ]
then
convert "$obj" -resize $MAX "reduced/$obj.jpg"
cd reduced
mv "$obj".jpg "${obj%.jpg}".jpg;
tam=$(expr `identify -format "%b" "$obj" | tr -d "B"` / 1024)
width=$(identify -format "%w" "$obj")
height=$(identify -format "%h" "$obj")
echo -e "\tNew dimensions: $width x $height px"
echo -e "\tNew file size: $tam kB"
cd ..
echo -e "\tOk!"
elif [ $height -gt $MAX ]
then
convert "$obj" -resize x$MAX "reduced/$obj.jpg"
cd reduced
mv "$obj".jpg "${obj%.jpg}".jpg;
tam=$(expr `identify -format "%b" "$obj" | tr -d "B"` / 1024)
width=$(identify -format "%w" "$obj")
height=$(identify -format "%h" "$obj")
echo -e "\tNew dimensions: $width x $height px"
echo -e "\tNew file size: $tam kB"
cd ..
echo -e "\tOk!"
else
cp "$obj" reduced/
echo -e "\tDo not modify!"
fi
done

Err, in answer to your question - "No, it probably wouldn't".
Firstly, PNG files can support transparency and JPEGs cannot, so if this was scripted to your specification, you could lose countless hours of work that went into creating transparent masks for thousands of images.
Secondly, PNG files do not support EXIF/IPTC data, so you would also lose all your Copyright, camera and lens settings, GPS data, dates, and oodles of other metadata.
Thirdly, your PNG file may contain 16 bits per channel, whereas JPEG can only store 8 bits per channel so you could potentially lose an awful lot of fine colour gradations by moving from PNG to JPEG.
Fourthly, you could potentially lose compatibility with older Web browsers which had spotty support for PNGs.

Related

Looping over all files of certain extension in a directory

I wrote a small script that unzips all the *.zip files in the current directory to extract only *.srt files directory to a newly created directory. It then loops over all the *.mkv files in the current directory to get their name and then changes subs/*.srt file name to produce a new file name that is exactly as *.mkv file name.
The script works when there is one zip file and one mkv file but when there are more files it produces bad filenames. I cannot track why this is the case. Now I figured out when this is the case.
EDIT
I managed to narrow down the scenarios when file names are changed in erroneous way.
Let's say in a current directory we have three *.mkv files: (sorted alphabetically)
$ ls -1a *.mkv
Home.S06E10.1080p.BluRay.x264-PRINTER.mkv
Home.S06E11.1080p.BluRay.x264-PRINTER.mkv
Home.S06E12.1080p.BluRay.x264-PRINTER.mkv
and three *.srt files:
$ ls -1a *.srt
Home.S06E10.srt
Home.S06E11.BDRip.X264-PRINTER.srt
Home.S06E12.BDRip.X264-PRINTER.srt
When I run the script, I get:
subs/Home.S06E10.srt -> subs/Home.S06E10.1080p.BluRay.x264-PRINTER.srt
subs/Home.S06E10.1080p.BluRay.x264-PRINTER.srt -> subs/Home.S06E11.1080p.BluRay.x264-PRINTER.srt
subs/Home.S06E11.1080p.BluRay.x264-PRINTER.srt -> subs/Home.S06E12.1080p.BluRay.x264-PRINTER.srt
As you see, Home.S06E10.srt is used twice
#!/usr/bin/env bash
mkdir -p subs
mkdir -p mkv-out
mkdir -p subs-bak
# unzip files, maybe there are subtitles in it...
for zip in *.zip; do
if [ -f "$zip" ]; then
unzip "$zip" -d subs "*.srt" >/dev/null
fi
done
# move all subtitles to subs catalog
for srt in *.srt; do
if [ -f "$srt" ]; then
mv "*.srt" subs
fi
done
mkvCount=(*.mkv)
srtCount=(subs/*.srt)
if [ ${#mkvCount[#]} != ${#srtCount[#]} ]; then
echo "Different number of srt and mkv files!"
exit 1
fi
for MOVIE in *.mkv; do
for SUBTITLE in subs/*.srt; do
NAME=$(basename "$MOVIE" .mkv)
SRT="subs/$NAME.srt"
if [ ! -f "$SRT" ]; then
echo "$SUBTITLE -> ${SRT}"
mv "$SUBTITLE" "$SRT"
fi
done
done
You seem to be relying on the lexicographical order of the files to associate one SRT with one MKV. If all you have are season-episode files for the same series, then I suggest a completely different approach: iterate a season and an episode counters, then generate masks in the form S##E## and find a movie and a subtitle files. If you find them, you move them.
for season in {01..06}; do
for episode in {01..24}; do
# Count how many movies and subtitles we have in the form S##E##
nummovies=$(find -name "*S${season}E${episode}*.mkv" | wc -l)
numsubs=$(find -name "*S${season}E${episode}*.srt" | wc -l)
if [[ $nummovies -gt 1 || $numsubs -gt 1 ]]; then
echo "Multiple movies/subtitles for S${season}E${episode}"
exit 1
fi
# Skip if there is no movie or subtitle for this particular
# season/episode combination
if [[ $nummovies -eq 0 ]]; then
continue
fi
if [[ $numsubs -eq 0 ]]; then
echo "No subtitle for S${season}E${episode}"
continue
fi
# Now actually take the MKV file, get its basename, then find the
# SRT file with the same S##E## and move it
moviename=$(find -name "*S${season}E${episode}*.mkv")
basename=$(basename -s .mkv "$moviename")
subfile=$(find -name "*S${season}E${episode}*.srt")
mv "${subfile}" "${basename}.mkv"
done
done
If you don't want to rewrite everything, just change your last loop:
Drop the inner loop
Take the movie name instead and use sed to find the particular S##E## substring
Use find to find one SRT file like in my code
Move it
This has the benefit of not relying on hard-coded number of seasons/episodes. I guessed six seasons and no season with more than 26 episodes. However I thought my code would do and would look more simple.
Make certain that there will be exactly one SRT file. Having zero or more than one file will probably just give an error from mv, but it's better to be safe. In my code I used a separate call to find with wc to count the number of lines, but if you are more knowledgeable in bash-foo, then perhaps there's a way to treat the output of find as an array instead.
In both my suggestions you can also drop that check for # movies = # subtitles. This gives you more flexibility. The subtitles can be in whatever directories you want, but the movies are assumed to the in the CWDIR. With find you can also use the -or operator to accept other extensions, such as AVI and MPG.

How to avoid header truncation on the tiff using GhostScript. Convert ps to tiff

In this section I add the header file to the top of the tiff file.
echo "/CourierLatin1 findfont 8 scalefont setfont" >>${PS}
echo "40 2 moveto (${DATE}) show" >>${PS}
echo "200 2 moveto (${NAME}) show" >>${PS}
echo "400 2 moveto (${FROM}) show" >>${PS}
echo "510 2 moveto (${PAGEINFO}) show showpage" >>${PS}
${BIN}/gs -dQUIET -dNOPAUSE -dBATCH -dSAFER -sDEVICE=tiffg4 -sOutputFile=${HDR_FILE} \
-dDEVICEWITHPOINTS=612 -dDEVICEHEIGHTPOINTS=11 ${PS} >>${LOG_FILE}
if [ $? -ne 0 ] ; then
cleanup
echo "${BIN}/gs failed" >>${LOG_FILE}
exit 1
fi
# Overlay the header onto the tiff page
SIZE=`${BIN}/tiffinfo ${f} | grep "Image Width:" | sed -e "s/Image Width//" | sed -e "s/Image Length//" | sed -e "s/ //g"`
WIDTH=`echo ${SIZE} | cut -d: -f2`
LENGTH=`echo ${SIZE} | cut -d: -f3`
OFFSET=`expr ${LENGTH} - 2156`
# no shifting of header line up or down is needed
OFFSET=0
if [ ${OFFSET} -lt 0 ] ; then
OFFSET=0
fi
echo "Adding ${HDR_FILE} to ${f} ${WIDTH}x${LENGTH} at 0,${OFFSET}" >>${LOG_FILE}
${CONVERT} tiff:${f} -draw "image over 0,${OFFSET} 0,0 'tiff:${HDR_FILE}'" -compress Group4 -flatten +matte tiff:${f} >>${LOG_FILE}
if [ $? -ne 0 ] ; then
cleanup
echo "${CONVERT} failed" >>${LOG_FILE}
exit 1
fi
Can come to me in any size page. So I first translate tiff post script to size.
I convert tiff again. Can see the code as below:
# Nasty compromise - Have to convert the tiff file to ps file so that I can turn around
# and create a 'faxable' tiff file (one that Radisys will accept)
${BIN}/tiff2ps -a -h11.0 -w8.5 ${LOCAL_FILE} >${PS_FILE}
if [ $? -ne 0 ] ; then
cleanup
echo "${BIN}/tiff2ps failed" >>${LOG_FILE}
exit 1
fi
# Part 2 of compromise
${BIN}/gs -dQUIET -dNOPAUSE -dBATCH -dSAFER -sDEVICE=tiffg4 -sPAPERSIZE=letter -r204x196 -sOutputFile=${LOCAL_FILE} ${PS_FILE} >>${LOG_FILE} 2>&1
if [ $? -ne 0 ] ; then
cp ${BKUP_FILE} ${ORIG_FILE}
cleanup
echo "${BIN}/gs ps2tiff failed" >>${LOG_FILE}
exit 1
fi
Output tiff is given the Radisys. I dont understand Why header cut for this output.pdf.
You can see output fax image as below. why is Header cutted ?
Looks to me like you are printing the 'header' first, then putting the image from the TIFF file (in this case, the text 'FAX') on top of it.
TIFF files aren't masks, so the white space is not transparent. When you turn a TIFF file into PostScript it encapsulates the TIFF bitmap image as a PostScript image. Where the white space in the image lies on top of the 'header', it prints over it, obscuring part of the 'header'.
Its a little hard to be sure what's going on, I'm not a shell script expert, but it appears you first use Ghostscript to render a piece of PostScript to TIFF. That is your 'header'.
You then invoke $(CONVERT) which I'm going to guess is ImageMagick's convert utility to combine the header and some original TIFF file.
I suspect at this point is where your problem occurs, I'm guessing that you need to put the two tiff files in the reverse order so that the header is placed 'on top' of the original, instead of 'under' it (in Z-order terms. In your invocation of $(CONVERT) try swapping $(f) and $(HDR_FILE).
You can of course check the TIFF file at that point, presumably, to see if all is well.
You then take that TIFF file, use tiff2ps (which I think is part of libtiff) to wrap the image up as PostScript, and then use Ghostscript to re-render the image at a different (fax) resolution.
I'd say that you are using the wrong tool for that last step anyway. You already have a bitmap, you'd be better off using an image manipulation application to downsample it to the desired resolution. While your solution undoubtedly works, I would suspect that something like ImageMagick would do a better job.
At the very least, checking the TIFF file before you send it through tiff2ps should tell you whether the result is correct at that point.

Bash linux, Hour, Minutes and Seconds of a JPG Image

I am trying to get the complete date of an image in JPG format. The format I want to get is 14:25:38 (Hour:Minutes:Seconds).
I tried with the commands:
$ stat -c %y DSC_0002.JPG | sed 's/^\([0-9\-]*\).*/\1/'
=> 2017-05-19 -Not that way
$ file DSC_0002.JPG
=> DSC_0002.JPG: JPEG image data, Exif standard: [TIFF image data, little-endian, direntries=11, manufacturer=NIKON CORPORATION, model=NIKON D5200, orientation=upper-left, xresolution=180, yresolution=188, resolutionunit=2, software=Ver.1.01 , datetime=2017:05:19 13:30:34, GPS-Data], baseline, precision 8, 6000x4000, frames 3
This last (file DSC_0002.JPG) command displays datetime=2017:05:19 13:30:34, but I need to get only 13:30:34
Preferably without using add-ons or programs external to Linux bash.
Thank you very much for any help.
My 2 cents...
As exif is a big part of my job and because I already spent a lot of time building my own scripts, time is comming to make a little bench.
Getting result of jhead using bash:
This use jhead because it's my prefered. See further to know why...
DateTime=$(jhead -nofinfo DSC_0002.JPG)
DateTime=${DateTime#*Date/Time*: }
DateTime=${DateTime%%$'\n'*}
This is the quickest way, (a lot quicker than using bash regex)!
echo $DateTime
2011:02:27 14:53:32
Different ways
Inquiring a jpeg file for knowing his date is possible by using at least 4 different tools:
file The magic file recognition command make some light inquiring on any file for determining is nature and print some more informations.
file DSC_0002.JPG
DSC_0002.JPG: JPEG image data, Exif standard: [TIFF image data, big-endian,
direntries=10, manufacturer=CANIKON, model=CANIKON AB12, orientation=upper-
left, xresolution=168, yresolution=176, resolutionunit=2, software=Ver.1.00
, datetime=2011:02:27 14:53:32], baseline, precision 8, 3872x2592, frames 3
file print size, resolution and dateTime from a jpeg file.
jhead is a jpeg's header dedicated tool:
jhead DSC_0002.JPG
File name : DSC_0002.JPG
File size : 4940925 bytes
File date : 2011:02:27 14:53:32
Camera make : CANIKON
Camera model : CANIKON AB12
Date/Time : 2011:02:27 14:53:32
Resolution : 3872 x 2592
Flash used : No
Focal length : 55.0mm (35mm equivalent: 82mm)
Exposure time: 0.0080 s (1/125)
Aperture : f/5.6
ISO equiv. : 180
Whitebalance : Auto
Metering Mode: pattern
JPEG Quality : 97
identify is part of ImageMagick package wich is a kind of all purpose tool for bitmap images... (Due to some security bugs in past and overall perfomance, I'ts not my personal choice).
identify DSC_0002.JPG
DSC_0002.JPG JPEG 3872x2592 3872x2592+0+0 8-bit sRGB 4.941MB 0.010u 0:00.020
identify -format "%[EXIF:DateTime]\n" DSC_0002.JPG
2011:02:27 14:53:32
exiftool is a dedicated tool using libimage.
exiftool DSC_0002.JPG
ExifTool Version Number : 9.74
File Name : DSC_0002.JPG
Directory : .
File Size : 4.7 MB
File Modification Date/Time : 2011:02:27 14:53:32+01:00
File Access Date/Time : 2017:06:05 08:40:26+02:00
File Inode Change Date/Time : 2017:06:05 08:40:04+02:00
File Permissions : rw-r--r--
File Type : JPEG
MIME Type : image/jpeg
Exif Byte Order : Big-endian (Motorola, MM)
...
Modify Date : 2011:02:27 14:53:32
...
Thumbnail Image : (Binary data 8965 bytes, use -b option to extract)
Circle Of Confusion : 0.020 mm
Depth Of Field : 15.07 m (8.58 - 23.65)
Field Of View : 24.7 deg (5.50 m)
Focal Length : 55.0 mm (35 mm equivalent: 82.0 mm)
Hyperfocal Distance : 26.80 m
Light Value : 11.1
The default output of this is 185 lines length, I've dropped a lot.
pure bash As requested and because Toby Speight recommend to not doing this:
FDate=;while IFS= read -d $'\0' -n 1024 raw;do
[ "$raw" ] && \
[ -z "${raw#[0-9][0-9][0-9][0-9]:[0-9][0-9]:[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]}" ] &&
FDate=$raw && break
done <DSC_0002.JPG
echo $FDate
2011:02:27 14:53:32
Ok, this a far away a perfect function: first date found in file is considered, regardless of fieldname.
Little bench
As the goal is to extract date time part of exif infos, the bench would only do this.
time for i in {1..100};do ... done >/dev/null
export TIMEFORMAT="R:%4R u:%4U s:%4S"
Like this (from quicker to slower):
# quicker: jhead
time for i in {1..100};do jhead DSC_0002.JPG ;done >/dev/null
R:0.115 u:0.000 s:0.028
# 2nd: file
time for i in {1..100};do file DSC_0002.JPG; done >/dev/null
R:0.226 u:0.000 s:0.044
# 3nd: pure bash
time for i in {1..100};do
while IFS= read -d $'\0' -n 1024 raw ;do
[ "$raw" ] &&
[ -z "${raw#[0-9][0-9][0-9][0-9]:[0-9][0-9]:[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]}" ] &&
ftim=$raw && break
done <DSC_0002.JPG
done >/dev/null
R:0.393 u:0.380 s:0.012
# 4nd: best dedicated: exiftool
time for i in {1..100};do exiftool -time:CreateDate DSC_0002.JPG ;done >/dev/null
R:14.921 u:13.064 s:0.956
# slower: imagemagick's identify
time for i in {1..100};do identify -format "%[EXIF:DateTime]\n" DSC_0002.JPG ;done >/dev/null
R:21.609 u:15.712 s:5.060
Sorry, I agree with Toby Speight: Doing this in pure bash is a not so good idea!
Setting file date
Doing date manipulation is very easy under pure bash, but working on file date. For this, I asked my system to set all photos files datetime to creation date found in exif.
For this, exiftool offer a specific syntax:
exiftool '-FileModifyDate<DateTimeOriginal /path/to/myphotos
and jhead too:
jhead -q -ft /path/to/myphotos/*.JPG
This will set all creation file date based on creation date in exif infos. Once done, you could use standard tools for filesystem inquiry:
ls -l DSC_0002.JPG
stat DSC_0002.JPG
find /path/to/myphotos -type f -name 'DSC*JPG' -mtime +400 -mtime -410 -ls
And so on...
Not all images have metadata, but those who have them can get them in the following ways:
identify -format "%[EXIF:DateTime]\n" image.jpg | awk '{print $2}
As said Toby Speight:
exiftool -time:CreateDate -a -G0:1 -s image.jpg | awk '{print $5}'
and
jhead image.jpg | awk '/^Date\/Time/{print $4}'
I'm sure there are other options, but I did not practice them
You can use cut command to parse output.
In your case stat gives output
2017-05-05 06:12:37.228033281 -0500
So to get desire output you can use stat -c %y popen.c | cut -f2 -d' ' | cut -f1 -d'.'
Refer: man cut

How can I extract the filename without extension (and add a counter) from a variable on the Linux command line?

I want to do a batch convert and resize of some images in a folder on the Linux (Mint 17) command line, so I am using:
for $file in *.jpg; do convert -resize 25% $file $file.png; done
but that leaves me with a bunch of files like:
image1.jpg.png
photo4.jpg.png
picture7.jpg.png
...
is there a way that I can easily clip the file extension from the file name so that it only has the .png instead of the preceding .jpg as well?
or, better yet, can I include a counter so that I can run the command with something like this:
for $file in *.jpg using $count; do convert -resize 25% $file image$count.png; done
so that I end up with something like:
image1.png
image2.png
image3.png
image4.png
...
I'd rather not have to worry about creating a batch script, so if that is too hard and there is a simple way of just removing the .jpg inline, then I'm happy with that..
thanks!
EDIT:
I think this question is okay to be a question in it's own right because it also has a counter element.
for file in *.jpg; do convert -resize 25% $file ${file%.*}.png; done
$ ls *.jpg
DSCa_.jpg* DSCb_.jpg* DSCc_.jpg*
$ count=0; for file in *.jpg; do (( count++ )); convert -resize 25% "$file" "${file%\.*}$count.png"; done
$ ls *.png
DSCa_1.png DSCb_2.png DSCc_3.png
${file%\.*} uses parameter substitution to remove the shortest matching string from the right of $file. You can read more about parameter substitution here
Something like this?
i=image1.jpg
echo ${i/.jpg/.png}
image1.png
Also, you can use the command basename (strip suffixe from filename) :
for file in *.jpg; do base=$( basename ${file} .jpg); convert -resize 25% $file ${base}.png; done

bash: ./script.sh: Argument list too long

I have the script below which I run with from within a directory of images with ./checkres.sh * or even *.jpg, but it only works if there is a small number of files, less than 100 or so. Anything more and it stops with error "Argument list too long" and I have 100,000's of images to process.
I been looking all over the web for hours and tried dozens of different changes, using a while instead of a for loop and various other changes to variables but nothing seems to work.
Can anyone help solve this mystery as I'm new to bash scripting and have run out of ideas.
#!/bin/bash
for img in "$#"
do
export height=`identify -format %h "$img"`
export width=`identify -format %w "$img"`
let ratio=$width/$height
echo Image "$img" = [ $width x $height ] = $ratio
if [ $ratio == 0 ]
then
echo Portrait 0
convert "$img" -geometry 600x800 -format jpeg -quality 80 "$img"
else
echo Landscape 1
convert "$img" -geometry 800x600 -format jpeg -quality 80 "$img"
fi
You don't need to change anything about your script; just change how you invoke it:
find . -maxdepth 1 -name '*.jpg' -exec ./checkres.sh '{}' +
This will put only as many filenames on each invocation as will fit, running the script more than once as necessary.
In fact -- unless you change the calling convention, you can't fix this bug by changing anything about your script, since the problem is happening while the shell is trying to start it!
However, there's another workaround -- instead of defining this as an external script, you could define it as a shell function (in your ~/.bashrc or similar), which means it doesn't need to be started as an external command, meaning the operating system limits in question don't apply.
checkres() {
for img; do
...
done
}
checkres *.jpg
...is executed entirely within the shell, mooting any limits on the length of the command line which can be passed to an external command during its execution.
If you want to take a directory argument:
for img in "$1"/*; do
Quoting correctly this way will ensure that your code can deal with directory names containing spaces.

Resources