I have tens of thousands of png images that are very similar to each other and I would like to archive them and compress them in the process. I am aware that png files can barely be compressed because it already is a compressed file format. In my case though, these are similar to each other which is why I thought that there may be some program out there that takes advantage of that. Any hints?
EDIT: example image: https://imgur.com/a/N9csZZH
Video compression also tries to take advantage of similar images to compress smaller. You should try a lossless video compression codec such as WebM.
I think it does not matter for PNG or other lossless formats how similar they appear for humans when you want to compress them (in a single tar for example). Only a few bits or pixels difference make each image mathematically a totally different object. When a high compression of zstd cannot do the trick, you won't need to search anymore. You cannot outperform entropy. There is a mathematical limit on compression, and zstd comes close to that.
Related
I want to reduce the size of jpg's I use on my website. Is there a way to reduce the size of jpg files so that I can reduce the data transfer charges without reducing much of clarity? I am hoping to do this without uploading my files somewhere.
[Nearly] Any application of JPEG is going to distort the image from the original. You can adjust the compression settings to balance compression to image distortion. The amount of compression you can get without visible distortion depends upon the type of image. If you have a cartoon with sharp color transitions, you are going to quickly see distortions with JPEG.
Things you can do to change compression:
Change the quantization tables ("Quality Settings" in many encoders)
Subsample the Cb and Cr components.
Use optimal Huffman tables (has no visible effect the image)
The best way to shrink JPG files on web is to use shrinkjpeg.com. The reason I recommend shrinkjpeg.com is because the site compresses JPG Images without actually uploading the files to their server (Who knew HTML5 magic can do this locally?)
I am currently using "convert" to convert a set of png files to an animated GIF:
system("convert -delay 4 -loop 0 /tmp/$$/*.png $out");
(from a perl script).
This turns out to be quite slow if there are too many PNG files, but it needs to be done "on-demand" from the web. It can take 10 seconds or so, during which the user has to wait.
Is there a better, faster way of concatenating all the PNG files? (Is this slow because of compression to GIF, if so, I don't mind using a faster compression scheme.)
I need to work with tiff images online. Tiff images are not supported by browsers. So i thought maybe i can convert them on the fly and stream them into the browser as pngs.
I found many image processing haskell libraries and JuicyPixels looks simple enough and supports reading from tiff and saving to many other formats including png.
The simplest case is to just save to png file and then serve it with sendFile
But i think involving hard drive in the process is going to add too much overhead and substantially slow down the response. SO my question is, how do i stream the image converted with JuicyPixels from tiff to png directly, without saving it into a file first.
Does JuicyPixels have any streaming interfaces? Or maybe there's a simple enough way to get to data representation in specific format and then pass it to any streaming libraries like conduit?
As i side question, anyone did streaming images from Yesod?
I don't have any experience with JuicyPixels, but it looks like it encodes to lazy ByteStrings. If that's the case, then you just need to return that lazy ByteString wrapped up in a DontFullyEvaluate.
I would like to compress scanned text (monochrome or few colours) and store it in pdf (maybe djvu) files. I remember that I got very good results with Windows/Acrobat and "ZRLE" compressed monochrome tiff embedded into pdf. The algorithm was loossless as far as I remember. Now I search a way to obtain good results on linux. It should be storage saving and avoid loss (I do not mind loosing colours, but I do not want e.g. jpeg compression which would create noisy results for text scans). I need it for batch conversion, so I was thinking of the ImageMagick convert command. But which output format should I use so I get good results and to be able to embed it into pdf files (for example using pdflatex)? Or is it generally better to use djvu files?
jbig2enc encoder for images using jbig2 compression,
was originally written for GoogleBooks by Adam Langley
https://github.com/agl/jbig2enc
I forked to include latest improvements By Rubypdf and others
https://github.com/DingoDog/jbig2enc
I also built several binaries of jbig2enc for puppy linux (it can be working also on other distributions)
http://dokupuppylinux.info/programs:encoders
DJVU is not a bad choice, but if you want to stay in PDF for better compatibility you may want to look into lossless JBIG2 compression.
Quote from Wikipedia:
Overall, the algorithm used by JBIG2 to compress text is very similar
to the JB2 compression scheme used in the DjVu file format for coding
binary images.
What is the difference between compressed and uncompressed .wav files?
The WAV format is a container format for audio files in Windows.
The WAV file consists of a header and the contents. The header contains information about the size, duration, sampling frequency, resolution, and other information about the audio contained in the WAV file. Generally, after the header is the actual audio data.
Since WAV is a container format, the data it contains can be stored in various formats. One of which is uncompressed PCM, but it can also store ADPCM, MP3 and other formats, and can be read and written if an audio codec for the format is available.
The difference between compressed and uncompressed WAV files is that the data contained within the WAV file is either uncompressed raw audio samples, or it is compressed using an audio codec, in which case, it must be decompressed before it can be played back.
Further reading:
Wikipedia: Audio compression (data)
Wikipedia: WAV
Wikipedia: Codec
There's a great explanation here. The basic difference is that an uncompressed wave file has just the raw bits in it as they "appear". There is nothing done to compress or shrink them. A compressed wave file uses some sort of codec to shrink down the data before putting it in the file.
The difference between these two things is basically in the size of object, the compressed one might have low size compared to uncompressed basically the content are the same.
You have to be very careful when using the word "uncompressed" when talking about media.
Basically ALL digital media is compressed in some way. Audio, or video. No matter what it is, it is compressed in some way. Its intrinsic to converting from analog to digital.
The problem isn't really technical, its lingual.
People think that uncompressed means "nothing done to it" when in reality there really isnt any way you can do this. There is always some kind of compression done when you convert the analog signal coming out of the mic and going into a file...Its essential.
What uncompressed means is very high quality. And different "Uncompressed" codecs do things differently.
I know more about video codecs, so i will base my example in those.
Black Magic (A company that makes video Out Cards) has an Uncompressed Codec. Its very good. Makes Beautiful images.. But its not really "uncompressed". Sure its big. But compare it to a DPX of TIFF image sequence...and it aint that big, and is quite compressed. Its only 10 bit, but something like an OpenEXR image sequence is like 32 bit...and coming from film, that is still technically compressed. It has to be.
Its just the nature of the beast.