I wish to exploit redundancy among a set of similar colored JPG images. Set redundancy compression has been used successfully for similar 8-bit grayscale images. They basically find the MAX and MIN of a set of images and encode the original images as differences with respect to either the MAX or MIN image, depending on whichever is the smaller difference. About 20-50% additional compression has been obtained for grayscale images using this approach besides normal compression tools like gzip or bzip. I do the following:
Decompress the JPG images to RGB char buffers
Compute the MIN and MAX char buffers
Encode the difference char buffers as JPG images
The problem is that to retrieve the original image from the difference image, ideally I would need to encode the difference losslessly. But, there is no lossless transformation from RGB->JPG in libjpeg and even at quality=1.0, because the difference coefficients are small (1~10), I end up losing almost all the information (almost all decoded data is 1~3).
To solve this, I tried using huffman encoding of the difference char buffers.
Encode the difference char buffers using Huffman encoding
The original JPG images are of size ~256 KB, the corresponding RGB buffer is ~7.8 MB and the huffman encoded difference images are of size ~2.2 MB. So, Huffman encoding does save lot of space w.r.t RGB buffers but the original JPG equivalents are much smaller.
Any suggestions to store these difference buffers efficiently comparable in size to the 256 KB original JPG files ?
Is there a way to save these difference buffers with low valued data coefficients as JPG images without losing substantial information ?
Related
I just tried to convert few JPEGs to a GIF image using some online services. For a collection of 1.8 MB of randomly selected JPEGs, the resultant GIF was about 3.8 MB in size (without any extra compression enabled).
I understand GIF is lossless compression. And that's why I expected the resultant output to be around 1.8 MB (input size). Can someone please help me understand what's happening with this extra space ?
Additionally, is there a better way to bundle a set of images which are similar to each other (for transmission) ?
JPEG is a lossy compressed file, but still it is compressed. When it uncompresses into raw pixel data and then recompressed into GIF, it is logical to get that bigger a size
GIF is worse as a compression method for photographs, it is suited for flat colored drawings mostly. It uses RLE [run-length encoding] if I remember well, that is you get entries in the compressed file that mean "repeat this value N times", so you need to have lots of same colored pixels in horizontal sequence to get good compression.
If you have images that are similar to each other, maybe you should consider packing them as consequtive frames (the more similar should be closer) of a video stream and use some lossless compressor (or even risk it with a lossy one) for video, but maybe this is an overkill.
If you have a color image, multiply the width x height x 3. That is the normal size of the uncompressed image data.
GIF and JPEG are two difference methods for compressing that data. GIF uses the LZW method of compression. In that method the encoder creates a dictionary of previously encountered data sequences. The encoder write codes representing sequences rather than the actual data. This can actual result in an file larger than the actual image data if the encode cannot find such sequences.
These GIF sequences are more likely to occur in drawing where the same colors are used, rather than in photographic images where the color varies subtly through out.
JPEG uses a series of compression steps. These have the drawback that you might not get out exactly what you put in. The first of these is conversion from RGB to YCbCr. There is not a 1-to-1 mapping between these colorspaces so modification can occur there.
Next is subsampling.The reason for going to YCbCr is that you can sample the Cb and Cr components at a lower rate than the Y component and still get good representation of the original image. If you do 1 Y to 4 Cb and 4 Cr you reduce the amount of data to compress by half.
Next is the discrete cosine transform. This is a real number calculation performed on integers. That can produce rounding errors.
Next is quantization. In this step less significant values from the DCT are discarded (less data to compress). It also introduces errors from integer division.
I used cwebp to convert my jpg image to web. Now I am using dwebp to convert it back but its increasing in size from original one. Is there any way to control the file size in dwebp.
Transcoding between lossy formats tends to increase the size unless the representation of data happens to be extremely compatible between the formats, be it audio, pictures, video or other lossy data. WebP uses a 4x4 Hadamard transform, whereas JPEG uses an 8x8 Discrete Cosine Transform (DCT). Quantization, which is the main form of data loss in these formats, produces different kind of artefacts in these transformations, and transcoding cannot be optimal. Particularly, if either WebP or JPEG was saved with extremely low quality, the other format will struggle to compete with it after transcoding -- the later format will not only have to codify the image signal, but the resulting artefacts from the other format, too.
So, while there is an inherent tendency for an increase in file size in such back-and-forth conversion, the exact amount of loss happening at every stage can be controlled. Which flags and tools (including versions) are you using exactly?
I have read that for reading a compressed image into memory it must be decompressed first and the original size will be allocated. So, since it is decompressed and it's size before compression is allocated in memory, why is the resolution of a compressed image when viewed with a certain software not the same of the original one?
Compression is not about reducing the resolution, or size in pixels, of an image. Compression is about reducing the amount of bytes required to represent a specific image.
You can see an image as an array of 4-byte structures, one for each pixel, where each byte represents one of the components of the color of each pixel, namely red, green, blue and alpha. The size required to represent an image with this scheme is
width * height * 4
So a 100x100 pixel image would have 10000 pixels, and therefore consume 40000 bytes. This is in fact, roughly the way in which the BMP format stores images.
However, this is not the only way you can represent those 10000 pixels. If, for example, the first 5000 pixels are blue, and the bottom pixels are brown, you could represent the image by saying something like "blue: 5000, brown 5000", and that would take much fewer bytes to represent. This scheme is roughly how RLE (Run Length Encoding) works, and is widely used in many formats such as GIF.
However, there is only so much you can do to reduce the amount of bytes required to represent the bytes in your image. The data is not always easy to represent with fewer bytes, so what some compression algorithms, like the one used in JPEG files do, is to modify the pixels just a bit, so the data is much easier to compress, yet the changes are not very noticeable. If such changes are acceptable, it is possible to achieve impressive results when compressing the image. This is what is called "lossy compression".
The entire point of compressing images is to make it easier to move images from one place to another, be it by storing them on a disc or sending them over the internet. However, when you are going to display an image, your computer has to tell the monitor what color each pixel has to get, so once you are going to display the image, you need to decompress it.
Technically, decompressing a compressed image does maintain the quality of the compressed image. However compressing an image may reduce quality (ie. the compressed image is slightly degraded from the original image).
The specific changes to the image depend on what sort of compression you apply; that said, I'm not familiar with any sort of compression that would change the resolution or size of the image. Are you sure you got that right?
I'm trying to understand the JPEG compression process and performed the following steps to verify a few things.
I take an input image img1.jpg and compress it by using IrfanView, say quality=50 (img1_compress.jpg).
Then I crop a small block from the input image img1.jpg (block.jpg of size 8x8 at X,Y=16,16) and compress it by using the same value of quality parameter (50). Let's call it block_compress.jpg.
Now when I compare this block's pixel values with the one in fully compressed image, they don't match.
To clarify, the pixel value at position 0,0 in block_compress.jpg should match with the pixel value at position 16,16 in img1_compress.jpg.
I'm confused why pixel values don't match? Any ideas?
I just did this experiment with my JPEG codec and the pixel values match. Irfanview may be applying some kind of noise filter or other modifications when it compresses JPEG images. Without seeing the source code to the codec you can't know what it's doing. Your experiment is valid, but by using other people's code to test your theory you can't know what's really going on inside their code.
JPEG is lossy compression algorithm. Compressing one image with identical compression settings in different tools can produce differ result. You need use one of lossless algorithms if you want pixel-to-pixel result. I.e. you can use PNG
"the DC component of each 8x8 block is predicted from the previous block.” : by Oli Charlesworth
I'm going to let users upload images of 300x300 compressed with JPEG. Is there a way to determine what the maximum file size of such an image would be?
I can imagine this can be tried by compressing random noise at 100 quality, but is there a theoretical maximum?
Say that the image is totally uncompressable random noise, could it be 3 bytes per pixel (24-bits colour) and a margin for the metadata? Or could such an image turn out larger than the original when compressed?
From wikipedia:
For highest quality images (Q=100), about 8.25 bits per color pixel is required
http://en.wikipedia.org/wiki/JPEG#Sample_photographs
So, for Q=100 on an 300x300 image, that would result in (300 * 300) px * 8.25 bits/px = 742,500 bits = ~ 93 kB
There are also lossless JPEG coding modes, which are practically not used (last sentence, second paragraph). But they would have the RGB typical 24 bits/pixel.
There is no limit on jpeg metadata size, which means there's no limit to the size of a jpeg file. See this answer I've linked for an explanation of why and also for an example of a realistic situation where the metadata might get large: What is the maximum size of JPEG metadata?
So if assuming a maximum practical/realistic size suits your purpose, then you should factor that example into your calculations. In many contexts it would be fine to just reject things outside of that maximum as outside the domain of your program's intended usage.
But if you absolutely must rely on theoretical, then unfortunately it's a big bold ∞
Note: I do not have a huge amount of personal experience with the jpeg specification, so I am going off of what people have said about repeated fields being allowed, as well as multiple comment fields. Please correct me if you find evidence to the contrary.