How to enhance the colors and contrast of an image - graphics

I need to process the first "Original" image to get something similar to the second "Enhanced" one. I applied some naif calculation and the new image has more contrast and more strong colors but in the higher color regions a color hole appears. I have no idea about image processing, it would be great if you can suggest me which concepts and/or algorithms I could apply to get the result without this problem.

Convert the image to the HSB (Hue, Saturation, Brightness) color space.
Multiply the saturation by some amount. Use a cutoff value if your platform requires it.
Example in Mathematica:
satMult = 4; (*saturation multiplier *)
imgHSB = ColorConvert[Import["http://i.imgur.com/8XkxR.jpg"], "HSB"];
cs = ColorSeparate[imgHSB]; (* separate in H, S and B*)
newSat = Image[ImageData[cs[[2]]] * satMult]; (* cs[[2]] is the saturation*)
ColorCombine[{cs[[1]], newSat, cs[[3]]}, "HSB"]] (* rebuild the image *)
A table increasing the saturation value:

The "holes" that you see in the processed picture are the darker areas of the original picture, which went to negative values with your darkening algorithm. I suspect these out of range values are then written to the new image as positive numbers, so they end up in the higher part of the brightness scale. For example, let's say a pixel value is 10, and you are substracting 12 from all pixels to darken them a bit. This pixel will underflow and become -2. When you write it back to the file, -2 gets represented as 0xfe in hex, and this is 254 if you take it as an unsigned number.
You should use an algorithm that keeps the pixel values within the valid range, or at least you should "clamp" the values to the valid range. A typical clamp function defined as a C macro would be:
#define clamp(p) (p < 0 ? 0 : (p > 255 ? 255 : p))
If you add the above macro to your processing function it will take care of the "holes", but instead you will now have dark colors in those places.
If you are ready for something a bit more advanced, here on Wikipedia they have the brightness and contrast formulas that are used by The GIMP. These which will do a pretty good job with your image if you choose the proper coefficients.

This wikipedia article does a good job of explaining histogram equalization for contrast enhancement.
Code for grayscale images:
unsigned char* EnhanceContrast(unsigned char* data, int width, int height)
{
int* cdf = (int*) calloc(256, sizeof(int));
for(int y = 0; y < height; y++) {
for(int x = 0; x < width; x++) {
int val = data[width*y + x];
cdf[val]++;
}
}
int cdf_min = cdf[0];
for(int i = 1; i < 256; i++) {
cdf[i] += cdf[i-1];
if(cdf[i] < cdf_min) {
cdf_min = cdf[i];
}
}
unsigned char* enhanced_data = (unsigned char*) malloc(width*height);
for(int y = 0; y < height; y++) {
for(int x = 0; x < width; x++) {
enhanced_data[width*y + x] = (int) round(cdf[data[width*y + x]] - cdf_min)*255.0/(width*height-cdf_min);
}
}
free(cdf);
return enhanced_data;
}

Related

Loss of data during the Inverse-FFT of an Image

I am using the following code to convert a Bitmap to Complex and vice versa.
Even though those were directly copied from Accord.NET framework, while testing these static methods, I have discovered that, repeated use of these static methods cause 'data-loss'. As a result, the end output/result becomes distorted.
public partial class ImageDataConverter
{
#region private static Complex[,] FromBitmapData(BitmapData bmpData)
private static Complex[,] ToComplex(BitmapData bmpData)
{
Complex[,] comp = null;
if (bmpData.PixelFormat == PixelFormat.Format8bppIndexed)
{
int width = bmpData.Width;
int height = bmpData.Height;
int offset = bmpData.Stride - (width * 1);//1 === 1 byte per pixel.
if ((!Tools.IsPowerOf2(width)) || (!Tools.IsPowerOf2(height)))
{
throw new Exception("Imager width and height should be n of 2.");
}
comp = new Complex[width, height];
unsafe
{
byte* src = (byte*)bmpData.Scan0.ToPointer();
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++, src++)
{
comp[y, x] = new Complex((float)*src / 255,
comp[y, x].Imaginary);
}
src += offset;
}
}
}
else
{
throw new Exception("EightBppIndexedImageRequired");
}
return comp;
}
#endregion
public static Complex[,] ToComplex(Bitmap bmp)
{
Complex[,] comp = null;
if (bmp.PixelFormat == PixelFormat.Format8bppIndexed)
{
BitmapData bmpData = bmp.LockBits( new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadOnly,
PixelFormat.Format8bppIndexed);
try
{
comp = ToComplex(bmpData);
}
finally
{
bmp.UnlockBits(bmpData);
}
}
else
{
throw new Exception("EightBppIndexedImageRequired");
}
return comp;
}
public static Bitmap ToBitmap(Complex[,] image, bool fourierTransformed)
{
int width = image.GetLength(0);
int height = image.GetLength(1);
Bitmap bmp = Imager.CreateGrayscaleImage(width, height);
BitmapData bmpData = bmp.LockBits(
new Rectangle(0, 0, width, height),
ImageLockMode.ReadWrite,
PixelFormat.Format8bppIndexed);
int offset = bmpData.Stride - width;
double scale = (fourierTransformed) ? Math.Sqrt(width * height) : 1;
unsafe
{
byte* address = (byte*)bmpData.Scan0.ToPointer();
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++, address++)
{
double min = System.Math.Min(255, image[y, x].Magnitude * scale * 255);
*address = (byte)System.Math.Max(0, min);
}
address += offset;
}
}
bmp.UnlockBits(bmpData);
return bmp;
}
}
(The DotNetFiddle link of the complete source code)
(ImageDataConverter)
Output:
As you can see, FFT is working correctly, but, I-FFT isn't.
That is because bitmap to complex and vice versa isn't working as expected.
What could be done to correct the ToComplex() and ToBitmap() functions so that they don't loss data?
I do not code in C# so handle this answer with extreme prejudice!
Just from a quick look I spotted few problems:
ToComplex()
Is converting BMP into 2D complex matrix. When you are converting you are leaving imaginary part unchanged, but at the start of the same function you have:
Complex[,] complex2D = null;
complex2D = new Complex[width, height];
So the imaginary parts are either undefined or zero depends on your complex class constructor. This means you are missing half of the data needed for reconstruction !!! You should restore the original complex matrix from 2 images one for real and second for imaginary part of the result.
ToBitmap()
You are saving magnitude which is I think sqrt( Re*Re + Im*Im ) so it is power spectrum not the original complex values and so you can not reconstruct back... You should store Re,Im in 2 separate images.
8bit per pixel
That is not much and can cause significant round off errors after FFT/IFFT so reconstruction can be really distorted.
[Edit1] Remedy
There are more options to repair this for example:
use floating complex matrix for computations and bitmap only for visualization.
This is the safest way because you avoid additional conversion round offs. This approach has the best precision. But you need to rewrite your DIP/CV algorithms to support complex domain matrices instead of bitmaps which require not small amount of work.
rewrite your conversions to support real and imaginary part images
Your conversion is really bad as it does not store/restore Real and Imaginary parts as it should and also it does not account for negative values (at least I do not see it instead they are cut down to zero which is WRONG). I would rewrite the conversion to this:
// conversion scales
float Re_ofset=256.0,Re_scale=512.0/255.0;
float Im_ofset=256.0,Im_scale=512.0/255.0;
private static Complex[,] ToComplex(BitmapData bmpRe,BitmapData bmpIm)
{
//...
byte* srcRe = (byte*)bmpRe.Scan0.ToPointer();
byte* srcIm = (byte*)bmpIm.Scan0.ToPointer();
complex c = new Complex(0.0,0.0);
// for each line
for (int y = 0; y < height; y++)
{
// for each pixel
for (int x = 0; x < width; x++, src++)
{
complex2D[y, x] = c;
c.Real = (float)*(srcRe*Re_scale)-Re_ofset;
c.Imaginary = (float)*(srcIm*Im_scale)-Im_ofset;
}
src += offset;
}
//...
}
public static Bitmap ToBitmapRe(Complex[,] complex2D)
{
//...
float Re = (complex2D[y, x].Real+Re_ofset)/Re_scale;
Re = min(Re,255.0);
Re = max(Re, 0.0);
*address = (byte)Re;
//...
}
public static Bitmap ToBitmapIm(Complex[,] complex2D)
{
//...
float Im = (complex2D[y, x].Imaginary+Im_ofset)/Im_scale;
Re = min(Im,255.0);
Re = max(Im, 0.0);
*address = (byte)Im;
//...
}
Where:
Re_ofset = min(complex2D[,].Real);
Im_ofset = min(complex2D[,].Imaginary);
Re_scale = (max(complex2D[,].Real )-min(complex2D[,].Real ))/255.0;
Im_scale = (max(complex2D[,].Imaginary)-min(complex2D[,].Imaginary))/255.0;
or cover bigger interval then the complex matrix values.
You can also encode both Real and Imaginary parts to single image for example first half of image could be Real and next the Imaginary part. In that case you do not need to change the function headers nor names at all .. but you would need to handle the images as 2 joined squares each with different meaning ...
You can also use RGB images where R = Real, B = Imaginary or any other encoding that suites you.
[Edit2] some examples to make my points more clear
example of approach #1
The image is in form of floating point 2D complex matrix and the images are created only for visualization. There is little rounding error this way. The values are not normalized so the range is <0.0,255.0> per pixel/cell at first but after transforms and scaling it could change greatly.
As you can see I added scaling so all pixels are multiplied by 315 to actually see anything because the FFT output values are small except of few cells. But only for visualization the complex matrix is unchanged.
example of approach #2
Well as I mentioned before you do not handle negative values, normalize values to range <0,1> and back by scaling and rounding off and using only 8 bits per pixel to store the sub results. I tried to simulate that with my code and here is what I got (using complex domain instead of wrongly used power spectrum like you did). Here C++ source only as an template example as you do not have the functions and classes behind it:
transform t;
cplx_2D c;
rgb2i(bmp0);
c.ld(bmp0,bmp0);
null_im(c);
c.mul(1.0/255.0);
c.mul(255.0); c.st(bmp0,bmp1); c.ld(bmp0,bmp1); i2iii(bmp0); i2iii(bmp1); c.mul(1.0/255.0);
bmp0->SaveToFile("_out0_Re.bmp");
bmp1->SaveToFile("_out0_Im.bmp");
t. DFFT(c,c);
c.wrap();
c.mul(255.0); c.st(bmp0,bmp1); c.ld(bmp0,bmp1); i2iii(bmp0); i2iii(bmp1); c.mul(1.0/255.0);
bmp0->SaveToFile("_out1_Re.bmp");
bmp1->SaveToFile("_out1_Im.bmp");
c.wrap();
t.iDFFT(c,c);
c.mul(255.0); c.st(bmp0,bmp1); c.ld(bmp0,bmp1); i2iii(bmp0); i2iii(bmp1); c.mul(1.0/255.0);
bmp0->SaveToFile("_out2_Re.bmp");
bmp1->SaveToFile("_out2_Im.bmp");
And here the sub results:
As you can see after the DFFT and wrap the image is really dark and most of the values are rounded off. So the result after unwrap and IDFFT is really pure.
Here some explanations to code:
c.st(bmpre,bmpim) is the same as your ToBitmap
c.ld(bmpre,bmpim) is the same as your ToComplex
c.mul(scale) multiplies complex matrix c by scale
rgb2i converts RGB to grayscale intensity <0,255>
i2iii converts grayscale intensity ro grayscale RGB image
I'm not really good in this puzzles but double check this dividing.
comp[y, x] = new Complex((float)*src / 255, comp[y, x].Imaginary);
You can loose precision as it is described here
Complex class definition in Remarks section.
May be this happens in your case.
Hope this helps.

Code explanation for bitmap conversion

https://stackoverflow.com/a/2574798/159072
public static Bitmap BitmapTo1Bpp(Bitmap img)
{
int w = img.Width;
int h = img.Height;
//
Bitmap bmp = new Bitmap(w, h, PixelFormat.Format1bppIndexed);
BitmapData data = bmp.LockBits(new Rectangle(0, 0, w, h), ImageLockMode.ReadWrite, PixelFormat.Format1bppIndexed);
Why this addition and division?
byte[] scan = new byte[(w + 7) / 8];
for (int y = 0; y < h; y++)
{
for (int x = 0; x < w; x++)
{////Why this condition check?
if (x % 8 == 0)
//Why divide by 8?
scan[x / 8] = 0;
Color c = img.GetPixel(x, y);
//Why this condition check?
if (c.GetBrightness() >= 0.5)
{
// What is going on here?
scan[x / 8] |= (byte)(0x80 >> (x % 8));
}
}
// Why Martial.Copy() called here?
Marshal.Copy(scan, 0, (IntPtr)((long)data.Scan0 + data.Stride * y), scan.Length);
}
bmp.UnlockBits(data);
return bmp;
}
The code uses some basic bit-hacking techniques, required because it needs to set bits and the minimum storage element you can address in C# is a byte. I intentionally avoided using the BitArray class.
int w = img.Width;
I copy the Width and Height properties of the bitmap into a local variable to speed up the code, the properties are too expensive. Keep in mind that w are the number of pixels across the bitmap, it represents the number of bits in the final image.
byte[] scan = new byte[(w + 7) / 8];
The scan variable stores the pixels in one scan line of the bitmap. The 1bpp format uses 1 bit per pixel so the total number of bytes in a scan line is w / 8. I add 7 to ensure the value is rounded up, necessary because integer division always truncates. w = 1..7 requires 1 byte, w = 8..15 requires 2 bytes, etcetera.
if (x % 8 == 0) scan[x / 8] = 0;
The x % 8 expression represents the bit number, x / 8 is the byte number. This code sets all the pixels to Black when it progresses to the next byte in the scan line. Another way to do it would be re-allocating the byte[] in the outer loop or resetting it back to 0 with a for-loop.
if (c.GetBrightness() >= 0.5)
The pixel should be set to White when the source pixel is bright enough. Otherwise it leaves it at Black. Using Color.Brightness is a simple way to avoid dealing with the human eye's non-linear perception of brightness (luminance ~= 0.299 * red + 0.587 * green + 0.114 * blue).
scan[x / 8] |= (byte)(0x80 >> (x % 8));
Sets a bit to White in the scan line. As noted x % 8 is the bit number, it shifts 0x80 to the right by the bit number, they are stored in reverse order in this pixel format.

Scrolling through colors effect in RGB

I want to create a colour scroller effect. I have a function that I give it RGB values (eg. setColor(189,234,45)) and I want to change the colour rapidly but I don't want to get many repeats to create an effect of scrolling through the colours.
I have tried tried the following but it doesn't quite generate the effect that I am looking for.
for (int i = 0; i < 256; i++) {
for (int j = 0; j < 256; j++) {
for (int k = 0; k < 256; k++) {
setColor(i, j, k);
}
}
}
I wanted to know if anyone knows how the colour scroller's colours are arranged next to each other. The arrangement I am looking for looks like the scroll on the right.
The colors you are working with are represented as R,G,B (red green blue) values. However, another
way to think about color is hue, saturation, value. In the scroll image you are trying to emulate,
it is the hue that is changing - the saturation and value (brightness) are unaffected.
Here is a function that happens to make a hue-cycle gradient like the one in the image you linked to:
int n = 256; // number of steps
float TWO_PI = 3.14159*2;
for (int i = 0; i < n; ++i) {
int red = 128 + sin(i*TWO_PI/n + 0) + 127;
int grn = 128 + sin(i*TWO_PI/n + TWO_PI/3) + 127;
int blu = 128 + sin(i*TWO_PI/n + 2*TWO_PI/3) + 127;
setColor(red, grn, blu);
}
To understand how that function works, I recommend that you read my color tutorial that GreenAsJade linked to.
However, that kind of gradient function isn't quite what you need, because you want to start from a particular color you are passing in, and then go to the next color in the sequence. It's much easier to do this kind of thing if you represent your colors as HSV triplets (or HSB triplets), instead of RGB triplets. Then you can manipulate just the hue component, and get those kind of rainbow effects. In helps to have a set of function that can convert from RGB to HSV and back again.
This site contains a bunch of color conversion source code, including the ones you need for those conversions. Using the two conversion functions supplied on that page, your code might look like:
void cycleMyColor(int *r, int *g, int *b) {
float h,s,v, fr,fg,fb;
RGBtoHSV(*r/255.0,*g/255.0,*b/255.0,&h,&s,&v);
h += 1/256.0; // increment the hue here
h -= (int) h; // and cycle around if necessary
HSVtoRGB(&fr,&fg,&fb,h,s,v);
*r = fr*255; *g = fg*255; *b = fb*255;
setColor(*r,*g,*b);
}
This code is a little more complicated than it needs to be because the color conversions on that site use floating point color components that go from 0-1, instead of integers that go from 0-255, as you were using, so I'm spending a few lines converting between those two representations. You may find it simpler to just keep your color in HSB space, and then convert to RGB when you want to display it.
As you mentioned in your edit, you don't like the sequence of colours, because you start from black an end at white, instead of starting at one end of the rainbow and going to the other.
So you are going to need to work out a sequence of RGB that goes from blue through green and yellow to red. That means you need to start with (0,0,255) and end at (255, 0, 0), and don't pass through (255,255,255) or (0,0,0) - in a nutshell, that's how its done.
There are many ways you could do this and get a pleasing effect - beyond the scope of an answer here. This article explores it in depth:
http://krazydad.com/tutorials/makecolors.php

Project Tango Point Cloud strange crash, and dense depth map

I am trying to use Project Tango C API, but the application crashed with no error if number of point cloud are more than ~6.5k (after some testing) with the following code
int width = mImageSource->getDepthImageSize().x;
int height = mImageSource->getDepthImageSize().y;
double fx = mImageSource->calib.intrinsics_d.projectionParamsSimple.fx;
double fy = mImageSource->calib.intrinsics_d.projectionParamsSimple.fy;
double cx = mImageSource->calib.intrinsics_d.projectionParamsSimple.px;
double cy = mImageSource->calib.intrinsics_d.projectionParamsSimple.py;
memset(inputRawDepthImage->GetData(MEMORYDEVICE_CPU), -1, sizeof(short)*width*height);
for (int i = 0; i < XYZ_ij->xyz_count; i++) {
float X = XYZ_ij->xyz[i*3][0];
float Y = XYZ_ij->xyz[i*3][1];
float Z = XYZ_ij->xyz[i*3][2];
if (Z < EPSILON || (X < EPSILON && -X < EPSILON) || (Y < EPSILON && -Y < EPSILON) || X != X || Y != Y || Z != Z)
continue;
int x_2d = (int)(fx*X/Z+cx);
int y_2d = (int)(fy*Y/Z+cy);
if (x_2d >=0 && x_2d < width && y_2d >= 0 && y_2d < height && (x_2d != 0 || x_2d != 0)) {
inputRawDepthImage->GetData(MEMORYDEVICE_CPU)[x_2d + y_2d*width] = (short) (Z*1000);
} else {
continue;
}
}
However, if I use for (int i = 0; i < XYZ_ij->xyz_count && i < 6500; i++) everything works fine. I am just wondering if there is an upper bound for access point cloud with C API or I did something wrong?
(width is 320, height is 180, and other intrinsics are loaded from Tango API)
In addition, Google mentioned to use nearest- neighbor filter to get dense depth map in bottom of this page, is there an interface in Tango API for this? Or would anyone suggest an open source implementation for it.
I am also wondering if there is anyway to "pull" colored image(1280x720) in onXYZijAvailable because I need a dense synchronized colored point cloud. Do I need to apply external matrix to align both coordinate frame, or I only need to subsample color image (assume their coordinate system are the same)?
Thank you for any advice!
In your code that looks up the depth sample coordinates...
for (int i = 0; i < XYZ_ij->xyz_count; i++) {
float X = XYZ_ij->xyz[i*3][0];
float Y = XYZ_ij->xyz[i*3][1];
float Z = XYZ_ij->xyz[i*3][2];
...you should be using an index of i, not i*3. It is a 2D array so you don't have to manage the stride for the higher dimension yourself.
The SDK does not provide a call to fill in locations with no depth samples, probably because there are many approaches with different tradeoffs. The Wikipedia page on nearest neighbor search is a reasonable place to start. There is an interface to FLANN in OpenCV.
The SDK will only deliver the latest color image to you. If you want a prior image (e.g. with a timestamp close to your depth samples) you will have to manage that yourself. Because you can never get a color image at exactly the same timestamp with your depth samples (as the same camera is used in different modes for both), you theoretically should apply the extrinsic pose to align them. In practice if the motion is small over the 0.5 frame time or less between the timestamps, I think most people are going to ignore it.

Average color (X11 colors)

I want to fill the intersection of two(or more filled) rectangles with the average color. I have the colors of each rectangle stored as unsigned ints. How can I get the average color?
Thank you for you help!
Technically, you might be running on a color-map device, which means you need to go through X11 color management for all of this. You need to query the XColor for your two input colors, compute the average, then look up the closest representable color:
// Query XColor for both input colors
XColor xcol1, xcol2, outcol;
xcol1.pixel = color1;
xcol2.pixel = color2;
XQueryColor(display, colormap, &xcol1);
XQueryColor(display, colormap, &xcol2);
// Average red/green/blue and look up nearest representable color
outcol.red = (xcol1.red + xcol2.red) / 2;
outcol.green = (xcol1.green + xcol2.green) / 2;
outcol.blue = (xcol1.blue + xcol2.blue) / 2;
XAllocColor(display, colormap, &outcol);
// outcol.pixel is now the color to use
On a paletted device, you also need to free the color afterwards etc. - it's a mess, basically.
But in all likelihood you're on a 32-bit truecolor device, which means the integer is just a bitfield of r, g, b and a (not necessarily in that order). You can compute their average like this:
UInt out_color = 0;
for (int i=0; i < 4; i++) {
// Extract channel i from both input colors
UInt in1 = (color1 >> (i*8)) & 0xff;
UInt in2 = (color2 >> (i*8)) & 0xff;
// Compute the average and or it into the output color
out_color |= ((in1 + in2) / 2) << (i*8);
}
Color color1 = Color.FromArgb(UInt1);
Color color2 = Color.FromArgb(UInt2);
Color averageColor = Color.FromArgb(255,(color1.R + color2.R)/2,(color1.G + color2.G)/2,(color1.B + color2.B)/2);
This is assuming that you need a fully opaque average color.

Resources