How to get moving object's mask using OpenCV BackgroundSubtractorMOG2 - object

I want to mask the moving objects from video.
I found that OpenCV has some built-in BackgroundSubtractors which could possibly saving my time a lot. However, according to the official reference, the function:
void BackgroundSubtractorMOG2::operator()(InputArray image, OutputArray fgmask, double learningRate=-1)
should output a mask, fgmask, but it doesn't. The fgmask variable will contain the "contour of the mask" instead after invoking above method. That's weird. All I want is a simple closed region filled with white color(for example) to represent the moving objects. How could I do that?
Any reply or recommendation would be very appreciate. Thanks a lot.
Here's my code:
int main(int argc, char *argv[])
{
cv::BackgroundSubtractorMOG2 bg = BackgroundSubtractorMOG2(30,16.0,false);
cv::VideoCapture cap(0);
cv::Mat frame, mask, _frame, _fmask;
cvNamedWindow("mask", CV_WINDOW_AUTOSIZE);
for(;;)
{
cap >> frame;
bg(frame,fmask,-1);
_frame = IplImage(frame);
_fmask = IplImage(fmask);
cvShowImage("mask", &_fmask);
if(cv::waitKey(30) >= 0) break;
}
return 0;
}
A snapshot of the output video is:
p.s. My working environment is OpenCV2.4.3 on OSX 10.8 and XCode 4.5.2 with apple LLVM compiler 4.1.

If you want to acquire the whole objects filled with white pixels in the foreground then I would ask you to tell me something about your experience.
My question is, for the code, you mentioned above, do you get more white pixels when you generate more motion in front of your camera?
If yes then there are two paramenters to learn about for your requirement.
First is the History parameter. which you have configured as 30 in the constructor BackgroundSubtractorMOG2(30,16.0,false);. You can test this param by incresing, say to 300. It will maintain the motion history of the object in the foreground. So if you have moved completely from your starting location within the 300 frames then you will get whole object covered with white pixels as you want. but it will be erased gradually. So it cannot give you the 100% solution.
The second parameter is called learning rate. In the code you mentioned bg(frame,fmask,-1); where -1 is your learning rate. you can set it to 0.0 to 1.0 and default is -1. When you set it 0, you will get what you want for the objects which are not part of the frame in the starting of the video. You can call this kind of object "foreign objects". You will get foreign object covered with white pixels.
Explore your testing from the information I have mentioned above and share your experience.

Related

Vuforia video playback with fixed dimension

I am using vuforia video playback demo with cloud recognition.
I have combined both projects and it is working properly. But currently video dimension is according to detected object. But i need fixed width and height when video plays.
Can anyone help me ?
Thanks in advance.
Well apparently Vuforia fixes the width and height at the start of the game no matter what the size of the object is. I could not find when exactly this operation is conducted but it is done at beginning of your game. When you change the size of the ImageTarget in runtime it is not fixed anymore. Add these lines to your OnTrackingFound function of DefaultTrackableEventHandler.cs
if (this.name == "WhateverTheNameOfYourRelatedImageTarget"&& !isScaled)
{
//Increase the size however you want i just added 1 to each dimension
this.transform.localScale += Vector3.one;
// make isScaled true not to scale every time it is found initially it shoud be false
isScaled = true;
}
Good luck!
What i usually do is , instead of Videoplayback , i play the video on canvas object and hook this object to Defaulttrackableeventhandler script. So when target is found, gameobject.setactive(true) and gameobject.setactive(false) when target is lost. By this method the size of the gameobject is fixed and it stays in a fixed location .
I just made an example you can get here (have to import it to any project and open the scene Assets/VideoExample/Examples). You can see there a bit clearer what the ScreenSpace - Overlay does ... it might be better to just switch to ScreenSpace - Camera in general

Linux: Xautomation with fake mouse pointer

I have created a fake mouse pointer using the xinput command as outlined here which produces a second pointer that hovers in the center of my screen.
I would now like to automate it using the xte command, but unfortunately xte only seems grab control of the hardware mouse that I wish to keep free.
The man page for xte does not have any flags to specify which pointer to take control of.
I was wondering if anybody had any ideas?
NB: The second pointer is purely for me to able to work on the same computer whilst running the graphical pipeline
Edit: So by looking at the xte source I've found references to XQueryPointer
Bool XQueryPointer(display, w, root_return, child_return, root_x_return, root_y_return,
win_x_return, win_y_return, mask_return)
Display *display;
Window w;
Window *root_return, *child_return;
int *root_x_return, *root_y_return;
int *win_x_return, *win_y_return;
unsigned int *mask_return;
//Arguments:
display Specifies the connection to the X server.
w Specifies the window.
root_return Returns the root window that the pointer is in.
child_return Returns the child window that the pointer is located in, if any.
root_x_return
root_y_return Return the pointer coordinates relative to the root window's origin.
win_x_return
win_y_return Return the pointer coordinates relative to the specified window.
mask_return Returns the current state of the modifier keys and pointer buttons.
from the Xlib class, which as you can see returns only the first mouse pointer and does not give option for another.
Edit2: Looking through the libx11-dev source I'm finding mentions of it at ./src/QuPntr.c and Xlibint.h, but the code is getting harder to read and I'm out of my depth here
xinput makes use of the XI2 extension to allow for multiple pointers.
This library (X11/extensions/XInput2.h) provides functions prefixed by "XI" which accept a device parameter (the same referenced by xinput --list), for instance XIQueryPointer, XIWarpPointer or XIGetClientPointer.
Thus you can control your fake pointer programmatically if your automation tool doesn’t support XI2.
In a simpler way, it could suffice to xinput --reattach the hardware device (a slave device) to the new pointer (a master device).
Very late answer I know, but I came across this question via web search so I add what I know for reference.
Actually there isn't any mention of XQueryPointer inside the source code of xte.c (there are only a few inside xmousepos.c, which is code for a different tool).
I checked a few older versions that existed before you asked the question.
The relevant functions are XIWarpPointer and XTestFakeDeviceButtonEvent. As seen here in the reduced source code:
#include <X11/Xlib.h>
#include <X11/extensions/XInput2.h>
#include <X11/extensions/XTest.h>
int main (int argc, char *argv[])
{
int delta_x = 500, delta_y = 160;
Display *display = XOpenDisplay(0);
Window root = DefaultRootWindow(display);
XIWarpPointer(display, 14, None, root, 0, 0, 0, 0, delta_x, delta_y);
XDevice *device = NULL;
device = XOpenDevice(display, 16);
XTestFakeDeviceButtonEvent(display, device, 1, True, 0, 0, CurrentTime);
XTestFakeDeviceButtonEvent(display, device, 1, False, 0, 0, CurrentTime);
XCloseDisplay(display);
}
which does the same thing as xte -i 16 "mousemove 500 160" "mouseclick 1".
But what might bother you (at least it bothers me): Windows that got clicked gain focus, effectively unfocusing the window that you were working on.
Actually I asked a question that asks for an alternative to XTestFakeDeviceButtonEvent just recently (Sending a button press event for a second pointing device; I also provided some example code that actually performs mouse clicks without giving focus to a window, so it's possible but dunno how it's done with another pointing device)..
An alternative might be to work with multiple X sessions like this user did it: Controlling multiple pointers with Xlib or xinput in ubuntu/linux. Sadly he hasn't shared any code.
I have some hope with uinput (https://www.kernel.org/doc/html/v4.12/input/uinput.html) but haven't tried it yet.

OpenCV store image in Object (OOP)

I'm programming in C++ using OpenCV in an object oriented approach. Basically I have an array of object called People[8]. For each array, I want to allocate an image to it by taking a picture using webcam. I did something like this:
for (int i=0; i<8; i++){
cvWaitKey(0); //wait for input then take picture
Mat grabbed = cam1.CamCapture();
People[i].setImage(grabbed);
imshow("picture", grabbed);
cvWaitKey(1);
}
I face 2 problems here:
1) The imshow does not display the 'latest' image captured, it display the image previously taken i.e (i-1) instead of i.
2) When I display all the images together, 8 windows appear and all of them are displaying the last image captured on the camera.
I do not have any clue what is wrong, could anyone please advice? Thank you in advance.
"all of them are displaying the last image captured on the camera."
the images you get from the capture point to driver memory. so the former image gets overwritten by the latter.
you need to store a clone() of the mat you get, like:
People[i].setImage( grabbed.clone() );
I have not worked with OpenCV for a while but I would move around cvWaitKey( 1 ), I also would not have 2 calls to it, from what I remember it is similar to glFlush(). Also I would change 1 to 10. For some reason I remember 1 not working.

How to print DIB backbuffer on printer - GDI, MFC

I'm using MFC's doc/view architecture to implement printing. I use double buffering, I draw everything onto my backbuffer which is DIB bitmap. Than I use StretchBlt to copy that DIB onto printer DC.
The strange thing is - print preview is working well! When I print on virtual PDF printer, it is working well! But when I print on actual printer (I'm testing on two different printers - same results) - it just prints "garbage". The "garbage" means sometimes it prints totally black page, sometimes it prints the first few pages repeatedly, i.e. it prints wrong part of DIB, just like if I messed up coordinates to StretchBlt, but I didn't mess anything up, I checked multiple times, plus why is print preview is working flawlessly then?
I tried many variations:
Using memory DC compatible to screen DC, when printing.
Using memory DC compatible to printer DC, and selecting my DIB into it.
Using memory DC compatible to printer DC, and using dedicated DIB onto which I copy my original backbuffer DIB.
etc.
But the results are same. Below is the code where i create the DIB. I think the DIB format might be the problem, so please advice if there is something wrong with it. I tried both 24 bits and 32 bits as values for bmiHeader.biBitCount.
// Setup proper backbuffer:
_CleanupBackBufferStuff();
_pMemDc = new CDC;
_pMemDc->CreateCompatibleDC(&aDC);
BITMAPINFO bmi;
memset(&bmi, 0, sizeof(BITMAPINFO));
bmi.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
bmi.bmiHeader.biWidth = _sizeBackBuffer.cx;
bmi.bmiHeader.biHeight = -_sizeBackBuffer.cy; // top-down
bmi.bmiHeader.biPlanes = 1;
bmi.bmiHeader.biBitCount = 24; // Tried 32 as well
bmi.bmiHeader.biCompression = BI_RGB;
unsigned char *pBitmapRawBits = 0;
HANDLE hMemBitmap = CreateDIBSection(aDC.GetSafeHdc(), &bmi, DIB_RGB_COLORS, (void**)&pBitmapRawBits, 0, 0);
_hOldSelBitmap = (HBITMAP)_pMemDc->SelectObject(hMemBitmap);
Also here is the code for StretchBlt (nothing special here):
pDC->SetStretchBltMode(HALFTONE);
SetBrushOrgEx(pDC->GetSafeHdc(), 0, 0, 0);
BOOL bSuccess = pDC->StretchBlt(rectClipBoxPlayground.left, rectClipBoxPlayground.top, rectClipBoxPlayground.Width(), rectClipBoxPlayground.Height(),
_pMemDc, rectClipBoxBackBuffer.left, rectClipBoxBackBuffer.top, rectClipBoxBackBuffer.Width(), rectClipBoxBackBuffer.Height(), SRCCOPY);
StretchBlt returns true, also (pDC->GetDeviceCaps(RASTERCAPS) & RC_STRETCHBLT) is true as well.
UPDATE: After Adrian's comment, I changed my code to use StretchDIBits. The problem is still the same! Below is the code I'm using currently:
// Copy back buffer to screen dc:
pDC->SetStretchBltMode(HALFTONE);
SetBrushOrgEx(pDC->GetSafeHdc(), 0, 0, 0);
HBITMAP hMemBitmap = (HBITMAP)_pMemDc->SelectObject(_hOldSelBitmap);
DWORD dwLines = StretchDIBits(pDC->GetSafeHdc(),
rectClipBoxPlayground.left, rectClipBoxPlayground.top, rectClipBoxPlayground.Width(), rectClipBoxPlayground.Height(),
rectClipBoxBackBuffer.left, _sizeBackBuffer.cy - rectClipBoxBackBuffer.top - rectClipBoxBackBuffer.Height(), rectClipBoxBackBuffer.Width(), rectClipBoxBackBuffer.Height(),
_pBitmapRawBits, &_bitmapInfo, DIB_RGB_COLORS, SRCCOPY);
_pMemDc->SelectObject(hMemBitmap);
It still behaives like the source coordinates are incorrect. It either prints one of first few pages (no matter what page I select), or prints almost-fully-black pages. The print preview is working perfectly, so this makes me think there should be no problems with my coordinate-calculation code. It works in preview, it works with virtual (pdf) printer, it fails when printing on actual printer. What the hell?....
Make sure you don't have the DIBSECTION selected into more than one DC at a time. That can cause all sorts of unpredictable behavior.
For printing, you can probably bypass the memory DC altogether if you keep your bmi and pBitmapRawBits handy. Make sure the DIBSECTION is not selected into any DC, and then call SetDIBitsToDevice or StretchDIBits to transfer the image to the printer DC.
If you're still having problems, you might want to check the capabilities of your printers. Not all the drivers support all the bitmap transfer methods. I believe the printing system is supposed to hide those differences from you, but perhaps not. Call GetDeviceCaps on your printer DC, and check the RASTERCAPS for RC_BITBLT and friends.

When I load an image in OpenCV it is always darker than the original. Why?

So I load a color .png file that has been taken with an iphone using cvLoadImage. And after it's been loaded, when I immediately display it in my X11 terminal, the image is definitely darker than the original png file.
I currently use this to load the image:
IplImage *img3 = cvLoadImage( "bright.png", 1);
For the second parameter I have tried all of the following:
CV_LOAD_IMAGE_UNCHANGED
CV_LOAD_IMAGE_GRAYSCALE
CV_LOAD_IMAGE_COLOR
CV_LOAD_IMAGE_ANYDEPTH
CV_LOAD_IMAGE_ANYCOLOR
but none of these have worked. Grayscale definitely made the image grayscale. But as suggested from http://www.cognotics.com/opencv/docs/1.0/ref/opencvref_highgui.htm, even using CV_LOAD_IMAGE_ANYDEPTH | CV_LOAD_IMAGE_ANYCOLOR to load the image as truthfully as possible resulted in a darker image being displayed in the terminal.
Does anyone have any ideas on how to get the original image to display properly?
Thanks a lot in advance.
Yes, OpenCV does not apply Gamma correction.
// from: http://gegl.org/
// value: 0.0-1.0
static inline qreal
linear_to_gamma_2_2 (qreal value){
if (value > 0.0030402477)
return 1.055 * pow (value, (1.0/2.4)) - 0.055;
return 12.92 * value;
}
// from: http://gegl.org/
static inline qreal
gamma_2_2_to_linear (qreal value){
if (value > 0.03928)
return pow ((value + 0.055) / 1.055, 2.4);
return value / 12.92;
}
It only happens when you load it in OpenCV? Opening with any other viewer doesn't show a difference?
I can't confirm this without a few tests but I believe the iPhone display gamma is 1.8 (source: http://www.colorwiki.com/wiki/Color_on_iPhone#The_iPhone.27s_Display). Your X11 monitor probably is adjusted for 2.2 (like the rest of the world).
If this theory holds, yes, images are going to appear darker on X11 than on the iPhone. You may change your monitor calibration or do some image processing to account for the difference.
Edit:
I believe OpenCV really does not apply gamma correction. My reference to this is here:
http://permalink.gmane.org/gmane.comp.lib.opencv.devel/837
You might want to implement it yourself or "correct" it with ImageMagick. This page instructs you on how to do so:
http://www.4p8.com/eric.brasseur/gamma.html
I usually load an image with:
cvLoadImage("file.png", CV_LOAD_IMAGE_UNCHANGED);
One interesting test you could do to detect if OpenCV is really messing with the image data, is simply creating another image with cvCreateImage(), then copy the data to this newly created image and save it to another file with cvLoadImage().
Maybe, it's just a display error. Of course, I would suggest you to update to the most recent version of OpenCV.

Resources