I am using PCL viewer (which uses VTK) for visualizing a 3D point cloud generated by SLAM algorithm. I am trying to render the view of point cloud as seen by the robot at a given pose (position and orientation). I am able to set the position and ViewUp vector of the camera, but I am unable to set the Focal point of the camera to the heading of the robot. Currently, I am using sliders to set the Focal Point, but I want to set it programmatically based on the heading.
I am trying understand the type (angle in rad / distance in m) and range of values VTKCamera Focal Point expects and how that's related to the heading.
Function where I am updating camera
void Widget::setcamView(){
//transfrom position
Eigen::Vector3d position = this->transformpose(Eigen::Vector3d(image_pose.at(pose_ittr).position[0], image_pose.at(pose_ittr).position[1], image_pose.at(pose_ittr).position[2]));
posx = position(0);
posy = position(1);
posz = position(2);
//transform the pose
Eigen::Vector3d attitude = this->transformpose(Eigen::Vector3d(image_pose.at(pose_ittr).orientation[0],image_pose.at(pose_ittr).orientation[1],image_pose.at(pose_ittr).orientation[2]));
roll = attitude(0);
pitch = attitude(1);
yaw = attitude(2);
viewx = ui->viewxhSlider->value();// * std::pow(10,-3);
viewy = ui->viewyhSlider->value();// * std::pow(10,-3);
viewz = ui->viewzhSlider->value();// * std::pow(10,-3);
// debug
std::cout<<"Positon: "<<posx<<"\t"<<posy<<"\t"<<posz<<std::endl<<
"View: "<<viewx<<"\t"<<viewy<<"\t"<<viewz<<std::endl<<
"Orientation: "<<roll<<"\t"<<pitch<<"\t"<<yaw<<std::endl;
point_cutoffy = ui->ptcutoffhSlider->value();
if(yaw <=0)
yaw = yaw * -1;
viewer->setCameraPosition(posx,posy,posz+1,
viewz,viewy,viewz,
0, 0, 1, 0);
viewer->setCameraFieldOfView(1);
viewer->setCameraClipDistances(point_cutoffx,point_cutoffy,0);
ui->qvtkWidget->update();
count++;
}
Any help is greatly appreciated.
-Thanks
P.S
PCL Viewer Set Camera Implementation (uses VTK)
void pcl::visualization::PCLVisualizer::setCameraPosition (
double pos_x, double pos_y, double pos_z,
double view_x, double view_y, double view_z,
double up_x, double up_y, double up_z,
int viewport)
{
rens_->InitTraversal ();
vtkRenderer* renderer = NULL;
int i = 0;
while ((renderer = rens_->GetNextItem ()) != NULL)
{
// Modify all renderer's cameras
if (viewport == 0 || viewport == i)
{
vtkSmartPointer<vtkCamera> cam = renderer->GetActiveCamera ();
cam->SetPosition (pos_x, pos_y, pos_z);
cam->SetFocalPoint (view_x, view_y, view_z);
cam->SetViewUp (up_x, up_y, up_z);
renderer->ResetCameraClippingRange ();
}
++i;
}
win_->Render ();
}
I'm working with very similar problem via opencv Viz, which also uses VTK. Relatively to your question, I think you can find an answer HERE
I have a virtual scanner that generates a 2.5-D view of a point cloud (i.e. a 2D-projection of a 3D point cloud) depending on camera position. I'm using the vtkCamera.GetProjectionTransformMatrix() to get transformation matrix from world/global to camera coordinates.
However, if the input point cloud has color information for points I would like to preserve it.
Here are the relevant lines:
boost::shared_ptr<pcl::visualization::PCLVisualizer> vis; // camera location, viewpoint and up direction for vis were already defined before
vtkSmartPointer<vtkRendererCollection> rens = vis->getRendererCollection();
vtkSmartPointer<vtkRenderWindow> win = vis->getRenderWindow();
win->SetSize(xres, yres); // xres and yres are predefined resolutions
win->Render();
float dwidth = 2.0f / float(xres),
dheight = 2.0f / float(yres);
float *depth = new float[xres * yres];
win->GetZbufferData(0, 0, xres - 1, yres - 1, &(depth[0]));
vtkRenderer *ren = rens->GetFirstRenderer();
vtkCamera *camera = ren->GetActiveCamera();
vtkSmartPointer<vtkMatrix4x4> projection_transform = camera->GetProjectionTransformMatrix(ren->GetTiledAspectRatio(), 0, 1);
Eigen::Matrix4f mat1;
for (int i = 0; i < 4; ++i)
for (int j = 0; j < 4; ++j)
mat1(i, j) = static_cast<float> (projection_transform->Element[i][j]);
mat1 = mat1.inverse().eval();
Now, mat1 is used to transform coordinates to camera-view:
pcl::PointCloud<pcl::PointXYZ>::Ptr &cloud;
int ptr = 0;
for (int y = 0; y < yres; ++y)
{
for (int x = 0; x < xres; ++x, ++ptr)
{
pcl::PointXYZ &pt = (*cloud)[ptr];
if (depth[ptr] == 1.0)
{
pt.x = pt.y = pt.z = std::numeric_limits<float>::quiet_NaN();
continue;
}
Eigen::Vector4f world_coords(dwidth * float(x) - 1.0f,
dheight * float(y) - 1.0f,
depth[ptr],
1.0f);
world_coords = mat1 * world_coords;
float w3 = 1.0f / world_coords[3];
world_coords[0] *= w3;
world_coords[1] *= w3;
world_coords[2] *= w3;
pt.x = static_cast<float> (world_coords[0]);
pt.y = static_cast<float> (world_coords[1]);
pt.z = static_cast<float> (world_coords[2]);
}
}
I want the virtual scanner to return pcl::PointXYZRGB point cloud with color information.
Any help on how to implement this from someone experienced in VTK would save some of my time.
It's possible that I missed a relevant question already asked here - in that case, please point me to it. Thanks.
If I understand correctly that you want to get the color in which the point was rendered into the win RenderWindow, you should be able to get the data from the rendering buffer by calling
float* pixels = win->GetRGBAPixelData(0, 0, xres - 1, yres - 1, 0/1).
This should give you each pixel of the rendering buffer as an array in the format [R0, G0, B0, A0, R1, G1, B1, A1, R2....]. The last parameter which I wrote as 0/1 is whether the data should be taken from front or back opengl buffers. I presume by default double buffering should be on, so then you want to read from back buffer (use '1'), but I am not sure.
Once you have that, you can get the color in your second loop for all pixels that belong to points (depth[ptr] != 1.0) as:
pt.R = pixels[4*ptr];
pt.G = pixels[4*ptr + 1];
pt.B = pixels[4*ptr + 2];
You should call win->ReleaseRGBAPixelData(pixels) once you're done with it.
In my app I am having situation where all the phone will have only Portrait, Tablet with height = 1024 ; width = 768 then Height/width(1024/768<=1.3333) I want to make the tab Landscape otherwise I need to make Portrait.
String userAgent = new WebView(activity).getSettings()
.getUserAgentString();
double screen_size = 1.3333333333333333;
if (userAgent.contains("Mobile")) {
activity.setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
mEditor.putString("Device_Mode", "Phone");
mEditor.commit();
} else {
activity.setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
Display display = activity.getWindowManager().getDefaultDisplay();
Point size = new Point();
display.getRealSize(size);
double height = size.y;
double width = size.x;
double aspect_Ratio = width / height;
if (aspect_Ratio <= screen_size) {
activity.setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE);
mEditor.putString("Device_Mode", "TAB-LANDSCAPE");
mEditor.commit();
quit();
} else {
activity.setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
mEditor.putString("Device_Mode", "TAB-PORTRAIT");
mEditor.commit();
quit();
}
}
Screen is getting flicking and app is getting crash at last.
Check weather the mobile or tablet by using following code.
TelephonyManager manager =(TelephonyManager)context.getSystemService(Context.TELEPHONY_SERVICE);
if(manager.getPhoneType() == TelephonyManager.PHONE_TYPE_NONE){
return "Tablet";
}else{
return "Mobile";
}
The screen got flickering because it is keep on switching between the orientation. So I've stopped by
android:configChanges="orientation|keyboardHidden|screenSize"
putting this line code in my manifest on the corresponding activity. So the activity flickering is stopped.
int i = getResources().getConfiguration().orientation;
Display display = getWindowManager().getDefaultDisplay();
Point size = new Point();
display.getRealSize(size);
double height = size.y;
double width = size.x;
double aspect_Ratio = 0.0;
if (i == 1)
aspect_Ratio = height / width;
else if (i == 2)
aspect_Ratio = width / height;
if (aspect_Ratio <= screen_size) {
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE);
} else {
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
}
By using this code I have achieved my task :)
For context: I am going to analyze the breathing movement of parents during kangaroo mother care and I wish to respect their privacy by not recording them, but only the movement of stickers I placed on their chest and stomach.
So far, I'm able to track 2 colours based on webcam input through the code below. However, I would like to record only the tracked colours instead of the webcam feed as to preserve the privacy of the parent.
Does anybody know how to add a background colour, whilst still being able to track colour?
import processing.video.*;
Capture video;
final int TOLERANCE = 20;
float XRc = 0;// XY coordinate of the center of the first target
float YRc = 0;
float XRh = 0;// XY coordinate of the center of the second target
float YRh = 0;
int ii=0; //Mouse click counter
color trackColor; //The first color is the center of the robot
color trackColor2; //The second color is the head of the robot
void setup() {
size(640,480);
video = new Capture(this,640,480);
video.start();
trackColor = color(255,0,0);
trackColor2 = color(255,0,0);
smooth();
}
void draw() {
background(0);
if (video.available()) {
video.read();
}
video.loadPixels();
image(video,0,0);
float r2 = red(trackColor);
float g2 = green(trackColor);
float b2 = blue(trackColor);
float r3 = red(trackColor2);
float g3 = green(trackColor2);
float b3 = blue(trackColor2);
int somme_x = 0, somme_y = 0;
int compteur = 0;
int somme_x2 = 0, somme_y2 = 0;
int compteur2 = 0;
for(int x = 0; x < video.width; x++) {
for(int y = 0; y < video.height; y++) {
int currentLoc = x + y*video.width;
color currentColor = video.pixels[currentLoc];
float r1 = red(currentColor);
float g1 = green(currentColor);
float b1 = blue(currentColor);
if(dist(r1,g1,b1,r2,g2,b2) < TOLERANCE) {
somme_x += x;
somme_y += y;
compteur++;
}
else if(compteur > 0) {
XRc = somme_x / compteur;
YRc = somme_y / compteur;
}
if(dist(r1,g1,b1,r3,g3,b3) < TOLERANCE) {
somme_x2 += x;
somme_y2 += y;
compteur2++;
}
else if(compteur2 > 0) {
XRh = somme_x2 / compteur2;
YRh = somme_y2 / compteur2;
}
}
}
if(XRc != 0 || YRc != 0) { // Draw a circle at the first target
fill(trackColor);
strokeWeight(0.05);
stroke(0);
ellipse(XRc,YRc,20,20);
}
if(XRh != 0 || YRh != 0) {// Draw a circle at the second target
fill(trackColor2);
strokeWeight(0.05);
stroke(0);
ellipse(XRh,YRh,20,20);
}
}
void mousePressed() {
if (mousePressed && (mouseButton == RIGHT)) { // Save color where the mouse is clicked in trackColor variable
if(ii==0){
if (mouseY>480){mouseY=0;mouseX=0;}
int loc = mouseX + mouseY*video.width;
trackColor = video.pixels[loc];
ii=1;
}
else if(ii==1){
if (mouseY>480){mouseY=0;mouseX=0;}
int loc2 = mouseX + mouseY*video.width;
trackColor2 = video.pixels[loc2];
ii=2;
}
}
}
Try adding the background(0); right before you draw the first circle. It should cover the video and you can draw the circles on top of it.
Regards
Jose
To begin with I am using the Chili Framework for lessons 1-15 as downloadable here:
http://www.planetchili.net/
I am using DirectX 9 on an old laptop running Windows XP SP3. I have set the Direct3D rendering to software in order to run the framework. I am using Visual Studio Express C++ 2010 with the first service pack installed.
This is the code I am having trouble with:
// Start moving reticle code
DrawReticle(itemLocX, itemLocY, 255, 255, 255);
if(itemLocX == pointA && itemLocX != pointAb)
{
itemLocX += 2;
}
else if(itemLocX == pointBc && itemLocX != pointDa)
{
itemLocX -= 2;
}
if(itemLocY == pointAb && itemLocY != pointBc)
{
itemLocY += 2;
}
else if(itemLocY == pointDa && itemLocX != pointA)
{
itemLocY -= 2;
}
// End moving reticle code
Now Chili's solution is to move along the y axis while checking for x, and x while checking for y. I may post that later, don't have it readily available. You can see it at the beginning of this video:
http://youtu.be/JEmwkQsi8l0
However I wanted to do this logically, as if I was walking the border along an invisible wall inside a box. I wanted it to make sense what was going on. But the cursor won't move, and I see no reason why it doesn't. Here is my game.h:
#pragma once
#include "D3DGraphics.h"
#include "Keyboard.h"
class Game
{
public:
Game( HWND hWnd,const KeyboardServer& kServer );
void Go();
private:
void ComposeFrame();
/********************************/
/* User Functions */
void DrawReticle(int xP, int yP, int cR, int cG, int cB);
/*
xP = x position,
yP = y position,
cR = color red,
cG = color green,
cB = color blue
*/
// TODO: User functions go here
/********************************/
private:
D3DGraphics gfx;
KeyboardClient kbd;
/********************************/
/* User Variables */
int pointA; // Starting at pointA (100, 100) - the top left
int pointAb; // Move from pointA to pointAb (700, 100) - the top right
int pointBc; // Move from pointAb to pointBc (700, 500) - the bottom right
int pointCd; // Move from pointBc to pointCd (100,500) - the bottom left
int pointDa; // Move from pointCd to pointDa (100,100) - the top left
/*
These points describe the process of starting, then four movements. The four points are A, B, C, D. We start at A, then go to B (pointAb, read as A to b), then go to C (pointBc, read as B to c), then go to D (pointCd, read as C to d) then go to A (pointDa, read as D to a).
This can be very confusing, because there are five varibles used. But if we drew it out there would only four points, as well as only four movements. The best way to think of it is that starting is itself a movement, and as you need a place to start from, it itself must have a point. Since you start at A, but haven't yet gone anywhere, pointA is our starting point. Once you start moving, you go from pointA to pointB. Now if we used pointB as our variable it would be confusing,because we would have to move from pointA to pointB to pointC to pointD and then back to pointA. Still five variables, one is repeating, but the first pointA describes where you start, and the last where you end. Since these are two different actions on the same point, I have elected to use two letter names for each of the points you move to, while the point you start at has a single letter name. It was the best way I could clearly think about this process.
*/
int itemLocX; // Initial position of item on the x axis
int itemLocY; // Initial position of item on the y axis
int reticleX; // Initial position of reticle on the x axis
int reticleY; // Initial position of reticle on the y axis
// TODO: User variables go here
/********************************/
};
Here is my game.cpp:
#include "Game.h"
Game::Game( HWND hWnd,const KeyboardServer& kServer )
: gfx(hWnd),
kbd(kServer),
itemLocX(100), // Initial position of item on the x axis
itemLocY(100), // Initial position of item on the y axis
reticleX(400), // Initial position of reticle on the x axis
reticleY(300), // Initial position of reticle on the y axis
pointA(100), // Movement from 0 to A, stopping at A
pointAb(700), // Movement from A to b, stopping at B
pointBc(500), // Movement from B to c, stopping at C
pointCd(700), // Movement from C to d, stopping at D
pointDa(500) // Movement from D to a, stopping at A
{}
void Game::Go()
{
gfx.BeginFrame();
ComposeFrame();
gfx.EndFrame();
}
void Game::DrawReticle(int xP, int yP, int cR, int cG, int cB)
/*
xP = x position,
yP = y position,
cR = color red,
cG = color green,
cB = color blue
*/
{
gfx.PutPixel(xP-5,yP,cR,cG,cB);
gfx.PutPixel(xP-4,yP,cR,cG,cB);
gfx.PutPixel(xP-3,yP,cR,cG,cB);
gfx.PutPixel(xP+3,yP,cR,cG,cB);
gfx.PutPixel(xP+4,yP,cR,cG,cB);
gfx.PutPixel(xP+5,yP,cR,cG,cB);
gfx.PutPixel(xP,yP,cR,cG,cB);
gfx.PutPixel(xP,yP-5,cR,cG,cB);
gfx.PutPixel(xP,yP-4,cR,cG,cB);
gfx.PutPixel(xP,yP-3,cR,cG,cB);
gfx.PutPixel(xP,yP+3,cR,cG,cB);
gfx.PutPixel(xP,yP+4,cR,cG,cB);
gfx.PutPixel(xP,yP+5,cR,cG,cB);
}
void Game::ComposeFrame()
{
// Start draw reticle code
DrawReticle(reticleX, reticleY, 100, 155, 255);
// End draw reticle code
// Start color change code
int yT = 200; // Border 200 pixels from top
int yB = 400; // Border 200 pixels from bottom
int xL = 300; // Border 200 pixels from left
int xR = 500; // Border 200 pixels from right
if(reticleX < xL || reticleX > xR) // Defining color change area for X
{
DrawReticle(reticleX, reticleY, 255, 255, 255);
}
if(reticleY < yT || reticleY > yB) // Defining color change area for Y
{
DrawReticle(reticleX, reticleY, 255, 255, 255);
}
// End color change code
// Start moving reticle code
DrawReticle(itemLocX, itemLocY, 255, 255, 255);
if(itemLocX == pointA && itemLocX != pointAb)
{
itemLocX += 2;
}
else if(itemLocX == pointBc && itemLocX != pointDa)
{
itemLocX -= 2;
}
if(itemLocY == pointAb && itemLocY != pointBc)
{
itemLocY += 2;
}
else if(itemLocY == pointDa && itemLocX != pointA)
{
itemLocY -= 2;
}
// End moving reticle code
// Start border code
if(reticleX < 6)
{
reticleX = 6;
}
else if(reticleX > 794)
{
reticleX = 794;
}
if(reticleY < 6)
{
reticleY = 6;
}
else if(reticleY > 594)
{
reticleY = 594;
}
// End border code
// Start speed change code
int cSpeed = 4; // Default cursor speed
if(kbd.EnterIsPressed()) // Change to high speed
{
cSpeed = 8;
}
if(kbd.SpaceIsPressed()) // Change to low speed
{
cSpeed = 1;
}
if(kbd.RightIsPressed())
{
reticleX += cSpeed;
}
if(kbd.LeftIsPressed())
{
reticleX -= cSpeed;
}
if(kbd.UpIsPressed())
{
reticleY -= cSpeed;
}
if(kbd.DownIsPressed())
{
reticleY += cSpeed;
}
// End speed change code
}
Now I should note here that this should be done without functions and only the basic C++ operators. That's as far as Chili has taught to this point. This is my second attempt to solve this myself, after hours thinking about it and working on it on paper. I'm stuck. Just not seeing it. I think there is a logic error here on my part. I want to understand where my thinking may be mistaken, but more than that, how to think correctly, like the computer, about this.
I am also open to advice regarding my coding style. If I am not being clear enough, or am doing something that should not become a bad habit - basically if there is something I should be doing differently in writing my code I would like to know about it.
Thank you for your help - it is very much appreciated!
I see how you have tried to do this. Personally you have over complexed it.
1: you don't need the != operator in your if statements.
2: try this:
if(itemLocX < 700)
{
itemLocX += 2;
}
3: This worked fine during testing. Another point is that the if statements could be in the wrong order. I changed it to the order in which it moved across the screen in. I have X Y X Y and you have X X Y Y. (unconfirmed) It executes the if statements in order. I have hard coded the answer. set them to variables if you really want to. Hope this helped!