Getting Depth of other objects Kinect - object

I'm a newbie in kinect programming, i am working on a ball tracking using kinect and opencv..we all know that kinect provides Depth data, and with the code below:
DepthImagePoint righthandDepthPoint =
sensor.CoordinateMapper.MapSkeletonPointToDepthPoint
(
me.Joints[JointType.HandRight].Position,
DepthImageFormat.Resolution640x480Fps30
);
double rightdepthmeters = (righthandDepthPoint.Depth);
using this, I am able to get the depth of a right hand, using the function MapSkeletonPointToDepthPoint() by specifing the jointtype..
Is it possible to get the depth of other objects by specifying in the image where?
given the coordinate..I want to get the depth of the object in that coordinate?

Pulling depth data from the Kinect SDK can be extracted from the DepthImagePixel structure.
The example code below loops through the entire DepthImageFrame to examine each of the pixels. If you have a specific coordinate you wish to look at, remove the for loop and set the x and y to a specific value.
// global variables
private const DepthImageFormat DepthFormat = DepthImageFormat.Resolution320x240Fps30;
private const ColorImageFormat ColorFormat = ColorImageFormat.RgbResolution640x480Fps30;
private DepthImagePixel[] depthPixels;
// defined in an initialization function
this.depthWidth = this.sensor.DepthStream.FrameWidth;
this.depthHeight = this.sensor.DepthStream.FrameHeight;
this.depthPixels = new DepthImagePixel[this.sensor.DepthStream.FramePixelDataLength];
private void SensorAllFramesReady(object sender, AllFramesReadyEventArgs e)
{
if (null == this.sensor)
return;
bool depthReceived = false;
using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
{
if (null != depthFrame)
{
// Copy the pixel data from the image to a temporary array
depthFrame.CopyDepthImagePixelDataTo(this.depthPixels);
depthReceived = true;
}
}
if (true == depthReceived)
{
// loop over each row and column of the depth
for (int y = 0; y < this.depthHeight; ++y)
{
for (int x = 0; x < this.depthWidth; ++x)
{
// calculate index into depth array
int depthIndex = x + (y * this.depthWidth);
// extract the given index
DepthImagePixel depthPixel = this.depthPixels[depthIndex];
Debug.WriteLine("Depth at [" + x + ", " + y + "] is: " + depthPixel.Depth);
}
}
}
}

Related

Profit Maximization based on dynamix programming

I have been trying to solve this problem :
" You have to travel to different villages to make some profit.
In each village, you gain some profit. But the catch is, from a particular village i, you can only move to a village j if and only if and the profit gain from village j is a multiple of the profit gain from village i.
You have to tell the maximum profit you can gain while traveling."
Here is the link to the full problem:
https://www.hackerearth.com/practice/algorithms/dynamic-programming/introduction-to-dynamic-programming-1/practice-problems/algorithm/avatar-and-his-quest-d939b13f/description/
I have been trying to solve this problem for quite a few hours. I know this is a variant of the longest increasing subsequence but the first thought that came to my mind was to solve it through recursion and then memoize it. Here is a part of the code to my approach. Please help me identify the mistake.
static int[] dp;
static int index;
static int solve(int[] p) {
int n = p.length;
int max = 0;
for(int i = 0;i<n; i++)
{
dp = new int[i+1];
Arrays.fill(dp,-1);
index = i;
max = Math.max(max,profit(p,i));
}
return max;
}
static int profit(int[] p, int n)
{
if(dp[n] == -1)
{
if(n == 0)
{
if(p[index] % p[n] == 0)
dp[n] = p[n];
else
dp[n] = 0;
}
else
{
int v1 = profit(p,n-1);
int v2 = 0;
if(p[index] % p[n] == 0)
v2 = p[n] + profit(p,n-1);
dp[n] = Math.max(v1,v2);
}
}
return dp[n];
}
I have used extra array to get the solution, my code is written in Java.
public static int getmaxprofit(int[] p, int n){
// p is the array that contains all the village profits
// n is the number of villages
// used one extra array msis, that would be just a copy of p initially
int i,j,max=0;
int msis[] = new int[n];
for(i=0;i<n;i++){
msis[i]=p[i];
}
// while iteraring through p, I will check in backward and find all the villages that can be added based on criteria such previous element must be smaller and current element is multiple of previous.
for(i=1;i<n;i++){
for(j=0;j<i;j++){
if(p[i]>p[j] && p[i]%p[j]==0 && msis[i] < msis[j]+p[i]){
msis[i] = msis[j]+p[i];
}
}
}
for(i=0;i<n;i++){
if(max < msis[i]){
max = msis[i];
}
}
return max;
}

Trouble with indices

I am writing a Maximum Value Knapsack algorithm. It takes in a Knapsack object with Items that have a value and cost. I declare a 2D array for calculating the max value. For the base cases I have set the zeroth row values to 0 and zeroth column values to 0. I am running into trouble when I grab an item in the knapsack because when I want to grab the zeroth item, I am really grabbing the first item in the knapsack and am consequently getting the wrong values in the 2D array. Can someone check out my code and see what I am missing?
public static double MaximumKnapsack(Knapsack knapsack) {
int numItems = knapsack.getNumOfItems();
int budget = (int) knapsack.getBudget();
double[][] DP = new double[numItems+1][budget+1];
boolean taken = false;
for (int i = 0; i < numItems + 1; i++) {
for (int b = 0; b < budget + 1; b++) {
if (i == 0 || b == 0) {
DP[i][b] = 0;
}
else
{
Item item = knapsack.getItem(i);
if (item.getCost() > b) {
DP[i][b] = DP[i-1][b];
}
else
{
DP[i][b] = Math.max(DP[i-1][b-(int) item.getCost()] + item.getValue(),
DP[i-1][b]);
if (DP[i][b] == DP[i-1][b-(int) item.getCost()] + item.getValue() && item.getCost() != 0.0) {
taken = true;
}
}
}
}
taken = false;
}
return DP[numItems][budget];
}
I think the problem is in
Item item = knapsack.getItem(i);
beacuse your loop will start with i = 1. You should use:
Item item = knapsack.getItem(i-1);

How to declare a set of points for an object to move to and from? [Android/Java]

What I am trying to do is create a handwriting application which allows a person to press on the object (circle) and move it up/down on a set path. If the user reaches the lowest point another circle object is created and another set of points is created to follow etc.
So far I have a onTouch event which moves my ImageView object (circle) to where ever the finger is on the touch screen.
https://gist.github.com/Temptex/9796403
How can i get my ImageView (Circle) object to go from Point A -> B smoothly using onTouch events?
Edit: A picture example of what I am asking: http://imgur.com/kjgGcba
created an algorithm for it.
Created an array of x integers and y integers and used a for loop to check if the object was to high (out of bounds in the y coordinates) and check if y and x were out of bounds in the arrays.
If it was out of bounds from the array of integers, set it to the current x/y array values. in the for loop.
If this helps anyone here is the code:
private final int yCoOrdinate[] = {310, 360, 410};
private final int xCoOrdinate[] = {905, 890, 875};
// Get finger position
int x = (int)event.getRawX();
int y = (int)event.getRawY();
// View x/y co-ordinate on TextView
coordinates.setText("X = " + x + " - Y = " + y);
for(int i = 0; i < yCoOrdinate.length; i++) {
// If object is to high set it to yCoOrdinate[0]
if(y <= yCoOrdinate[0]) {
y = yCoOrdinate[0];
}
// Checks top left corner of A
if(y == yCoOrdinate[0] && x <= xCoOrdinate[0]) {
x = xCoOrdinate[0];
Log.d("Coordinate check", "X = " + x + "Y = " + y);
// Checks if current x/y position if out of bounds and sets them to i
} else if (y <= yCoOrdinate[i] && x <= xCoOrdinate[i]) {
y = yCoOrdinate[i];
x = xCoOrdinate[i];
Log.d("Coordinate check", "X = " + x + "Y = " + y);
}
}
I can now control my ImageView with in bounds.

Epplus SetPosition picture issue

I am using Epplus library to generate Excel 2010 and up compatible files in Asp.Net C#.
I am using version 3.1.2 which is the latest at this moment.
I am setting the row height first, before adding any pictures like this:
ExcelPackage pck = new ExcelPackage();
var ws = pck.Workbook.Worksheets.Add("sheet 1");
while (i < dt.Rows.Count + offset)
{
ws.Row(i).Height = 84;
i++;
}
dt is my DataTable with DataRows.
After setting the height, I am looping again through the rows to add the pictures
while (i < dt.Rows.Count + offset)
{
var prodImg = ws.Drawings.AddPicture(dr["code"].ToString(), new FileInfo(path));
prodImg.SetPosition(i - 1, 0, 14, 0);
prodImg.SetSize(75);
}
This works, but this does not:
var prodImg = ws.Drawings.AddPicture(dr["code"].ToString(), new FileInfo(path));
int w = prodImg.Image.Width;
int h = prodImg.Image.Height;
if (h > 140) // because height of 84 is 140 pixels in excel
{
double scale = h / 140.0;
w = (int)Math.Floor(w / scale);
h = 140;
}
int xOff = (150 - w) / 2;
int yOff = (140 - h) / 2;
prodImg.SetPosition(i - 1, xOff, 11, yOff);
prodImg.SetSize(w, h);
This results in off center pictures and unresized images. And this code then which is in the same loop:
var prodImgDm = ws.Drawings.AddPicture("bcdm" + dr["code"].ToString(), new FileInfo(pathDm));
prodImgDm.SetPosition(i - 1, 25, 15, 40);
prodImgDm.SetSize(100);
This does work sometimes. the pictures prodImgDm are datamatrix images with a static width and height and do not need to be resized because they are always small/tiny. So also without the SetSize in some rows, it works and in some other rows, it does not work. Really strange because the code is the same. It might be something in the library and/or Excel. Perhaps I am using it wrong? Any epplus picture expert?
Thanks in advance!!
edit sometimes a picture is worth a thousand words, so here is the screenshot. As you can see the product images are not horizontal and vertical aligned in the cell. And the datamatrix on the far right is sometimes scaled about 120% even when I set SetSize(100) so it is really strange to me. So the last datamatrix has the correct size... I already found this SO thread but that does not help me out, I think.
edit 2013/04/09 Essenpillai gave me a hint to set
pck.DoAdjustDrawings = false;
but that gave me even stranger images:
the datamatrix is still changing on row basis. on row is ok, the other is not. and the ean13 code is too wide.
public static void CreatePicture(ExcelWorksheet worksheet, string name, Image image, int firstColumn, int lastColumn, int firstRow, int lastRow, int defaultOffsetPixels)
{
int columnWidth = GetWidthInPixels(worksheet.Cells[firstRow, firstColumn]);
int rowHeight = GetHeightInPixels(worksheet.Cells[firstRow, firstColumn]);
int totalColumnWidth = columnWidth * (lastColumn - firstColumn + 1);
int totalRowHeight = rowHeight * (lastRow - firstRow + 1);
double cellAspectRatio = Convert.ToDouble(totalColumnWidth) / Convert.ToDouble(totalRowHeight);
int imageWidth = image.Width;
int imageHeight = image.Height;
double imageAspectRatio = Convert.ToDouble(imageWidth) / Convert.ToDouble(imageHeight);
int pixelWidth;
int pixelHeight;
if (imageAspectRatio > cellAspectRatio)
{
pixelWidth = totalColumnWidth - defaultOffsetPixels * 2;
pixelHeight = pixelWidth * imageHeight / imageWidth;
}
else
{
pixelHeight = totalRowHeight - defaultOffsetPixels * 2;
pixelWidth = pixelHeight * imageWidth / imageHeight;
}
int rowOffsetPixels = (totalRowHeight - pixelHeight) / 2;
int columnOffsetPixels = (totalColumnWidth - pixelWidth) / 2;
int rowOffsetCount = 0;
int columnOffsetCount = 0;
if (rowOffsetPixels > rowHeight)
{
rowOffsetCount = (int)Math.Floor(Convert.ToDouble(rowOffsetPixels) / Convert.ToDouble(rowHeight));
rowOffsetPixels -= rowHeight * rowOffsetCount;
}
if (columnOffsetPixels > columnWidth)
{
columnOffsetCount = (int)Math.Floor(Convert.ToDouble(columnOffsetPixels) / Convert.ToDouble(columnWidth));
columnOffsetPixels -= columnWidth * columnOffsetCount;
}
int row = firstRow + rowOffsetCount - 1;
int column = firstColumn + columnOffsetCount - 1;
ExcelPicture pic = worksheet.Drawings.AddPicture(name, image);
pic.SetPosition(row, rowOffsetPixels, column, columnOffsetPixels);
pic.SetSize(pixelWidth, pixelHeight);
}
public static int GetHeightInPixels(ExcelRange cell)
{
using (Graphics graphics = Graphics.FromHwnd(IntPtr.Zero))
{
float dpiY = graphics.DpiY;
return (int)(cell.Worksheet.Row(cell.Start.Row).Height * (1 / 72.0) * dpiY);
}
}
public static float MeasureString(string s, Font font)
{
using (var g = Graphics.FromHwnd(IntPtr.Zero))
{
g.TextRenderingHint = System.Drawing.Text.TextRenderingHint.AntiAlias;
return g.MeasureString(s, font, int.MaxValue, StringFormat.GenericTypographic).Width;
}
}
public static int GetWidthInPixels(ExcelRange cell)
{
double columnWidth = cell.Worksheet.Column(cell.Start.Column).Width;
Font font = new Font(cell.Style.Font.Name, cell.Style.Font.Size, FontStyle.Regular);
double pxBaseline = Math.Round(MeasureString("1234567890", font) / 10);
return (int)(columnWidth * pxBaseline);
}
enter image description here
I have the same problem with Epplus library.
After I find no solution how to solve this in my code, I checked source code of this library.
Epplus create excel picture always as twoCellAnchor drawing. In xlsx files you can find drawingXYZ.xml with this code:
<xdr:twoCellAnchor editAs="oneCell">
<xdr:from> ... </xdr:from>
<xdr:to> ... </xdr:to>
<xdr:pic>
...
</xdr:twoCellAnchor>
So, picture is always connected to two cells, and this is not variable part of Epplus library. You can find this part of code in ExcelDrawing.cs file.
XmlElement drawNode = _drawingsXml.CreateElement(
"xdr", "twoCellAnchor", ExcelPackage.schemaSheetDrawings);
colNode.AppendChild(drawNode);
You can easy create your own copy of this dll. The good news is that you need to modify only two files to fix this problem. So..
Download your copy of source codes for Epplus library from this site and open in Visual Studio.
We need to generate code of drawing as oneCellAnchor, so we must remove <xdr:to> element for pictures and create element <xdr:ext /> with picture dimensions as parameters.
New xml structure will looks like:
<xdr:oneCellAnchor editAs="oneCell">
<xdr:from> ... </xdr:from>
<xdr:ext cx="1234567" cy="7654321" />
<xdr:pic>
...
</xdr:oneCellAnchor>
Ok, so, how to do this?
Changes in Epplus code
ExcelDrawings.cs (link to file here)
At first we modify method CreateDrawingXml() inside ExcelDrawings.cs. Order to preserve the original functionality we add an optional parameter (if create oneCellAnchor) with default value. And in method, based this parameter, we create one or tow cell anchor and create or not <xdr:to> element.
Important part of this method code:
private XmlElement CreateDrawingXml(bool twoCell = true) {
if (DrawingXml.OuterXml == "")
{ ... } // not changed
XmlNode colNode= _drawingsXml.SelectSingleNode("//xdr:wsDr", NameSpaceManager);
//First change in method code
XmlElement drawNode;
if (twoCell)
drawNode = _drawingsXml.CreateElement(
"xdr", "twoCellAnchor", ExcelPackage.schemaSheetDrawings);
else
drawNode = _drawingsXml.CreateElement(
"xdr", "oneCellAnchor", ExcelPackage.schemaSheetDrawings);
colNode.AppendChild(drawNode);
//Add from position Element; // Not changed
XmlElement fromNode = _drawingsXml.CreateElement(
"xdr", "from", ExcelPackage.schemaSheetDrawings);
drawNode.AppendChild(fromNode);
fromNode.InnerXml = "<xdr:col>0</xdr:col><xdr:colOff>0</xdr:colOff>"
+ "<xdr:row>0</xdr:row><xdr:rowOff>0</xdr:rowOff>";
//Add to position Element;
//Second change in method
if (twoCell)
{
XmlElement toNode = _drawingsXml.CreateElement(
"xdr", "to", ExcelPackage.schemaSheetDrawings);
drawNode.AppendChild(toNode);
toNode.InnerXml = "<xdr:col>10</xdr:col><xdr:colOff>0</xdr:colOff>"
+ "<xdr:row>10</xdr:row><xdr:rowOff>0</xdr:rowOff>";
}
return drawNode;
}
Then we modify two methods for AddPicture inside the same file:
public ExcelPicture AddPicture(string Name, Image image, Uri Hyperlink)
{
if (image != null) {
if (_drawingNames.ContainsKey(Name.ToLower())) {
throw new Exception("Name already exists in the drawings collection");
}
XmlElement drawNode = CreateDrawingXml(false);
// Change: we need create element with dimensions
// like: <xdr:ext cx="3857625" cy="1047750" />
XmlElement xdrext = _drawingsXml.CreateElement(
"xdr", "ext", ExcelPackage.schemaSheetDrawings);
xdrext.SetAttribute("cx",
(image.Width * ExcelDrawing.EMU_PER_PIXEL).ToString());
xdrext.SetAttribute("cy",
(image.Height * ExcelDrawing.EMU_PER_PIXEL).ToString());
drawNode.AppendChild(xdrext);
// End of change, next part of method is the same:
drawNode.SetAttribute("editAs", "oneCell");
...
}
}
And this method with FileInfo as input parameter:
public ExcelPicture AddPicture(string Name, FileInfo ImageFile, Uri Hyperlink)
{
if (ImageFile != null) {
if (_drawingNames.ContainsKey(Name.ToLower())) {
throw new Exception("Name already exists in the drawings collection");
}
XmlElement drawNode = CreateDrawingXml(false);
// Change: First create ExcelPicture object and calculate EMU dimensions
ExcelPicture pic = new ExcelPicture(this, drawNode, ImageFile, Hyperlink);
XmlElement xdrext = _drawingsXml.CreateElement(
"xdr", "ext", ExcelPackage.schemaSheetDrawings);
xdrext.SetAttribute("cx",
(pic.Image.Width * ExcelDrawing.EMU_PER_PIXEL).ToString());
xdrext.SetAttribute("cy",
(pic.Image.Height * ExcelDrawing.EMU_PER_PIXEL).ToString());
drawNode.AppendChild(xdrext);
// End of change, next part of method is the same (without create pic object)
drawNode.SetAttribute("editAs", "oneCell");
...
}
}
So, this are all important code. Now we must change code for searching nodes and preserve order in elements.
In private void AddDrawings() we change xpath from:
XmlNodeList list = _drawingsXml.SelectNodes(
"//xdr:twoCellAnchor", NameSpaceManager);
To this:
XmlNodeList list = _drawingsXml.SelectNodes(
"//(xdr:twoCellAnchor or xdr:oneCellAnchor)", NameSpaceManager);
It is all in this file, now we change
ExcelPicture.cs (link to file here)
Original code find node for append next code in constructor like this:
node.SelectSingleNode("xdr:to",NameSpaceManager);
Because we do not create <xdr:to> element always, we change this code:
internal ExcelPicture(ExcelDrawings drawings, XmlNode node
, Image image, Uri hyperlink)
: base(drawings, node, "xdr:pic/xdr:nvPicPr/xdr:cNvPr/#name")
{
XmlElement picNode = node.OwnerDocument.CreateElement(
"xdr", "pic", ExcelPackage.schemaSheetDrawings);
// Edited: find xdr:to, or xdr:ext if xdr:to not exists
XmlNode befor = node.SelectSingleNode("xdr:to",NameSpaceManager);
if (befor != null && befor.Name == "xdr:to")
node.InsertAfter(picNode, befor);
else {
befor = node.SelectSingleNode("xdr:ext", NameSpaceManager);
node.InsertAfter(picNode, befor);
}
// End of change, next part of constructor is unchanged
_hyperlink = hyperlink;
...
}
And the same for second constructor with FileInfo as input parameter:
internal ExcelPicture(ExcelDrawings drawings, XmlNode node
, FileInfo imageFile, Uri hyperlink)
: base(drawings, node, "xdr:pic/xdr:nvPicPr/xdr:cNvPr/#name")
{
XmlElement picNode = node.OwnerDocument.CreateElement(
"xdr", "pic", ExcelPackage.schemaSheetDrawings);
// Edited: find xdr:to, or xdr:ext if xdr:to not exists
XmlNode befor = node.SelectSingleNode("xdr:to", NameSpaceManager);
if (befor != null && befor.Name == "xdr:to")
node.InsertAfter(picNode, befor);
else {
befor = node.SelectSingleNode("xdr:ext", NameSpaceManager);
node.InsertAfter(picNode, befor);
}
// End of change, next part of constructor is unchanged
_hyperlink = hyperlink;
...
Now, pictures are created as oneCellAnchor. If you want, you can create multiple AddPicture methods for booth variants. Last step is build this project and create your own custom EPPlus.dll. Then close your project which use this dll and copy new files EPPlus.dll, EPPlus.pdb, EPPlus.XML inside your project (backup and replace your original dll files) at the same place (so you don't need do any change in your project references or settings).
Then open and rebuild your project and try if this solve your problem.
Maybe I am too late, but here is mine answer..
you can read it on codeplex issue as well
(https://epplus.codeplex.com/workitem/14846)
I got this problem as well.
And after some research I figured out where the bug is.
It's in the ExcelRow class on the 149 line of code (ExcelRow.cs file).
There is a mistake, when row's Height got changed it recalcs all pictures heights but uses pictures widths inplace of heights, so it's easy to fix.
Just change the line
var pos = _worksheet.Drawings.GetDrawingWidths();
to
var pos = _worksheet.Drawings.GetDrawingHeight();
See the code changes on image
P.S. Actual for version 4.0.4

Unity3d Box Collider Vertex

How do I get the vertexes of a box collider in world space in Unity3d? (with correct rotation and scale). Just doing local-world on the object does not work.
obj.transform.localToWorldMatrix.MultiplyPoint3x4(extents); and doing similar things for the other corners is what I tried.
Since you can easily get the BoxCollider's centre and extents you can get the local vertex position easily by adding or subtracting the extent from the centre, then transforming it into global world space with a method like Vector3.TransformPoint()
The following script prints the global position of each vertex of a BoxCollider attached to the same GameObject as the script, every Update() call.
using System.Collections;
public class DebugPrintColliderVertices : MonoBehaviour
{
const uint NUM_VERTICES = 8;
private BoxCollider boxCollider;
private Transform[] vertices;
void Awake()
{
boxCollider = (BoxCollider)this.gameObject.GetComponent(typeof(BoxCollider));
if (boxCollider == null)
{
Debug.Log ("Collider not found on " + this.gameObject.name + ". Ending script.");
this.enabled = false;
}
vertices = new Transform[NUM_VERTICES];
}
void Update()
{
Vector3 colliderCentre = boxCollider.center;
Vector3 colliderExtents = boxCollider.extents;
for (int i = 0; i != NUM_VERTICES ; ++i)
{
Vector3 extents = colliderExtents;
extents.Scale (new Vector3((i & 1) == 0 ? 1 : -1, (i & 2) == 0 ? 1 : -1, (i & 4) == 0 ? 1 : -1));
Vector3 vertexPosLocal = colliderCentre + extents;
Vector3 vertexPosGlobal = boxCollider.transform.TransformPoint(vertexPosLocal);
// display vector3 to six decimal places
Debug.Log ("Vertex " + i + " # " + vertexPosGlobal.ToString("F6"));
}
}
}
Note that the BoxCollider's size is multiplied by the transform.scale of the GameObject it's attached to. I haven't included that calculation in said script, but finding the overall rotation/scale should be fairly trivial from this point.

Resources