I am playing with some models for the game glest.
These models are made up of one or more meshes; each mesh is made up of many frames which describe the position of each vertex for each frame of animation. In the model shown below, the position of each vertex in each wheel in each frame is in an array.
These models have been exported from 3D tools like Blender. Someone somewhere has the originals.
But I am wondering, for simple animation such as a wheel turning, how can you compute the transforms - the steps of rotate, scale and translate, or the matrix that when applied to the previous frame will result in the new frame?
(Obviously not all frames will have such transforms, because they may distort the models and such.)
Also, how can you detect mirroring and other opportunities to reduce the amount of vertex data by applying a matrix and rendering the same vertices again?
Running speed - if its measured in just minutes - won't be a problem.
First off, some assumptions:
You're dealing with 3D affine transformations (linear transformation plus translation).
You have the vertices for each frame in your animation
You can associate at least 4 vertices in a frame with 4 vertices in the next frame
Then you can take 4 vertices as 4D collumn vectors (appending a 1 in each vector's 4th element) in the original space and concatenate them to create a 4x4 matrix, called X. Do the same for their corresponding vectors in the tranformed space and call them Y, which will also be a 4x4 matrix. A little linear algebra provides you with a method to find the 4x4 matrix A that when applied to X gives you Y. Thus:
AX = Y
A = YX-1
Using this to get rotations and scaling is not trivial. However, the rightmost column of A will contain the translation for the object between the successive frames.
Related
3D Density maps of course can be plotted as heatmap, but when data itself is homogeneous (near 0) except for a small part (2D cross section for example):
This should give a letter 'E' shape as 2D "model". The original data is not saved as point-cloud however.
A naive approach would be to use the pixels that are more than a certain value, and then smooth the border. However this does not take into account of the border pixels being small.
Another would be to use some point-cloud based algorithms that come with modeling softwares, but then the point-cloud's probability function would still be discontinuous on pixel border, and not taking into account that only one side have signal.
Is there any tested solution to this (the example is 2D, the actual case is many 2D slices that compose a low-res 3D density map)? I was thinking of making border pixels have area proportional to signal data, and border should be defined from gradient? Any suggestions?
I was thinking of model visualization results similar to this (seems to be based on established point-cloud algorithm):
Many questions deal with generating normal from depth or depth from normal, but I want to ask about a simple way to generate all the planar surfaces given the depth and normal of an image.
I already have depth and normal of each pixel in the image. For each pixel (ui, vi), assume that we can get its 3D coordinates (xi, yi, zi) with zi as the depth and normal vector (nix, niy, niz). Thus, a unique tangent plane is defined by: nix(x - xi) + niy(y - yi) + niz(z - zi) = 0. Then, for each pixel we can define a unique planar surface by the above equation.
What is a common practice in finding the function f such that f(u, v) = (x, y, z) (from pixel to 3D coordinates)? Is pinhole model (plus the depth data) an effective and accurate one?
How does one generate all the planar surfaces effectively? One way is to iterate through all the pixels in the image and find all the planes, but this seems like an ineffective method.
If its pinhole model
make sure your 3D data is not distorted by projection.
group your points by normal
this is easy or hard depending on the points/normal accuracy. Simply sort the points by normals which leads to O(n.log(n)) where n is number of points.
test/group by planes in single normal group
The idea is to pick 3 points from a group compute plane from it and test which points of the group belongs to it. If too low count you got wrong points picked (not belonging to the same plane) and need to pick different ones. Also if the picked points are too close to each or on the same line you can not get correct plane from it.
The math function for plane is:
x*nx + y*ny + z*nz + d = 0
where (nx,ny,nz) is your normal of the group (unit vector) and (x,y,z) is your point position. So you just compute d from a known point (one of the picked ones (x0,y0,z0) ) ...
d = -x0*nx -y0*ny -z0*nz
and then just test which points are sattisfying this condition:
threshod=1e-20; // just accuracy margin
fabs(x*nx + y*ny + z*nz + d) <= threshod
now remove matched points from the group (move them into found plane object) and apply this bullet again on the remaining points until they count is low or no valid plane is found...
then test another group until no groups are left...
I think RANSAC can speed things up to avoid brute force in this case but never used it myself so google ...
A possible approach for the planes is to consider the set of normal vectors and perform clustering on them (for instance by k-means). Then every cluster can correspond to several parallel surfaces. By evaluating the distance from the origin (a scalar function), you can form sub-clusters which will separate those surfaces. Finally, points at constant distance can belong to different coplanar patches, which you can separate by connected component labelling.
It is likely that clustering on the normal vectors and distance simultaneously (hence in a 4D space) will yield better results and be simpler. Be sure to normalize the vectors. Another option is to represent the vectors by just two parameters (such as spherical angles), but this will lead to a quite non-uniform mapping, and create phase wrapping issues.
I have an SVG <path> with points in "model" coordinate system. For simplicity let my path consist of x, sin(x) pairs - note the lack of any scaling and offsets.
To render it on screen I calculated a SVGMatrix and put it into SVGTransformList of my path element. Also I use CSS vector-effect: non-scaling-stroke.
Now I want to pan my sine chart using a mouse, so I got the shift vector in SVG screen coordinates.
My idea is to put one more matrix in my SVGTransformList and calculate it from the screen shift vector.
Should I put this new matrix before or after my original matrix? What is considered a good style? (I know that the coefficients of the second matrix will be different in the two cases)
Also to transform my shift vector to model coordinates I transform back two SVGPoints: zero and with coordinates of my delta vector, and manually subtract the images coordinate-wise. Is it the way to transform vectors, e.g. there are
no better math or API approach?
I'm working on implementing Akush Gupta's synthetic data generation dataset (http://www.robots.ox.ac.uk/~vgg/data/scenetext/gupta16.pdf). In his work. he used a convolutional neural network to extract a point cloud from a 2-dimensional scenery image, segmented the point clouds to isolate different planes, used RANSAC to fit a 3d plane to the point cloud segments, and then warped the pixels for the segment, given the 3D plane, to a fronto-parallel view.
I'm stuck in this last part- warping my extracted 3D plane to a fronto-parallel view. I have X, Y, and Z vectors as well as a normal vector. I'm thinking what I need to do is perform some type of perspective transform or rotation that would bring all the pixels on the plane to a complete 0 Z-axis while the X and Y would remain the same. I could be wrong about this, it's been a long time since I've had any formal training in geometry or linear algebra.
It looks like skimage's Perspective Transform requires me to know the dimensions of the final segment coordinates in 2d space. It looks like AffineTransform requires me to know the rotation. All I have at this point is my X,Y,Z and normal vector and the suspicion that I may know my destination plane by just setting the Z axis to all zeros. I'm not sure if my assumption is correct but I need to be able to warp all the pixels in the segment of interest to fronto-parallel, fit a bounding box, place text inside of it, then warp the final segment back to the original perspective in 3d space.
Any help with how to think about this or implement it would be massively useful.
What i am trying to do http://www.youtube.com/watch?v=CaTI2d0tQME 3:15
In my 3D api there is quad.rotation[x,y,z], quad[x,y,z] which is center of it and width/height. I understand that vertices are being calculated from all of the given. And normal can be calculated from vertices but i have a feeling i should be able to get it just from the rotation?
Yes you can !
Your quad must be axis-oriented (along the X, Y or Z axis, which is its normal vector in its local space). Compose this vector with the quad rotation matrix and you will have your new, nice and shiny normal vector in world space !
A little warning : if the quad transformation matrix is generated by any 3D engine, it could contain scaling factors that will mess the normal vector up. In this case, the classical solution is to compute the transposed inverse of the matrix, or to generate your custom transformation matrix with the quad's rotation values.