I am using wgpu-rs.
I am drawing a 2D point that's within a triangle. I want this point to have a single color throughout.
My current implementation is repeating some bytes and seems wasteful. I think there must be a better way.
For each (x,y) pair, it is also including a Vec3<f32> for color, as (r,g,b), a Vec2<f32> for the center, and an f32 value for the radius. My problem is that this data is repeated for each of the 3 vertices, when I only need it once.
I've written a struct that looks like this to hold the data:
pub struct Point {
pub position: [f32; 2],
pub color: [f32; 3],
pub center: [f32; 2],
pub radius: f32,
}
And the memory layout that goes to build the wgpu::VertexAttributes looks like this:
&[
(mem::size_of::<[f32; 2]>(), wgpu::VertexFormat::Float32x2),
(mem::size_of::<[f32; 3]>(), wgpu::VertexFormat::Float32x3),
(mem::size_of::<[f32; 2]>(), wgpu::VertexFormat::Float32x2),
(mem::size_of::<[f32; 1]>(), wgpu::VertexFormat::Float32),
]
(This code is incomplete as is and is only given to clarify my question.)
My question is:
As I am passing the same radius, the same center, and the same color for each vertex, is there a way to bundle the data onto the vertex buffer such that this information is not repeated for each vertex? I cannot use a uniform here (I would think) because I might be drawing many points of different radii and colors at once, so these variables vary. For every point, the radius, center, and color data is repeated 3 times when I only need it once.
I don't know how to do this as I need the color, center, and radius information for each vertex within the shader module.
I'm not sure if I can, for example, separate the data into distinct structs or use 2 buffers, or something else. It seems there must be some way to reduce the usage of these excess bytes as it would be quite wasteful if many points were drawn.
You can use an index buffer to repeat vertices. An index buffer should contain an array of 16 or 32-bit indices into a vertex buffer, and is specified using set_index_buffer(). When you draw_indexed() instead of draw(), the GPU iterates over the index buffer instead of the vertex buffer, and looks up vertex data from the vertex buffer using each index in the index buffer.
Index buffers are normally used so that each vertex in a complex mesh may be shared by many triangles, but they will also work for your situation; the index buffer would contain
[0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, ...]
No other part of your code needs to change to use an index buffer; you just need to create it, set_index_buffer() it, and call draw_indexed() instead of draw().
It's also possible to leave out the index buffer entirely and do vertex lookups using vertex shader code, but that requires using the vertex buffer differently and I haven't yet used that technique myself.
I think you might be already doing this, but just in case you aren't: a triangle whose vertices all have identical positions as output by the vertex shader will collapse to no pixels. In order to use 3 identical vertices to draw anything, you'll need to have the vertex shader offset the position for each corner of the triangle, using the vertex_index built-in value modulo 3 to identify which corner of the triangle it's processing.
Related
I'm working on a procedural planet generator that uses a quad sphere, or quadrilateralized spherical cube, to represent its surface.
Most authors seem to number and arrange the faces arbitrarily, and so did I. For example, this is the arrangement that Wikipedia shows for cube maps (apparently the Direct3D convention, although Wikipedia presents it as "the way" without mentioning the zillion alternatives):
But this leads to an issue when you want to know the neighbours of a given pixel, for example for normal mapping, or all sorts of simulations. Given a triple (face, u, v) that identifies a pixel (where u and v are integer indices, not texture coordinates), the task is to find the four triples that identify its four neighbours.
In the face interior, this is easy. But on the edges, you have to take 24 cases of wrapping into account: 6 cube faces × 4 edges per face. In pseudo-C:
Index neighbor(idx: Index, direction: Direction) -> Index {
switch (direction) {
case UP: if (idx.v < SIZE - 1) {
return Index { face: idx.face, u: idx.u, v: idx.v + 1 };
} else {
switch (face) {
case 0: return Index { face: 2, u: SIZE - 1, v: u };
// And so on for the other five faces
}
}
// And so on for the other three directions
}
}
It's tedious and error-prone, and the branching makes it potentially slower than needed.
Then I found the 2007 SIGGRAPH sketch Creating Spherical Worlds (sap_0251) by Compton et al., which mentions:
Further, by choosing face mappings to be permutations of the corresponding axes, it is possible to formulate efficient algorithms for wrapping between faces, and projecting a 3D point into a chart.
This tantalizing sentence is all we get; there's no further explanation, and I can't find any follow-up articles by these authors either.
How can we choose the face mapping to allow for efficient wrapping?
UPDATE
Adding a separate answer because of the difference in approach.
(This answer assumes that the UV coordinates are per-face in the range 0..<SIZE, with SIZE being constant for all faces.)
I can't think of a good way of efficiently computing the the neighbouring (face, u, v) across the boundaries for arbitrary cuboid mappings, but it should be relatively easy to just store a mapping of it for each face. For each face, store four mappings, one for each primary direction in UV space (i.e, +U, -U, +V, -V). Each of these mappings should contain a reference to the next face in that direction along with mapping coefficients for transform (u0, v0) -> (u1, v1).
For the example mapping above, face 2 (the top one) would have the following mappings:
up:
faceID: 5
u: SIZE-u
v: SIZE-1
down:
faceID: 4
u: u
v: SIZE-1
left:
faceID: 1
u: SIZE-v
v: SIZE-1
right:
faceID: 0
u: v
v: SIZE-1
When doing a neighbour lookup, check if the lookup falls outside the dimensions (0..<SIZE) and if it does, use the lookup structure defined above. So if you're looking up the next position along the U dimension on the boundary of face 2, just check the mapping for 'right': face 0, with a u value equal to the original v value, and a v value equal to SIZE-1.
You will need some method for precomputing this data when creating the geometry.
OLD ANSWER
Assuming the following:
This is for doing texture lookups.
You have control over the way textures are generated.
Then I propose an alternate solution.
Instead of finding an efficient way of mapping the neighbour relationship across the discontinuity, simply make the texel value outside the UV map region the same as the value in its corresponding neighbour polygon.
After some quality time with pencil and paper, I figured out a reasonably elegant way to do this. We still need to distinguish between even and odd faces, but that can be done with a modulo operation (compiled down to a bitwise AND), so it doesn't need branching.
The arrangement is as follows:
The mapping of global axes to local axes:
u v normal
+X +Y +Z
-X +Z +Y
-Y -Z +X
+Y +X -Z
+Z -X -Y
-Z -Y -X
Each global axis is used once in each column, so this is indeed a permutation. Whether it's the same one that the Spore developers designed, remains an open question.
There are certain rules now, for example, moving UP from face f,
if f is even (0, 2, 4), we always find face (f + 1) % 6 (1, 3, 5),
if f is odd (1, 3, 5), we always find face (f + 5) % 6 (0, 2, 4).
So the neighbour lookup becomes much simpler:
Index neighbor(idx: Index, direction: Direction) -> Index {
switch (direction) {
case UP: if (idx.v < SIZE - 1) {
return Index { face: idx.face, u: idx.u, v: idx.v + 1 };
} else {
return Index { face: (face + 1 + 4 * (face % 2)) % 6, u: SIZE - 1 - u, v: SIZE - 1 }
}
// And similar for the other three directions
}
}
There might still be room for improvement, so better answers are quite welcome!
Does the technique that vulkan uses (and I assume other graphics libraries too) to interpolate vertex attributes in a perspective-correct manner require that the vertex shader must normalize the homogenous camera-space vertex position (ie: divide through by the w-coordinate such that the w-coordinate is 1.0) prior to multiplication by a typical projection matrix of the form...
g/s 0 0 0
0 g 0 n
0 0 f/(f-n) -nf/(f-n)
0 0 1 0
...in order for perspective-correctness to work properly?
Or, will perspective-correctness continue to work on any homogeneous vertex position in camera-space (with a w-coordinate other than 1.0)?
(I didn't completely follow the perspective-correctness math, so it is unclear which to me which is the case.)
Update:
In order to clarify terminology:
vec4 modelCoordinates = vec4(x_in, y_in, z_in, 1);
mat4 modelToWorld = ...;
vec4 worldCoordinates = modelToWorld * modelCoordinates;
mat4 worldToCamera = ...;
vec4 cameraCoordinates = worldToCamera * worldCoordinates;
mat4 cameraToProjection = ...;
vec4 clipCoordinates = cameraToProjection * cameraCoordinates;
output(clipCoordinates);
cameraToProjection is a matrix like the one shown in the question
The question is does cameraCoordinates.w have to be 1.0?
And consequently the last row of both the modelToWorld and worldToCamera matricies have to be 0 0 0 1?
You have this exactly backwards. Doing the perspective divide in the shader is what prevents perspective-correct interpolation. The rasterizer needs the perspective information provided by the W component to do its job. With a W of 1, the interpolation is done in window space, without any regard to perspective.
Provide a clip-space coordinate to the output of your vertex processing stage, and let the system do what it exists to do.
the vertex shader must normalize the homogenous camera-space vertex position (ie: divide through by the w-coordinate such that the w-coordinate is 1.0) prior to multiplication by a typical projection matrix of the form...
If your camera-space vertex position does not have a W of 1.0, then one of two things has happened:
You are deliberately operating in a post-projection world space or some similar construct. This is a perfectly valid thing to do, and the math for a camera space can be perfectly reasonable.
Your code is broken somewhere. That is, you intend for your world and camera space to be a normal, Euclidean, non-homogeneous space, but somehow the math didn't work out. Obviously, this is not a perfectly valid thing to do.
In both cases, dividing by W is the wrong thing to do. If your world space that you're placing a camera into is post-projection (such as in this example), dividing by W will break your perspective-correct interpolation, as outlined above. If your code is broken, dividing by W will merely mask the actual problem; better to fix your code than to hide the bug, as it may crop up elsewhere.
To see whether or not the camera coordinates need to be in normal form, let's represent the camera coordinates as multiples of w, so they are (wx,wy,wz,w).
Multiplying through by the given projection matrix, we get the clip coordinates (wxg/s, wyg, fwz/(f-n)-nfw/(f-n)), wz)
Calculating the x-y framebuffer coordinates as per the fixed Vulkan formula we get (P_x * xg/sz +O_x, P_y * Hgy/z + O_y). Notice this does not depend on w, so the position in the framebuffer of a polygons verticies doesn't require the camera coordinates be in normal form.
Likewise calculation of the barycentric coordinates of fragments within a polygon only depends on x,y in framebuffer coordinates, and so is also independant of w.
However perspective-correct perspective interpolation of fragment attributes does depend on W_clip of the verticies as this is used in the formula given in the Vulkan spec. As shown above W_clip is wz which does depend on w and scales with it, so we can conclude that camera coordinates must be in normal form (their w must be 1.0)
I have a binary mask of a road, the mask is a little irregular(sometimes even more than depicted in the image).
I have tried houghLine in OpenCV to detect boundary lines, but the boundary lines are not as expected. I tried erosion and dilation to smooth out things, but no luck. Also since the path is curved it becomes even difficult to detect boundary lines using houghLines. How can I modify the code to detect lines better?
img2=cv2.erode(img2,None,iterations=2)
img2=cv2.dilate(img2,None,iterations=2)
can=cv2.Canny(img2,150,50)
lines=cv2.HoughLinesP(can,1,np.pi/180,50,maxLineGap=50,minLineLength=10)
if(lines is not None):
for x in lines:
#print(lines[0])
#mask=np.zeros(frame2.shape,dtype=np.uint8)
#roi=lines
#cv2.fillPoly(mask,roi,(255,255,255))
#cv2.imshow(mask)
for x1,y1,x2,y2 in x:
cv2.line(frame2,(x1,y1),(x2,y2),(255,0,0),2)
You say that Hough is failing but you don't say why. Why is your output "not as expected"? In my experience, Hough Line Detection’s critical points are two: 1) The edges mask you pass to it and 2) how you filter the resulting lines. You should be fine-tuning those two steps and Hough should be enough for your problem.
I don't know what kind of problems the line detector is giving you, but suppose you are interested (as your question suggests) in other methods for lane detection. There are at least two things you could try: 1) Bird's eye transform of the road – which makes line detection much easier since all your lines are now parallel lines. And 2) Contour detection (instead of lines).
Let's examine 2 and what kind of results you can obtain. Listen, man, I offer my answer in C++, but I make notes along with it. I try to highlight the important ideas, so you can implement them in your language of choice. However, if all you want is a CTRL+C and CTRL+V solution, that's ok, but this answer won't help you.
Ok, let's start by reading the image and converting it to binary. Our goal here is to first obtain the edges. Pretty standard stuff:
//Read input image:
std::string imagePath = "C://opencvImages//lanesMask.png";
cv::Mat testImage = cv::imread( imagePath );
//Convert BGR to Gray:
cv::Mat grayImage;
cv::cvtColor( testImage, grayImage, cv::COLOR_RGB2GRAY );
//Get binary image via Otsu:
cv::Mat binaryImage;
cv::threshold( grayImage, binaryImage, 0, 255, cv::THRESH_OTSU );
Now, simply pass this image to Canny's Edge detector. The parameters are also pretty standard. As per Canny's documentation, the ratios between lower and upper thresholds are related by a factor of 3:
//Get Edges via Canny:
cv::Mat testEdges;
//Setup lower and upper thresholds for edge detection:
float lowerThreshold = 30;
float upperThreshold = 3 * lowerThreshold;
cv::Canny( binaryImage, testEdges, lowerThreshold, upperThreshold );
Your mask is pretty good; these are the edges Canny finds:
Now, here's where we are trying something different. We won't use Hough's line detection, instead, let's find the contours of the mask. Each contour is made of points. What we are looking for is actually lines, straight lines that can be fitted to these points. There's more than a method for achieving that. I propose K-means, a clustering algorithm.
The idea is that the points, as you can see, can be clustered in 4 groups: The vanishing point of the lanes (those should be 2 endpoints there) and the 2 starting points of the road. If we give K-means the points of the contour and tell it to cluster the data in 4 separate groups, we should get the means (location) of those 4 points.
Let's try it out. The first step is to find the contours in the edges mask:
//Get contours:
std::vector< std::vector<cv::Point> > contours;
std::vector< cv::Vec4i > hierarchy;
cv::findContours( testEdges, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0) );
K-means needs a specific data type on its input. I'll use a cv::Point2f vector to store all the contour points. Let's set up the variables used by K-means:
//Set up the data containers used by K-means:
cv::Mat centers; cv::Mat labels;
std::vector<cv::Point2f> points; //the data for clustering is stored here
Next, let's loop through the contours and store each point inside the Point2f vector, so we can further pass it to K-means. Let’s use the loop to also draw the contours and make sure we are not messing things up:
//Loop thru the found contours:
for( int i = 0; i < (int)contours.size(); i++ ){
//Set a color & draw contours:
cv::Scalar color = cv::Scalar( 0, 256, 0 );
cv::drawContours( testImage, contours, i, color, 2, 8, hierarchy, 0, cv::Point() );
//This is the current vector of points that is being processed:
std::vector<cv::Point> currentVecPoint = contours[i];
//Loop thru it and store each point as a float point inside a plain vector:
for(int k = 0; k < (int)currentVecPoint.size(); k++){
cv::Point currentPoint = currentVecPoint[k];
//Push (store) the point into the vector:
points.push_back( currentPoint );
}
}
These are the contours found:
There, now, I have the contour points in my vector. Let's pass the info on to K-means:
//Setup K-means:
int clusterCount = 4; //Number of clusters to split the set by
int attempts = 5; //Number of times the algorithm is executed using different initial labels
int flags = cv::KMEANS_PP_CENTERS;
cv::TermCriteria criteria = cv::TermCriteria( CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 10, 0.01 );
//The call to kmeans:
cv::kmeans( points, clusterCount, labels, criteria, attempts, flags, centers );
And that's all. The result of K-means is in the centers matrix. Each row of the matrix should have 2 columns, denoting a point center. In this case, the matrix is of size 4 x 2. Let's draw that info:
As expected, 4 center points, each is the mean of a cluster. Very cool, now, is this approximation enough for your application? Only you know that! You could work with those points and extend both lines, but that's a possible improvement of this result.
I have a polyhedron which has one face missing. I have a library function which can help close the border after sending in an array of points in CCW order:
class Mesh {
// points need to be in CCW order when see from the ouside of this polyhedron
void addAFace(std::vector<Point> points);
}
I have found the vertices on the border and have put them in an array one after the other. How can I know if the order of vertices in this array is in counter-clockwise or clockwise order when seeing from the outside of the polyhedron?
For example, the vertices should be in the order of 0, 1, 2, 3
The polyhedron may be non-convex.
For convex polyhedron: get any point Px in polyhedron not belonging to that face (for example, vertex 4, 5, 6 or 7 for your example) and check sign of triple product - should be negative for CCW order
((P1 - P0) .cross. (P2 - P1)) .dot. (Px - P1)
If there is no guarantee that P0-P3 are properly ordered, then it is needed to check signs of triple products for all face vertices triplets
If polygon is concave, you need some point near given face but lying inside polyhedron. For example, choose any triplet of consequent vertices with acute internal angle, get center of triangle and shift this center along normal by small distance. Check that shifted point is inside, make reflection otherwise.
Is there any example out there of a HLSL written .fx file that splats a tiled texture with different tiles?Like this: http://messy-mind.net/blog/wp-content/uploads/2007/10/transitions.jpg you can see theres a different tile type in each square and there's a little blurring between them to make a smoother transition,but right now I just need to find a way to draw the tiles on a texture.I have a 2D array of integers,each integer equals a corresponding tile type(0 = grass,1 = stone,2 = sand).I opened up a few HLSL examples and they were really confusing.Everything is running fine on the C++ side,but HLSL is proving to be difficult.
You can use a technique called 'texture splatting'. It mixes several textures (color maps) using another texture which contains alpha values for each color map. The texture with alpha values is an equivalent of your 2D array. You can create a 3-channel RGB texture and use each channel for a different color map (in your case: R - grass, G - stone, B - sand). Every pixel of this texture tells us how to mix the color maps (for example R=0 means 'no grass', G=1 means 'full stone', B=0.5 means 'sand, half intensity').
Let's say you have four RGB textures: tex1 - grass, tex2 - stone, tex3 - sand, alpha - mixing texture. In your .fx file, you create a simple vertex shader which just calculates the position and passes the texture coordinate on. The whole thing is done in pixel shader, which should look like this:
float tiling_factor = 10; // number of texture's repetitions, you can also
// specify a seperate factor for each texture
float4 PS_TexSplatting(float2 tex_coord : TEXCOORD0)
{
float3 color = float3(0, 0, 0);
float3 mix = tex2D(alpha_sampler, tex_coord).rgb;
color += tex2D(tex1_sampler, tex_coord * tiling_factor).rgb * mix.r;
color += tex2D(tex2_sampler, tex_coord * tiling_factor).rgb * mix.g;
color += tex2D(tex3_sampler, tex_coord * tiling_factor).rgb * mix.b;
return float4(color, 1);
}
If your application supports multi-pass rendering you should use it.
You should use a multi-pass shader approach where you render the base object with the tiled stone texture in the first pass and on top render the decal passes with different shaders and different detail textures with seperate transparent alpha maps.
(Transparent map could also be stored in your detail texture, but keeping it seperate allows different tile-levels and more flexibility in reusing it.)
Additionally you can use different texture coordinate channels for each decal pass one so that you do not need to hardcode your tile level.
So for minimum you need two shaders, whereas Shader 2 is used as often as decals you need.
Shader to render tiled base texture
Shader to render one tiled detail texture using a seperate transparency map.
If you have multiple decals z-fighting can occur and you should offset your polygons a little. (Very similar to basic simple fur rendering.)
Else you need a single shader which takes multiple textures and lays them on top of the base tiled texture, this solution is less flexible, but you can use one texture for the mix between the textures (equals your 2D-array).