I want to set different blurFilter.texelSpacingMultiplier for different regions in image in GPUImageCannyEdgeDetection filter is there a way to do that.
The texelSpacingMultiplier is defined as a uniform in the fragment shaders used for this operation. That will remain constant across the image.
If you wished to have this vary in parts of the image, you will need to create a custom version of this operation and its sub-filters that takes in a varying value for this per-pixel.
Probably the easiest way to do this would be to have your per-pixel values for the multiplier be encoded into a texture that would be input as a secondary image. This texture could be read from within the fragment shaders and the decoded value from the RGBA input converted into a floating point value to set this multiplier per-pixel. That would allow you to create a starting image (drawn or otherwise) that would be used as a mask to define how this is applied.
It will take a little effort to do this, since you will need to rewrite several of the sub-filters used to construct the Canny edge detection implementation here, but the process itself is straightforward.
Related
All other possible values of VkColorSpaceKHR specify a recognised colour space encoding, and I assume that creating a swapchain with those values would mean that the final pixel data submitted by the application to the presentation queue would be transformed, somewhere down the line, into a decoded space, through a known EOTF.
The specification defines VK_COLOR_SPACE_PASS_THROUGH_EXT as follows:
"VK_COLOR_SPACE_PASS_THROUGH_EXT specifies that color components are used 'as is'. This is intended to allow applications to supply data for color spaces not described here."
What would this mean for my application? Let's say (entirely hypothetically) that I was building some fancy spectrum renderer and I was using a custom system for representing colours, and then that would all be transformed into a final RGB format, with just the three channels of intensity values, independent of colour space or any encoding.
Would my image really just be blitted to the screen "as is", as the specification claims? If so, how does the implementation know what to do with the given pixel data? Would it result in every pixel on my display being set to those raw RGB intensities, with no transfer functions being applied?
Thanks, and as always, I do not really know what I'm talking about so please bear with me.
Pass-through means pass it through. In terms of display presentation, the Vulkan implementation is an intermediary between the user and the display engine (ie: the part of the OS that deals with how a window accesses and manipulates the screen). Pass-through means for Vulkan to pass the data through as-is. The user is providing the data in the color space that the display engine expects it to be.
I loaded a model using tinyobjloader in a Vulkan application. The color of each vertex simply equals its 3d position. Using RenderDoc I verified that the depth buffer is working correctly:
But the color output shows some weird artifacts where you see vertices that are occluded:
This is how the artifacts look when using phong lighting:
Face orientation and culling is correct
I've tried both SRGB and SFLOAT image formats, both yield the same results
I don't explicitly transition the layouts (and thus don't change the access masks using VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT) but let the subpasses take care of it
Since Vulkan code is commonly very long, I've created a gist so you can look at the main application code. Let me know if you need to see more.
Color blending is order dependent operation, and so tricky when used with depth buffering.
Your code is:
vk::PipelineColorBlendAttachmentState colorBlendAttachment(true,
vk::BlendFactor::eSrcColor, vk::BlendFactor::eOneMinusSrcColor,
vk::BlendOp::eAdd,
vk::BlendFactor::eOne, vk::BlendFactor::eZero,
vk::BlendOp::eAdd,
Primitives (triangles) are processed in primitime order. Here notably, the triangle that is first in your index buffer will be processed first.
Now, as depth testing works is that a fragment proceeds if it passes the depth test. That means one fragment could suceed. Then other fragment with even better depth value could overwrite it.
This affects your Dst blend value. In your case it will either be the clear color, or the previous fragment color, depending on whichever happens first, per the primitive order.
Your blend op is srcColor * srcColor + dstColor * (1-srcColor). If your previous color is 0, then it results in 2*srcColor, which is probably non-sense, but not noticable. But if dstColor is something, then your output becomes some bright artifact color with more of a Dst's tint.
I'm new to WebGL and for an assignment I'm trying to write a function which takes as argument an object, let's say "objectA". ObjectA will not be rendered but if it overlaps with another object in the scene, let’s say “objectB”, the part of objectB which is inside objectA will disappear. So the effect is that there is a hole in ObjectB without modifying its mesh structure.
I've managed to let it work on my own render engine, based on ray tracing, which gives the following effect:
image initial scene:
image with objectA removed:
In the first image, the green sphere is "objectA" and the blue cube is "objectB".
So now I'm trying to program it in WebGL, but I'm a bit stuck. Because WebGL is based on rasterization rather than ray tracing, it has to be calculated in another way. A possibility could be to modify the Z-buffer algorithm, where the fragments with a z-value lying inside objectA will be ignored.
The algorithm that I have in mind works as follows: normally only the fragment with the smallest z-value will be stored at a particular pixel containing the colour and z-value. A first modification is that at a particular pixel, a list of all fragments belonging to that pixel is maintained. No fragments will be discarded. Secondly per fragment an extra parameter is stored containing the object where it belongs to. Next the fragments are sorted in increasing order according to their z-value.
Then, if the first fragment belongs to objectA, it will be ignored. If the next one belongs to objectB, it will be ignored as well. If the third one belongs to objectA and the fourth one to objectB, the fourth one will be chosen because it lies outside objectA.
So the first fragment belonging to objectB will be chosen with the constraint that the amount of previous fragments belonging to objectA is even. If it is uneven, the fragment will lie inside objectA and will be ignored.
Is this somehow possible in WebGL? I've also tried to implement it via a stencil buffer, based on this blog:
WebGL : How do make part of an object transparent?
But this is written for OpenGL. I transformed the code instructions to WebGL code instructions, but it didn't work at all. But I'm not sure whether it will work with a 3D object instead of a 2D triangle.
Thanks a lot in advance!
Why wouldn't you write raytracer inside the fragment shader (aka pixel shader)?
So you would need to render a fullscreen quad (two triangles) and then the fragment shader would be responsible for raytracing. There are plenty of resources to read/learn from.
This links might be useful:
Distance functions - by iq
How shadertoy works
Simple webgl raytracer
EDIT:
Raytracing and SDFs (signed distance functions aka constructive solid geometry (CSGs)) are good way to handle what you need and how is generally achieved to intersect objects. Intersections, and boolean operators in general, for mesh geometry (i.e. made of polygons) is not done during the rendering, rahter it uses special algorithms that do all the processing ahead of rendering, so the resulting mesh actually exists in the memory and its topology is actually calculated and then just rendered.
Depending on the specific scenario that you have, you might be able to achieve the effect under some requirements and restrictions.
There are few important things to take into account: depth peeling (i.e. storing depth values of multiple fragments per single pixel, triangle orientation (CW or CCW) and polygon face orientation (front-facing or back-facing).
Say, for example, that both of your polygons are convex, then rendering backfacing polygons of ObjectA, then of ObjectB, then frontfacing polygons of A, then of B might achieve the desired effect (I'm not including full calculations for all cases of overlaps that can exist).
Under some other sets of restrictions you might be able to achieve the effect.
In your specific example in question, you have shown frontfacing faces of the cube, then in the second image you can see the backface of the cube. That already implies that you have at least two depth values per pixel stored somehow.
There is also a distinction between intersecting in screen-space, or volumes, or faces. Your example works with faces and is the hardest (there are two cases: the one you've shown where mesh A's pixels who are inside mesh B are simply discarded (i.e. you drilled a hole inside its surface), and there is a case where you do boolean operation where you never put a hole in the surface, but in the volume) and is usually done with algorithm that computes output mesh. SDFs are great for volumes. Screen-space is achieved by simply using depth test to discard some fragments.
Again, too many scenarios and depends on what you're trying to achieve and what are the constraints that you're working with.
In a real-time graphics application, I believe a frame buffer is the memory that holds the final rasterised image that will be displayed for a single frame.
References to deep frame buffers seem to imply there's some caching going on (vertex and material info), but it's not clear what this data is used for, or how.
What specifically is a deep frame buffer in relation to a standard frame buffer, and what are its uses?
Thank you.
Google is your friend.
It can mean two things:
You're storing more than just RGBA per pixel. For example, you might be storing normals or other lighting information so you can do re-lighting later.
Interactive Cinematic Relighting with Global Illumination
Deep Image Compositing
You're storing more than one color and depth value per pixel. This is useful, for example, to support order-independent transparency.
A z buffer is similar to a color buffer which is usually used to store the "image" of a 3D scene, but instead of storing color information (in the form a 2D array of rgb pixels), it stores the distance from the camera to the object visible through each pixel of the framebuffer.
Traditionally, z-buffer only sore the distance from the camera to the nearest object in the 3D for any given pixel in the frame. The good thing about this technique is that if 2 images have been rendered with their z-buffer, then they can be re-composed using a 2D program for instance, but pixels from the image A which are in "front" of the pixels from image "B", will be composed on top of the re-composed image. To decide whether these pixels are in front, we can use the information stored in the images' respective z-buffer. For example, imagine we want to compose pixels from image A and B at pixel coordinates (100, 100). If the distance (z value) stored in the z-buffer at coordinates (100, 100) is 9.13 for image A and 5.64 for image B, the in the recomposed image C, at pixel coordinates (100, 100) we shall put the pixel from the image B (because it corresponds to a surface in the 3D scene which is in front of the object which is visible through that pixel in image A).
Now this works great when objects are opaque but not when they are transparent. So when objects are transparent (such as when we render volumes, clouds, or layers of transparent surfaces) we need to store more than one z value. Also note, that "opacity" changes as the density of the volumetric object or the number of transparent layers increase. Anyway, just to say that a deep image or deep buffer is technically just like a z-buffer but rather than storing only one depth or z values it stores not only more than one depth value but also stores the opacity of the object at each one of these depth value.
Once we have stored this information, it is possible in post-production to properly (that is accurately) recompose 2 or more images together with transparencies. For instance if you render 2 clouds and that these clouds overlap in depth, then their visibility will be properly recomposed as if they had been rendered together in the same scene.
Why would we use such technique at all? Often because rendering scenes containing volumetric elements is generally slow. Thus it's good to render them seprately from other objects in the scene, so that if you need to make tweaks to the solid objects you do not need to re-render the volumetrics elements again.
This technique was mostly made popular by Pixar, in the renderer they develop and sell (Prman). Avatar (Weta Digital in NZ) was one of the first film to make heavy use of deep compositing.
See: http://renderman.pixar.com/resources/current/rps/deepCompositing.html
The cons of this technique: deep images are very heavy. It requires to store many depth values per pixels (and these values are stored as floats). It's not uncomon for such images to be larger than a few hundred to a a couple of gigabytes depending on the image resolution and scene depth complexity. Also you can recompose volume object properly but they won't cast shadow on each other which you would get if you were rendering objects together in the same scene. This make scene management slightly more complex that usual, ... but this is generally dealt with properly.
A lot of this information can be found on scratchapixel.com (for future reference).
Does, anyone know, how I can change brightness and contrast of color image. I know about vtkImageMapToWindowLevel, but after setting level or window of image in this class, the color image becomes grayscale.
Thanks for answers;
By definition, a color image is already color mapped, and you cannot change the brightness/contrast of the image without decomposition and recomposition.
First, define a pair of numbers called brightness and contrast in whatever way you want. Normally, I'd take brightness as the maximum value, and contrast as the ratio between minimum and maximum. Similarly, if you want to use Window/Level semantics, "level" is the minimum scalar value, and window is the difference between maximum and minimum.
Next, you find the scalar range - the minimum and maximum values in your desired output image, using the brightness and contrast. If you're applying brightness/contrast, the scalar range is:
Maximum = brightness
Minimum = Maximum / contrast
Assume a color lookup table (LUT), with a series of colors at different proportional values, say, in the range of 0 to 1. Now, since we know the brightness and contrast, we can setup the LUT with the lower value (range 0) mapping to "minimum" and the upper value (range 1) mapping to "maximum". When this is done, a suitable class, like vtkImageMapToColors can take the single-component input and map it to a 3 or 4 component image.
Now, all this can happen only for a single-component image, as the color LUT classes (vtkScalarsToColors and related classes) are defined only on single-component images.
If you have access to the original one-component image, and you're using vtkImageMapToColors or some similar class, I'd suggest handling it at that stage.
If you don't, there is one way I can think of:
Extract the three channels as three different images using vtkImageExtractComponents (you'll need three instances, each with the original image as input).
Independently scale the 3 channels using vtkImageShiftScale (shift by brightness, scale by contrast)
Combine the channels back using vtkImageAppendComponents
Another possibility is to use vtkImageMagnitude, which will convert the image back to grey-scale (by taking the magnitude of the three channels together), and re-applying the color table using vtkImageMapToColors and any of the vtkScalarsToColors classes as your lookup table.
The first method is better if your image is a real photograph or something similar, where the colors are from some 3-component source, and the second would work better if your input image is already using false colors (say an image from an IR camera, or some procedurally generated fractal that's been image mapped).