I'm working on developing software for an electron microscope.This works by focusing a beam of electrons at a specific part of the sample and then recording the image using a sensor.
The scan data is saved as a 4D array where the first 2 dimensions are the location (x and y) at which the beam was focused and the other 2 dimensions are the raw sensor output which is a 2D image.
while analyzing the data, I realized that there are some stuck pixels which I would ideally be able to repair automatically via software.
Here is an example:
As you can see, the data shape is 256,256,256,256 which means we scanned 256x256 points and the sensor data is a 256x256 image.
On the right data browser window (called nav), you can see the scan location which is 0,0 (also marked on the left window by scanY and scanX). I drew circles around a few of the stuck pixels. here is another scan location for reference:
I can automatically detect these pixels by unraveling the sensor data and checking for locations where the scan values are always the same, but I'm not sure how to repair these.
My first guess was reading the data from all the pixels that are next to this pixel and averaging them, then storing the average value instead of the stuck value, but I'm not sure if this is a good approach.
How do professional software such as Photoshop "repair" or "hide" a defect in the picture? Are there any known algorithms for this issue? I did a bit of searching but didn't manage to find much.
Related
I am trying to do camera-lidar calibration. For that i took some lidar data of a checkerboard. For the extrinsic calibration I selected the corner points of checkerboard in image and in pcd. While selecting the pcd points i noticed that the points for ckecker board is not on the same depth. (I have attached the top view and front view of checkerboard below). Since the points are in different depths, the ectrinsic calibration values I am getting are wrong.
This is the top view..
This is the front view..
Is there any preprocessing step that can be done for reducing this kind of spreading in clusters?
Lidar used here : Benewake CE30D
I'm using python 3.7 to process data from a set of ultrasonic sensors (HC-SR04) and an encoder placed on a moving robot. There are 8 ultrasonic sensors which are located on all four corners of the robot, facing front, sideways and backwards, and these give me the distance from their location to any nearby walls or lack thereof. The encoder is placed on a wheel and gives the distance travelled at the instant of sampling.
The objective here is to use the data from these sensors to map the path taken by the robot through a labyrinth and thus create the map for the labyrinth. And that's where I'm stuck, I've already ran the data through a filter to clean up the noise and I've managed to visually determine the path taken by the robot, i.e. if it's driving towards a wall, if it's turned left or right, but having upwards of four thousands data entries this is not ideal.
I'm looking for guidance or ideas on how to determine wether the robot is moving North, South, East or West and also how to create an image to display the path taken and furthermore the map of the labyrinth.
Any help will be greatly appreciated!
Edit: I'm adding graphs from the first 120 data points 1,2,3,4, as well as the placement of the sensors5
I have been struggling to find a proper image comparison technique. I have a directory in my system with couple of pictures and I am trying to recreate those pictures with same same objects, same lighting and same camera position. I want to know whether the correct camera frame is same as mentioned reference image.
for example, assume, we have a camera mounted in a fixed position, we took a picture using that camera and stored that picture with named 'reference.jpg', now when i run this image comparison algorithm, without changing the camera orientation or any of the surroundings, the algorithm should return the correlation between the referenced image and the current frame, in this scenario it must return something like 1 as nothing is changed and everything is same.
Until now i have been using SSIM technique, although the precision of the technique is very bad, for example if i take a picture and then run SSIM technique in a loop, the deviation between the correlation foctor is very high somewhere like it says 0.72 or so which is very bad for precision.
I need to be able to turn a black and white image into series of lines (start, end points) and circles (start point, radius). I have a "pen width" that's constant.
(I'm working with a screen that can only work with this kind of graphics).
Problem is, I don't want to over complicate things - I could represent any image with loads of small lines, but it would take a lot of time to draw, so I basically want to "approximate" the image using those lines and circles.
I've tried several approaches (guessing lines, working area by area, etc) but none had any reasonable results without using a lot of lines and circles.
Any idea on how to approach this problem?
Thanks in advance!
You don't specify what language you are working in here but I'd suggest OpenCV if possible. If not, then most decent CV libraries ought to support the features that I'm about to describe here.
You don't say if the input is already composed of simple shapes ( lines and polygons) or not. Assuming that it's not, i.e. it's a photo or frame from a video for example, you'll need to do some edge extraction to find the lines that you are going to model. Use a Canny or other edge detector to convert the image into a series of lines.
I suggest that you then extract Circles as they are the richest feature that you can model directly. You should consider using a Hough Circle transform to locate circles in your edge image. Once you've located them you need to remove them from the edge image (to avoid duplicating them in the line processing section below).
Now, for each pixel in the edge image that's 'on' you want to find the longest line segment that it's a part of. There are a number of algorithms for doing this, simplest would be Probabilistic Hough Transform (also available in openCV) to extract line segments which will give you control over the minimum length, allowed gaps etc. You may also want to examine alternatives like LSWMS which has OpenCV source code freely available.
Once you have extracted the lines and circles you can plot them into a new image or save the coordinates for your output device.
Back story: I'm creating a Three.js based 3D graphing library. Similar to sigma.js, but 3D. It's called graphosaurus and the source can be found here. I'm using Three.js and using a single particle representing a single node in the graph.
This was the first task I had to deal with: given an arbitrary set of points (that each contain X,Y,Z coordinates), determine the optimal camera position (X,Y,Z) that can view all the points in the graph.
My initial solution (which we'll call Solution 1) involved calculating the bounding sphere of all the points and then scale the sphere to be a sphere of radius 5 around the point 0,0,0. Since the points will be guaranteed to always fall in that area, I can set a static position for the camera (assuming the FOV is static) and the data will always be visible. This works well, but it either requires changing the point coordinates the user specified, or duplicating all the points, neither of which are great.
My new solution (which we'll call Solution 2) involves not touching the coordinates of the inputted data, but instead just positioning the camera to match the data. I encountered a problem with this solution. For some reason, when dealing with really large data, the particles seem to flicker when positioned in front/behind of other particles.
Here are examples of both solutions. Make sure to move the graph around to see the effects:
Solution 1
Solution 2
You can see the diff for the code here
Let me know if you have any insight on how to get rid of the flickering. Thanks!
It turns out that my near value for the camera was too low and the far value was too high, resulting in "z-fighting". By narrowing these values on my dataset, the problem went away. Since my dataset is user dependent, I need to determine an algorithm to generate these values dynamically.
I noticed that in the sol#2 the flickering only occurs when the camera is moving. One possible reason can be that, when the camera position is changing rapidly, different transforms get applied to different particles. So if a camera moves from X to X + DELTAX during a time step, one set of particles get the camera transform for X while the others get the transform for X + DELTAX.
If you separate your rendering from the user interaction, that should fix the issue, assuming this is the issue. That means that you should apply the same transform to all the particles and the edges connecting them, by locking (not updating ) the transform matrix until the rendering loop is done.