Install OV7251 driver in RPI-3B+ to use with ROS - linux

Question

I wanted to install the OV7251 camera driver to work with a module I've recently purchased, the Arducam OV7251 MIPI, as I need to perform SLAM-like system called Virtual Inertial Navigation (VIN) and global shutter cameras are preferred for this. As far as my system goes, I'm using ROS Kinetic on an RPI-3B+ running Ubuntu 16.04 . I am using this camera as it is near my price point (<20$), and goes through the RPI's CSI Port, which sources say is easier and faster than ones going through USB.
I wanted to take this camera and publish its data to a topic, that way the repository I'm using for VIN, OpenVINS, can track the camera's position. Now, the camera that I'm using doesn't have much on it other than the manufacturer's github page, which does not work on Ubuntu, and cannot connect to ROS. Now, I'm decently inexperienced with RPI's, ROS included, since I wanted to originally perform this on an Arduino, but that was majorly impossible, so I doubt I would be able to write a simple ROS node, let alone one that connected with the CSI port.
Currently, I am unable to find many libraries for this, and help given to me has proved to be un-substantial. The camera does not natively have drivers supported on RPI, which is why I cannot find any /dev/video libraries, cheese turns up nothing, and the command $ Vcgencmd get_camera returns no detected devices. Someone suggested kernel hacking, in order to enable the module in menuconfig using libraries like the ones here. While I do not know much about kernel hacking, he reccomended that I follow this guide and after I run the defconfig line, I should search for "OV7251" in menuconfig and modularize the only one which popped up. And despite flashing and repeating this process multiple times to ensure I did not choose the wrong branch, the rpi-5.4.y branch, or wrong model, the RPI-3B+, I ended up being stuck on the rainbow screen after I rebooted every time. I know that the rainbow screen either means low power, which it wasn't because I had it run before, or a kernel error, which would most likely make sense.
Now, while I would most definitely like to fix the rainbow screen error, I would also like to know, how after installing the OV7251 driver, how do I get it working with ROS to send data to topics? Since I doubt I could write my own node, is there a library that I could look for to perform this, or would libraries that did not work previously due to a missing driver suddenly work now, or would I have to take an existing one and modify it? In any case, A more low-level tutorial to accomplish this would be quite handy seeing as I am new.
But, in the case this is not software, and the reason this camera is not supported is for good reason, is there any other cheap global shutter camera I can work with? I couldn't seem to find many over my various searches, but maybe you all have better luck/experience in this field. Although, I did manage to find another library by this same manufacturer which happens to support my camera model and even has a ROS node that works on ubuntu. However, I believe that if this can be done, then so can doing so by just the CSI port rather than buying an additional 40$ USB camera hat for the pi, and along with that, I am starting to doubt the validity of this companies repositories.
Yet the fact I am finding little information on the topic of this camera alone on the CSI port of an RPI and how renowned this company it scares me that it could be impossible, which if it is, do link me some other good and hopefully well-documented cameras, which could very well be a lot to ask for. And if it is just simply impossible to get the results I want with the parameters I have set, then how badly would a rolling shutter camera affect VIN'S performance, and furthermore is there any special dataset designed for rolling shutter which could minimize the drop in quality? This terrain is all too new to me.

Answer 1

Ok, so I got a rpi engineer to add a dtoverlay for the ov7251 in the rpi's firmware, and the most recent rpi-update has the overlay in the kernel.
I did sudo rpi-update to install the update, i then added dtoverlay=ov7251 to /boot/config.txt in order to enable the overlay, and i edited it by running sudo nano /boot/config.txt. And the repository only has one dependency, v4l-utils, which is installed easily enough by running sudo apt-get install v4l-utils. Finally i ran sudo reboot to initialize the changes.
And in order to pull the images into ROS, i edited a v4l2 node called usb_cam in order to accept the pixel format that the ov7251 camera uses (Y10). My fork can be found here. In order to install it, (since the docs for the original repo say very little on installation), i ran:
cd ~/catkin_ws/src
git clone https://github.com/ai-are-better-than-humans/usb_cam.git
cd ..
catkin_make
and then after that all you have to do is roslaunch usb_cam usb_cam-test.launch to start the node. Mine started out dark, so i had to go into the launch file and mess around with the brightness for a bit. And while youre there, make sure the pixel_format parameter has a value of Y10
You should get a sensor_msgs::Image message being published to a topic named "<camera_name>/image_raw", you can run rqt_graph to visualize it. Big thanks to 6by9 over at raspberry pi forums, dont think i could have gotten it done without him, he did alot of work that im very thankfull for. Thought id share the knowledge back here in case anyone finds it usefull.
EDIT: I hear you can also compile with catkin_make --pkg usb_cam -DCMAKE_BUILD_TYPE=Release instead of catkin_make if the node takes too much CPU. Also, if you see a ton of error messages while compiling, its fine, it still should work, but if you want to get rid of them you can refer to this answer from a ros thread:
It looks like you need to install libavcodec. I don't know the exact
command to install it off the top of my head, but the format will look
like this:
sudo apt-get install libavcodec
The exact package name might not be
libavcodec. It maybe looks something like libavcodec-VERSION-NUMBER or
libavcodec-dev. In these situations you can search for packages with a
command like this:
apt-cache search libavcodec
This will find all packages that have text
containing "libavcodec". This should find the correct package for you
to install.