How to convert raw YUV image to jpg - linux

Question

I have a raw image that was taken with v4l2-ctl after the camera had been setup like:
# media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]"
# media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/1280x960 field:none]"
# media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/1280x960 field:none]"
and then the picture got snapped with:
# v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=frame.raw --stream-count=1
now I've tried multiple methods to convert this into a jpeg but nothing seems to yield the expected output
the raw file can be downloaded here: https://drive.google.com/file/d/1VqXnrJDYbzdtSsWfTlm2mX9rl1-Rl_7F/view?usp=sharing
I tried out the following command:
convert -verbose -size 1280x960 UYVY:frame.raw frame.bmp
which I found on Converting from YUV(UYVY) to RGB using imagemagick
but it doesn't do the trick

Answer 1

Your frame is 2457600 bytes and your pixel dimensions are 1280x960, so you have:
bits per pixel = 2457600 * 8 / (1280 * 960) = 16
You can get a list of the pixel formats that ffmpeg supports using:
ffmpeg -pix_fmts 2> /dev/null
Sample Output
FLAGS NAME NB_COMPONENTS BITS_PER_PIXEL
-----
IO... yuv420p 3 12
IO... yuyv422 3 16
IO... rgb24 3 24
IO... bgr24 3 24
IO... yuv422p 3 16
IO... yuv444p 3 24
IO... yuv410p 3 9
...
...
That means you can get a list of pixel formats that contain Y, U and V with 16 bits per pixel like this:
ffmpeg -pix_fmts 2> /dev/null | awk '/y/ && /u/ && /16$/ {print}'
IO... yuyv422 3 16
IO... yuv422p 3 16
IO... yuvj422p 3 16
IO... uyvy422 3 16
IO... yuv440p 3 16
IO... yuvj440p 3 16
IO... yvyu422 3 16
Now you can run a loop, iterating over all the 16-bit per pixel YUV formats and see what ffmpeg makes of your image - naming each result after the format so you can identify which is which:
ffmpeg -pix_fmts 2> /dev/null |
awk '/y/ && /u/ && /16$/ {print $2}' |
while read f; do
ffmpeg -y -s:v 1280x960 -pix_fmt $f -i frame.raw $f.jpg
done
That gives you these files:
-rw-r--r-- 1 mark staff 304916 3 Feb 09:38 yuv440p.jpg
-rw-r--r-- 1 mark staff 227123 3 Feb 09:38 yuvj422p.jpg
-rw-r--r-- 1 mark staff 39543 3 Feb 09:38 yuyv422.jpg
-rw-r--r-- 1 mark staff 39545 3 Feb 09:38 yvyu422.jpg
And I guess that yuyv422.jpg is your image, so that means you can extract it with:
ffmpeg -y -s:v 1280x960 -pix_fmt yuyv422 -i frame.raw result.jpg
If you wanted to do that with ImageMagick, you could do something like this:
#!/bin/bash
python3 <<EOF
import numpy as np
h, w = 960, 1280
# Load raw file into Numpy array
raw = np.fromfile('frame.raw', np.uint8)
raw[0::2].tofile('Y') # Starting at the 1st byte, write every 2nd byte to file "Y"
raw[1::4].tofile('U') # Starting at the 2nd byte, write every 4th byte to file "U"
raw[3::4].tofile('V') # Starting at the 3rd byte, write every 4th byte to file "V"
EOF
# Load the Y channel, then the U and V channels forcibly resizing them, then combine and go to sRGB
magick -depth 8 -size 1280x960 gray:Y \
\( -size 640x960 gray:U gray:V -resize 1280x960\! \) \
-set colorspace YUV -combine -colorspace sRGB result.jpg
If yo don't like/have Python, that part can be replaced with some basic C as follows:
#include <stdint.h>
#include <stdio.h>
// Split YUYV file called "frame.raw" into separate channels with filenames "Y", "U" and "V"
// Compile with: clang -O3 splitter.c -o splitter
int main(){
FILE *in, *Y, *U, *V;
uint8_t buffer[4];
size_t bytesRead;
// Open input file and 1 output file per channel
in = fopen("frame.raw", "rb");
Y = fopen("Y", "wb");
U = fopen("U", "wb");
V = fopen("V", "wb");
// read up to sizeof(buffer) bytes
while ((bytesRead = fread(buffer, 1, sizeof(buffer), in)) > 0)
{
fputc(buffer[0], Y);
fputc(buffer[1], U);
fputc(buffer[2], Y);
fputc(buffer[3], V);
}
}
Having had so much fun doing ffmpeg, Python, and C versions, I thought I'd try just doing it in the shell - converting bytes to lines and so I could pick alternate lines instead of alternate bytes. This works the same as the above:
#!/bin/bash
# Build JPEG image from YUYV image with packed bytes in order YUYVYUYV...
# Use "xxd" to convert bytes into lines, then extract alternate lines - which is easier than extracting bytes
H=960
W=1280
INPUT="frame.raw"
# Take top byte of every uint16 and put into "Y.pgm"
xxd -c1 -p "$INPUT" | sed -n 'p;n' | xxd -r -p | magick -size ${W}x${H} -depth 8 gray:- Y.pgm
# Take bottom byte of every 2nd uint16, starting at the 1st, resize up to full width and put into "U.pgm"
xxd -c1 -p "$INPUT" | sed -n 'n;p' | sed -n 'p;n' | xxd -r -p | magick -size $((W/2))x${H} -depth 8 gray:- -resize ${W}x${H}\! U.pgm
# Take bottom byte of every 2nd uint16, starting at the 2nd, resize up to full width and put into "V.pgm"
xxd -c1 -p "$INPUT" | sed -n 'n;p' | sed -n 'n;p' | xxd -r -p | magick -size $((W/2))x${H} -depth 8 gray:- -resize ${W}x${H}\! V.pgm
# Load the 3 channels, combine and convert to JPEG
magick {Y,U,V}.pgm -set colorspace YUV -combine -colorspace sRGB result.jpg
# Remove litter
rm {Y,U,V}.pgm
As regards colour cast removal, as I said in the comments, the " normal" way, AFAIK, is to get the average colour of the image and invert its Hue then blend that "negated cast" back with the original image to offset the original colour cast. Here is a crude attempt - if anyone knows better please ping me!
Step 1: Get average colour cast
magick result.jpg -resize 1x1\! cast.png
Step 2: Invert the cast
magick cast.png -modulate 100,100,0 correction.png
Step 3: Blend the original with the correction and brighten maybe
magick result.jpg correction.png -define compose:args=50,50 -compose blend -composite -auto-level result.jpg
Here are the original and corrected versions:
Obviously you can change the percentages for different degrees of "correction".