How do I use read_scanline from the piston image crate? - rust

I just started learning Rust and I really like to learn by building something 'real'. So I read through the Book, installed Rust, played around with the language and got cargo run working.
I then decided to try and read an image from disk and convert that image into a Vec. In this case, I want to detect the color of the pixels and store that somehow.
I broke it into multiple parts to learn Rust and its syntax:
Starting with:
Read an image from disk
Get the pixels of an image
Show colour code
This led me to the following code using the image crate:
extern crate image;
use std::path::Path;
use image::GenericImage;
fn main() {
let img = image::open(&Path::new("src/maze.gif")).unwrap();
let pixels = img.pixels();
for e in pixels {
let (_, _, color) = e;
println!("Pixel colour {:?}", color);
}
println!("Dimensions {:?}", img.dimensions());
}
So proud as I am, I see some information popping up:
* snip *
Pixel colour Rgba { data: [255, 255, 255, 255] }
Pixel colour Rgba { data: [0, 0, 0, 255] }
Pixel colour Rgba { data: [255, 255, 255, 255] }
* snip *
Now I want to store for each line of the image its pixel information. I would like to have a Vec (is that correct?) with the info. A PHP array would look like this:
$a = [
0 => [ Color, Color, Color, Color],
1 => [ Color, Color, Color, Color]
];
Therefore my assumption is to use read_scanline. This is where reading AND understanding the documentation fails me completely.
What I think I need to do is:
Get the decoder for the current opened image
Call read_scanline on that decoder
Iterate over the results
But how?!
The code reads this:
read_scanline(&mut self, buf: &mut [u8]) -> ImageResult<u32>
I break this down as follows:
First argument, an ImageDecoder object.
Second argument a mutable 8 bit value.
It returns an ImageResult object.
So I tried adapting the code slightly:
extern crate image;
use std::path::Path;
use image::GenericImage;
use image::ImageDecoder;
fn main() {
let img = image::open(&Path::new("src/maze.gif")).unwrap();
let pixels = img.pixels();
let something: &mut [u8];
let result = image::ImageDecoder::read_scanline(img, something);
for e in pixels {
let (_, _, color) = e;
println!("Pixel colour {:?}", color);
}
println!("Dimensions {:?}", img.dimensions());
}
Which, as you might have guessed fails miserably.
error: mismatched types:
expected `&mut _`,
found `image::dynimage::DynamicImage`
(expected &-ptr,
found enum `image::dynimage::DynamicImage`) [E0308]
src/main.rs:13 let result = image::ImageDecoder::read_scanline(img, something);
Obviously this is due to the fact I didn't pass an ImageDecoder object. But how can I? How should I read and understand the documentation. I think it is due to not understanding &mut self. I also do not understand how I should initiate this.
I hope someone can explain what it is I am missing and point me into the right direction.

Note I'm having trouble actually opening a GIF as each line says it is 0 bytes, so there's the possibility I've missed something important... I'll use a JPEG to demonstrate instead.
ImageDecoder is a trait. You need to use a concrete implementation of the trait. The documentation lists all known implementors of the trait, one of which is image::gif::Decoder, another is image::jpeg::JPEGDecoder.
read_scanline accepts a mutable self, which means that you normally will call it with method syntax: object.method(arg1). The first non-self argument is a mutable slice of bytes. The documentation states:
Reads one row from the image into buf and returns the row index
So the pixel data will be stored in the buffer. The next trick is to figure out how many bytes we need for a row and how many rows there are. ImageDecoder::row_len and ImageDecoder::dimensions address that respectively. Put together, we get something like this:
extern crate image;
use std::fs::File;
use image::jpeg;
use image::{GenericImage, ImageDecoder};
use image::{ColorType, Rgb, Pixel};
fn main() {
let f = File::open("/tmp/cat.jpg").unwrap();
let mut decoder = jpeg::JPEGDecoder::new(f);
let (width, height) = decoder.dimensions().unwrap();
let row_len = decoder.row_len().unwrap();
println!("{} x {}, {}", width, height, row_len);
let rows: Vec<_> = (0..height).map(|_| {
let mut row = vec![0; row_len];
decoder.read_scanline(&mut row).unwrap();
row
}).collect();
}
This loads the JPEG, row-by-row, into a Vec<Vec<u8>> - a vector of vectors of bytes.
To convert the raw data into pixels, we can use Pixel::from_slice, another trait method. In this case, I only handle one type of pixel - 8-bit RGB:
let colortype = decoder.colortype().unwrap();
assert_eq!(colortype, ColorType::RGB(8)); // Others are left to you!
let row_colors: Vec<Vec<_>> = rows.iter().map(|r| {
r.chunks(3).map(|p| Rgb::from_slice(p)).collect()
}).collect();
I'm not in love with this because of the hard-coded 3. There should be some way of knowing that this pixel type only takes 3 bytes, but I'm not seeing anything obvious.

Related

Copy a slice of i32 pixels into an [u8] slice

How to copy a row of pixels in an i32 slice into an existing slice of pixels in an [u8] slice ?
Both slices are in the same memory layout (i.e. RGBA) but I don't know the unsafe syntax to copy one efficiently into the other. In C it would just be a memcpy().
You can flat_map the byte representation of each i32 into a Vec<u8>:
fn main() {
let pixels: &[i32] = &[-16776961, 16711935, 65535, -1];
let bytes: Vec<u8> = pixels
.iter()
.flat_map(|e| e.to_ne_bytes())
.collect();
println!("{bytes:?}");
}
There are different ways to handle the endianess of the system, I left to_ne_bytes to preserve the native order, but there are also to_le_bytes and to_be_bytes if that is something that needs to be controlled.
Alternatively, if you know the size of your pixel buffer ahead of time, you can use an unsafe transmute:
const BUF_LEN: usize = 4; // this is your buffer length
fn main() {
let pixels: [i32; BUF_LEN] = [-16776961, 16711935, 65535, -1];
let bytes = unsafe {
std::mem::transmute::<[i32; BUF_LEN], [u8; BUF_LEN * 4]>(pixels)
};
println!("{bytes:?}");
}
Assuming that you in fact do not need any byte reordering, the bytemuck library is the tool to use here, as it allows you to write the i32 to u8 reinterpretation without needing to consider safety (because bytemuck has checked it for you).
Specifically, bytemuck::cast_slice() will allow converting &[i32] to &[u8].
(In general, the function may panic if there is an alignment or size problem, but there never can be such a problem when converting to u8 or any other one-byte type.)

Rusttype how to position glyphs vertically

I'm trying to use the rusttype crate to render text. So far it's been fantastic, but I am running into a wall when trying to correctly position individual glyphs.
I am rendering text by rendering each glyph to an RgbaImage (from the image crate, like in the rusttype image example) and rendering it to a quad mesh.
pub struct Glyph {
pub image: image::RgbaImage,
pub glyph: rusttype::PositionedGlyph<'static>,
pub vertical_offset: f32,
}
// &self.font is a rusttype::Font type
pub fn draw_glyph(&self, ch: char, font_size: f32) -> Option<Glyph> {
// Set size
let scale = rusttype::Scale::uniform(font_size);
let v_metrics = self.font.v_metrics(scale);
// Position and scale glyph
let offset = rusttype::point(0.0, 0.0 + v_metrics.ascent);
let glyph = self.font.glyph(ch).scaled(scale).positioned(offset);
// Get glyph dimensions
let bounds = glyph.pixel_bounding_box()?;
let glyph_height = (v_metrics.ascent - v_metrics.descent).ceil() as u32;
let glyph_width = (bounds.max.x - bounds.min.x) as u32;
// Try to align glyphs on a baseline
let offset_y = bounds.height() as f32 - v_metrics.ascent;
// Generate image
let mut image =
image::ImageBuffer::from_pixel(glyph_width, glyph_height, image::Rgba([200; 4]));
glyph.draw(|x, y, v| {
image.put_pixel(x, y, image::Rgba([0 + (v * 255.0) as u8; 4]));
});
Some(Glyph {
glyph: glyph,
image: image,
vertical_offset: offset_y,
})
}
However, without the vertical_offset calculation, I receive the following text:
As you can see, the characters are all over the place.
I need to take into account the ascent and descent size of the font. However, this is where I run into issues.
I can compensate for the ascent using the following calculation:
let bounds = glyph.pixel_bounding_box()?;
let offset_y = bounds.height() as f32 - v_metrics.ascent;
And then shifting the quad meshes downwards by offset_y.
This improves the output when there are no descenders like "g" in the text:
But does not help when there are descending characters involved:
I cannot figure out how the rusttype examples handle this. They use the Font::layout method, but this doesn't have any special vertical align code.
I'm definitely doing something wrong. Please help!
Try compensating for the descent as well:
let offset_y = bounds.height() as f32 - v_metrics.ascent + v_metrics.descent;

Determine The Bit Depth of a DynamicImage in Rust

How could one determine the bit depth of a image::DynamicImage in rust? I'm using fltk-rs function for creating an RgbImageand RgbImage::new() requires that I specify the bit depth of the Rgb byte data its going to be created from. The images I'm loading in my program have various bit depths, so I need to be able to determine the bit depth value for image dynamically.
impl RgbImage
pub fn new(
data: &[u8],
w: u32,
h: u32,
depth: u32
) -> Result<RgbImage, FltkError>
Here's a typical use case where I would like to know the bit depth of my DynamicImage:
image = "0.23.12"
fltk = "0.12.0"
use image::*;
use fltk::*;
//...
let img = image::open("my_img.png").unwrap(); //<---find the bit depth of this loaded image
let (imgx, imgy) = img.dimensions();
let fltk_rgb = RgbImage::new(&img.to_bytes(), imgx, imgy, 4).unwrap(); //<---then use bit depth value here
//...
As #Ivan C said, the easiest way is with a look-up table in a match statement.
use image::DynamicImage;
pub fn bit_depth(image: &image::DynamicImage) -> u32 {
use DynamicImage::*;
match image {
ImageLuma8(_) => 1,
ImageLumaA8(_) => 2,
ImageRgb8(_) => 3,
ImageRgba8(_) => 4,
ImageBgr8(_) => 3,
ImageBgra8(_) => 4,
ImageLuma16(_) => 2,
ImageLumaA16(_) => 4,
ImageRgb16(_) => 6,
ImageRgba16(_) => 8,
}
}
Rust Playground
That said, you'll have to be a little bit careful. FLTK makes some assumptions about what the bit depth says about color order in each pixel (documented here, note in particular that it assumes RGB color order and will reject any bit depths >4). It might be better to create a function that takes a DynamicImage and converts it if necessary to the appropriate FLTK-compatible representation in the image crate (essentially one of ImageLuma8, ImageLumaA8, ImageRgb8 or ImageRgba8), then hard-codes the bit depth for the resulting image type.
DynamicImage has a color() method to get the color type, which itself has a bytes_per_pixel() method:
img.color().bytes_per_pixel()

How to convert Vec<Rgb<u8>> to Vec<u8>

Using the Piston image crate, I can write an image by feeding it a Vec<u8>, but my actual data is Vec<Rgb<u8>> (because that is a lot easier to deal with, and I want to grow it dynamically).
How can I convert Vec<Rgb<u8>> to Vec<u8>? Rgb<u8> is really [u8; 3]. Does this have to be an unsafe conversion?
The answer depends on whether you are fine with copying the data. If copying is not an issue for you, you can do something like this:
let img: Vec<Rgb<u8>> = ...;
let buf: Vec<u8> = img.iter().flat_map(|rgb| rgb.data.iter()).cloned().collect();
If you want to perform the conversion without copying, though, we first need to make sure that your source and destination types actually have the same memory layout. Rust makes very few guarantees about the memory layout of structs. It currently does not even guarantee that a struct with a single member has the same memory layout as the member itself.
In this particular case, the Rust memory layout is not relevant though, since Rgb is defined as
#[repr(C)]
pub struct Rgb<T: Primitive> {
pub data: [T; 3],
}
The #[repr(C)] attribute specifies that the memory layout of the struct should be the same as an equivalent C struct. The C memory layout is not fully specified in the C standard, but according to the unsafe code guidelines, there are some rules that hold for "most" platforms:
Field order is preserved.
The first field begins at offset 0.
Assuming the struct is not packed, each field's offset is aligned to the ABI-mandated alignment for that field's type, possibly creating unused padding bits.
The total size of the struct is rounded up to its overall alignment.
As pointed out in the comments, the C standard theoretically allows additional padding at the end of the struct. However, the Piston image library itself makes the assumption that a slice of channel data has the same memory layout as the Rgb struct, so if you are on a platform where this assumption does not hold, all bets are off anyway (and I couldnt' find any evidence that such a platform exists).
Rust does guarantee that arrays, slices and vectors are densely packed, and that structs and arrays have an alignment equal to the maximum alignment of their elements. Together with the assumption that the layout of Rgb is as specified by the rules I quotes above, this guarantees that Rgb<u8> is indeed laid out as three consecutive bytes in memory, and that Vec<Rgb<u8>> is indeed a consecutive, densely packed buffer of RGB values, so our conversion is safe. We still need to use unsafe code to write it:
let p = img.as_mut_ptr();
let len = img.len() * mem::size_of::<Rgb<u8>>();
let cap = img.capacity() * mem::size_of::<Rgb<u8>>();
mem::forget(img);
let buf: Vec<u8> = unsafe { Vec::from_raw_parts(p as *mut u8, len, cap) };
If you want to protect against the case that there is additional padding at the end of Rgb, you can check whether size_of::<Rgb<u8>>() is indeed 3. If it is, you can use the unsafe non-copying version, otherwise you have to use the first version above.
You choose the Vec<Rgb<u8>> storage format because it's easier to deal with and you want it to grow dynamically. But as you noticed, there's no guarantee of compatibility of its storage with a Vec<u8>, and no safe conversion.
Why not take the problem the other way and build a convenient facade for a Vec<u8> ?
type Rgb = [u8; 3];
#[derive(Debug)]
struct Img(Vec<u8>);
impl Img {
fn new() -> Img {
Img(Vec::new())
}
fn push(&mut self, rgb: &Rgb) {
self.0.push(rgb[0]);
self.0.push(rgb[1]);
self.0.push(rgb[2]);
}
// other convenient methods
}
fn main() {
let mut img = Img::new();
let rgb : Rgb = [1, 2, 3];
img.push(&rgb);
img.push(&rgb);
println!("{:?}", img);
}

How do I get a vector of u8 RGB values when using the image crate?

I need to take an image and get a list of RGB byte values. I am using the image crate. This is what I have:
extern crate image;
fn main() {
let im = image::open("wall.jpg").unwrap().to_rgb();
let data: Vec<[u8; 3]> = im.pixels().flat_map(|p| vec![p.data]).collect();
let rgb: Vec<&u8> = data.iter().flat_map(|p| p.iter()).collect();
println!("First Pixel: {} {} {}", rgb[0], rgb[1], rgb[2]);
}
This seems pretty ugly. I have to introduce an intermediate variable and I get a vector of pointers to the values I really need, so before I can do something else, I would have to map over it again to get the actual values.
All I want is a vector of u8. How do I get that?
As of 0.23.12, to_rgb has been deprecated use DynamicImage::to_rgb8 or DynamicImage::to_bytes instead:
let im = image::open("wall.jpg").unwrap().to_rgb8();
let rgb: Vec<u8> = im.into_raw();
// Alternatively
let bytes = image::open("wall.jpg").unwrap().to_bytes();
Prior to 0.23.12, if you just want the raw data itself, you can just call DynamicImage::raw_pixels:
let im = image::open("wall.jpg").unwrap().to_rgb();
let rgb: Vec<u8> = im.raw_pixels();
If all you're actually interested in is the first pixel though, I'd recommend calling GenericImage::get_pixel:
let im = image::open("wall.jpg").unwrap();
let first_pixel = im.get_pixel(0, 0);
Which you can then turn into a [u8; 3] array of RGB data:
let rgb = first_pixel.to_rbg();
println!("First Pixel: {} {} {}", rgb.data[0], rgb.data[1], rgb.data[2]);

Resources