I want to create a Test ref from the array ref with the same size and keep the lifetime checking.
I can do this by using a function and I know the function can deduce the lifetime. The code below is intentionally designed to fail when compiling because of use after move. It works.
struct Test {
a: i32,
}
/// 'a can be removed for simplification
fn create_test<'a>(ptr: &'a mut [u8]) -> &'a mut Test {
assert_eq!(ptr.len(), size_of::<Test>());
unsafe { &mut *(ptr as *mut [u8] as *mut Test) }
}
fn main() {
let mut space = Box::new([0 as u8; 100]);
let (s1, _s2) = space.split_at_mut(size_of::<Test>());
let test = create_test(s1);
drop(space);
test.a += 1;
}
My question is how can I do this without declaring an extra function to constrain the lifetime.
fn main() {
let mut space = Box::new([0 as u8; 100]);
let (s1, _s2): (&'a mut [u8], _) = space.split_at_mut(size_of::<Test>());
let test: &'a mut Test = unsafe { &mut *(s1 as *mut [u8] as *mut Test) };
drop(space);
}
such `a is not allowed.
The following code works. And it holds the borrowing check.
fn main() {
let mut space = Box::new([0 as u8; 100]);
let layout = Layout::new::<Test>();
println!("{}", layout.align());
let (_prefix, tests, _suffix) = unsafe { space.align_to_mut::<Test>() };
assert!(tests.len() > 0);
let test = &mut tests[0];
let (_, suffix, _) = unsafe { tests[1..].align_to_mut::<u8>() };
}
You cannot do that, but this is not needed either. Lifetimes are used to ensure safety across function boundaries. In the same function you can just ensure safety manually.
Theoretically, we would not need a borrow checker if the compiler could just inspect the called functions and follow the execution path to deterime whether we invoke Undefined Behavior. Practically, this can't be done because of problems like the Halting Problem and performance.
Related
I know the priciple of Pin is not to expose mutable reference of its inner type. If I want to actually pin a type, I need to implement !Unpin for it.
While doing some research, I know I can create a !Unpin object by Pin::new_unchecked. However, I need to make sure we can not move in P::Deref/DerefMut. What if I both implement !Unpin and a DerefMut that actually move inner fields? Will it make Pin no longer taking effects?
So I write the following code. It defines a EvilNUnpin which it a pointer to String which do evil things. It actually mutates the String b by setting it to 3.
What I expect is by calling std::mem::swap
xp.deref() and xp2.deref() will equal to 3, since we have called DerefMut.
If we save the pointer of String x1.b and x2.b before we wrapped it into a Pin. I think we should finally observe the value of ptr1 and ptr2 is finally 3. However, it is not. I wonder why.
#[derive(Default, Debug)]
struct EvilNUnpin {
b: String,
}
impl !Unpin for EvilNUnpin {}
impl Deref for EvilNUnpin {
type Target = String;
fn deref(&self) -> &Self::Target {
&self.b
}
}
impl DerefMut for EvilNUnpin {
fn deref_mut(&mut self) -> &mut Self::Target {
self.b = "3".to_owned();
&mut self.b
}
}
fn main() {
let mut x1 = EvilNUnpin { b: "1".to_owned() };
let mut x2 = EvilNUnpin { b: "2".to_owned() };
let ptr1 = &x1.b as *const _ as isize;
let ptr2 = &x2.b as *const _ as isize;
// Note that we consider x1 and x2 as a pointer, so we don't use &x1
let mut xp = unsafe { Pin::new_unchecked(x1) };
let mut xp2 = unsafe { Pin::new_unchecked(x2) };
std::mem::swap(&mut xp.as_mut(), &mut xp2.as_mut());
assert_eq!(xp.deref(), "3");
assert_eq!(xp2.deref(), "3");
unsafe {
let n1 = &*(ptr1 as *const String);
let n2 = &*(ptr2 as *const String);
assert_eq!(n1, "3"); // fire, actually 1
assert_eq!(n2, "3"); // fire, actually 2
}
}
Maybe it is because there are actually some copy here?
EDIT
I have changed my implementation, and make sure the inner String is not moved. What I want to check is what will happen if I break the constraint of Pin::new_unchecked. I think !Unpin will not take effect, so we will end up with the two value swapped. However, it is still not correct.
#[derive(Debug)]
struct EvilNUnpin<'a> {
b: &'a mut String,
}
impl !Unpin for EvilNUnpin<'_> {}
impl Deref for EvilNUnpin<'_> {
type Target = String;
fn deref(&self) -> &Self::Target {
&self.b
}
}
impl DerefMut for EvilNUnpin<'_> {
fn deref_mut(&mut self) -> &mut Self::Target {
self.b.push('a');
&mut self.b
}
}
fn main() {
let mut s1 = "1".to_owned();
let mut s2 = "2".to_owned();
let ptr1 = &s1 as *const _ as isize;
let ptr2 = &s2 as *const _ as isize;
let mut x1 = EvilNUnpin { b: &mut s1 };
let mut x2 = EvilNUnpin { b: &mut s2 };
// Note that we don't use a reference.
let mut xp = unsafe { Pin::new_unchecked(x1) };
let mut xp2 = unsafe { Pin::new_unchecked(x2) };
std::mem::swap(&mut xp.as_mut(), &mut xp2.as_mut());
unsafe {
let n1 = &*(ptr1 as *const String);
let n2 = &*(ptr2 as *const String);
assert_eq!(n1, "1a"); // why not 2a
assert_eq!(n2, "2a"); // why not 1a
}
}
From the documentation on Pin::as_mut (emphasis mine):
This is a generic method to go from &mut Pin<Pointer<T>> to Pin<&mut T>. It is safe because, as part of the contract of Pin::new_unchecked, the pointee cannot move after Pin<Pointer<T>> got created. “Malicious” implementations of Pointer::DerefMut are likewise ruled out by the contract of Pin::new_unchecked.
And for Pin::new_unchecked we see this:
This constructor is unsafe because we cannot guarantee that the data pointed to by pointer is pinned, meaning that the data will not be moved or its storage invalidated until it gets dropped. If the constructed Pin<P> does not guarantee that the data P points to is pinned, that is a violation of the API contract and may lead to undefined behavior in later (safe) operations.
By using this method, you are making a promise about the P::Deref and P::DerefMut implementations, if they exist. Most importantly, they must not move out of their self arguments: Pin::as_mut and Pin::as_ref will call DerefMut::deref_mut and Deref::deref on the pinned pointer and expect these methods to uphold the pinning invariants.
And your implementation clearly moves the value inside DerefMut - it drops this value, to be precise.
So, it's assumed that EvilNUnpin is simply unsound to ever be pinned; in other words, Pin::new_unchecked is to blame for the output you see.
It is possible to coerce &mut T into &T but it doesn't work if the type mismatch happens within a type constructor.
playground
use ndarray::*; // 0.13.0
fn print(a: &ArrayView1<i32>) {
println!("{:?}", a);
}
pub fn test() {
let mut x = array![1i32, 2, 3];
print(&x.view_mut());
}
For the above code I get following error:
|
9 | print(&x.view_mut());
| ^^^^^^^^^^^^^ types differ in mutability
|
= note: expected reference `&ndarray::ArrayBase<ndarray::ViewRepr<&i32>, ndarray::dimension::dim::Dim<[usize; 1]>>`
found reference `&ndarray::ArrayBase<ndarray::ViewRepr<&mut i32>, ndarray::dimension::dim::Dim<[usize; 1]>>`
It is safe to coerce &mut i32 to &i32 so why it is not applied in this situation? Could you provide some examples on how could it possibly backfire?
In general, it's not safe to coerce Type<&mut T> into Type<&T>.
For example, consider this wrapper type, which is implemented without any unsafe code and is therefore sound:
#[derive(Copy, Clone)]
struct Wrapper<T>(T);
impl<T: Deref> Deref for Wrapper<T> {
type Target = T::Target;
fn deref(&self) -> &T::Target { &self.0 }
}
impl<T: DerefMut> DerefMut for Wrapper<T> {
fn deref_mut(&mut self) -> &mut T::Target { &mut self.0 }
}
This type has the property that &Wrapper<&T> automatically dereferences to &T, and &mut Wrapper<&mut T> automatically dereferences to &mut T. In addition, Wrapper<T> is copyable if T is.
Assume that there exists a function that can take a &Wrapper<&mut T> and coerce it into a &Wrapper<&T>:
fn downgrade_wrapper_ref<'a, 'b, T: ?Sized>(w: &'a Wrapper<&'b mut T>) -> &'a Wrapper<&'b T> {
unsafe {
// the internals of this function is not important
}
}
By using this function, it is possible to get a mutable and immutable reference to the same value at the same time:
fn main() {
let mut value: i32 = 0;
let mut x: Wrapper<&mut i32> = Wrapper(&mut value);
let x_ref: &Wrapper<&mut i32> = &x;
let y_ref: &Wrapper<&i32> = downgrade_wrapper_ref(x_ref);
let y: Wrapper<&i32> = *y_ref;
let a: &mut i32 = &mut *x;
let b: &i32 = &*y;
// these two lines will print the same addresses
// meaning the references point to the same value!
println!("a = {:p}", a as &mut i32); // "a = 0x7ffe56ca6ba4"
println!("b = {:p}", b as &i32); // "b = 0x7ffe56ca6ba4"
}
Full playground example
This is not allowed in Rust, leads to undefined behavior and means that the function downgrade_wrapper_ref is unsound in this case. There may be other specific cases where you, as the programmer, can guarantee that this won't happen, but it still requires you to implement it specifically for those case, using unsafe code, to ensure that you take the responsibility of making those guarantees.
Consider this check for an empty string that relies on content staying unchanged for the runtime of the is_empty function (for illustration purposes only, don't use this in production code):
struct Container<T> {
content: T
}
impl<T> Container<T> {
fn new(content: T) -> Self
{
Self { content }
}
}
impl<'a> Container<&'a String> {
fn is_empty(&self, s: &str) -> bool
{
let str = format!("{}{}", self.content, s);
&str == s
}
}
fn main() {
let mut foo : String = "foo".to_owned();
let container : Container<&mut String> = Container::new(&mut foo);
std::thread::spawn(|| {
container.content.replace_range(1..2, "");
});
println!("an empty str is actually empty: {}", container.is_empty(""))
}
(Playground)
This code does not compile since &mut String does not coerce into &String. If it did, however, it would be possible that the newly created thread changed the content after the format! call but before the equal comparison in the is_empty function, thereby invalidating the assumption that the container's content was immutable, which is required for the empty check.
It seems type coercions don't apply to array elements when array is the function parameter type.
playground
The following code fails to compile because MutRef is not Copy. It can not be made copy because &'a mut i32 is not Copy. Is there any way give MutRef similar semantics to &'a mut i32?
The motivation for this is being able to package up a large set of function parameters into a struct so that they can be passed as a group instead of needing to be passed individually.
struct MutRef<'a> {
v: &'a mut i32
}
fn wrapper_use(s: MutRef) {
}
fn raw_use(s: &mut i32) {
}
fn raw_ref() {
let mut s: i32 = 9;
let q = &mut s;
raw_use(q);
raw_use(q);
}
fn wrapper() {
let mut s: i32 = 9;
let q = MutRef{ v: &mut s };
wrapper_use(q);
wrapper_use(q);
}
No.
The name for this feature is "implicit reborrowing" and it happens when you pass a &mut reference where the compiler expects a &mut reference of a possibly different lifetime. The compiler only implicitly reborrows when the actual type and the expected type are both &mut references. It does not work with generic arguments or structs that contain &mut references. There is no way in current Rust to make a custom type that can be implicitly reborrowed. There is an open issue about this limitation dating from 2015, but so far nobody has proposed any way to lift it.
You can always implement your own method to explicitly reborrow:
impl<'a> MutRef<'a> {
// equivalent to fn reborrow(&mut self) -> MutRef<'_>
fn reborrow<'b>(&'b mut self) -> MutRef<'b> {
MutRef {v: self.v}
}
}
fn wrapper() {
let mut s: i32 = 9;
let mut q = MutRef{ v: &mut s };
wrapper_use(q.reborrow()); // does not move q
wrapper_use(q); // moves q
}
See also
Why is the mutable reference not moved here?
Type inference and borrowing vs ownership transfer
I am having this error, other times I had something similar and I have been able to solve, in different ways but now is not how to solve in this case:
borrowed value does not live long enough in
I moved the code that fails one more simple, but I can not find the error:
fn main(){
let mut v: Vec<&Fn(i32) -> i32> = Vec::new();
v.push(&ops_code1);
//v.push(&ops_code2);
//v.push(&ops_code3);
}
fn ops_code1(value: i32) -> i32 {
..//
error: borrowed value does not live long enough
v.push(&ops_code1);
play.rust
What you are doing here is creating a Vec of closures. In Rust static functions are treated slightly differently from closures, so when we create the reference a closure is actually created. If we do that after creating the Vec the resulting closure gets a shorter lifetime than the Vec, which is an error. We can instead use a let to create the closure before the Vec, giving a long enough lifetime, outliving the Vec:
fn main() {
let extended = &ops_code1;
let mut v: Vec<&Fn(i32) -> i32> = Vec::new();
// Note that placing it here does not work:
// let extended = &ops_code1;
v.push(extended);
//v.push(&ops_code2);
//v.push(&ops_code3);
}
fn ops_code1(value: i32) -> i32 {
println!("ops_code1 {}", value);
value
}
Rust Playground
However, if you only use static functions - and not closures - the following also works fine, and lets you avoid the extra let:
fn main() {
let mut v: Vec<fn(i32) -> i32> = Vec::new();
v.push(ops_code1);
v.push(ops_code2);
}
fn ops_code1(value: i32) -> i32 {
println!("ops_code1 {}", value);
value
}
fn ops_code2(value: i32) -> i32 {
println!("ops_code2 {}", value);
value
}
Rust Playground
A third option is to use boxed closures, which let's you use both closures and static functions without the extra lets, but with its own trade-offs:
fn main() {
let mut v: Vec<Box<Fn(i32) -> i32>> = Vec::new();
v.push(Box::new(ops_code1));
v.push(Box::new(ops_code2));
for f in v {
f(1);
}
}
fn ops_code1(value: i32) -> i32 {
println!("ops_code1 {}", value);
value
}
fn ops_code2(value: i32) -> i32 {
println!("ops_code2 {}", value);
value
}
Rust Playground
I don't know how to convert a Vec<i32> into a &[u8] slice.
fn main() {
let v: Vec<i32> = vec![1; 100_000_000];
let v_bytes: &[u8] = /* ... */;
}
I want to write a large Vec<i32> to a file so I can read it back at a future time.
You can use std::slice::from_raw_parts:
let v_bytes: &[u8] = unsafe {
std::slice::from_raw_parts(
v.as_ptr() as *const u8,
v.len() * std::mem::size_of::<i32>(),
)
};
Following the comments on this answer, you should wrap this code in a function and have the return value borrow the input, so that you use the borrow checker as far as possible:
fn as_u8_slice(v: &[i32]) -> &[u8] {
unsafe {
std::slice::from_raw_parts(
v.as_ptr() as *const u8,
v.len() * std::mem::size_of::<i32>(),
)
}
}
Since Rust 1.30, the best solution is to use slice::align_to:
fn main() {
let v: Vec<i32> = vec![1; 8];
let (head, body, tail) = unsafe { v.align_to::<u8>() };
assert!(head.is_empty());
assert!(tail.is_empty());
println!("{:#x?}", body);
}
This properly handles the cases where the alignment of the first type and the second type do not match. In this example, I ensure that the alignment of the i32 is greater than that of the u8 via the assert! statements.
I took #swizards answer and ran with it a bit to get the other side of the coin - reading the vector back in:
use std::fs::File;
use std::io::{Read, Write};
use std::{mem, slice};
fn as_u8_slice(v: &[i32]) -> &[u8] {
let element_size = mem::size_of::<i32>();
unsafe { slice::from_raw_parts(v.as_ptr() as *const u8, v.len() * element_size) }
}
fn from_u8(v: Vec<u8>) -> Vec<i32> {
let data = v.as_ptr();
let len = v.len();
let capacity = v.capacity();
let element_size = mem::size_of::<i32>();
// Make sure we have a proper amount of capacity (may be overkill)
assert_eq!(capacity % element_size, 0);
// Make sure we are going to read a full chunk of stuff
assert_eq!(len % element_size, 0);
unsafe {
// Don't allow the current vector to be dropped
// (which would invalidate the memory)
mem::forget(v);
Vec::from_raw_parts(
data as *mut i32,
len / element_size,
capacity / element_size,
)
}
}
fn do_write(filename: &str, v: &[i32]) {
let mut f = File::create(filename).unwrap();
f.write_all(as_u8_slice(v)).unwrap();
}
fn do_read(filename: &str) -> Vec<i32> {
let mut f = File::open(filename).unwrap();
let mut bytes = Vec::new();
f.read_to_end(&mut bytes).unwrap();
from_u8(bytes)
}
fn main() {
let v = vec![42; 10];
do_write("vector.dump", &v);
let v2 = do_read("vector.dump");
assert_eq!(v, v2);
println!("{:?}", v2)
}