I'm creating a library that tries to be a Cache struct that wraps some structure that would be the component that retrieves "fresh data" for example through some HTTP requests; and also a backend field that is the actual cache backend for the data retrieved through the wrapped component.
I'm facing some problems when trying to "refresh" the data in the cache after retrieving some new data.
use std::collections::HashMap;
pub trait Cacheable<K, V>
where K: std::cmp::Eq + std::hash::Hash
{
fn get(&self, k: &K) -> Option<&V>;
fn set(&mut self, k: K, v: V);
fn del(&mut self, k: &K);
}
pub struct Cache<'a, K, V>
where K: std::cmp::Eq + std::hash::Hash
{
wrapped: &'a dyn Cacheable<K, V>,
backend: Box<dyn Cacheable<K, V>>,
}
impl<'a, K, V> Cache<'a, K, V>
where K: std::cmp::Eq + std::hash::Hash + 'static,
V: 'static,
{
fn New(wrapped: &'a dyn Cacheable<K, V>, backend_type: BackendType) -> Self {
let backend = match backend_type {
BackendType::Memory => MemoryBackend::<K, V>::New(),
};
Cache {
wrapped: wrapped,
backend: Box::new(backend),
}
}
fn get(&mut self, k: &K) -> Option<&V> {
// PROBLEM 1:
// Borrow backend here as immutable
if let Some(v) = self.backend.get(k) {
return Some(v)
}
if let Some(v) = self.wrapped.get(k) {
let cache_k = *k.clone(); // PROBLEM 2: How to clone reference value?
let cache_v = *v.clone(); // PROBLEM 2: How to clone reference value?
// PROBLEM 1:
// Borrow backend here as mutable
self.backend.set(cache_k, cache_v);
return Some(v)
}
None
}
}
pub enum BackendType {
Memory
}
pub struct MemoryBackend<K, V>
where K: std::cmp::Eq + std::hash::Hash
{
data: HashMap<K, V>,
}
impl<K, V> Cacheable<K, V> for MemoryBackend<K, V>
where K: std::cmp::Eq + std::hash::Hash
{
fn get(&self, k: &K) -> Option<&V> {
self.data.get(k)
}
fn set(&mut self, k: K, v: V) {
self.data.insert(k, v);
}
fn del(&mut self, k: &K) {
self.data.remove(k);
}
}
impl<K, V> MemoryBackend<K, V>
where K: std::cmp::Eq + std::hash::Hash
{
fn New() -> Self {
MemoryBackend {
data: HashMap::<K, V>::new(),
}
}
}
Problem 1
Possible coexistence of mutable and immutable references. What would be the best option here?
I tried changing the get method signature to return an Option<V> instead of Option<&V> but that leads me also to the problem 2, for which I can't seem to be able to clone the value of a shared reference.
Problem 2
move occurs because value has type `K`, which does not implement the `Copy` trait
help: consider borrowing here: `&*k.clone()`"
AFAIK I don't want to set Copy trait as bound of V because it might not be a simple type and would much rather clone its value than setting that as an API restriction. I can't seem to work around this.
Problem 1: Possible coexistence of mutable and immutable references. What would be the best option here? I tried changing the get method signature to return an Option<V> instead of Option<&V> but that leads me also to the problem 2, for which I can't seem to be able to clone the value of a shared reference.
You could add a get_or_insert method to the trait, and implement it for MemoryBackend using the hash_map::Entry API:
fn get_or_insert(&mut self, k: K, v: Option<V>) -> Option<&V> {
match self.data.entry(k) {
Entry::Occupied(entry) => Some(entry.into_mut()),
Entry::Vacant(entry) => v.map(|v| &*entry.insert(v)),
}
}
Or, to avoid evaluating v unless it's actually required (which an optimisation pass might already do if the above is inlined), you could instead pass in a function that returns Option<V>:
fn get_or_insert_with(&mut self, k: K, f: &dyn Fn() -> Option<V>) -> Option<&V> {
match self.data.entry(k) {
Entry::Occupied(entry) => Some(entry.into_mut()),
Entry::Vacant(entry) => f().map(|v| &*entry.insert(v)),
}
}
Note that, in order for Cacheable to be object-safe, we have to use the indirection of a function-trait object here (which, again, an optimisation pass might eliminate if the above is inlined).
Problem 2:
"move occurs because value has type K, which does not implement the Copy trait
help: consider borrowing here: &*k.clone()"
AFAIK I don't want to set Copy trait as bound of V because it might not be a simple type and would much rather clone its value than setting that as an API restriction.
So I can't seem to work around this.
You could add K: Clone and V: Clone constraints to the implementation:
impl<'a, K, V> Cache<'a, K, V>
where
K: Clone + std::cmp::Eq + std::hash::Hash + 'static,
V: Clone + 'static,
{
// etc
}
(Playground)
Related
I'm trying to write a class in Rust with custom drop logic that wraps a mutable reference, but no matter what I do, I can't get the lifetimes to work out, and the compiler error messages haven't helped. Can anyone tell what I'm doing wrong, and how to fix it?
Note: I already tried every modification to this code that I could think of, such as removing or reversing the 'b: 'a constraint, but no matter what I do, the compiler produces one kind of inscrutable lifetime error message or another.
pub struct MapRef<'a, K: Eq + Hash, V>{
p: &'a mut HashMap<K, V>,
}
impl<'a, K: Eq + Hash, V> MapRef<'a, K, V> {
pub fn new(p: &'a mut HashMap<K, V>) -> Self {Self{p}}
}
impl<'a, K: Eq + Hash, V> MapRef<'a, K, V> {
pub fn reborrow<'b: 'a>(&'b mut self) -> MapRef<'b, K, V> {
Self::new(self.p)
}
}
impl<'a, K: Eq + Hash, V> Drop for MapRef<'a, K, V> {
fn drop(&mut self) {
println!("dropping ref");
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test() -> Result<(), String> {
let mut m: HashMap<(), ()> = HashMap::new();
let mut r1 = MapRef::new(&mut m);
{
let r2 = r1.reborrow();
}
Ok(())
}
}
Example error message:
|
37 | let r2 = r1.reborrow();
| ^^ borrowed value does not live long enough
...
41 | }
| -
| |
| `r1` dropped here while still borrowed
| borrow might be used here, when `r1` is dropped and runs the `Drop` code for type `util::MapRef`
It turns out that the reason the compiler suggests adding the erroneous 'b: 'a constraint is because the Self in Self::new is implicitly referring to the 'a lifetime. The solution is to change Self::new to MapRef::new, which allows you to drop the bad constraint.
impl<'a, K: Eq + Hash, V> MapRef<'a, K, V> {
pub fn reborrow<'b>(&'b mut self) -> MapRef<'b, K, V> {
MapRef::new(self.p)
}
}
Note that 'b can now be elided, but I left it in here for clarity.
I have a function that takes a collection of &T (represented by IntoIterator) with the requirement that every element is unique.
fn foo<'a, 'b, T: std::fmt::Debug, I>(elements: &'b I)
where
&'b I: IntoIterator<Item = &'a T>,
T: 'a,
'b: 'a,
I would like to also write a wrapper function which can work even if the elements are not unique, by using a HashSet to remove the duplicate elements first.
I tried the following implementation:
use std::collections::HashSet;
fn wrap<'a, 'b, T: std::fmt::Debug + Eq + std::hash::Hash, J>(elements: &'b J)
where
&'b J: IntoIterator<Item = &'a T>,
T: 'a,
'b: 'a,
{
let hashset: HashSet<&T> = elements.into_iter().into_iter().collect();
foo(&hashset);
}
playground
However, the compiler doesn't seem happy with my assumption that HashSet<&T> implements IntoIterator<Item = &'a T>:
error[E0308]: mismatched types
--> src/lib.rs:10:9
|
10 | foo(&hashset);
| ^^^^^^^^ expected type parameter, found struct `std::collections::HashSet`
|
= note: expected type `&J`
found type `&std::collections::HashSet<&T>`
= help: type parameters must be constrained to match other types
= note: for more information, visit https://doc.rust-lang.org/book/ch10-02-traits.html#traits-as-parameters
I know I could use a HashSet<T> by cloning all the input elements, but I want to avoid unnecessary copying and memory use.
If you have a &HashSet<&T> and need an iterator of &T (not &&T) that you can process multiple times, then you can use Iterator::copied to convert the iterator's &&T to a &T:
use std::{collections::HashSet, fmt::Debug, hash::Hash, marker::PhantomData};
struct Collection<T> {
item: PhantomData<T>,
}
impl<T> Collection<T>
where
T: Debug,
{
fn foo<'a, I>(elements: I) -> Self
where
I: IntoIterator<Item = &'a T> + Clone,
T: 'a,
{
for element in elements.clone() {
println!("{:?}", element);
}
for element in elements {
println!("{:?}", element);
}
Self { item: PhantomData }
}
}
impl<T> Collection<T>
where
T: Debug + Eq + Hash,
{
fn wrap<'a, I>(elements: I) -> Self
where
I: IntoIterator<Item = &'a T>,
T: 'a,
{
let set: HashSet<_> = elements.into_iter().collect();
Self::foo(set.iter().copied())
}
}
#[derive(Debug, Hash, PartialEq, Eq)]
struct Foo(i32);
fn main() {
let v = vec![Foo(1), Foo(2), Foo(4)];
Collection::<Foo>::wrap(&v);
}
See also:
Using the same iterator multiple times in Rust
Does cloning an iterator copy the entire underlying vector?
Why does cloning my custom type result in &T instead of T?
Note that the rest of this answer made the assumption that a struct named Collection<T> was a collection of values of type T. OP has clarified that this is not true.
That's not your problem, as shown by your later examples. That can be boiled down to this:
struct Collection<T>(T);
impl<T> Collection<T> {
fn new(value: &T) -> Self {
Collection(value)
}
}
You are taking a reference to a type (&T) and trying to store it where a T is required; these are different types and will generate an error. You are using PhantomData for some reason and accepting references via the iterator, but the problem is the same.
In fact, PhantomData makes the problem harder to see as you can just make up values that don't work. For example, we never have any kind of string here but we "successfully" created the struct:
use std::marker::PhantomData;
struct Collection<T>(PhantomData<T>);
impl Collection<String> {
fn new<T>(value: &T) -> Self {
Collection(PhantomData)
}
}
Ultimately, your wrap function doesn't make sense, either:
impl<T: Eq + Hash> Collection<T> {
fn wrap<I>(elements: I) -> Self
where
I: IntoIterator<Item = T>,
This is equivalent to
impl<T: Eq + Hash> Collection<T> {
fn wrap<I>(elements: I) -> Collection<T>
where
I: IntoIterator<Item = T>,
Which says that, given an iterator of elements T, you will return a collection of those elements. However, you put them in a HashMap and iterate on a reference to it, which yields &T. Thus this function signature cannot be right.
It seems most likely that you want to accept an iterator of owned values instead:
use std::{collections::HashSet, fmt::Debug, hash::Hash};
struct Collection<T> {
item: T,
}
impl<T> Collection<T> {
fn foo<I>(elements: I) -> Self
where
I: IntoIterator<Item = T>,
for<'a> &'a I: IntoIterator<Item = &'a T>,
T: Debug,
{
for element in &elements {
println!("{:?}", element);
}
for element in &elements {
println!("{:?}", element);
}
Self {
item: elements.into_iter().next().unwrap(),
}
}
}
impl<T> Collection<T>
where
T: Eq + Hash,
{
fn wrap<I>(elements: I) -> Self
where
I: IntoIterator<Item = T>,
T: Debug,
{
let s: HashSet<_> = elements.into_iter().collect();
Self::foo(s)
}
}
#[derive(Debug, Hash, PartialEq, Eq)]
struct Foo(i32);
fn main() {
let v = vec![Foo(1), Foo(2), Foo(4)];
let c = Collection::wrap(v);
println!("{:?}", c.item)
}
Here we place a trait bound on the generic iterator type directly and a second higher-ranked trait bound on a reference to the iterator. This allows us to use a reference to the iterator as an iterator itself.
See also:
How does one generically duplicate a value in Rust?
Is there any way to return a reference to a variable created in a function?
How do I write the lifetimes for references in a type constraint when one of them is a local reference?
There were a number of orthogonal issues with my code that Shepmaster pointed out, but to solve the issue of using a HashSet<&T> as an IntoIterator<Item=&T>, I found that one way to solve it is with a wrapper struct:
struct Helper<T, D: Deref<Target = T>>(HashSet<D>);
struct HelperIter<'a, T, D: Deref<Target = T>>(std::collections::hash_set::Iter<'a, D>);
impl<'a, T, D: Deref<Target = T>> Iterator for HelperIter<'a, T, D>
where
T: 'a,
{
type Item = &'a T;
fn next(&mut self) -> Option<Self::Item> {
self.0.next().map(|x| x.deref())
}
}
impl<'a, T, D: Deref<Target = T>> IntoIterator for &'a Helper<T, D> {
type Item = &'a T;
type IntoIter = HelperIter<'a, T, D>;
fn into_iter(self) -> Self::IntoIter {
HelperIter((&self.0).into_iter())
}
}
Which is used as follows:
struct Collection<T> {
item: PhantomData<T>,
}
impl<T: Debug> Collection<T> {
fn foo<I>(elements: I) -> Self
where
I: IntoIterator + Copy,
I::Item: Deref<Target = T>,
{
for element in elements {
println!("{:?}", *element);
}
for element in elements {
println!("{:?}", *element);
}
return Self { item: PhantomData };
}
}
impl<T: Debug + Eq + Hash> Collection<T> {
fn wrap<I>(elements: I) -> Self
where
I: IntoIterator + Copy,
I::Item: Deref<Target = T> + Eq + Hash,
{
let helper = Helper(elements.into_iter().collect());
Self::foo(&helper);
return Self { item: PhantomData };
}
}
fn main() {
let v = vec![Foo(1), Foo(2), Foo(4)];
Collection::<Foo>::wrap(&v);
}
I'm guessing that some of this may be more complicated than it needs to be, but I'm not sure how.
full playground
I am reading the section on closures in the second edition of the Rust book. At the end of this section, there is an exercise to extend the Cacher implementation given before. I gave it a try:
use std::clone::Clone;
use std::cmp::Eq;
use std::collections::HashMap;
use std::hash::Hash;
struct Cacher<T, K, V>
where
T: Fn(K) -> V,
K: Eq + Hash + Clone,
V: Clone,
{
calculation: T,
values: HashMap<K, V>,
}
impl<T, K, V> Cacher<T, K, V>
where
T: Fn(K) -> V,
K: Eq + Hash + Clone,
V: Clone,
{
fn new(calculation: T) -> Cacher<T, K, V> {
Cacher {
calculation,
values: HashMap::new(),
}
}
fn value(&mut self, arg: K) -> V {
match self.values.clone().get(&arg) {
Some(v) => v.clone(),
None => {
self.values
.insert(arg.clone(), (self.calculation)(arg.clone()));
self.values.get(&arg).unwrap().clone()
}
}
}
}
After creating a version that finally works, I am really unhappy with it. What really bugs me is that cacher.value(...) has 5(!) calls to clone() in it. Is there a way to avoid this?
Your suspicion is correct, the code contains too many calls to clone(), defeating the very optimizations Cacher is designed to achieve.
Cloning the entire cache
The one to start with is the call to self.values.clone() - it creates a copy of the entire cache on every single access.
After non-lexical lifetimes
Remove this clone.
Before non-lexical lifetimes
As you likely discovered yourself, simply removing .clone() doesn't compile. This is because the borrow checker considers the map referenced for the entire duration of match. The shared reference returned by HashMap::get points to the item inside the map, which means that while it exists, it is forbidden to create another mutable reference to the same map, which is required by HashMap::insert. For the code to compile, you need to split up the match in order to force the shared reference to go out of scope before insert is invoked:
// avoids unnecessary clone of the whole map
fn value(&mut self, arg: K) -> V {
if let Some(v) = self.values.get(&arg).map(V::clone) {
return v;
} else {
let v = (self.calculation)(arg.clone());
self.values.insert(arg, v.clone());
v
}
}
This is much better and probably "good enough" for most practical purposes. The hot path, where the value is already cached, now consists of only a single clone, and that one is actually necessary because the original value must remain in the hash map. (Also, note that cloning doesn't need to be expensive or imply deep copying - the stored value can be an Rc<RealValue>, which buys object sharing for free. In that case, clone() will simply increment the reference count on the object.)
Clone on cache miss
In case of cache miss, the key must be cloned, because calculation is declared to consume it. A single cloning will be sufficient, though, so we can pass the original arg to insert without cloning it again. The key clone still feels unnecessary, though - a calculation function shouldn't require ownership of the key it is transforming. Removing this clone boils down to modifying the signature of the calculation function to take the key by reference. Changing the trait bounds of T to T: Fn(&K) -> V allows the following formulation of value():
// avoids unnecessary clone of the key
fn value(&mut self, arg: K) -> V {
if let Some(v) = self.values.get(&arg).map(V::clone) {
return v;
} else {
let v = (self.calculation)(&arg);
self.values.insert(arg, v.clone());
v
}
}
Avoiding double lookups
Now are left with exactly two calls to clone(), one in each code path. This is optimal, as far as value cloning is concerned, but the careful reader will still be nagged by one detail: in case of cache miss, the hash table lookup will effectively happen twice for the same key: once in the call to HashMap::get, and then once more in HashMap::insert. It would be nice if we could instead reuse the work done the first time and perform only one hash map lookup. This can be achieved by replacing get() and insert() with entry():
// avoids the second lookup on cache miss
fn value(&mut self, arg: K) -> V {
match self.values.entry(arg) {
Entry::Occupied(entry) => entry.into_mut(),
Entry::Vacant(entry) => {
let v = (self.calculation)(entry.key());
entry.insert(v)
}
}.clone()
}
We've also taken the opportunity to move the .clone() call after the match.
Runnable example in the playground.
I was solving the same exercise and ended with the following code:
use std::thread;
use std::time::Duration;
use std::collections::HashMap;
use std::hash::Hash;
use std::fmt::Display;
struct Cacher<P, R, T>
where
T: Fn(&P) -> R,
P: Eq + Hash + Clone,
{
calculation: T,
values: HashMap<P, R>,
}
impl<P, R, T> Cacher<P, R, T>
where
T: Fn(&P) -> R,
P: Eq + Hash + Clone,
{
fn new(calculation: T) -> Cacher<P, R, T> {
Cacher {
calculation,
values: HashMap::new(),
}
}
fn value<'a>(&'a mut self, key: P) -> &'a R {
let calculation = &self.calculation;
let key_copy = key.clone();
self.values
.entry(key_copy)
.or_insert_with(|| (calculation)(&key))
}
}
It only makes a single copy of the key in the value() method. It does not copy the resulting value, but instead returns a reference with a lifetime specifier, which is equal to the lifetime of the enclosing Cacher instance (which is logical, I think, because values in the map will continue to exist until the Cacher itself is dropped).
Here's a test program:
fn main() {
let mut cacher1 = Cacher::new(|num: &u32| -> u32 {
println!("calculating slowly...");
thread::sleep(Duration::from_secs(2));
*num
});
calculate_and_print(10, &mut cacher1);
calculate_and_print(20, &mut cacher1);
calculate_and_print(10, &mut cacher1);
let mut cacher2 = Cacher::new(|str: &&str| -> usize {
println!("calculating slowly...");
thread::sleep(Duration::from_secs(2));
str.len()
});
calculate_and_print("abc", &mut cacher2);
calculate_and_print("defghi", &mut cacher2);
calculate_and_print("abc", &mut cacher2);
}
fn calculate_and_print<P, R, T>(intensity: P, cacher: &mut Cacher<P, R, T>)
where
T: Fn(&P) -> R,
P: Eq + Hash + Clone,
R: Display,
{
println!("{}", cacher.value(intensity));
}
And its output:
calculating slowly...
10
calculating slowly...
20
10
calculating slowly...
3
calculating slowly...
6
3
If you remove the requirement of returning values, you don't need to perform any clones by making use of the Entry:
use std::{
collections::{hash_map::Entry, HashMap},
fmt::Display,
hash::Hash,
thread,
time::Duration,
};
struct Cacher<P, R, T>
where
T: Fn(&P) -> R,
P: Eq + Hash,
{
calculation: T,
values: HashMap<P, R>,
}
impl<P, R, T> Cacher<P, R, T>
where
T: Fn(&P) -> R,
P: Eq + Hash,
{
fn new(calculation: T) -> Cacher<P, R, T> {
Cacher {
calculation,
values: HashMap::new(),
}
}
fn value<'a>(&'a mut self, key: P) -> &'a R {
let calculation = &self.calculation;
match self.values.entry(key) {
Entry::Occupied(e) => e.into_mut(),
Entry::Vacant(e) => {
let result = (calculation)(e.key());
e.insert(result)
}
}
}
}
fn main() {
let mut cacher1 = Cacher::new(|num: &u32| -> u32 {
println!("calculating slowly...");
thread::sleep(Duration::from_secs(1));
*num
});
calculate_and_print(10, &mut cacher1);
calculate_and_print(20, &mut cacher1);
calculate_and_print(10, &mut cacher1);
let mut cacher2 = Cacher::new(|str: &&str| -> usize {
println!("calculating slowly...");
thread::sleep(Duration::from_secs(2));
str.len()
});
calculate_and_print("abc", &mut cacher2);
calculate_and_print("defghi", &mut cacher2);
calculate_and_print("abc", &mut cacher2);
}
fn calculate_and_print<P, R, T>(intensity: P, cacher: &mut Cacher<P, R, T>)
where
T: Fn(&P) -> R,
P: Eq + Hash,
R: Display,
{
println!("{}", cacher.value(intensity));
}
You could then choose to wrap this in another struct that performed a clone:
struct ValueCacher<P, R, T>
where
T: Fn(&P) -> R,
P: Eq + Hash,
R: Clone,
{
cacher: Cacher<P, R, T>,
}
impl<P, R, T> ValueCacher<P, R, T>
where
T: Fn(&P) -> R,
P: Eq + Hash,
R: Clone,
{
fn new(calculation: T) -> Self {
Self {
cacher: Cacher::new(calculation),
}
}
fn value(&mut self, key: P) -> R {
self.cacher.value(key).clone()
}
}
use std::collections::HashMap;
use std::hash::Hash;
struct Watchable<'a, K, V, W: Watcher<'a, K, V>> {
data: HashMap<K, V>,
watchers: Vec<W>,
}
trait Watcher<'a, K, V> {
fn before_new(&mut self, key: &'a K, value: &'a V);
}
struct IndexWatcher<'a, I: 'a, V: 'a> {
data: HashMap<&'a I, &'a V>,
indexer: fn(&V) -> &I,
}
impl<'a, K, V, I> Watcher<'a, K, V> for IndexWatcher<'a, I, V>
where I: Eq + Hash
{
fn before_new(&mut self, key: &'a K, value: &'a V) {
let index = (self.indexer)(value);
self.data.insert(index, value);
}
}
error[E0392]: parameter `'a` is never used
--> src/main.rs:4:18
|
4 | struct Watchable<'a, K, V, W: Watcher<'a, K, V>> {
| ^^ unused type parameter
|
= help: consider removing `'a` or using a marker such as `std::marker::PhantomData`
Is there any way to remove some lifetime annotation? It seems everything has the same lifetime a.
At first, I didn't put any specific lifetime:
struct IndexWatcher<I, V> {
data: HashMap<&I, &V>,
indexer: fn(&V) -> &I,
}
The compiler complained about lifetimes, so I added it:
struct IndexWatcher<'a, I: 'a, V: 'a> {
data: HashMap<&'a I, &'a V>,
indexer: fn(&V) -> &I,
}
When I tried to implement the trait without lifetimes:
trait Watcher<K, V> {
fn before_new(&mut self, key: &K, value: &V);
}
impl<'a, K, V, I> Watcher<K, V> for IndexWatcher<'a, I, V>
where I: Eq + Hash
{
fn before_new(&mut self, key: &K, value: &V) {
let index = (self.indexer)(value);
self.data.insert(index, value);
}
}
I got the error:
error[E0312]: lifetime of reference outlives lifetime of borrowed content...
--> <anon>:18:33
|
18 | self.data.insert(index, value);
| ^^^^^
|
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
--> <anon>:17:21
|
17 | let index = (self.indexer)(value);
| ^^^^^^^^^^^^^^^^^^^^^
|
Hence my final version with lifetimes everywhere.
Ideally, I would like to use the trait in Watchable like the following:
impl<K, V, W: Watcher<K, V>> Watchable<K, V, W>
where K: Eq + Hash {
fn insert(&mut self, k: K, v: V) -> Option<V> {
match self.data.entry(k) {
Occupied(mut occupied) => {
{
let k = occupied.key();
let old = occupied.get();
for watcher in &mut self.watchers {
watcher.before_change(k, &v, old);
}
}
let old = occupied.insert(v);
Some(old)
},
Vacant(vacant) => {
{
let k = vacant.key();
for watcher in &mut self.watchers {
watcher.before_new(k, &v);
}
}
vacant.insert(v);
None
}
}
}
}
trait Watcher<K, V> {
fn before_new(&mut self, key: &K, value: &V);
fn before_change(&mut self, key: &K, value: &V, old: &V);
}
You can get rid of the error by using a higher-rank trait bound instead of a lifetime parameter:
struct Watchable<K, V, W: for<'a> Watcher<'a, K, V>> {
data: HashMap<K, V>,
watchers: Vec<W>,
}
That doesn't solve your problem though. IndexWatcher will not satisfy the bound for<'a> Watcher<'a, K, V> because a IndexWatcher<'a, I, V> only implements Watcher<'a, K, V> for one specific lifetime, not for all possible lifetimes.
Fundamentally, the problem is that you're trying to put a value and a reference to that value in the same struct (indirectly). That is, your idea is that the watchers are expected to borrow data from the Watchable, but the Watchable also owns the watchers. Please take the time to read this question and Shepmaster's answer to understand why that idea is not going to work.
In particular, be aware that inserting an entry in or removing an entry from the Watchable's HashMap might invalidate the references in any of the watchers due to HashMap needing to reallocate storage, which may cause the address of the keys and values to change.
What I would do instead is wrap the keys and values in an Rc (or an Arc if you want to share a Watchable across threads). Getting a shared index value out of a Rc<V> might be problematic though. Consider changing your index function to fn(&V) -> I, and have the index functions return clones (they can be clones of an Rc if cloning the index value is too expensive) or handles.
Given the implementation below, where essentially I have some collection of items that can be looked up via either a i32 id field or a string field. To be able to use either interchangeably, a trait "IntoKey" is used, and a match dispatches to the appropriate lookup map; this all works fine for my definition of get within the MapCollection impl:
use std::collections::HashMap;
use std::ops::Index;
enum Key<'a> {
I32Key(&'a i32),
StringKey(&'a String),
}
trait IntoKey<'a> {
fn into_key(&'a self) -> Key<'a>;
}
impl<'a> IntoKey<'a> for i32 {
fn into_key(&'a self) -> Key<'a> { Key::I32Key(self) }
}
impl<'a> IntoKey<'a> for String {
fn into_key(&'a self) -> Key<'a> { Key::StringKey(self) }
}
#[derive(Debug)]
struct Bar {
i: i32,
n: String,
}
struct MapCollection
{
items: Vec<Bar>,
id_map: HashMap<i32, usize>,
name_map: HashMap<String, usize>,
}
impl MapCollection {
fn new(items: Vec<Bar>) -> MapCollection {
let mut is = HashMap::new();
let mut ns = HashMap::new();
for (idx, item) in items.iter().enumerate() {
is.insert(item.i, idx);
ns.insert(item.n.clone(), idx);
}
MapCollection {
items: items,
id_map: is,
name_map: ns,
}
}
fn get<'a, K>(&self, key: &'a K) -> Option<&Bar>
where K: IntoKey<'a> //'
{
match key.into_key() {
Key::I32Key(i) => self.id_map.get(i).and_then(|idx| self.items.get(*idx)),
Key::StringKey(s) => self.name_map.get(s).and_then(|idx| self.items.get(*idx)),
}
}
}
fn main() {
let bars = vec![Bar { i:1, n:"foo".to_string() }, Bar { i:2, n:"far".to_string() }];
let map = MapCollection::new(bars);
if let Some(bar) = map.get(&1) {
println!("{:?}", bar);
}
if map.get(&3).is_none() {
println!("no item numbered 3");
}
if let Some(bar) = map.get(&"far".to_string()) {
println!("{:?}", bar);
}
if map.get(&"baz".to_string()).is_none() {
println!("no item named baz");
}
}
However, if I then want to implement std::ops::Index for this struct, if I attempt to do the below:
impl<'a, K> Index<K> for MapCollection
where K: IntoKey<'a> {
type Output = Bar;
fn index<'b>(&'b self, k: &K) -> &'b Bar {
self.get(k).expect("no element")
}
}
I hit a compiler error:
src/main.rs:70:18: 70:19 error: cannot infer an appropriate lifetime for automatic coercion due to conflicting requirements
src/main.rs:70 self.get(k).expect("no element")
^
src/main.rs:69:5: 71:6 help: consider using an explicit lifetime parameter as shown: fn index<'b>(&'b self, k: &'a K) -> &'b Bar
src/main.rs:69 fn index<'b>(&'b self, k: &K) -> &'b Bar {
src/main.rs:70 self.get(k).expect("no element")
src/main.rs:71 }
I can find no way to specify a distinct lifetime here; following the compiler's recommendation is not permitted as it changes the function signature and no longer matches the trait, and anything else I try fails to satisfy the lifetime specification.
I understand that I can implement the trait for each case (i32, String) separately instead of trying to implement it once for IntoKey, but I am more generally trying to understand lifetimes and appropriate usage. Essentially:
Is there actually an issue the compiler is preventing? Is there something unsound about this approach?
Am I specifying my lifetimes incorrectly? To me, the lifetime 'a in Key/IntoKey is dictating that the reference need only live long enough to do the lookup; the lifetime 'b associated with the index fn is stating that the reference resulting from the lookup will live as long as the containing MapCollection.
Or am I simply not utilizing the correct syntax to specify the needed information?
(using rustc 1.0.0-nightly (b63cee4a1 2015-02-14 17:01:11 +0000))
Do you intend on implementing IntoKey on struct's that are going to store references of lifetime 'a? If not, you can change your trait and its implementations to:
trait IntoKey {
fn into_key<'a>(&'a self) -> Key<'a>;
}
This is the generally recommended definition style, if you can use it. If you can't...
Let's look at this smaller reproduction:
use std::collections::HashMap;
use std::ops::Index;
struct Key<'a>(&'a u8);
trait IntoKey<'a> { //'
fn into_key(&'a self) -> Key<'a>;
}
struct MapCollection;
impl MapCollection {
fn get<'a, K>(&self, key: &'a K) -> &u8
where K: IntoKey<'a> //'
{
unimplemented!()
}
}
impl<'a, K> Index<K> for MapCollection //'
where K: IntoKey<'a> //'
{
type Output = u8;
fn index<'b>(&'b self, k: &K) -> &'b u8 { //'
self.get(k)
}
}
fn main() {
}
The problem lies in get:
fn get<'a, K>(&self, key: &'a K) -> &u8
where K: IntoKey<'a>
Here, we are taking a reference to K that must live as long as the Key we get out of it. However, the Index trait doesn't guarantee that:
fn index<'b>(&'b self, k: &K) -> &'b u8
You can fix this by simply giving a fresh lifetime to key:
fn get<'a, 'b, K>(&self, key: &'b K) -> &u8
where K: IntoKey<'a>
Or more succinctly:
fn get<'a, K>(&self, key: &K) -> &u8
where K: IntoKey<'a>