Consider the following scenarios.
let mut map = HashMap::new();
map.insert(2,5);
thread::scope(|s|{
s.spawn(|_|{
map.insert(1,5);
});
s.spawn(|_|{
let d = map.get(&2).unwrap();
});
}).unwrap();
This code cannot be compiled because we borrow the variable map mutably in h1 and borrow again in h2. The classical solution is wrapping map by Arc<Mutex<...>>. But in the above code, we don't need to lock whole hashmap. Because, although two threads concurrently access to same hashmap, they access completely different region of it.
So I want to share map through thread without using lock, but how can I acquire it? I'm also open to use unsafe rust...
in the above code, we don't need to lock whole hashmap
Actually, we do.
Every insert into the HashMap may possibly trigger its reallocation, if the map is at that point on its capacity. Now, imagine the following sequence of events:
Second thread calls get and retrieves reference to the value (at runtime it'll be just an address).
First thread calls insert.
Map gets reallocated, the old chunk of memory is now invalid.
Second thread dereferences the previously-retrieved reference - boom, we get UB!
So, if you need to insert something in the map concurrently, you have to synchronize that somehow.
For the standard HashMap, the only way to do this is to lock the whole map, since the reallocation invalidates every element. If you used something like DashMap, which synchronizes access internally and therefore allows inserting through shared reference, this would require no locking from your side - but can be more cumbersome in other parts of API (e.g. you can't return a reference to the value inside the map - get method returns RAII wrapper, which is used for synchronization), and you can run into unexpected deadlocks.
Related
I'm wrapping a Rust object to be used from Lua. I need the object to be destroyed when neither Rust code nor Lua still has a reference to it, so the obvious (to me) solution is to use Rc<T>, stored in Lua-managed memory.
The Lua API (I'm using rust-lua53 for now) lets you allocate a chunk of memory and attach methods and a finalizer to it, so I want to store an Rc<T> into that chunk of memory.
My current attempt looks like. First, creating an object:
/* Allocate a block of uninitialized memory to use */
let p = state.new_userdata(mem::size_of::<Rc<T>>() as size_t) as *mut Rc<T>;
/* Make a ref-counted pointer to a Rust object */
let rc = Rc::<T>::new(...);
/* Store the Rc */
unsafe { ptr::write(p, rc) };
And in the finaliser:
let p: *mut Rc<T> = ...; /* Get a pointer to the item to finalize */
unsafe { ptr::drop_in_place(p) }; /* Release the object */
Now this seems to work (as briefly tested by adding a println!() to the drop method). But is it correct and safe (as long as I make sure it's not accessed after finalization)? I don't feel confident enough in unsafe Rust to be sure that it's ok to ptr::write an Rc<T>.
I'm also wondering about, rather than storing an Rc<T> directly, storing an Option<Rc<T>>; then instead of drop_in_place() I would ptr::swap() it with None. This would make it easy to handle any use after finalization.
Now this seems to work (as briefly tested by adding a println!() to the drop method). But is it correct and safe (as long as I make sure it's not accessed after finalisation)? I don't feel confident enough in unsafe Rust to be sure that it's ok to ptr::write an Rc<T>.
Yes, you may ptr::write any Rust type to any memory location. This "leaks" the Rc<T> object, but writes a bit-equivalent to the target location.
When using it, you need to guarantee that no one modified it outside of Rust code and that you are still in the same thread as the one where it was created. If you want to be able to move across threads, you need to use Arc.
Rust's thread safety cannot protect you here, because you are using raw pointers.
I'm also wondering about, rather than storing an Rc<T> directly, storing an Option<Rc<T>>; then instead of drop_in_place() I would ptr::swap() it with None. This would make it easy to handle any use after finalisation.
The pendant to ptr::write is ptr::read. So if you can guarantee that no one ever tries to ptr::read or drop_in_place() the object, then you can just call ptr::read (which returns the object) and use that object as you would use any other Rc<T> object. You don't need to care about dropping or anything, because now it's back in Rust's control.
You should also be using new_userdata_typed instead of new_userdata, since that takes the memory handling off your hands. There are other convenience wrapper functions ending with the postfix _typed for most userdata needs.
Your code will work; of course, note that the drop_in_place(p) will just decrease the counter of the Rc and only drop the contained T if and only if it was the last reference, which is the correct action.
I'm using Kuchiki to parse some HTML and making HTTP requests using hyper to concurrently operate on results through scoped_threadpool.
I select and iterate over listings. I decide the number of threads to allocate in the threadpool based on the number of listings:
let listings = document.select("table.listings").unwrap();
let mut pool = Pool::new(listings.count() as u32);
pool.scoped(|scope| {
for listing in listings {
do_stuff_with(listing);
}
});
When I try to do this I get capture of moved value: listings. listings is kuchiki::iter::Select<kuchiki::iter::Elements<kuchiki::iter::Descendants>>, which is non-copyable -- so I get neither an implicit clone nor an explicit .clone.
Inside the pool I can just do document.select("table.listings") again and it will work, but this seems unnecessary to me since I already used it to get the count. I don't need listings after the loop either.
Is there any way for me to use a non-copyable value in a closure?
Sadly, I think it's not possible the way you want to do it.
Your listings.count() consumes the iterator listings. You can avoid this by writing listings.by_ref().count(), but this won't have the desired effect, since count() will consume all elements of the iterator, so that the next call to next() will always yield None.
The only way to do achieve your goal is to somehow get the length of the iterator listings without consuming its elements. The trait ExactSizeIterator was built for this purpose, but it seems that kuchiki::iter::Select doesn't implement it. Note that this may also be impossible for that kind of iterator.
Edit: As #delnan suggested, another possibility is of course to collect the iterator into a Vec. This has some disadvantages, but may be a good idea in your case.
Let me also note, that you probably shouldn't create one thread for every line in the SELECT result set. Usually threadpools use approximately as many threads as there are CPUs.
Just to get better understanding of the Send and Sync traits, are there examples of types that either:
Implement Send and do not implement Sync.
Implement Sync and do not implement Send.
First of all, it is important to realize that most structs (or enums) are Send:
any struct that does not contain any reference can be Send + 'static
any struct that contain references with a lower-bound lifetime of 'a can be Send + 'a
As a result, you would generally expect any Sync struct to be Send too, because Send is such an easy bar to reach (compared to the much harder bar of being Sync which requires safe concurrent modification from multiple threads).
However, nothing prevents the creator of a type to specifically mark it as not Send. For example, let's resuscitate conditions!
The idea of conditions, in Lisp, is that you setup a handler for a given condition (say: FileNotFound) and then when deep in the stack this condition is met then your handler is called.
How would you implement this in Rust?
Well, to preserve threads independence, you would use thread-local storage for the condition handlers (see std::thread_local!). Each condition would be a stack of condition handlers, with either only the top one invoked or an iterative process starting from the top one but reaching down until one succeeds.
But then, how would you set them?
Personally, I'd use RAII! I would bind the condition handler in the thread-local stack and register it in the frame (for example, using an intrusive doubly-linked list as the stack).
This way, when I am done, the condition handler automatically un-registers itself.
Of course, the system has to account for users doing unexpected things (like storing the condition handlers in the heap and not dropping them in the order they were created), and this is why we use a doubly-linked list, so that the handler can un-register itself from the middle of the stack if necessary.
So we have a:
struct ConditionHandler<T> {
handler: T,
prev: Option<*mut ConditionHandler<T>>,
next: Option<*mut ConditionHandler<T>>,
}
and the "real" handler is passed by the user as T.
Would this handler be Sync?
Possibly, depends how you create it but there is no reason you could not create a handler so that a reference to it could not be shared between multiple threads.
Note: those threads could not access its prev/next data members, which are private, and need not be Sync.
Would this handler be Send?
Unless specific care is taken, no.
The prev and next fields are not protected against concurrent accesses, and even worse if the handler were to be dropped while another thread had obtained a reference to it (for example, another handler trying to un-register itself) then this now dangling reference would cause Undefined Behavior.
Note: the latter issue means that just switching Option<*mut Handler<T>> for AtomicPtr<ConditionHandler<T>> is not sufficient; see Common Pitfalls in Writing Lock-Free Algorithms for more details.
And there you have it: a ConditionHandler<T> is Sync if T is Sync but will never be Send (as is).
For completeness, many types implement Send but not Sync (most Send types, actually): Option or Vec for example.
Cell and RefCell implement Send but not Sync because they can be safely sent between threads but not shared between them.
As I've read all objects in D are fully location independent. How this requirement is achieved?
One thing that comes to my mind, is that all references are not pointers to the objects, but to some proxy, so when you move object (in memory) you just update that proxy, not all references used in program.
But this is just my guess. How it is done in D for real?
edit: bottom line up front, no proxy object, objects are referenced directly through regular pointers. /edit
structs aren't allowed to keep a pointer to themselves, so if they get copied, they should continue to just work. This isn't strictly enforced by the language though:
struct S {
S* lol;
void beBad() {
lol = &this; // this compiler will allow this....
}
}
S pain() {
S s;
s.beBad();
return s;
}
void main() {
S s;
s = pain();
assert(s.lol !is &s); // but it will also move the object without notice!
}
(EDIT: actually, I guess you could use a postblit to update internal pointers, so it isn't quite without notice. If you're careful enough, you could make it work, but then again, if you're careful enough, you can shoot between your toes without hitting your foot too. EDIT2: Actually no, the compiler/runtime is still allowed to move it without even calling the postblit. One example of where this happens is if it copies a stack frame to the heap to make a closure. The struct data is moved to a new address without being informed. So yeah. /edit)
And actually, that assert isn't guaranteed to pass, the compiler might choose to call pain straight on the local object declared in main, so the pointer would work (though I'm not able to force this optimization here for a demo, generally, when you return a struct from a function, it is actually done via a hidden pointer the caller passes - the caller says "put the return value right here" thus avoiding a copy/move in some cases).
But anyway, the point just is that the compiler is free to copy or not to copy a struct at its leisure, so if you do keep the address of this around in it, it may become invalid without notice; keeping that pointer is not a compile error, but it is undefined behavior.
The situation is different with classes. Classes are allowed to keep references to this internally since a class is (in theory, realized by the garbage collector implementation)) an independent object with an infinite lifetime. While it may be moved (such as be a moving GC (not implemented in D today)), if it is moved, all references to it, internal and external, would also be required to be updated.
So classes can't have the memory pulled out from under them like structs can (unless you the programmer take matters into your own hands and bypass the GC...)
The location independent thing I'm pretty sure is referring only to structs and only to the rule that they can't have pointers to themselves. There's no magic done with references or pointers - they indeed work with memory addresses, no proxy objects.
This is a continuation from here: Golang: Shared communication in async http server
Assuming I have a hashmap w/ locking:
//create async hashmap for inter request communication
type state struct {
*sync.Mutex // inherits locking methods
AsyncResponses map[string]string // map ids to values
}
var State = &state{&sync.Mutex{}, map[string]string{}}
Functions that write to this will place a lock. My question is, what is the best / fastest way to have another function check for a value without blocking writes to the hashmap? I'd like to know the instant a value is present on it.
MyVal = State.AsyncResponses[MyId]
Reading a shared map without blocking writers is the very definition of a data race. Actually, semantically it is a data race even when the writers will be blocked during the read! Because as soon as you finish reading the value and unblock the writers - the value may not exists in the map anymore.
Anyway, it's not very likely that proper syncing would be a bottleneck in many programs. A non-blocking lock af a {RW,}Mutex is probably in the order of < 20 nsecs even on middle powered CPUS. I suggest to postpone optimization not only after making the program correct, but also after measuring where the major part of time is being spent.