Assume we have a struct capable of self-mutation that has to happen as part of a background operation:
struct Thing {
var something = 0
mutating func operation(block: () -> Void) {
// Start some background operation
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0)) {
// Mutate self upon background task completion
self.something += 1
block()
}
}
}
Now, when I use such a struct in context:
var myThing = Thing()
myThing.operation {
println(myThing.something)
}
The println gives me 0, as if myThing was never mutated. Printing self.something from within the dispatch_async obviously yields 1.
How do I work around this issue, preferably without having to pass the updated struct's self in the operation competition block and overriding the original variable in the main context?
// Ew
var myThing = Thing()
myThing.operation {
(mutatedThing) in
myThing = mutatedThing
println(myThing.something)
}
I'm adding a second answer because my first answer addressed a different point.
I have just encountered this difficulty myself in circumstances almost identical to yours.
After working and working and working to try to find out what was going on, and fix it, I realized that the problem was, basically, using a value type where a reference type should be used.
The closure seems to create a duplicate of the struct and operate on that, leaving the original untouched--which is more in line with the behavior of a value type.
On the other hand, the desired behavior is for the actions performed in the closure to be retained by the environment outside the closure--in other words two different contexts (inside the closure and out of it) need to refer to the same objects--which is more in line with the behavior of a reference type.
Long story short, I changed the struct to a class. The problem vanished, no other code needed.
I have seen this exact problem many times, but without more detail I can't say if it was happening for the same reason that you are having it.
It drove me crazy until I realized that the dispatched operation was taking place at an illogical time, i.e. usually before the current time. In the dispatch block's frame of reference, it correctly updated the variable, which is why it prints out correctly from inside the block. But in the "real world", for some reason, the dispatch is considered to have never happened, and the value change is discarded. Or perhaps it's because the mutation implicitly creates a new struct and because it was "time travelling" the reference to it never updated. Can't say.
In my case, the problem went away once I correctly scheduled the dispatch. I hope that helps!
Related
Question is general, no particular example.
If we have a function that returns a closure and closure itself returns some captured variable, does that variable type have to implement Copy˙trait?
From documentation:
https://doc.rust-lang.org/rust-by-example/fn/closures/output_parameters.html
fn create_fn() -> impl Fn() {
let text = "Fn".to_owned();
move || println!("This is a: {}", text)
}
this works, but for some types this solution give error. For example, if a text would be of type Option<T> it returns error saying something like:
`move occurs because `text` has type `std::option::Option<Box<dyn Something>>`, which does not implement the `Copy` trait
captured by this `Fn` closure`
One solution to solve the error is either to change closure trait to FnOnce but that introduces restriction that closure can be called only once.
But, if we want to call it multiple times, does that necessarily means that text must implement Copy trait?
I thought that move keyword does exactly that but i'm obviously wrong so if someone could explain details about how this moving captured variables actually works?
No, the type must not necessarily implement Copy. The thing is, when you have a Fn, it must be callable several times. If you actually move out something, then it's not going to work unless you're not actually moving it but just copying it (thus the error). But, the important thing is, you're not necessarily moving something out. You could also, for instance, create it on the fly (well, technically speaking, you'd still be moving it out, but maybe the example will help).
fn create_fn() -> impl Fn() -> String {
let text = "something";
move || text.to_string()
}
(see the playground)
Note that, here, String does not implement Copy (and it could also not implement Clone). What makes it work is that this function can be called over and over again, because each time it won't move out something it will need in a future call (here, by creating a brand new String to be given out).
But, if you're moving out a captured variable, you won't be able to do that twice. In that specific case, it would have to implement Copy (or just Clone and then you'd have to .clone() it).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
Rust's .expect() is one of the most surprising names in the Result/Option space. While unwrap makes sense -- get a value from a Result by "unwrapping" it -- expect is surprisingly counterintuitive (to me).
Since so much of Rust is inspired by conventions from functional programming languages, I have been assuming that this is another example of "strange homage to obscurity," but when I asked the Duck it couldn't find an answer for me.
So, I give up. Why is .expect() the name for Rust's .unwrap_or_panic_with_this_message() function? Is this a reference to a feature in yet another functional language? Is it back-handed shade (or a complement) to Tcl? The result of too many late nights and too much Espresso at Mozilla?
What is the etymology of this plucky (but fierce!) little member of the standard library?
Edit:
Great news, everyone! After more than a year, we now have both an "explanation" in #GManNickG's tremendously well-researched and documented answer, below, and a "mental model" provided by #Peng's comment and link:
https://doc.rust-lang.org/std/error/index.html#common-message-styles
In short, it is now recommended that diagnostics for expect() should be written as "expect...should" or maybe "expect...should have". This makes more sense in the diagnostic output (see the link for examples) but also makes the source code scan better:
foo.expect("something bad happened") // NO!
foo.expect("gollywumpus should be current") // YES!
Summary:
No explicit reason for the name is given. However, it is incredibly likely the name comes from the world of parsers, where one "expects" to see a particular token (else the compilation fails).
Within rustc, the use of expect-like functions long predate use within Option. These are functions like expect(p, token::SEMI) to expect to parse a semicolon and expect_word(p, "let") to expect to parse the let keyword. If the expectation isn't met, compilation fails with an error message.
Eventually a utility function was added within the compiler that would expect not a specific token or string, but that a given Option contained a value (else, fail compilation with the given error message). Over time this was moved to the Option struct itself, where it remains today.
Personally, I don't find it unusual at all. It's just another verb you can do to the object, like unwrapping or taking or mapping its value. Expecting a value (else, fail) from your Option seems quite natural.
History:
The oldest commit of note is the following:
https://github.com/rust-lang/rust/commit/b06dc884e57644a0c7e9c5391af9e0392e5f49ac
Which adds this function within the compiler:
fn expect<T: copy>(sess: session, opt: option<T>, msg: fn() -> str) -> T {
alt opt {
some(t) { t }
none { sess.bug(msg()); }
}
}
As far as I can tell, this is the first function named "expect" that deals with inspecting an Option. Observe in particular this example use-case within the commit (which was implementing support for class methods!):
#debug("looking up %? : %?", def, class_doc);
let the_field = expect(tcx.sess,
decoder::maybe_find_item(def.node, class_doc),
{|| #fmt("get_field_type: in class %?, field ID %? not found",
class_id, def)});
If the result of decoder::maybe_find_item is None, compilation will fail with the given error.
I encourage you to look at the parser code in this commit - there is extensive use of other expect-esque functions: e.g., expect(p, token::RPAREN) and expect_word(p, "let"). The name of this new function is almost obvious in this environment.
Eventually, the utility of this function was extracted and placed within Option itself:
https://github.com/rust-lang/rust/commit/e000d1db0ab047b8d2949de4ab221718905ce3b1
Which looked like:
pure fn expect<T: copy>(opt: option<T>, reason: str) -> T {
#[doc = "
Gets the value out of an option, printing a specified message on failure
# Failure
Fails if the value equals `none`
"];
alt opt { some(x) { x } none { fail reason; } }
}
It's worth noting that sometime later, there was eventually an (additional) function named unwrap_expect added in:
https://github.com/rust-lang/rust/commit/be3a71a1aa36173ce2cd521f811d8010029aa46f
pure fn unwrap_expect<T>(-opt: option<T>, reason: ~str) -> T {
//! As unwrap, but with a specified failure message.
if opt.is_none() { fail reason; }
unwrap(opt)
}
Over time these were both subsumed by an Expect trait, which Option implemented:
https://github.com/rust-lang/rust/commit/0d8f5fa618da00653897be2050980c800389be82
/// Extension trait for the `Option` type to add an `expect` method
// FIXME(#14008) should this trait even exist?
pub trait Expect<T> {
/// Unwraps an option, yielding the content of a `Some`
///
/// # Failure
///
/// Fails if the value is a `None` with a custom failure message provided by
/// `msg`.
fn expect<M: Any + Send>(self, m: M) -> T;
}
Spoiler for that TODO: that trait no longer exists. It was removed shortly after per:
https://github.com/rust-lang/rust/issues/14008.
More or less this is where we are at today.
I think the most likely conclusion is that the use of expect as a meaningful function name long predates its use in Option. Given it says what it does (expect a value or fail) there is little reason to break the pattern.
I'm using the cursive_table_view crate, and the table has a set_on_submit method to which you pass a closure to be executed when the user press over a selected row in the table. The signature of said method is:
pub fn set_on_submit<F>(&mut self, cb: F)
where
F: Fn(&mut Cursive, usize, usize) + 'static,
Inside the closure, I'm trying to use something that actually comes as a parameter in the function that contains this code. I simply declare that parameter as &'static, and that's all good and expected. Now of course, I've just moved the problem to the caller of my function.
fn my_function(param: &'static MyType) {
let table = TableView::new();
table.set_on_sumbit(|cur, row, index| {
do_something_with(param);
}
}
Eventually I get to a point where I have to declare my variable as static to pass it as a parameter to that function, but because I need to modify this variable before the call (and after declaration), it needs to be static mutable. Now the problem is, when I try to modify the variable, the compiler says that I can only do that with unsafe.
I believe I understand the why of everything above. The closure is going to be called upon some event in the UI, so it makes sense that this lives for as long as the entire program.
I've tried several options with Rc and read a ton about lifetimes and even threads and messages, but I felt I was blindly trying things without really understanding at some point. Where I'm at now is that I believe I understand, but I'm uncomfortable with having to go the unsafe route suggestion.
Is it avoidable? Is my pattern wrong to begin with?
I didn't want to add more code to keep the explanation agnostic to everything else, but of course I'm happy to edit if more code helps understand the issue. Also, I don't really need to modify the struct any longer by the time I've passed the reference to the closure, in case that helps find a solution down that path.
Think of references as mostly (excluding &'static and a few other cases) being for temporary things, especially temporary things that live only as long as a stack frame. Therefore, don't try to use references for things like user interface callbacks.
Here's how to use std::rc::Rc instead, which ensures that the data lives as long as needed.
use std::rc::Rc;
fn my_function(param: Rc<MyType>) { // changed parameter type
let table = TableView::new();
table.set_on_submit(|cur, row, index| {
do_something_with(¶m); // & enables conversion to &MyType
}
}
Now, in order to use this, you need to arrange so your data is owned by the Rc. It is not possible to have the above sort of callback and also have let data = MyType {...} directly, because the data must be owned by the Rc. (However, if useful, you can initialize it and then later put it in the Rc.)
fn main() {
// Wrap it as soon as you construct it.
// (Or at least before you start using callbacks.)
let my_app_data = Rc::new(MyType {
// whatever fields or constructor...
});
// Call the function. `.clone()` here makes a copy of the `Rc` refcounted pointer,
// not the underlying data.
my_function(my_app_data.clone());
}
If your program uses threads, then you must use the thread-safe reference counter std::sync::Arc instead of Rc.
If you want to modify the data while the callback exists — whether in a callback or elsewhere — you need to also use RefCell (or thread-safe std::sync::Mutex / RwLock) inside the Rc, which adds a few complications.
I've seen a bunch of questions related to this subject, but none of them offers anything that would be an acceptable solution (please, no loading external Groovy scripts, no calling to sh step etc.)
The operation I need to perform is a oneliner, but pipeline limitations made it impossible to write anything useful in that unter-language...
So, here's minimal example:
#NonCPS
def encodeProperties(Map properties) {
properties.collect { k, v -> "$k=$v" }.join('|')
}
node('dockerized') {
stage('Whatever') {
properties = [foo: 123, bar: "foo"]
echo encodeProperties(properties)
}
}
Depending on whether I add or remove #NonCPS annotation, or type declaration of the argument, the error changes, but it never gives any reason for what happened. It's basically random noise, that contradicts the reality of the situation (at times it would claim that some irrelevant object doesn't have a method encodeProperties, other times it would say that it cannot find a method encodeProperties with a signature that nobody was trying to call it with (like two arguments instead of one) and so on.
From reading the documentation, which is of disastrous quality, I sort of understood that maybe functions in general aren't serializable, and that is why you need to explain this explicitly to the Groovy interpreter... I'm sorry, this makes no sense, but this is roughly what documentation says.
Obviously, trying to use collect inside stage creates a load of new errors... Which are, at least understandable in that the author confesses that their version of Groovy doesn't implement most of the Groovy standard...
It's just a typo. You defined encodeProperties but called encodeProprties.
(Pseudo-)Code
Here is a non-compilable code-sketch of the concepts I am having trouble with:
struct Data {};
struct A {};
struct B {};
struct C {};
/* and many many more...*/
template<typename T>
class Listener {
public:
Listener(MyObject* worker):worker(worker)
{ /* do some magic to register with RTI DDS */ };
public:
// This function is used ass a callback from RTI DDS, i.e. it will be
// called from other threads when new Data is available
void callBackFunction(Data d)
{
T t = extractFromData(d);
// Option 1: direct function call
// works somewhat, but shows "QObject::startTimer: timers cannot be started
// from another thread" at the console...
worker->doSomeWorkWithData(t); //
// Option 2: Use invokeMethod:
// seems to fail, as the macro expands including '"T"' and that type isn't
// registered with the QMetaType system...
// QMetaObject::invokeMethod(worker,"doSomeGraphicsWork",Qt::AutoConnection,
// Q_ARG(T, t)
// );
// Option 3: use signals slots
// fails as I can't make Listener, a template class, a QObject...
// emit workNeedsToBeDone(t);
}
private:
MyObject* worker;
T extractFromData(Data d){ return T(d);};
};
class MyObject : public QObject {
Q_OBJECT
public Q_SLOTS:
void doSomeWorkWithData(A a); // This one affects some QGraphicsItems.
void doSomeWorkWithData(B b){};
void doSomeWorkWithData(C c){};
public:
MyObject():QObject(nullptr){};
void init()
{
// listeners are not created in the constructor, but they should have the
// same thread affinity as the MyObject instance that creates them...
// (which in this example--and in my actual code--would be the main GUI
// thread...)
new Listener<A>(this);
new Listener<B>(this);
new Listener<C>(this);
};
};
main()
{
QApplication app;
/* plenty of stuff to set up RTI DDS and other things... */
auto myObject = new MyObject();
/* stuff resulting in the need to separate "construction" and "initialization" */
myObject.init();
return app.exec();
};
Some more details from the actual code:
The Listener in the example is a RTI DataReaderListener, the callback
function is onDataAvailable()
What I would like to accomplish
I am trying to write a little distributed program that uses RTI's Connext DDS for communication and Qt5 for the GUI stuff--however, I don't believe those details do matter much as the problem, as far as I understood it, boils down to the following:
I have a QObject-derived object myObject whose thread affinity might or might not be with the main GUI thread (but for simplicity, let's assume that is the case.)
I want that object to react to event's which happen in another, non-Qt 3rd-party library (in my example code above represented by the functions doSomeWorkWithData().
What I understand so far as to why this is problematic
Disclaimer: As usual, there is always more than one new thing one learns when starting a new project. For me, the new things here are/were RTI's Connext and (apparently) my first time where I myself have to deal with threads.
From reading about threading in Qt (1,2,3,4, and 5 ) it seems to me that
QObjects in general are not thread safe, i.e. I have to be a little careful about things
Using the right way of "communicating" with QObjects should allow me to avoid having to deal with mutexes etc myself, i.e. somebody else (Qt?) can take care of serializing access for me.
As a result from that, I can't simply have (random) calls to MyClass::doSomeWorkWithData() but I need to serialize that. One, presumably easy, way to do so is to post an event to the event queue myObject lives in which--when time is available--will trigger the execution of the desired method, MyClass::doSomeWorkWithData() in my case.
What I have tried to make things work
I have confirmed that myObject, when instantiated similarly as in the sample code above, is affiliated with the main GUI thread, i.e. myObject.thread() == QApplication::instance()->thread().
With that given, I have tried three options so far:
Option 1: Directly calling the function
This approach is based upon the fact that
- myObject lives in the GUI thread
- All the created listeners are also affiliated with the GUI thread as they are
created by `myObject' and inherit its thread that way
This actually results in the fact that doSomeWorkWithData() is executed. However,
some of those functions manipulate QGraphicsItems and whenever that is the case I get
error messages reading: "QObject::startTimer: timers cannot be started from another
thread".
Option 2: Posting an event via QMetaObject::invokeMethod()
Trying to circumvent this problem by properly posting an event for myObject, I
tried to mark MyObject::doSomeWorkWithData() with Q_INVOKABLE, but I failed at invoking the
method as I need to pass arguments with Q_ARG. I properly registered and declared my custom types
represented by struct A, etc. in the example), but I failed at the fact the
Q_ARG expanded to include a literal of the type of the argument, which in the
templated case didn't work ("T" isn't a registered or declared type).
Trying to use conventional signals and slots
This approach essentially directly failed at the fact that the QMeta system doesn't
work with templates, i.e. it seems to me that there simply can't be any templated QObjects.
What I would like help with
After spending about a week on attempting to fix this, reading up on threads (and uncovering some other issues in my code), I would really like to get this done right.
As such, I would really appreciate if :
somebody could show me a generic way of how a QObject's member function can be called via a callback function from another 3rd-party library (or anything else for that matter) from a different, non QThread-controlled, thread.
somebody could explain to me why Option 1 works if I simply don't create a GUI, i.e. do all the same work, just without a QGraphcisScene visualizing it (and the project's app being a QCoreApplication instead of a QApplication and all the graphics related work #defineed out).
Any, and I mean absolutely any, straw I could grasp on is truly appreciated.
Update
Based on the accepted answer I altered my code to deal with callbacks from other threads: I introduced a thread check at the beginning of my void doSomeWorkWithData() functions:
void doSomeWorkWithData(A a)
{
if( QThread::currentThread() != this->thread() )
{
QMetaObject::invokeMethod( this,"doSomeWorkWithData"
,Qt::QueuedConnection
,Q_ARG(A, a) );
return;
}
/* The actual work this function does would be below here... */
};
Some related thoughts:
I was contemplating to introduce a QMutexLocker before the if statement, but decided against it: the only part of the function that is potentially used in parallel (anything above the return; in the if statement) is--as far as I understand--thread safe.
Setting the connection type manually to Qt::QueuedConnection: technically, if I understand the documentation correctly, Qt should do the right thing and the default, Qt::AutoConnection, should end up becoming a Qt::QueuedConnection. But since would always be the case when that statement is reached, I decided to put explicitly in there to remind myself about why this is there.
putting the queuing code directly in the function and not hiding it in an interim function: I could have opted to put the call to invokeMethod in another interim function, say queueDoSomeWorkWithData()', which would be called by the callback in the listener and then usesinvokeMethodwith anQt::AutoConnection' on doSomeWorkWithData(). I decided against this as there seems no way for me to auto-code this interim function via templates (templates and the Meta system was part of the original problem), so "the user" of my code (i.e. the person who implements doSomeWorkWithData(XYZ xyz)) would have to hand type the interim function as well (as that is how the templated type names are correctly resolved). Including the check in the actual function seems to me to safe typing an extra function header, keeps the MyClass interface a little cleaner, and better reminds readers of doSomeWorkWithData() that there might be a threading issue lurking in the dark.
It is ok to call a public function on a subclass of QObject from another thread if you know for certain that the individual function will perform only thread-safe actions.
One nice thing about Qt is that it will handle foreign threads just as well as it handles QThreads. So, one option is to create a threadSafeDoSomeWorkWithData function for each doSomeWorkWithData that does nothing but QMetaMethod::invoke the non-threadsafe one.
public:
void threadSafeDoSomeWorkWithData(A a) {
QMetaMethod::invoke("doSomeWorkWithData", Q_ARG(A,a));
}
Q_INVOKABLE void doSomeWorkWithData(A a);
Alternatively, Sergey Tachenov suggests an interesting way of doing more or less the same thing in his answer here. He combines the two functions I suggested into one.
void Obj2::ping() {
if (QThread::currentThread() != this->thread()) {
// not sure how efficient it is
QMetaObject::invoke(this, "ping", Qt::QueuedConnection);
return;
}
// thread unsafe code goes here
}
As to why you see normal behaviour when not creating a GUI? Perhaps you're not doing anything else that is unsafe, aside from manipulating GUI objects. Or, perhaps they're the only place in which your thread-safety problems are obvious.