In my sample application, I have basically two threads.
The main thread contains a Lua engine (which is not thread-safe) and registers some C++ functions into this engine. However, one of these functions takes too long to perform (since it downloads some file over the internet) and I want the Lua engine to continue doing other stuff without blocking during the download process.
Therefore, I want to make it asynchronous: When the downloadFile() function is called from Lua, I create a new thread which performs the download. Then, the function returns and the Lua engine can process other work. When the download is finished, the second thread somehow needs to tell the main thread that it should somehow call some additional function processFile() to complete it.
This is where I'm struggling now: What is the easiest / cleanest solution to achieve this?
A new user-data object can be returned by your downloadFile() or similarly named function for kicking off the thread. This new user-data object would contain the thread handle and have an associated meta-table with an __index entry that would have a function for checking the completion status of the download, and contain other synchronization functions.
Possibly looking like this:
local a = downloadFile("foo")
-- do other things
a:join() -- now let the download finish
processFile()
or this:
local a = downloadFile("foo")
local busywork = coroutine.create(doOtherStuff)
while(not a:finished()) do
coroutine.resume(busywork)
end
processFile()
Try LuaLanes for instance. Also here.
Without integrating multithreading into lua ( which isn't that hard, Lua is already prepared for it ), your only solution is to handle the signaling in c++.
From what you've said, there isn't any interaction with Lua in the thread downloadFile() creates, does Lua have to call processFile()? If not, what's stopping you from handling it all in c++? If you need to notify Lua, you can always use a callback function ( save it in the registry ), handle the signal in c++ and run the callback.
Since your engine is not thread-safe, I don't think there is a way to handle it in Lua.
Return a "future" object from the downloadFile function. This can be used to check the result of the long-running asynchronous operation. See java.util.concurrent.Future or QFuture for example implementations. Also, check out this mailing list thread for a discussion of implementing futures/promises in Lua using coroutines.
Related
I'm writing a .NET wrapper to a C++ lib for accessing custom DB. The lib uses callbacks to return data queried.
void Query(Params p, Func callback);
It calls specified function for every record in returned dataset.
I need to write an interface like so:
Record ReadOne();
It's supposed to block until it gets one record and then return.
To my understanding, such a method should run until library calls back, then freeze this execution and switch context to return single record. On next call to ReadOne() it should switch back and return from callback.
I could see this done with additional thread. My question is: could it be done without additional thread? Are there any context manipulation mechanisms in CLR? Could async/await serve this purpose?
I have the same situation like this: stop thread started by qtconcurrent::run
I need to close child thread (started with QtConcurrent::run) on closeEvent in QMainWindow.
But my function in child thread use code from *.dll: I can`t use loop because all that I do - is calling the external dll like
QFuture<void> = QtConcurrent::run(obj->useDllfunc_with_longTermJob());
And when I close the app with x-button my gui is closed, but second thread with_longTermJob() still worked and when is finished I have an error.
I know some decisions for this:
using other functions like map() or something else with
QFuture.cancel/stop functionality, not QtConcurrent::run().But I need only one function call. run() is what I need.
or use QThread instead Concurrent.But it`s not good for me.
What method more simple and better and how can I implement this? Is there a method that I don`t listed?
Could you provide small code sample for decision. Thx!
QtConcurrent::run isn't a problem here. You must have means of stopping the dllFuncWithLongTermJob. If you don't have such means, then the API you're using is broken, and you're out of luck. There's nothing you can do that'd be generally safe. Forcibly terminating a thread can leave the heap in an inconsistent state, etc. - if you need to terminate a thread, you need to immediately abort the application.
Hopefully, you can call something like stopLongTermJob that sets some flag that interrupts the dllFuncWithLongTermJob.
Then:
auto obj = new Worker;
auto objFuture = QtConcurrent::run([=]{obj->dllFuncWithLongTermJob();});
To interrupt:
obj->stopLongTermJob(); // must be thread-safe, sets a flag
objFuture.waitForFinished();
When I create a new thread in a program... in it's thread handle function, why do I pass variables that I want that thread to use through the thread function prototype as parameters (as a void pointer)? Since threads share the same memory segments (except for stack) as the main program, shouldn't I be able to just use the variables directly instead of passing parameters from main program to new thread?
Well, yes, you could use the variables directly. Maybe. Assuming that they aren't changed by some other thread before your thread starts running.
Also, a big part of passing parameters to functions (including thread functions) is to limit the amount of information the called function has to know about the outside world. If you pass the thread function everything it needs in order to do its work, then you can change the rest of the program with relative impunity and the thread will still continue to work. If, however, you force the thread to know that there is a global list of strings called MyStringList, then you can't change that global list without also affecting the thread.
Information hiding. Encapsulation. Separation of concerns. Etc.
You cannot pass parameters to a thread function in any kind of normal register/stack manner because thread functions are not called by the creating thread - they are given execution directly by the underlying OS and the API's that do this copy a fixed number of parameters, (usually only one void pointer), to the new and different stack of the new thread.
As Jim says, failure to understand this mechanism often results in disaster. There are numnerous questions on SO where the vars that devs. hope would be used by a new thread are RAII'd away before the new thread even starts.
Hi need some help on my lua script. I have a script here that will run a server like application (infinite loop). Problem here is it doesn't execute the second coroutine.
Could you tell me whats wrong Thank you.
function startServer()
print( "...Running server" )
--run a server like application infinite loop
os.execute( "server.exe" )
end
function continue()
print("continue")
end
co = coroutine.create( startServer() )
co1 = coroutine.create( continue() )
Lua have cooperative multithreading. Threads are not swtiched automatically, but must yield to others. When one thread is running, every other thread is waiting for it to finish or yield. Your first thread in this example seems to run server.exe, which, I assume, never finishes until interrupted. Thus second thread never gets its turn to run.
You also run threads wrong. In your example you're not running any threads at all. You execute function and then would try to create coroutine with its output, which naturally would fail. But since you never get back from server.exe you didn't notice this problem yet. Remove those brackets after startServer and continue to fix it.
As already noted, there are several issues with the script that prevent you from getting what you want:
os.execute("...") is blocked until the command is completed and in your case it doesn't complete (as it runs an infinite loop). Solution: you need to detach that process from yours by using something like io.popen() instead of os.execute()
co = coroutine.create( startServer() ) doesn't create a coroutine in your case. coroutine.create call accepts a function reference and you pass it the result of startServer call, which is nil. Solution: use co = coroutine.create( startServer ) (note that parenthesis are dropped, so it's not a function call anymore).
You are not yielding from your coroutines; if you want several coroutines to work together, they need to be cooperating by giving control to each other when appropriate. That's what yield command is for and that's why it's called non-preemptive multithreading. Solution: you need to use a combination of resume and yield calls after you create your coroutine.
startServer doesn't need to be a coroutine as you are not giving control back to it; its only purpose is to start the server.
In your case, the solution may not even need coroutines as all you need to do is: (1) start the server and let it detach from your process (for example, using popen) and (2) work with your process using whatever communication protocol it requires (pipes, sockets, etc.).
There are more complex and complete solutions (like LuaLanes) and also several good descriptions on creating simple coroutine dispatchers.
Your coroutine is not yielding
I'm creating a node C++ addon that is supposed to call back a js function as events are triggered by some other thread that is not under my control (it is actually a .net thread from a managed dll I use). As the JS world is single-threaded, I can't just keep the js function reference around and call it from that non-js thread. I can't even create the parameters I want to pass to that callback function on that thread.
I've seen uv_queue_work being used in cases where you want to move work off of the js thread and once finished and get called back from the js thread to do whatever you need to do on it, like e.g. calling back a js function.
For later reference, here's that function's signature:
int uv_queue_work(uv_loop_t* loop, uv_work_t* req, uv_work_cb work_cb,
uv_after_work_cb after_work_cb);
Calling uv_queue_work is fine for one-off jobs, and I could probably get it to call me arbitrarily often by chaining calls to uv_queue_work from after_work_cb, while work_cb synchronizes with that other thread, but I wonder if there isn't a more straightforward way to do it.
Like a fn provided by node.js that could be called directly by any other thread with a fn pointer that points to code to be executed on the main js thread at the next occasion. Any idea?
Rather than using uv_queue_work, you should look at the uv_async_* functions (Check out https://github.com/joyent/libuv/blob/master/include/uv.h).
A very nice implementation of this can be found in the node sqllite project https://github.com/developmentseed/node-sqlite3/blob/master/src/async.h.
Just a note however, if you are not EXTREMELY careful with this stuff, you will end up with some TERRIBLE loop reference count bugs (Application exits to early, or not at all)