I'm creating a node C++ addon that is supposed to call back a js function as events are triggered by some other thread that is not under my control (it is actually a .net thread from a managed dll I use). As the JS world is single-threaded, I can't just keep the js function reference around and call it from that non-js thread. I can't even create the parameters I want to pass to that callback function on that thread.
I've seen uv_queue_work being used in cases where you want to move work off of the js thread and once finished and get called back from the js thread to do whatever you need to do on it, like e.g. calling back a js function.
For later reference, here's that function's signature:
int uv_queue_work(uv_loop_t* loop, uv_work_t* req, uv_work_cb work_cb,
uv_after_work_cb after_work_cb);
Calling uv_queue_work is fine for one-off jobs, and I could probably get it to call me arbitrarily often by chaining calls to uv_queue_work from after_work_cb, while work_cb synchronizes with that other thread, but I wonder if there isn't a more straightforward way to do it.
Like a fn provided by node.js that could be called directly by any other thread with a fn pointer that points to code to be executed on the main js thread at the next occasion. Any idea?
Rather than using uv_queue_work, you should look at the uv_async_* functions (Check out https://github.com/joyent/libuv/blob/master/include/uv.h).
A very nice implementation of this can be found in the node sqllite project https://github.com/developmentseed/node-sqlite3/blob/master/src/async.h.
Just a note however, if you are not EXTREMELY careful with this stuff, you will end up with some TERRIBLE loop reference count bugs (Application exits to early, or not at all)
Related
I'm writing a .NET wrapper to a C++ lib for accessing custom DB. The lib uses callbacks to return data queried.
void Query(Params p, Func callback);
It calls specified function for every record in returned dataset.
I need to write an interface like so:
Record ReadOne();
It's supposed to block until it gets one record and then return.
To my understanding, such a method should run until library calls back, then freeze this execution and switch context to return single record. On next call to ReadOne() it should switch back and return from callback.
I could see this done with additional thread. My question is: could it be done without additional thread? Are there any context manipulation mechanisms in CLR? Could async/await serve this purpose?
In Unity, the thread cannot operate the object provided by UnityEngine like transform.position etc, which caused an exception that get_transform can only be called from the main thread.
However, these methods can be used in some async function like BeginReceive, so is there anyone can tell me why? The async function is not thread or sth else?
I try the code below:
void Start(){
AsyncCallback callback = new AsyncCallback (demo);
callback.BeginInvoke (null, null, null);
}
void demo(IAsyncResult result){
Debug.Log(Thread.CurrentThread.ManagedThreadId);
Debug.Log(gb.transform.position.ToString());
}
It does throw an exception when I run the code on Unity Editor. However, when I run this code on an Android phone directly, it didn't throw any exception and the code was performed correctly.
The log in applogcat shows that:
Line 13497: 02-20 14:37:49.973 31027 31697 I Unity : 3
Line 13501: 02-20 14:37:49.975 31027 31697 I Unity : (0.0, 0.0, 0.0)
So it seems that the function runs on another thread instead of main thread, so could anyone tell me why transform works on this situation?
Unity doesn't allow calling most API functions from threads other than the main thread, period. All of the event/message processing is actually done on the main thread.
The coroutine system based on IEnumerator is a bit of a hack and doesn't actually allow for multi-threading (keep in mind that even the .NET 4.5 async/await feature doesn't necessarily imply multithreaded execution either).
If calling the UnityEngine API works, you're on the main thread.
UI APIs aren't allowed to be called from a different thread than the UI one.
This simplifies how Unity works behind the scenes and actually makes it faster.
Some async methods are dispatched using an event loop and not a different thread. Just because a method is async it doesn't mean it gets to run on a different thread.
The most obvious example of this in Unity are Coroutines. They do run async, but on the main thread. This is possible because Unity adds all of to a list and executes them every frame.
You can call the Unity API from other threads, but NOT if you're running the game from within the Unity Editor. Release builds do not check which thread the call to the Unity API originated from. I assume they don't bother to avoid the performance hit.
I haven't tested this much myself though. The Unity documentation is quite clear that the API is not thread-safe. Therefore, definitely don't make any property assignments or calls that change the game state from other threads. Merely reading values might be OK, but it depends on the unknown internal caching behavior of UnityEngine, ie. hashtables/dictionaries would be bad for multi-threading.
When I create a new thread in a program... in it's thread handle function, why do I pass variables that I want that thread to use through the thread function prototype as parameters (as a void pointer)? Since threads share the same memory segments (except for stack) as the main program, shouldn't I be able to just use the variables directly instead of passing parameters from main program to new thread?
Well, yes, you could use the variables directly. Maybe. Assuming that they aren't changed by some other thread before your thread starts running.
Also, a big part of passing parameters to functions (including thread functions) is to limit the amount of information the called function has to know about the outside world. If you pass the thread function everything it needs in order to do its work, then you can change the rest of the program with relative impunity and the thread will still continue to work. If, however, you force the thread to know that there is a global list of strings called MyStringList, then you can't change that global list without also affecting the thread.
Information hiding. Encapsulation. Separation of concerns. Etc.
You cannot pass parameters to a thread function in any kind of normal register/stack manner because thread functions are not called by the creating thread - they are given execution directly by the underlying OS and the API's that do this copy a fixed number of parameters, (usually only one void pointer), to the new and different stack of the new thread.
As Jim says, failure to understand this mechanism often results in disaster. There are numnerous questions on SO where the vars that devs. hope would be used by a new thread are RAII'd away before the new thread even starts.
I have a fundamental confusion in Node.js technology. Let me explain with this small example.
I have this piece of code
setTimeout(function (){
console.log('world');
}, 2000);
console.log('hello');
When I start executing this code, it immedietely prints 'hello' and after 2 seconds it prints 'world'.
Now I just want to know that if node.js is said to be the single threaded framework, then in which context or where(thread/process) this setTimeout function gets executed since the only single thread is executing the remaining code (printing world).
In case of I/O calls like DB hit, node.js uses the Libeio which in turn use threads internally. So it is not single threaded at all.
Am I right??
Please suggest.
Thanks
Node.js does indeed use threads internally. When they say it's "single threaded" they mean that the javascript is run within only one of those internal threads.
This thread runs an event loop which can be summed up as this:
while true
foreach events as event
call event js callback
endforeach
endwhile
your setTimeout function creates a timer in the event loop, triggering it to call your callback later.
The timers are executed in the same thread, in the "javascript single thread": Understanding javascript timers.
The file system calls occur at a C/C++ level. Internally nodejs uses a pool of threads. The libuv library provides that functionality and therefore, asynchorous calls at a C/C++ level. Then, node.js exposes a public api to the "javascript single thread".
In my sample application, I have basically two threads.
The main thread contains a Lua engine (which is not thread-safe) and registers some C++ functions into this engine. However, one of these functions takes too long to perform (since it downloads some file over the internet) and I want the Lua engine to continue doing other stuff without blocking during the download process.
Therefore, I want to make it asynchronous: When the downloadFile() function is called from Lua, I create a new thread which performs the download. Then, the function returns and the Lua engine can process other work. When the download is finished, the second thread somehow needs to tell the main thread that it should somehow call some additional function processFile() to complete it.
This is where I'm struggling now: What is the easiest / cleanest solution to achieve this?
A new user-data object can be returned by your downloadFile() or similarly named function for kicking off the thread. This new user-data object would contain the thread handle and have an associated meta-table with an __index entry that would have a function for checking the completion status of the download, and contain other synchronization functions.
Possibly looking like this:
local a = downloadFile("foo")
-- do other things
a:join() -- now let the download finish
processFile()
or this:
local a = downloadFile("foo")
local busywork = coroutine.create(doOtherStuff)
while(not a:finished()) do
coroutine.resume(busywork)
end
processFile()
Try LuaLanes for instance. Also here.
Without integrating multithreading into lua ( which isn't that hard, Lua is already prepared for it ), your only solution is to handle the signaling in c++.
From what you've said, there isn't any interaction with Lua in the thread downloadFile() creates, does Lua have to call processFile()? If not, what's stopping you from handling it all in c++? If you need to notify Lua, you can always use a callback function ( save it in the registry ), handle the signal in c++ and run the callback.
Since your engine is not thread-safe, I don't think there is a way to handle it in Lua.
Return a "future" object from the downloadFile function. This can be used to check the result of the long-running asynchronous operation. See java.util.concurrent.Future or QFuture for example implementations. Also, check out this mailing list thread for a discussion of implementing futures/promises in Lua using coroutines.