Related
Basically, I need a way to call Matlab functions from an indefinitely-long separate thread.
First, I'm aware that I could use the TCPIP or UDP functionality to communicate between two instances of Matlab. I'll explain why that doesn't really help.
Background: I've written a Matlab class that acts as an interface for a USB device. Matlab was chosen because I need it to run on Mac/Linux/Windows, and the target users are only familiar with Matlab. Because of some inconsistencies in Matlab across platforms, I'm not using the BytesAvailableFcn or BytesAvailableFcnMode (I need as near realtime as possible, and with the aforementioned there can be delays up to 100s of milliseconds to send and receive data), and am instead sending and polling the port at a fixed interval using a timer. This introduces some overhead, and, if the user holds onto the main thread, the sending/receiving will also stop. Now, one of the most important function of the class is to set callbacks that are based on the input received from the device. The user sets their function and a given condition to match, and the object will call it automatically.
Problem: This object works well, completely in the background. However, as mentioned, it consumes some resources on the Matlab thread. I'm curious about making just the serial wrapper and callback functionality run on its own thread. However, if I compile it as a standalone application (for all 3 platforms) I believe my only solution will be TCPIP/UDP communication. Which then requires the object running on the main thread to poll the port in order to handle the callbacks in realtime - thus negating the benefit of moving it to a standalone application.
Any suggestions?
Threading in matlab is a nightmare. Doing anything in realtime, with the kind of latencies you're describing is not advised. Under the hood, Matlab uses Java for all it's platform independence. If you want to do this right, you'll write your app natively in Java, and call your java from Matlab (to deal with the fact that your users are incapable of installing a JRE, but can install matlab.)
That said, there is a better way to handle callbacks than what you are doing. My preferred architecture in this scenario is to have one thread service the hardware, and communicate with other threads via message queues (one for input, one for output, and one for command/control if you need to get super fancy.) Basically, the hardware thread then just focuses on servicing the queues. You have a second thread handle the callbacks. It reads the output queue of the hardware thread, and services the callbacks. I've never done this in matlab (see first paragraph) but it works very well in Java contexts.
In the interest of ease of programming (local function calls instead of IPC) and performance (e.g. avoiding copies of large buffers), I'd like to have a Java VM call native code using JNI instead of through interprocess communication. There would be lots of worker threads, each doing computer vision on some image and sending back a list of detected features.
I've found a few other posts about this topic:
How to implement a native code sandbox?
Linux: Is it possible to sandbox shared library code
but in all cases, the agreed upon solution is to use multiple processes.
But I would like to explore the feasibility of partly sand boxing threads. Clearly, this goes against common sense, but I think if your client processes aren't malicious and if you can recover from faults, and in the worst case, are willing to tolerate a whole system crash once in a blue moon, it might work.
There are some hints that this is possible such as from jmajnert in #2. You would have to capture segfaults and other crashes, and terminate and restart the crashed thread. But I also want to reset the heap of the thread. That means each thread should have a private heap, but I don't know of any common malloc implementation that lets you create multiple heaps (AIX seems to).
Then I would want to close all files opened by the thread when it gets restarted.
Also, if Java objects get compromised by the native code, would it be practical to provide some fault tolerance like recreating them?
Because if complexity of hopping models between some native code, and the JVM -- The idea itself is really not even feasible.
To be feasible, you'd need to be within a single machine/threading model.
Lets assume you're in posix/ansi c.
You'd need to write a custom allocator that allocated from pools. Each time you launched a thread you'd allocate a new pool and set that pool as a thread local variable that all your custom_malloc() functions would allocate from. This way, when your thread died you could crush all of it's memory along with it.
Next, you'll need to set up some niftyness with setjmp/longjmp and signal to catch all those segfaults etc. exit the thread, crush it's memory and restart.
If you have objects from the "parent process" that you don't want to get corrupted, you'd have to create some custom mutexes that would have rollback functions that could be triggered when a threads signal handler was triggered to destroy the thread.
This question already has answers here:
Why are most UI frameworks single threaded?
(6 answers)
Closed 7 years ago.
In every GUI library I've used (Swing, Android, Windows Forms, WPF) there's this golden rule saying that one cannot access or modify GUI elements from another thread (other than the GUI thread). I suppose this rule applies to any GUI library. Breaking this rule will most likely cause application to crash. However, I've been wondering recently, why is it so? I couldn't find any profound explanation. So what is the low-level explanation of this rule?
No piece of software is thread-safe unless it is explicitly designed and build to be so.
A GUI is a complex and stateful beast, making it thread-safe would be 'prohibitively expensive'.
There is a very simple reason for this. Usually UI functions are not thread-safe (as making them thread-safe would pessimize performance).
Of those you listed, some may be wrappers around existing mechanisms, so you have to answer the question indirectly via the underlying GUI framework. In case of multi-platform GUI frameworks like e.g. Qt, you will also have the lowest-common denominator that determines what is possible and what isn't.
Now, why is access to the GUI not thread-safe? In the cases where I'm most familiar with (win32 and X11), accesses are often performed indirectly by sending requests and sometimes waiting for the according answer. This usually works in an atomic way, even across process boundaries, so that is not directly cause of the problem. However, if you do so from multiple threads, the worst that can happen is that data is modified in an uncoordinated way. For example, if you read, modify and write the same widget from two threads, these operations might be interleaved, so that only one thread's modifications will actually be applied.
There are other reasons for not supporting cross-thread access:
In win32, the queue with the messages is thread-local, which means that only the thread that created a window will actually find and be able to handle messages for that window. I guess this a legacy from times where processes were single-threaded and the message queue was simply a global. Making it thread-local is the same approach as the one used for making errno thread-safe.
Another reason is that support objects are created inside a process that represent some GUI element. For example, the MFC (on top of win32) use a map from the OS' widget handle to a C++ object representing that object. That map is stored in thread-local storage (which follows the thread-local message queue) and the access to the C++ objects is not guarded by a mutex. Accessing these objects from different threads is bad, not because they represent GUI objects but because they are not synchronized, simple as that.
If you think about modifying the structure of a widget tree (like e.g. the DOM tree in a browser), you either have very detailed knowledge of what other parts of the application are doing or you need to lock access to the whole tree before every operation just to be safe. Needless to say, this effectively prevents any parallel operations, so you can also take the next step and require all operations to come from one thread and thus save the whole multithreading overhead.
That said, I believe that Qt and C# (and probably others) actually do support some cross-thread operations. They will work some (more or less obscure) magic that forwards the calls to the GUI thread and forwards the results back to the calling thread again. In other words, they try to make the necessary inter-thread communication more convenient for the programmer, while retaining the efficiency and simplicity of the single-threaded GUI. This is not restricted to GUI handling though but rather a general approach, only that it is especially important for the GUI.
As far as I know, that is simply not true: Every object in Java might be accesed concurrently, as far as thread-safe techniques are correctly applied. The fact is that Java Swing objects are mostly not prepared for multithreading, so you'll have to perform external synchronization.
There are several instances in which you need several threads to interoperate in a GUI: Games, visual effects, user events...
More information about the GUI and multithreading:
https://docs.oracle.com/javase/tutorial/uiswing/concurrency/dispatch.html
We need to build a software framework (or middleware) that will enable messaging between different software components (or modules) running on a single machine. This framework will provide such features:
Communication between modules are through 'messaging'.
Each module will have its own message queue and message handler thread that will synchronously handle each incoming message.
With the above requirements, which of the following approach is the correct one (with its reasoning)?:
Implementing modules as processes, and messaging through shared memory
Implementing modules as threads in a single process, and messaging by pushing message objects to the destination module's message queue.
Of source, there are some apparent cons & pros:
In Option-2, if one module causes segmentation fault, the process (thus the whole application) will crash. And one module can access/mutate another module's memory directly, which can lead to difficult-to-debug runtime errors.
But with Option-1, you need to take care of the states where a module you need to communicate has just crashed. If there are N modules in the software, there can be 2^N many alive/crashed states of the system that affects the algorithms running on the modules.
Again in Option-1, sender cannot assume that the receiver has received the message, because it might have crashed at that moment. (But the system can alert all the modules that a particular module has crashed; that way, sender can conclude that the receiver will not be able to handle the message, even though it has successfully received it)
I am in favor of Option-2, but I am not sure whether my arguments are solid enough or not. What are your opinions?
EDIT: Upon requests for clarification, here are more specification details:
This is an embedded application that is going to run on Linux OS.
Unfortunately, I cannot tell you about the project itself, but I can say that there are multiple components of the project, each component will be developed by its own team (of 3-4 people), and it is decided that the communication between these components/modules are through some kind of messaging framework.
C/C++ will be used as programming language.
What the 'Module Interface API' will automatically provide to the developers of a module are: (1) An message/event handler thread loop, (2) a synchronous message queue, (3) a function pointer member variable where you can set your message handler function.
Here is what I could come up with:
Multi-process(1) vs. Single-process, multi-threaded(2):
Impact of segmentation faults: In (2), if one module causes segmentation fault, the whole application crashes. In (1), modules have different memory regions and thus only the module that cause segmentation fault will crash.
Message delivery guarantee: In (2), you can assume that message delivery is guaranteed. In (1) the receiving module may crash before the receival or during handling of the message.
Sharing memory between modules: In (2), the whole memory is shared by all modules, so you can directly send message objects. In (1), you need to use 'Shared Memory' between modules.
Messaging implementation: In (2), you can send message objects between modules, in (1) you need to use either of network socket, unix socket, pipes, or message objects stored in a Shared Memory. For the sake of efficiency, storing message objects in a Shared Memory seems to be the best choice.
Pointer usage between modules: In (2), you can use pointers in your message objects. The ownership of heap objects (accessed by pointers in the messages) can be transferred to the receiving module. In (1), you need to manually manage the memory (with custom malloc/free functions) in the 'Shared Memory' region.
Module management: In (2), you are managing just one process. In (1), you need to manage a pool of processes each representing one module.
Sounds like you're implementing Communicating Sequential Processes. Excellent!
Tackling threads vs processes first, I would stick to threads; the context switch times are faster (especially on Windows where process context switches are quite slow).
Second, shared memory vs a message queue; if you're doing full synchronous message passing it'll make no difference to performance. The shared memory approach involves a shared buffer that gets copied to by the sender and copied from by the reader. That's the same amount of work as is required for a message queue. So for simplicity's sake I would stick with the message queue.
in fact you might like to consider using a pipe instead of a message queue. You have to write code to make the pipe synchronous (they're normally asynchronous, which would be Actor Model; message queues can often be set to zero length which does what you want for it to be synchronous and properly CSP), but then you could just as easily use a socket instead. Your program can then become multi-machine distributed should the need arise, but you've not had to change the architecture at all. Also named pipes between processes is an equivalent option, so on platforms where process context switch times are good (e.g. linux) the whole thread vs process question goes away. So working a bit harder to use a pipe gives you very significant scalability options.
Regarding crashing; if you go the multiprocess route and you want to be able to gracefully handle the failure of a process you're going to have to do a bit of work. Essentially you will need a thread at each end of the messaging channel simply to monitor the responsiveness of the other end (perhaps by bouncing a keep-awake message back and forth between themselves). These threads need to feed status info into their corresponding main thread to tell it when the other end has failed to send a keep-awake on schedule. The main thread can then act accordingly. When I did this I had the monitor thread automatically reconnect as and when it could (e.g. the remote process has come back to life), and tell the main thread that too. This means that bits of my system can come and go and the rest of it just copes nicely.
Finally, your actual application processes will end up as a loop, with something like select() at the top to wait for message inputs from all the different channels (and monitor threads) that it is expecting to hear from.
By the way, this sort of thing is frustratingly hard to implement in Windows. There's just no proper equivalent of select() anywhere in any Microsoft language. There is a select() for sockets, but you can't use it on pipes, etc. like you can in Unix. The Cygwin guys had real problems implementing their version of select(). I think they ended up with a polling thread per file descriptor; massively inefficient.
Good luck!
Your question lacks a description of how the "modules" are implemented and what do they do, and possibly a description of the environment in which you are planning to implement all of this.
For example:
If the modules themselves have some requirements which makes them hard to implement as threads (e.g. they use non-thread-safe 3rd party libraries, have global variables, etc.), your message delivery system will also not be implementable with threads.
If you are using an environment such as Python which does not handle thread parallelism very well (because of its global interpreter lock), and running on Linux, you will not gain any performance benefits with threads over processes.
There are more things to consider. If you are just passing data between modules, who says your system needs to use either multiple threads or multiple processes? There are other architectures which do the same thing without either of them, such as event-driven with callbacks (a message receiver can register a callback with your system, which is invoked when a message generator generates a message). This approach will be absolutely the fastest in any case where parallelism isn't important and where receiving code can be invoked in the execution context of the caller.
tl;dr: you have only scratched the surface with your question :)
I am writing a multithreaded desktop application.
Generally
I am unsure about the implications that multitreading has on the architecture. There is a lot of literature on architecture, but I know none that takes multithreading into account. There is a lot of literature on the low level stuff of multithreading (mutexes, semaphores, etc.) but I know none that describes how these concepts are embedded into an architecture.
Which literature do you recommend to fill this gap?
Particularly
My application consist of
Presentation that creates and manages dialogs using a GUI toolkit,
Kernel that knows all about the domain of the application,
Controller that knows the Kernel and the Presentation and moderates between these two.
To be more precise, here is how files are opened:
The Presentation signals a FileOpenCommand.
The ApplicationController recieves this signal and
uses the ApplicationKernel to create a File object,
uses the ApplicationPresentation to create a FilePresentation object,
creates a FileController object, passing the File and the FilePresentation to the constructor.
The FileController registers itself as an observer on its File and FilePresentation.
Let's say File provides a long running operation Init() that should not block the user interface. There are two approaches that came to my mind:
File::Init() returns an object that encapsulates a thread and can be used to register an observer that is notified about progress, errors, completion etc. This puts a lot of responsibility into the FileController (who would be the observer), because it is now accessed from both the main thread as well as from the working thread.
Hide the working thread completely from the Controller. File::Init() would return nothing, but the ApplicationKernel would signal creation, progress and errors of long running operations in the main thread. This would drag a lot of communication though the ApplicationKernel, turning it into something like a god object.
Which of these two is the common approach to multithreading in a desktop application (if any)? Which alternative approaches do you recommend?
I suggest that you consider using the actor model. It is a concurrency abstraction that hides a lot of the details associated with threads, locks, etc.
Edit
Some additional comments spurred by #CMR's comment...
Under an actor model, I imagine that the application would still be structured using the same components as suggested in the question: Presentation, ApplicationController, etc. The difference with the actor model is that the components (now actors) would not hold direct references to each other. Rather, they would hold channels to which they could post asynchronous, immutable, messages to one another.
The sequence of events in the "open a file" case would be essentially the same in the actor model, except that channels would be passed to the FileController in step 2.3 instead of direct object references. Similarly, observer registration in occurs through channels.
So what's the difference? The main difference is that none of the code needs to be thread-aware. Threads are invisible to the application logic, being the concern of the actor framework. If one can follow the discipline of only passing immutable objects through the channels (which some actor frameworks enforce), then virtually all the difficult logic associated with thread synchronization disappears. Of course, one has to switch mindsets from a synchronous programming model to an asynchronous one -- not necessarily a trivial task. However, it is my opinion that the cost of that switch is outweighed by the benefit of not having to think about thread-safety (at least in systems of some complexity).
In UI programming in particular, asynchronous models make it much easier to give nice user feedback. For example, a UI element may kick off a long-running task, display a "working..." message, and then go to sleep. Some time later, a message arrives delivering the results of the long running task which the UI element then displays in place of the "working..." message. In similar fashion, tree views can be built incrementally as each tree node's data arrives in an incoming message.
You can think of an actor model as a generalization of the classic UI "event pump" approach -- except that every component (actor) is running its own event pump simultaneously instead of one pump dispatching to a bunch of components. Actor frameworks provide a way to run large or even huge numbers of such "simultaneous pumps" with very low overhead. In particular, a small number of threads (say, one per cpu) service a large number of actors.