Private variable disappears after thread starts - multithreading

Problem:
I have a private variable that is available during the startup of a threaded object, but is out of scope when it is used later (via a signal and slot call).
Details:
I have an application that I'm developing in Qt5 for both linux and windows.
Currently it works as expected under linux (where development began), but now
that I'm trying to stand it up on windows 7 (I didn't have a copy of windows initially) I have run into this problem where (on windows only) my private variables go out of scope after the thread initializes.
Question:
What is wrong with my object/thread structure such that the variable scope is fine under Linux, but not in windows? I thought that was the kind of "behind the scenes" stuff Qt took care of? (clearly not)
More Detail:
The order of operation goes like this
Instantiate an object
Move the object into a thread
Get the thread's start signal and call an init function in the object
Later, get data and emit a signal to the threaded object
Threaded object processes data
The code outlining the steps above is summarized below.
void MyWorkerClass::init()
{
// ... bunchOCode
procThread = new QThread; // <-- procThread - private to MyWorkerClass
procObj = new Processor(startupData); // <-- procObj - private to MyWorkerClass
procObj->moveToThread(procThread);
connect(procThread, SIGNAL(started()), procObj, SLOT(doStart()));
connect(this, SIGNAL(dataIsReady(void *)), procObj, SLOT(processMsgs(void *)));
procThread->start();
ok = waitforProcSetup();
// ... Life is good, do more stuff
}
class Processor : public QObject
{
// ... Other
// ... stuff
private slots:
void doStart();
void processMsgs(void * buffer);
private:
QHash<QString, bool> process;
}
void Processor::doStart() // <-- private slot
{
// ... take care of init stuff that couldn't be done in constructor
// Variable is valid here and I can work with it.
foreach(site, locations.uniqueKeys()) {
process[site] = true; // <-- works like a champ
qDebug() << QString("%1 => %2").arg(site).arg(process[site]);
}
}
void Processor::processMsgs(void * buffer) // <-- buffer is malloc'd memory and works fine
{
// ... When MyWorkerClass gets some data it emits a signal that is connected
// to this private slot.
// Simply trying to examine the variable causes a segfault (because it's uninitialized here)
qDebug() << "... processMsgs:" << process.isEmpty(); // <-- wets the bed
}
.

In trying to improve my question, by following the suggestions from the people who commented, I found out what was going on. I was working on creating a small working version of the example I posted (thanks Kuba Ober). The "error" I was encountering was a segmentation fault that could consistently be recreated with the debug line:
qDebug() << "... processMsgs:" << process.isEmpty(); // <-- wets the bed
Specifically, the private QHash variable "process" was useable when I called it the first time (after the thread was up and running)
connect(procThread, SIGNAL(started()), procObj, SLOT(doStart()));
but that same variable acted like it had gone out of scope when I tried to call it the second time
connect(this, SIGNAL(dataIsReady(void *)), procObj, SLOT(processMsgs(void *)));
The signal (dataIsReady(void *)) for this second call is an explicit "emit" that the worker class does when it's collected some data that can be processed. I tried to make that clear in the comments of the example pseudocode, but I didn't take into account that the comment code I included wouldn't stand out that well since it's all grey.
What was really going on was right after I filled "process[site]" with data I also looped over a quint64 array filling it with data too. The loop went 1 element too far and wrote into the "process" variable, making it look to me like it had gone out of scope. In linux it was purely coincidental that it didn't segfault (likely there was padding between the array and the QHash), but the windows runtime exposed the error for the first time.

Related

Capturing every frame to recognize text ARcore - Do i need to use semaphores?

I am developing an app in which I want to recognize text in real-time.
At first I was using onTapListener so whenever the user was tapping on the screen the current frame was being captured and after that the text recognition was called.
Right now I want to do this completely real-time so the user will not tap on the screen to capture the current frame. But every current frame is going to be captured until the moment the text of one captured current frame will be recognized.
For this reason, I created a global boolean field name locked that is initialized in false and I use it as a "locker" as you will see in a bit.
private boolean locked = false;
And in a method onUpdateFrame(FrameTime frameTime) I use the above global variable locked. When the first feature points are tracked I "lock" the update. So only the current thread gonna capture the current frame. And if the recognized data is null the I put locked = true so the next frame gonna be captured.
This is my code
public void onUpdateFrame(FrameTime frameTime) {
Frame frame = arFragment.getArSceneView().getArFrame();
// If there is no frame, just return.
if (frame == null) {
return;
}
//Making sure ARCore is tracking some feature points, makes the augmentation little stable.
if(frame.getCamera().getTrackingState()==TrackingState.TRACKING && !locked) {
locked = true;
if (mProgressDialog == null) {
mProgressDialog = ProgressDialog.show(this, "Processing",
"OCR...", true);
} else {
mProgressDialog.show();
}
executor.execute(() -> {
Bitmap b = captureImage();
final String[] text = {getOCRResult(b)};
handler.post(() -> {
if(text[0] != null && !text[0].equals("")){
doSomething();
}
else{
locked = false;
}
}
This is not working though. And my app is crashing immediately when is detecting the surface.
I am getting the following error, and the Toast that the error talks about it refers to a Toast that I have inside the method captureImage()
E/AndroidRuntime: FATAL EXCEPTION: pool-1-thread-1
Process: opencv.org, PID: 27860
java.lang.RuntimeException: Can't toast on a thread that has not called Looper.prepare()
I cant understand what I am doing wrong.
I heard of semaphores and this is why I asked that on my question. Should i use semaphores , do i need something like that so my app will work. As i understand i need one thread anytime to do the capture of the current frame.
Could someone help me i am a bit lost?
thank you
The reason you are getting this error is that you can't show a toast, or any UI, on a non-UI thread.
The usual way to handle this is to create a 'Handler' on the UI thread and send a message to its message queue asking it to post the thread.
You can see examples in both Java and Kotlin in this answer: Can't create handler inside thread that has not called Looper.prepare()
More info on Handler here: https://developer.android.com/reference/android/os/Handler

SHGetFileInfo doesn't return correct handle to icon

I'm using SHGetFileInfo function for getting icons for folders and different file types. According to MSDN call of this function should be done from background thread and before call Component Object Model (COM) must be initialized with CoInitialize or OleInitialize.
My code looks like this:
public void SetHlinkImage(string path)
{
Shell32.OleInitialize(IntPtr.Zero);
Task task = Task.Factory.StartNew(() => { LoadIcons(path); });
}
private void LoadIcons(string path)
{
image = GetHlinkImage(path);
if (OwnerControl.InvokeRequired)
layout.ModuleControl.BeginInvoke((MethodInvoker)delegate ()
{
Shell32.OleUninitialize();
});
}
public Icon GetHlinkImage(string path)
{
uint flags = Shell32.SHGFI_ICON | Shell32.SHGFI_ATTRIBUTES | Shell32.SHGFI_SMALLICON;
Shell32.SHFILEINFO shfi = new Shell32.SHFILEINFO();
IntPtr result = Shell32.SHGetFileInfo(path,
Shell32.FILE_ATTRIBUTE_DIRECTORY,
ref shfi,
(uint)Marshal.SizeOf(shfi),
flags);
Icon icon = (Icon)Icon.FromHandle(shfi.hIcon).Clone();
WinApi.DestroyIcon(shfi.hIcon); // cleanup
return icon;
}
Mostly the problem appears after first call of the code and as result I get an exception when I tried to create Icon from icon handle:
System.ArgumentException: Win32 handle that was passed to Icon is not
valid or is the wrong type
And further calls of the code work without problems.
Actually behaviour also somehow depends on the test system. It is hardly possible to reproduce this issue on Windows10 systems but on Windows 7 it happens quite often.
Has anyone experienced this problem?
From comment of Hans Passant:
Calling OleInitialize() is pointless, the CLR already initializes COM before it starts a thread. And it failed, something you cannot see because you are not checking its return value. Not knowing that, it just spirals into undiagnosable misery from there. Yes, more of it on Win7. You must provide an STA thread, if this needs to run in the background then consider this solution.

SAPI 5 TTS Events

I'm writing to ask you some advices for a particular problem regarding SAPI engine. I have an application that can speak both to the speakers and to a WAV file. I also need some events to be aware, i.e. word boundary and end input.
m_cpVoice->SetNotifyWindowMessage(m_hWnd, TTS_MSG, 0, 0);
hr = m_cpVoice->SetInterest(SPFEI_ALL_EVENTS, SPFEI_ALL_EVENTS);
Just for test I added all events! When the engine speaks to speakers all events are triggered and sent to the m_hWnd window, but when I set output to the WAV file, none of them are sent
CSpStreamFormat fmt;
CComPtr<ISpStreamFormat> pOld;
m_cpVoice->GetOutputStream(&pOld);
fmt.AssignFormat(pOld);
SPBindToFile(file, SPFM_CREATE_ALWAYS, &m_wavStream, &fmt.FormatId(), fmt.WaveFormatExPtr());
m_cpVoice->SetOutput(m_wavStream, false);
m_cpVoice->Speak(L"Test", SPF_ASYNC, 0);
Where file is a path passed as argument.
Really this code is taken from the TTS samples found on the SAPI SDK. It seems a little bit obscure the part setting the format...
Can you help me in finding the problem? Or does anyone of you know a better way to write TTS to WAV? I can not use manager code, it should be better to use the C++ version...
Thank you very much for help
EDIT 1
This seems to be a thread problem and searching in the spuihelp.h file, that contains the SPBindToFile helper I found that it uses the CoCreateInstance() function to create the stream. Maybe this is where the ISpVoice object looses its ability to send event in its creation thread.
What do you think about that?
I adopted an on-the-fly solution that I think should be acceptable in most of the cases, In fact when you write speech on files, the major event you would be aware is the "stop" event.
So... take a look a the class definition:
#define TTS_WAV_SAVED_MSG 5000
#define TTS_WAV_ERROR_MSG 5001
class CSpeech {
public:
CSpeech(HWND); // needed for the notifications
...
private:
HWND m_hWnd;
CComPtr<ISpVoice> m_cpVoice;
...
std::thread* m_thread;
void WriteToWave();
void SpeakToWave(LPCWSTR, LPCWSTR);
};
I implemented the method SpeakToWav as follows
// Global variables (***)
LPCWSTR tMsg;
LPCWSTR tFile;
long tRate;
HWND tHwnd;
ISpObjectToken* pToken;
void CSpeech::SpeakToWave(LPCWSTR file, LPCWSTR msg) {
// Using, for example wcscpy_s:
// tMsg <- msg;
// tFile <- file;
tHwnd = m_hWnd;
m_cpVoice->GetRate(&tRate);
m_cpVoice->GetVoice(&pToken);
if(m_thread == NULL)
m_thread = new std::thread(&CSpeech::WriteToWave, this);
}
And now... take a look at the WriteToWave() method:
void CSpeech::WriteToWav() {
// create a new ISpVoice that exists only in this
// new thread, so we need to
//
// CoInitialize(...) and...
// CoCreateInstance(...)
// Now set the voice, i.e.
// rate with global tRate,
// voice token with global pToken
// output format and...
// bind the stream using tFile as I did in the
// code listed in my question
cpVoice->Speak(tMsg, SPF_PURGEBEFORESPEAK, 0);
...
Now, because we did not used the SPF_ASYNC flag the call is blocking, but because we are on a separate thread the main thread can continue. After the Speak() method finished the new thread can continue as follow:
...
if(/* Speak is went ok */)
::PostMessage(tHwn, TTS_WAV_SAVED_MSG, 0, 0);
else
::PostMessage(tHwnd, TTS_WAV_ERROR_MSG, 0, 0);
}
(***) OK! using global variables is not quite cool :) but I was going fast. Maybe using a thread with the std::reference_wrapper to pass parameters would be more elegant!
Obviously, when receiving the TTS messages you need to clean the thread for a next time call! This can be done using a CSpeech::CleanThread() method like this:
void CSpeech::CleanThread() {
m_thread->join(); // I prefer to be sure the thread has finished!
delete m_thread;
m_thread = NULL;
}
What do you think about this solution? Too complex?

Does Arduino support threading?

I have a couple of tasks to do with arduino but one of them takes very long time, so I was thinking to use threads to run them simultaneously.
I have an Arduino Mega
[Update]
Finally after four years I can install FreeRTOS in my arduino mega. Here is a link
In short: NO.
But you may give it a shot at:
http://www.kwartzlab.ca/2010/09/arduino-multi-threading-librar/
(Archived version: https://web.archive.org/web/20160505034337/http://www.kwartzlab.ca/2010/09/arduino-multi-threading-librar
Github: https://github.com/jlamothe/mthread
Not yet, but I always use this Library with big projects:
https://github.com/ivanseidel/ArduinoThread
I place the callback within a Timer interrupt, and voilá! You have pseudo-threads running on the Arduino...
Just to make this thread more complete: there are also protothreads which have very small memory footprint (couple bytes if I remember right) and preserve variables local to thread; very handy and time saving (far less finite state machines -> more readable code).
Examples and code:
arduino-class / ProtoThreads wiki
Just to let you know what results you may expect: serial communication # 153K6 baudrate with threads for: status diodes blinking, time keeping, requested functions evaluation, IO handling and logic and all on atmega328.
Not real threading but TimedActions are a good alternative for many uses
http://playground.arduino.cc/Code/TimedAction#Example
Of course, if one task blocks, the others will too, while threading can let one task freeze and the others will continue...
No you can't but you can use Timer interrupt.
Ref : https://www.teachmemicro.com/arduino-timer-interrupt-tutorial/
The previous answer is correct, however, the arduino generally runs pretty quick, so if you properly time your code, it can accomplish tasks more or less simultaneously.
The best practice is to make your own functions and avoid putting too much real code in the default void loop
You can use arduinos
It is designed for Arduino environment. Features:
Only static allocation (no malloc/new)
Support context switching when delaying execution
Implements semaphores
Lightweight, both cpu and memory
I use it when I need to receive new commands from bluetooth/network/serial while executing the old ones and the old ones have delay in them.
One thread is the sever thread that does the following loop:
while (1) {
while ((n = Serial.read()) != -1) {
// do something with n, like filling a buffer
if (command_was_received) {
arduinos_create(command_func, arg);
}
}
arduinos_yield(); // context switch to other threads
}
The other is the command thread that executes the command:
int command_func(void* arg) {
// move some servos
arduinos_delay(1000); // wait for them to move
// move some more servos
}
Arduino does not support multithread programming.
However there have been some workarounds, for example the one in this project (you can install it also from the Arduino IDE).
It seems you have to define the schedule time yourself while in a real multithread environment it is the OS that decides when to execute tasks.
Alternatively you can use protothreads
The straight answer is No No No!. There are some alternatives but you can't expect a perfect multi threading functionality from an arduino mega. You can use arduino due or lenado for multithreading like below-
void loop1(){
}
void loop2(){
}
void loop3(){
}
Normally, I handle those types of cases in backend. You can run the main code in a server while using Arduino to just collect inputs and show outputs. In such cases I would prefer nodemcu which has built in wifi.
Thread NO!
Concurrent YES!
You can run different tasks concurrently with FreeRTOS library.
https://www.arduino.cc/reference/en/libraries/freertos/
void TaskBlink( void *pvParameters );
void TaskAnalogRead( void *pvParameters );
// Now set up two tasks to run independently.
xTaskCreate(
TaskBlink
, (const portCHAR *)"Blink" // A name just for humans
, 128 // Stack size
, NULL
, 2 // priority
, NULL );
xTaskCreate(
TaskAnalogRead
, (const portCHAR *) "AnalogRead"
, 128 // This stack size can be checked & adjusted by reading Highwater
, NULL
, 1 // priority
, NULL );
void TaskBlink(void *pvParameters) // This is a task.
{
(void) pvParameters;
// initialize digital pin 13 as an output.
pinMode(13, OUTPUT);
for (;;) // A Task shall never return or exit.
{
digitalWrite(13, HIGH); // turn the LED on (HIGH is the voltage level)
vTaskDelay( 1000 / portTICK_PERIOD_MS ); // wait for one second
digitalWrite(13, LOW); // turn the LED off by making the voltage LOW
vTaskDelay( 1000 / portTICK_PERIOD_MS ); // wait for one second
}
}
void TaskAnalogRead(void *pvParameters) // This is a task.
{
(void) pvParameters;
// initialize serial communication at 9600 bits per second:
Serial.begin(9600);
for (;;)
{
// read the input on analog pin 0:
int sensorValue = analogRead(A0);
// print out the value you read:
Serial.println(sensorValue);
vTaskDelay(1); // one tick delay (15ms) in between reads for stability
}
}
Just take care!
When different tasks tried to reach variables at the same time, like i2c communication line or sd card module. Use Semaphores and mutexes
https://www.geeksforgeeks.org/mutex-vs-semaphore/.
Arduino does not supports threading. However, you can do the next best thing and structure your code around state machines running in interleaving.
While there are lots of ways to implement your tasks as state machines, I recommend this library (https://github.com/Elidio/StateMachine). This library abstracts most of the process.
You can create a state machine as a class like this:
#include "StateMachine.h"
class STATEMACHINE(Blink) {
private:
int port;
int waitTime;
CREATE_STATE(low);
CREATE_STATE(high);
void low() {
digitalWrite(port, LOW);
*this << &STATE(high)<< waitTime;
}
void high() {
digitalWrite(port, HIGH);
*this << &STATE(low)<< waitTime;
}
public:
Blink(int port = 0, int waitTime = 0) :
port(port),
waitTime(waitTime),
INIT_STATE(low),
INIT_STATE(high)
{
pinMode(port, OUTPUT);
*this << &STATE(low);
}
};
The macro STATEMACHINE() abstracts the class inheritances, the macro CREATE_STATE() abstracts the state wrapper creation, the macro INIT_STATE() abstracts method wrapping and the macro STATE() abstracts state wrapper reference within the state machine class.
State transition is abstracted by << operator between the state machine class and the state, and if you want a delayed state transition, all you have to do is to use that operator with an integer, where the integer is the delay in millisseconds.
To use the state machine, first you have to instantiate it. Declaring an reference to the class in global space while instantiating it with new on setup function might do the trick
Blink *led1, *led2, *led3;
void setup() {
led1 = new Blink(12, 300);
led2 = new Blink(11, 500);
led3 = new Blink(10, 700);
}
Then you run the states on loop.
void loop() {
(*led2)();
(*led1)();
(*led3)();
}

Multithreaded code - force execution order

I have some code that will be accessed from two threads:
class Timer{
public:
void Start(){
start_ = clock_->GetCurrentTime();
running_ = true;
}
void Read(){
if(running_){
time_duration remaining = clock_->GetCurrentTime() - start_;
Actions(remaining)
}
}
private:
void Actions(time_duration remaining);
time start_;
bool running;
};
I've looked at some other timers available in various libraries, but didn't find any that fit my requirements, thus I'm rolling my own...
The Start() method is called (once only) from one thread. The Read() method is called very rapidly from another thread, the calls will start coming in before Start() is called.
Obviously it is very important that the start_ variable is initialized before the running_ flag is set. This could be solved by adding a mutex that gets grabbed when the Start() method is entered...and is grabbed before running_ is checked in the Read() method...but it seems somewhat unnecessary. If everything here executes in order, then there are no problems. I have no problem with the fact that a Read() could happen while the other thread is in the Start() routing, getting the time from the clock for example...the Read()s happen fast enough that its just not a big deal.
Anyway, I was looking for a way to ensure that the compiler/processor will execute the
start_ = clock_->GetCurrentTime();
running_ = true;
instructions in order they are listed above. (Or if I'm overlooking something else).
You need to make start_ and running_ volatile, and then introduce a memory barrier between the two assignments in Start().
Why not get rid of the "running" flag and use whether the "start" variable is empty as the condition? You don't specify a language but some sort of volatile marker for "start" would be a good idea too. This also assumes that "start" can be written atomically. For example:
class Timer{
public:
void Start(){
start_ = clock_->GetCurrentTime();
}
void Read(){
if(nullptr != start_){
time_duration remaining = clock_->GetCurrentTime() - start_;
Actions(remaining)
}
}
private:
void Actions(time_duration remaining);
volatile time start_;
};
I'm not sure whether I understand your question, but I think you want to prevent that Read is being executed when the Start method hasn't been set (or hasn't been executed completely) ?
If that's the case, can't you solve your problem by using an AutoResetEvent or ManualResetEvent (depending on what behavior you want).
In the Read method, you could specify that the Read method should wait when the Auto/ManualResetEvent hasn't been set, and at the end of the Start method, you can set the Event.
Why not just stick a
if (_running == false) Start();
In the Read() method.
And either protect Start with a mutex or define it as "critical" to ensure its single threaded.

Resources