Currently I have a code that is crashing (SEGFAULT) on me.
I am trying to compare a big amount of images that are in my drive to their counter parts in a server.
To speed up the process I get the image from the server and compare the images on a different thread.
From what I already tried and debugged, the issue is in getting the image from the server (that is why the other calls are commented out).
Also if I run without the QtConcurrent::run it does not crash, but if I put semaphore concurrentComparisons with only one resorce, it will crash.
Finally I also get the following errors
QObject::connect: Cannot connect (null)::configurationAdded(QNetworkConfiguration) to QNetworkConfigurationManager::configurationAdded(QNetworkConfiguration)
QObject::connect: Cannot connect (null)::configurationRemoved(QNetworkConfiguration) to QNetworkConfigurationManager::configurationRemoved(QNetworkConfiguration)
QObject::connect: Cannot connect (null)::configurationChanged(QNetworkConfiguration) to QNetworkConfigurationManager::configurationChanged(QNetworkConfiguration)
QObject::connect: Cannot connect (null)::onlineStateChanged(bool) to QNetworkConfigurationManager::onlineStateChanged(bool)
QObject::connect: Cannot connect (null)::configurationUpdateComplete() to QNetworkConfigurationManager::updateCompleted()
Any help would be very appreciated.....
Relevant code:
QSemaphore FileComparisonInfo::concurrentComparisons(1);
QtConcurrent::run( [this, localPath, imageURL]()
{
ImageComparer cmp;
FileComparisonInfo::concurrentComparisons.acquire();
//cmp.setImageLeftPath(localPath);
cmp.setImageRightPath(imageURL);
//cmp.createDifferenceImage();
FileComparisonInfo::concurrentComparisons.release();
});
void ImageComparer::setImageRightPath(QString path)
{
this->rightImagePath = path;
this->imageRight = getImage(path);
}
QImage* ImageComparer::getImage(QString path)
{
QUrl url(path);
QFile file(path);
if(file.exists())
{
return new QImage(path);
}
else if(url.isValid())
{
return getImageFromURL(path);
}
}
QImage* ImageComparer::getImageFromURL(QString url)
{
QNetworkAccessManager * tempNAM = new QNetworkAccessManager();
QNetworkReply *imageConnection = tempNAM->get( QNetworkRequest( QUrl( url ) ));
QEventLoop loop;
connect(imageConnection, SIGNAL(finished()), &loop, SLOT(quit()));
loop.exec();
QImage * downloadedImage;
if(imageConnection->error() != QNetworkReply::NoError)
{
qDebug() << imageConnection->errorString();
downloadedImage = new QImage();
}
else
{
QByteArray data = imageConnection->readAll();
downloadedImage = new QImage(QImage::fromData(data));
}
tempNAM->deleteLater();
imageConnection->deleteLater();
return downloadedImage;
}
Unfortunately this had nothing to do with the code.
One of the images was corrupted and was segfaulting in the comparison.
Related
Im trying to create a multithreaded namedpipe server as outlined in the msdn sample here https://learn.microsoft.com/en-us/windows/win32/ipc/multithreaded-pipe-server but Im trying to restrict the namedpipe to access by adminstrators group members only.
The example works correctly when no SECURITY_ATTRIBUTES structure is specified but when an SA is specified the first call is successful, but following calls to CreateNamedPipe fail as long as the first pipe is listening or communicating with a client. The create call fails, usually with ACCESS_DENIED, but sometimes with error 1305 The revision level is unknown. When the first pipe closes due to client disconnecting the following call will be successful for the next createnamedpipe call but will in turn fail once that pipe has a client.
I have tried multiple values for the grfInheritance field with no avail. This is my first adventure into explicitly specifying SECURITY so forgive me if I have missed something obvious. Note that in the Function that calls createnamedpipe I create a new SA structure with each create attempt but I have also tried creating one and sharing it outside the create loop.
Relevant code follows:
function that creates the pipe:
HRESULT DapiSettingsSvr::DapiSettingsListener()
{
while(m_run)
{
//find an unused control array member. If they are all used we have max connection so dont create a pipe.
UINT connectId = 0;
for (connectId; connectId < MAX_CONNECTIONS; connectId++)
{
if (m_controlArray[connectId].inuse == false)
break;
}
SECURITY_ATTRIBUTES sa;
HRESULT hr = InitializeSecurity(&sa);
if (FAILED(hr))
{
return hr;
}
if (connectId < MAX_CONNECTIONS)
{
HANDLE hpipe;
hpipe = CreateNamedPipe(
lpszPipename, // pipe name
PIPE_ACCESS_DUPLEX, // read/write access
PIPE_TYPE_BYTE | // byte bipe
PIPE_READMODE_BYTE | // read as bytes
PIPE_WAIT | // do not return until data is recieved
PIPE_REJECT_REMOTE_CLIENTS, // no remote connections
MAX_CONNECTIONS, // max. instances
OUTPUT_BUFFER_SIZE, // output buffer size
INPUT_BUFFER_SIZE, // input buffer size
0, // client time-out
&sa); // default security attribute
// CleanUpSecurityResources();
if (hpipe == INVALID_HANDLE_VALUE)
{
swprintf(logbuffer, ARRAYSIZE(logbuffer), L"CreateNamedPipe failed, GLE=%d.\n", GetLastError());
DapiSettingLogger(logbuffer);
}
else
{
m_controlArray[connectId].inuse = true;
m_controlArray[connectId].pThis = this;
m_controlArray[connectId].connectId = connectId;
m_controlArray[connectId].pipehandle = hpipe;
swprintf(logbuffer, ARRAYSIZE(logbuffer), L"\nPipe Server: Main thread awaiting client connection on %s\n", lpszPipename);
DapiSettingLogger(logbuffer);
// block until a client tries to connect.success is non zero. However a client can connect between the create call and ConnectNamedPipe call.
// In this case ConnectNamedPipe returns zero but GLE = ERROR_PIPE_CONNECTED and a valid connection exists. Check for this case.
fConnected = ConnectNamedPipe(hpipe, NULL) ? TRUE : (GetLastError() == ERROR_PIPE_CONNECTED);
if (fConnected)
{
// Create a thread for this client.
m_controlArray[connectId].threadHandle = CreateThread(
NULL, // no security attribute
0, // default stack size
WorkerInstance, // thread proc
(LPVOID)&m_controlArray[connectId], // thread parameter
0, // not suspended
&m_controlArray[connectId].threadId); // returns thread ID
if (m_controlArray[connectId].threadHandle == NULL)
{
swprintf_s(logbuffer, ARRAYSIZE(logbuffer), L"CreateThread failed, GLE=%d.\n", GetLastError());
DapiSettingLogger(logbuffer);
CloseHandle(m_controlArray[connectId].pipehandle);
ZeroMemory(&m_controlArray[connectId], sizeof(WORKER_INFO));
}
}
else
{
// The client could not connect, so close the pipe.
CloseHandle(m_controlArray[connectId].pipehandle);
ZeroMemory(&m_controlArray[connectId], sizeof(WORKER_INFO));
}
} //else valid connection
}
else
{
DapiSettingLogger((LPWSTR) L"Max Connections reached\n");
}
}
return S_OK;
}
Function that creates the SA
HRESULT DapiSettingsSvr::InitializeSecurity(SECURITY_ATTRIBUTES* psa)
{
HRESULT result = S_OK;
DWORD res, error;
EXPLICIT_ACCESS ea[1];
SID_IDENTIFIER_AUTHORITY SIDAuthNT = SECURITY_NT_AUTHORITY;
// Create a SID for the BUILTIN\Administrators group.
if (!AllocateAndInitializeSid(&SIDAuthNT, 2,
SECURITY_BUILTIN_DOMAIN_RID,
DOMAIN_ALIAS_RID_ADMINS,
0, 0, 0, 0, 0, 0,
&m_pAdminSID))
{
error = GetLastError();
swprintf(logbuffer, ARRAYSIZE(logbuffer), L"AllocateAndInitializeSid Error %u\n", error);
DapiSettingLogger(logbuffer);
result = HRESULT_FROM_WIN32(error);
goto Cleanup;
}
ea[0].grfAccessPermissions = GENERIC_ALL;
ea[0].grfAccessMode = GRANT_ACCESS;
ea[0].grfInheritance = SUB_CONTAINERS_AND_OBJECTS_INHERIT; //changing
ea[0].Trustee.TrusteeForm = TRUSTEE_IS_SID;
ea[0].Trustee.TrusteeType = TRUSTEE_IS_GROUP;
ea[0].Trustee.ptstrName = (LPTSTR)m_pAdminSID;
// Create a new ACL that contains the new ACE.
res = SetEntriesInAcl(1, ea, NULL, &m_pACL);
if (ERROR_SUCCESS != res)
{
swprintf(logbuffer, ARRAYSIZE(logbuffer),L"SetEntriesInAcl Error %u\n", res);
DapiSettingLogger(logbuffer);
result = HRESULT_FROM_WIN32(res);
goto Cleanup;
}
// Initialize a descriptor Use localalloc as it allows memory moving without changing handle value
m_pSD = (PSECURITY_DESCRIPTOR) LocalAlloc(LPTR,
SECURITY_DESCRIPTOR_MIN_LENGTH);
if (NULL == m_pSD)
{
error = GetLastError();
swprintf(logbuffer, ARRAYSIZE(logbuffer), L"LocalAlloc Error %u\n", error);
result = HRESULT_FROM_WIN32(error);
goto Cleanup;
}
if (!InitializeSecurityDescriptor(m_pSD,
SECURITY_DESCRIPTOR_REVISION))
{
error = GetLastError();
swprintf(logbuffer, ARRAYSIZE(logbuffer), L"InitializeSecurityDescriptor Error %u\n", error);
result = HRESULT_FROM_WIN32(error);
goto Cleanup;
}
// Add the ACL to the security descriptor.
if (!SetSecurityDescriptorDacl(m_pSD,
TRUE, // bDaclPresent flag
m_pACL,
FALSE)) // not a default DACL
{
error = GetLastError();
swprintf(logbuffer, ARRAYSIZE(logbuffer), L"SetSecurityDescriptorDacl Error %u\n", error);
result = HRESULT_FROM_WIN32(error);
goto Cleanup;
}
Cleanup:
if (FAILED(result))
{
CleanUpSecurityResources();
}
else
{
// Initialize a security attributes structure.
psa->nLength = sizeof(SECURITY_ATTRIBUTES);
psa->lpSecurityDescriptor = m_pSD;
psa->bInheritHandle = TRUE; /// NOTE I have toyed with this value also
}
return result;
}
Any input on what Im doing incorrectly would be greatly appriciated!!
Thanks!
According to Named Pipe Security and Access Rights,
In addition to the requested access rights, the DACL must allow the
calling thread FILE_CREATE_PIPE_INSTANCE access to the named pipe.
Ok, I figured this out. Im going to mark YangXiaoPo's answer as correct as this pointed me in the right direction but for clarification GENERIC_ALL already includes the right to FILE_CREATE_PIPE_INSTANCE or at least thats what my testing indcates. So setting the EXPICIT_ACCESS structure field to ea[0].grfAccessPermissions = GENERIC_ALL | FILE_CREATE_PIPE_INSTANCE; does not resolve this issue.
The answer lies in the fact that I was running the PipeServer program from within visual studio ( debug ) and thus as a generic user. So the first time through the loop a pipe gets created and the SA with the local administrators group ACE is then applied to the pipe.
So we get a pipe created in the listening state. As soon as a client connects the working thread is created and then the the while(m_run) loop does another iteration and tries to create a new pipe instance. This attempt fails ( actually a looping fail ) because the security attribute with the administrators only ACL is now looked at and the program is not running as an administrator. As soon as the first client disconnects the working thread closes the pipe handle ( effectively destroying the pipe ) and then in the next iteration a pipe is again created.
Running the program as Administrator ( or starting Visual studio as Admin and then debugging ) resolves the issue, though I think a fully correct solution would be to create a second ACE that specified Creator Owner in addition to Admin for the SA DACL.
Thanks!!
Trying to figure out why I can login with my rest API just fine on the main thread but not in a worker. All communication channels are operating fine and I am able to load it up no problem. However, when it tries to send some data it just hangs.
[Embed(source="../bin/BGThread.swf", mimeType="application/octet-stream")]
private static var BackgroundWorker_ByteClass:Class;
public static function get BackgroundWorker():ByteArray
{
return new BackgroundWorker_ByteClass();
}
On a test script:
public function Main()
{
fBCore.init("secrets", "my-firebase-id");
trace("Init");
//fBCore.auth.addEventListener(FBAuthEvent.LOGIN_SUCCES, hanldeFBSuccess);
fBCore.auth.addEventListener(AuthEvent.LOGIN_SUCCES, hanldeFBSuccess);
fBCore.auth.addEventListener(IOErrorEvent.IO_ERROR, handleIOError);
fBCore.auth.email_login("admin#admin.admin", "password");
}
private function handleIOError(e:IOErrorEvent):void
{
trace("IO error");
trace(e.text); //Nothing here
}
private function hanldeFBSuccess(e:AuthEvent):void
{
trace("Main login success.");
trace(e.message);//Complete success.
}
When triggered by a class via an internal worker channel passed from Main on init:
Primordial:
private function handleLoginClick(e:MouseEvent):void
{
login_mc.buttonMode = false;
login_mc.play();
login_mc.removeEventListener(MouseEvent.CLICK, handleLoginClick);
log("Logging in as " + email_mc.text_txt.text);
commandChannel.send([BGThreadCommand.LOGIN, email_mc.text_txt.text, password_mc.text_txt.text]);
}
Worker:
...
case BGThreadCommand.LOGIN:
log("Logging in with " + message[1] + "::" + message[2]); //Log goes to a progress channel and comes to the main thread reading the outputs successfully.
fbCore.auth.email_login(message[1], message[2]);
fbCore.auth.addEventListener(AuthEvent.LOGIN_SUCCES, loginSuccess); //Nothing
fbCore.auth.addEventListener(IOErrorEvent.IO_ERROR, handleLoginIOError); //Fires
break;
Auth Rest Class: https://github.com/sfxworks/FirebaseREST/blob/master/src/net/sfxworks/firebaseREST/Auth.as
Is this a worker limitation or a security sandbox issue? I have a deep feeling it is the latter of the two. If that's the case how would I load the worker in a way that also gives it the proper permissions to act?
Completely ignored the giveAppPrivelages property in the createWorker function. Sorry Stackoverflow. Sometimes I make bad questions when I get little (or none in this case) sleep the night before.
I want to call a function to delete files and folders from the system in a parallel thread called by QtConcurrent::run() (Qt for Embedded Linux 4.8). Using only a QFuture<bool> with QFuture::waitForFinished() to grab the result (I need to run some code right after the operation), I was able to make the system work.
But I want to display the ongoing result of the operation in a QProgressBar derived class with its setValue(int) called via signals and slots mechanism from inside the delete function, and what I get with the above method is a freezed main thread while the operation isn't finished, and that is inadmissible.
So I though about using QFutureWatcher<bool> and connecting it's finished() signal to another slot containing the remaining code to be run after the delete operation is finished.
The problem I'm facing is that, when I do this, the delete function is simply not run by QtConcurrent::run()! I checked that with printed messages to Terminal. Everything that occurs is that the QFutureWatcher calls its finished() signal without any execution of the delete files function (and that also happens if I use QFutureWatcher::waitForFinished()).
Is this some bug from Qt?
Regarding code, it's pretty exactly as in Qt Assistant: create the QFuture and QFutureWatcher globally, connect the finished() signal with the slot, call QtConcurrent::run(), and setFuture() to the future. Nothing special.
Any help appreciated.
EDIT
Following the request of Kuba, here is the relevant part of the code:
//Declared globally in the .cpp
QFuture<bool> future;
QFutureWatcher<bool> watcher;
//
void SelectRecordToDeleteWidget::slotDeleteRecordStateMachine()
{
switch (deleteRecordStateMachine)
{
case PrepareToDelete:
{
//...
connect(&watcher,SIGNAL(finished()),this,SLOT(slotDeleteRecordStateMachine()),Qt::UniqueConnection);
//...
}
break;
case DeleteRecords:
{
//...
future = QtConcurrent::run(removeFiles, QString(DEFAULT_RECORD_DIR) + "/" + recordList.at(aaa).second.second, poProgressDialog, &itemCounter);
watcher.setFuture(future);
qApp->processEvents();
//...
}
break;
case FinishDelete:
{
//Run code after deleting files
}
break;
default:
break;
}
}
This is all the code using QFuture and QFutureWatcher. The removeFiles is as follows (not forgetting that it works well without QFutureWatcher):
bool removeFiles(const QString dirName, Interface::ProgressDialog* const poProgressDialog, qint32* const itemDeletedCounter)
{
bool result = true;
try
{
QDir dir(dirName);
if (dir.exists())
{
Q_FOREACH (QFileInfo info, dir.entryInfoList(QDir::NoDotAndDotDot | QDir::System | QDir::Hidden | QDir::AllDirs | QDir::Files, QDir::DirsFirst))
{
// if (Q_UNLIKELY(poProgressDialog->wasCanceled()))
// break;
if (info.isDir())
{
result = removeFiles(info.absoluteFilePath(),poProgressDialog,itemDeletedCounter);
if (!result)
return result;
}
else
{
result = QFile::remove(info.absoluteFilePath());
if (!result)
return result;
if (!QMetaObject::invokeMethod(poProgressDialog, "setValue",
Qt::BlockingQueuedConnection,
Q_ARG(qint32, *itemDeletedCounter)))
{
mDebugS(QString("removeFiles: %1QMetaObject::invokeMethod(poProgressDialog, \"setValue\"... failed!"));
}
++(*itemDeletedCounter);
// mDebugS(QString("removeFiles: %1").arg(*itemDeletedCounter));
}
}
result = dir.rmdir(dirName);
}
}
catch (...)
{
const QString strTemp = QString("An error in a call to removeFiles");
mDebugS(strTemp);
mLog(strTemp);
}
return result;
}
I have a client application that receives video stream from a server via UDP or TCP socket.
Originally, when it was written using .NET 2.0 the code was using BeginReceive/EndReceive and IAsyncResult.
The client displays each video in it's own window and also using it's own thread for communicating with the server.
However, since the client is supposed to be up for a long period of time, and there might be 64 video streams simultaneously, there is a "memory leak" of IAsyncResult objects that are allocated each time the data receive callback is called.
This causes the application eventually to run out of memory, because the GC can't handle releasing of the blocks in time. I verified this using VS 2010 Performance Analyzer.
So I modified the code to use SocketAsyncEventArgs and ReceiveFromAsync (UDP case).
However, I still see a growth in memory blocks at:
System.Net.Sockets.Socket.ReceiveFromAsync(class System.Net.Sockets.SocketAsyncEventArgs)
I've read all the samples and posts about implementing the code, and still no solution.
Here's how my code looks like:
// class data members
private byte[] m_Buffer = new byte[UInt16.MaxValue];
private SocketAsyncEventArgs m_ReadEventArgs = null;
private IPEndPoint m_EndPoint; // local endpoint from the caller
Initializing:
m_Socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
m_Socket.Bind(m_EndPoint);
m_Socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, MAX_SOCKET_RECV_BUFFER);
//
// initalize the socket event args structure.
//
m_ReadEventArgs = new SocketAsyncEventArgs();
m_ReadEventArgs.Completed += new EventHandler<SocketAsyncEventArgs>(readEventArgs_Completed);
m_ReadEventArgs.SetBuffer(m_Buffer, 0, m_Buffer.Length);
m_ReadEventArgs.RemoteEndPoint = new IPEndPoint(IPAddress.Any, 0);
m_ReadEventArgs.AcceptSocket = m_Socket;
Starting the read process:
bool waitForEvent = m_Socket.ReceiveFromAsync(m_ReadEventArgs);
if (!waitForEvent)
{
readEventArgs_Completed(this, m_ReadEventArgs);
}
Read completion handler:
private void readEventArgs_Completed(object sender, SocketAsyncEventArgs e)
{
if (e.BytesTransferred == 0 || e.SocketError != SocketError.Success)
{
//
// we got error on the socket or connection was closed
//
Close();
return;
}
try
{
// try to process a new video frame if enough data was read
base.ProcessPacket(m_Buffer, e.Offset, e.BytesTransferred);
}
catch (Exception ex)
{
// log and error
}
bool willRaiseEvent = m_Socket.ReceiveFromAsync(e);
if (!willRaiseEvent)
{
readEventArgs_Completed(this, e);
}
}
Basically the code works fine and I see the video streams perfectly, but this leak is a real pain.
Did I miss anything???
Many thanks!!!
Instead of recursively calling readEventArgs_Completed after !willRaiseEvent use goto to return to the top of the method. I noticed I was slowly chewing up stack space when I had a pattern similar to yours.
I have a BlackBerry App that sends data over a web service when a button has it state set to ON. When the button is ON a timer is started which is running continuously in the background at fixed intervals. The method for HttpConnection is called as follows:
if(C0NNECTION_EXTENSION==null)
{
Dialog.alert("Check internet connection and try again");
return;
}
else
{
confirmation=PostMsgToServer(encryptedMsg);
}
The method PostMsgToServer is as follows:
public static String PostMsgToServer(String encryptedGpsMsg) {
//httpURL= "https://prerel.track24c4i.com/track24prerel/service/spi/post?access_id="+DeviceBoardPassword+"&IMEI="+IMEI+"&hex_data="+encryptedGpsMsg+"&device_type=3";
httpURL= "https://t24.track24c4i.com/track24c4i/service/spi/post?access_id="+DeviceBoardPassword+"&IMEI="+IMEI+"&hex_data="+encryptedGpsMsg+"&device_type=3";
//httpURL= "http://track24.unit1.overwatch/track24/service/spi/post?access_id="+DeviceBoardPassword+"&IMEI="+IMEI+"&hex_data="+encryptedGpsMsg+"&device_type=3";
try {
String C0NNECTION_EXTENSION = checkInternetConnection();
if(C0NNECTION_EXTENSION==null)
{
Dialog.alert("Check internet connection and try again");
return null;
}
else
{
httpURL=httpURL+C0NNECTION_EXTENSION+";ConnectionTimeout=120000";
//Dialog.alert(httpURL);
HttpConnection httpConn;
httpConn = (HttpConnection) Connector.open(httpURL);
httpConn.setRequestMethod(HttpConnection.POST);
DataOutputStream _outStream = new DataOutputStream(httpConn.openDataOutputStream());
byte[] request_body = httpURL.getBytes();
for (int i = 0; i < request_body.length; i++) {
_outStream.writeByte(request_body[i]);
}
DataInputStream _inputStream = new DataInputStream(
httpConn.openInputStream());
StringBuffer _responseMessage = new StringBuffer();
int ch;
while ((ch = _inputStream.read()) != -1) {
_responseMessage.append((char) ch);
}
String res = (_responseMessage.toString());
responce = res.trim();
httpConn.close();
}
}catch (Exception e) {
//Dialog.alert("Connection Time out");
}
return responce;
}
My Question: The App freezes whenever the method is called, i.e. whenever the timer has to execute and send data to the web service the App freezes - at times for a few seconds and at times for a considerable amount of time applying to the user as if the handset has hanged. Can this be solved? Kindly help!!
You are running your networking operation on the Event Thread - i.e. the same Thread that processes your application's Ui interactions. Networking is a blocking operation so effectively this is stopping your UI operation. Doing this on the Event Thread is not recommended and to be honest, I'm surprised it is not causing your application to be terminated, as this is often what the OS will do, if it thinks the application has blocked the Event Thread.
The way to solve this is start your network processing using a separate Thread. This is generally the easy part, the difficult part is
blocking the User from doing anything else while waiting for the
response (assuming you need to do this)
updating the User Interface with the results of your networking
processing
I think the second of these issues are discussed in this Thread:
adding-field-from-a-nonui-thread-throws-exception-in-blackberry
Since it appears you are trying to do this update at regular intervals in the background, I don't think the first is an issue, - for completeness, you can search SO for answers including this one:
blackberry-please-wait-screen-with-time-out
There is more information regarding the Event Thread here:
Event Thread