Desktop duplication (DirectX) screen capture fails to deliver screen updates - visual-c++

I'm working on an application that would capture the screen through Desktop duplication APIs (using DirectX 11) (only the diff to the previous screen update) and render it on another window (The viewer might be running on another machine connected via LAN). The code is an improved version of the sample provided in MSDN. Everything works fine except the device did not give any screen update though there is one some times in the mid, that happens around 10% of the time on some machines (mostly on windows 8/8.1 machines and rarely on windows 10 machines). I tried all the possible ways to sort out this problem. Reduced the number of device resets, that provided me some what reliable output but not always work fine for 100%.
The device fails to provide an initial screen (a full screen) some times (This happens 60% of the time on all windows operating systems where Desktop duplication is supported), I came up with a work around that retried for an initial update from the device until it provides one but that too resulted in multiple issues, the device might not even give the initial screen ever.
I have already invested weeks of my efforts to fix the problem but did not figure out a proper solution and there are no forums I know that discusses these kind of issues. Any help would be appreciated.
Below is my code to get the screen diff to the previous one, init the device, populating the adapters and monitors.
Please bear with me for a very long code snippet, Thanks in advance.
To Get the screen update:
INT getChangedRegions(int timeout, rectangles &dirtyRects, std::vector <MOVE_RECT> &moveRects, UINT &rect_count, RECT ScreenRect)
{
UINT diffArea = 0;
FRAME_DATA currentFrameData;
bool isTimeOut = false;
TRY
{
m_LastErrorCode = m_DuplicationManager.GetFrame(&currentFrameData, timeout, &isTimeOut);
if(SUCCEEDED(m_LastErrorCode) && (!isTimeOut))
{
if(currentFrameData.FrameInfo.TotalMetadataBufferSize)
{
m_CurrentFrameTexture = currentFrameData.Frame;
if(currentFrameData.MoveCount)
{
DXGI_OUTDUPL_MOVE_RECT* moveRectArray = reinterpret_cast<DXGI_OUTDUPL_MOVE_RECT*> (currentFrameData.MetaData);
if (moveRectArray)
{
for(UINT index = 0; index < currentFrameData.MoveCount; index++)
{
//WebRTC
// DirectX capturer API may randomly return unmoved move_rects, which should
// be skipped to avoid unnecessary wasting of differing and encoding
// resources.
// By using testing application it2me_standalone_host_main, this check
// reduces average capture time by 0.375% (4.07 -> 4.055), and average
// encode time by 0.313% (8.042 -> 8.016) without other impacts.
if (moveRectArray[index].SourcePoint.x != moveRectArray[index].DestinationRect.left || moveRectArray[index].SourcePoint.y != moveRectArray[index].DestinationRect.top)
{
if(m_UseD3D11BitmapConversion)
{
MOVE_RECT moveRect;
moveRect.SourcePoint.x = moveRectArray[index].SourcePoint.x * m_ImageScalingFactor;
moveRect.SourcePoint.y = moveRectArray[index].SourcePoint.y * m_ImageScalingFactor;
moveRect.DestinationRect.left = moveRectArray[index].DestinationRect.left * m_ImageScalingFactor;
moveRect.DestinationRect.top = moveRectArray[index].DestinationRect.top * m_ImageScalingFactor;
moveRect.DestinationRect.bottom = moveRectArray[index].DestinationRect.bottom * m_ImageScalingFactor;
moveRect.DestinationRect.right = moveRectArray[index].DestinationRect.right * m_ImageScalingFactor;
moveRects.push_back(moveRect);
diffArea += abs((moveRect.DestinationRect.right - moveRect.DestinationRect.left) *
(moveRect.DestinationRect.bottom - moveRect.DestinationRect.top));
}
else
{
moveRects.push_back(moveRectArray[index]);
diffArea += abs((moveRectArray[index].DestinationRect.right - moveRectArray[index].DestinationRect.left) *
(moveRectArray[index].DestinationRect.bottom - moveRectArray[index].DestinationRect.top));
}
}
}
}
else
{
return -1;
}
}
if(currentFrameData.DirtyCount)
{
RECT* dirtyRectArray = reinterpret_cast<RECT*> (currentFrameData.MetaData + (currentFrameData.MoveCount * sizeof(DXGI_OUTDUPL_MOVE_RECT)));
if (!dirtyRectArray)
{
return -1;
}
rect_count = currentFrameData.DirtyCount;
for(UINT index = 0; index < rect_count; index ++)
{
if(m_UseD3D11BitmapConversion)
{
RECT dirtyRect;
dirtyRect.bottom = dirtyRectArray[index].bottom * m_ImageScalingFactor;
dirtyRect.top = dirtyRectArray[index].top * m_ImageScalingFactor;
dirtyRect.left = dirtyRectArray[index].left * m_ImageScalingFactor;
dirtyRect.right = dirtyRectArray[index].right * m_ImageScalingFactor;
diffArea += abs((dirtyRect.right - dirtyRect.left) *
(dirtyRect.bottom - dirtyRect.top));
dirtyRects.push_back(dirtyRect);
}
else
{
diffArea += abs((dirtyRectArray[index].right - dirtyRectArray[index].left) *
(dirtyRectArray[index].bottom - dirtyRectArray[index].top));
dirtyRects.push_back(dirtyRectArray[index]);
}
}
}
}
return diffArea;
}
CATCH_ALL(e)
{
LOG(CRITICAL) << _T("Exception in getChangedRegions");
}
END_CATCH_ALL
return -1;
}
Here is the code to init the device
//
// Initialize duplication interfaces
//
HRESULT cDuplicationManager::InitDupl(_In_ ID3D11Device* Device, _In_ IDXGIAdapter *_pAdapter, _In_ IDXGIOutput *_pOutput, _In_ UINT Output)
{
HRESULT hr = E_FAIL;
if(!_pOutput || !_pAdapter || !Device)
{
return hr;
}
m_OutputNumber = Output;
// Take a reference on the device
m_Device = Device;
m_Device->AddRef();
/*
// Get DXGI device
IDXGIDevice* DxgiDevice = nullptr;
HRESULT hr = m_Device->QueryInterface(__uuidof(IDXGIDevice), reinterpret_cast<void**>(&DxgiDevice));
if (FAILED(hr))
{
return ProcessFailure(nullptr, _T("Failed to QI for DXGI Device"), _T("Error"), hr);
}
// Get DXGI adapter
IDXGIAdapter* DxgiAdapter = nullptr;
hr = DxgiDevice->GetParent(__uuidof(IDXGIAdapter), reinterpret_cast<void**>(&DxgiAdapter));
DxgiDevice->Release();
DxgiDevice = nullptr;
if (FAILED(hr))
{
return ProcessFailure(m_Device, _T("Failed to get parent DXGI Adapter"), _T("Error"), hr);//, SystemTransitionsExpectedErrors);
}
// Get output
IDXGIOutput* DxgiOutput = nullptr;
hr = DxgiAdapter->EnumOutputs(Output, &DxgiOutput);
DxgiAdapter->Release();
DxgiAdapter = nullptr;
if (FAILED(hr))
{
return ProcessFailure(m_Device, _T("Failed to get specified output in DUPLICATIONMANAGER"), _T("Error"), hr);//, EnumOutputsExpectedErrors);
}
DxgiOutput->GetDesc(&m_OutputDesc);
IDXGIOutput1* DxgiOutput1 = nullptr;
hr = DxgiOutput->QueryInterface(__uuidof(DxgiOutput1), reinterpret_cast<void**>(&DxgiOutput1));
*/
_pOutput->GetDesc(&m_OutputDesc);
// QI for Output 1
IDXGIOutput1* DxgiOutput1 = nullptr;
hr = _pOutput->QueryInterface(__uuidof(DxgiOutput1), reinterpret_cast<void**>(&DxgiOutput1));
if (FAILED(hr))
{
return ProcessFailure(nullptr, _T("Failed to QI for DxgiOutput1 in DUPLICATIONMANAGER"), _T("Error"), hr);
}
// Create desktop duplication
hr = DxgiOutput1->DuplicateOutput(m_Device, &m_DeskDupl);
DxgiOutput1->Release();
DxgiOutput1 = nullptr;
if (FAILED(hr) || !m_DeskDupl)
{
if (hr == DXGI_ERROR_NOT_CURRENTLY_AVAILABLE)
{
return ProcessFailure(nullptr, _T("Maximum number of applications using Desktop Duplication API"), _T("Error"), hr);
}
return ProcessFailure(m_Device, _T("Failed to get duplicate output in DUPLICATIONMANAGER"), _T("Error"), hr);//, CreateDuplicationExpectedErrors);
}
return S_OK;
}
Finally to get the current frame and difference to the previous one:
//
// Get next frame and write it into Data
//
_Success_(*Timeout == false && return == DUPL_RETURN_SUCCESS)
HRESULT cDuplicationManager::GetFrame(_Out_ FRAME_DATA* Data, int timeout, _Out_ bool* Timeout)
{
IDXGIResource* DesktopResource = nullptr;
DXGI_OUTDUPL_FRAME_INFO FrameInfo;
try
{
// Get new frame
HRESULT hr = m_DeskDupl->AcquireNextFrame(timeout, &FrameInfo, &DesktopResource);
if (hr == DXGI_ERROR_WAIT_TIMEOUT)
{
*Timeout = true;
return S_OK;
}
*Timeout = false;
if (FAILED(hr))
{
return ProcessFailure(m_Device, _T("Failed to acquire next frame in DUPLICATIONMANAGER"), _T("Error"), hr);//, FrameInfoExpectedErrors);
}
// If still holding old frame, destroy it
if (m_AcquiredDesktopImage)
{
m_AcquiredDesktopImage->Release();
m_AcquiredDesktopImage = nullptr;
}
if (DesktopResource)
{
// QI for IDXGIResource
hr = DesktopResource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void **>(&m_AcquiredDesktopImage));
DesktopResource->Release();
DesktopResource = nullptr;
}
if (FAILED(hr))
{
return ProcessFailure(nullptr, _T("Failed to QI for ID3D11Texture2D from acquired IDXGIResource in DUPLICATIONMANAGER"), _T("Error"), hr);
}
// Get metadata
if (FrameInfo.TotalMetadataBufferSize)
{
// Old buffer too small
if (FrameInfo.TotalMetadataBufferSize > m_MetaDataSize)
{
if (m_MetaDataBuffer)
{
delete [] m_MetaDataBuffer;
m_MetaDataBuffer = nullptr;
}
m_MetaDataBuffer = new (std::nothrow) BYTE[FrameInfo.TotalMetadataBufferSize];
if (!m_MetaDataBuffer)
{
m_MetaDataSize = 0;
Data->MoveCount = 0;
Data->DirtyCount = 0;
return ProcessFailure(nullptr, _T("Failed to allocate memory for metadata in DUPLICATIONMANAGER"), _T("Error"), E_OUTOFMEMORY);
}
m_MetaDataSize = FrameInfo.TotalMetadataBufferSize;
}
UINT BufSize = FrameInfo.TotalMetadataBufferSize;
// Get move rectangles
hr = m_DeskDupl->GetFrameMoveRects(BufSize, reinterpret_cast<DXGI_OUTDUPL_MOVE_RECT*>(m_MetaDataBuffer), &BufSize);
if (FAILED(hr))
{
Data->MoveCount = 0;
Data->DirtyCount = 0;
return ProcessFailure(nullptr, L"Failed to get frame move rects in DUPLICATIONMANAGER", L"Error", hr);//, FrameInfoExpectedErrors);
}
Data->MoveCount = BufSize / sizeof(DXGI_OUTDUPL_MOVE_RECT);
BYTE* DirtyRects = m_MetaDataBuffer + BufSize;
BufSize = FrameInfo.TotalMetadataBufferSize - BufSize;
// Get dirty rectangles
hr = m_DeskDupl->GetFrameDirtyRects(BufSize, reinterpret_cast<RECT*>(DirtyRects), &BufSize);
if (FAILED(hr))
{
Data->MoveCount = 0;
Data->DirtyCount = 0;
return ProcessFailure(nullptr, _T("Failed to get frame dirty rects in DUPLICATIONMANAGER"), _T("Error"), hr);//, FrameInfoExpectedErrors);
}
Data->DirtyCount = BufSize / sizeof(RECT);
Data->MetaData = m_MetaDataBuffer;
}
Data->Frame = m_AcquiredDesktopImage;
Data->FrameInfo = FrameInfo;
}
catch (...)
{
return S_FALSE;
}
return S_OK;
}
Update :
Failed to acquire next frame in DUPLICATIONMANAGER is getting printed whenever the device has hung (That is in the mid of streaming the screens, Ex: Continuously capturing a video and sending it to the other end)
// Get new frame
HRESULT hr = m_DeskDupl->AcquireNextFrame(timeout, &FrameInfo, &DesktopResource);
if (hr == DXGI_ERROR_WAIT_TIMEOUT)
{
*Timeout = true;
return S_OK;
}
*Timeout = false;
if (FAILED(hr))
{
return ProcessFailure(m_Device, _T("Failed to acquire next frame in DUPLICATIONMANAGER"), _T("Error"), hr);//, FrameInfoExpectedErrors);
}
here is the detailed error info :
Id3d11DuplicationManager::ProcessFailure - Error: Failed to acquire next frame in DUPLICATIONMANAGER, Detail: The keyed mutex was abandoned.
Update 2 :
I have got the error code whenever the device failed to give screen updates forever, And here is the same
Id3d11DuplicationManager::ProcessFailure - Error: Failed to get duplicate output in DUPLICATIONMANAGER, Detail: Access is denied.
The error code is E_ACCESSDENIED.
I do not understand why I am getting this error as I am running in SYSTEM mode already and the SetThreadDesktop had been executed twice (One during the init and another after detecting a failure)
This is what the explanation of the error on MSDN : E_ACCESSDENIED if the application does not have access privilege to the current desktop image. For example, only an application that runs at LOCAL_SYSTEM can access the secure desktop.
Is there anything else that would result in this kind of issue?

It's always good to check the return codes and immediately fall back to GDI or any other available screen capturing approach in case of non-recoverable errors. Retrying doesn't work most of the time for certain hardware errors like max limit reached, out of memory, device removed, etc, I learned it in a hard way. Furthermore, DirectX device takes a few iterations before producing an initial frame on rare occasions. It wouldn't be useful to retry more than 10 times, you can safely fallback or try re-initializing the device to check one more time before falling back.
Here are some basic checks to do:
Handle DXGI_ERROR_NOT_CURRENTLY_AVAILABLE error:
_pOutput->GetDesc(&m_OutputDesc);
// QI for Output 1
IDXGIOutput1* DxgiOutput1 = nullptr;
hr = _pOutput->QueryInterface(__uuidof(DxgiOutput1), reinterpret_cast<void**>(&DxgiOutput1));
if (FAILED(hr))
{
return ProcessFailure(nullptr, _T("Failed to QI for DxgiOutput1 in DUPLICATIONMANAGER"), _T("Error"), hr);
}
// Create desktop duplication
hr = DxgiOutput1->DuplicateOutput(m_Device, &m_DeskDupl);
DxgiOutput1->Release();
DxgiOutput1 = nullptr;
if (FAILED(hr) || !m_DeskDupl)
{
if (hr == DXGI_ERROR_NOT_CURRENTLY_AVAILABLE)
{
return ProcessFailure(nullptr, _T("Maximum number of applications using Desktop Duplication API"), _T("Error"), hr);
}
return ProcessFailure(m_Device, _T("Failed to get duplicate output in DUPLICATIONMANAGER"), _T("Error"), hr);//, CreateDuplicationExpectedErrors);
}
Check for device removed(DXGI_ERROR_DEVICE_REMOVED) or Device reset(DXGI_ERROR_DEVICE_RESET) & Out of memory(E_OUTOFMEMORY) error codes (I have received E_OUTOFMEMORY sometimes, though it's uncommon):
HRESULT ProcessFailure(_In_opt_ ID3D11Device* Device, _In_ LPCWSTR Str, _In_ LPCWSTR Title, HRESULT hr)//, _In_opt_z_ HRESULT* ExpectedErrors = NULL)
{
HRESULT TranslatedHr;
// On an error check if the DX device is lost
if (Device)
{
HRESULT DeviceRemovedReason = Device->GetDeviceRemovedReason();
switch (DeviceRemovedReason)
{
case DXGI_ERROR_DEVICE_REMOVED:
case DXGI_ERROR_DEVICE_RESET:
case static_cast<HRESULT>(E_OUTOFMEMORY) :
{
// Our device has been stopped due to an external event on the GPU so map them all to
// device removed and continue processing the condition
TranslatedHr = DXGI_ERROR_DEVICE_REMOVED;
break;
}
case S_OK:
{
// Device is not removed so use original error
TranslatedHr = hr;
break;
}
default:
{
// Device is removed but not a error we want to remap
TranslatedHr = DeviceRemovedReason;
}
}
}
else
{
TranslatedHr = hr;
}
_com_error err(TranslatedHr);
LPCTSTR errMsg = err.ErrorMessage();
return TranslatedHr;
}
Furthermore, Desktop duplication requires a real graphics device to be active in order to work. You may get E_ACCESSDENIED otherwise.
There are also other scenarios you may get this error, like, Desktop switch cases, abandoned keyed mutex. You can try reinitializing the device in such cases.
I have also uploaded my sample project here.

Related

Member Variables in Class Get Blown Away When Using std::thread

I have defined a base class using std::thread. For the child class, I perform some initialization of member variables and then start the thread using m_thread.reset(new std::thread(&MyClass::ThreadMain, this)); where m_thread is a member of MyClass. The purpose of the class is to read data from a serial port and report to a parent. The posix message queue handle of the parent is passed to MyClass during initialization before the thread is created. On running I get exceptions and I see that member variables that were initialized before the thread started appear to be no longer valid using the watch in GDB.
It appears as if the first message on the serial port is received and passed validation in order to get to the SendToParent call. At this call, it appears that I lose the stack. I tried running cppcheck to see if I have any memory leaks or buffer overflows and found nothing.
void MyClass::ThreadMain(void)
{
ssize_t bytesRead = 0;
UINT8 buffer[256];
UINT8 message[256];
BOOL partialMessage = FALSE;
UINT8 messageIndex = 0;
UINT8 payloadLength = 0;
// read data from the UART
while(1)
{
// the UART is setup to pend until data is available
bytesRead = read(m_radioFileDescriptor, buffer, sizeof(buffer));
if (FAIL == bytesRead)
{
LOG_SYSTEM_INFO("UART Read interrupted by a system call");
}
else if (bytesRead > 0)
{
// build the message
for(ssize_t i = 0 ; i < bytesRead ; i++)
{
if (FALSE == partialMessage)
{
// have we found the start of the message?
if(START_BYTE == buffer[i])
{
// start of new message
messageIndex = 0;
message[messageIndex] = buffer[i];
partialMessage = TRUE;
messageIndex++;
}
}
else
{
// keep building the message until the expected length is reached
if(LENGTH_POSITION == messageIndex)
{
// capture the expected message length
message[messageIndex] = buffer[i];
messageIndex++;
payloadLength = buffer[i];
}
else
{
message[messageIndex] = buffer[i];
messageIndex++;
// check for expected length and end byte
if((messageIndex == payloadLength) && (END_BYTE == buffer[i]))
{
// this should be a valid message but need to confirm by checking for a valid checksum
UINT8 messageChecksum = message[messageIndex - CHKSUM_POS_FROM_END];
UINT8 calculatedChecksum = RadioProtocol::Instance().GenerateRadioChecksum(message, (payloadLength - CHKSUM_POS_FROM_END));
if (messageChecksum == calculatedChecksum)
{
SendToParent(message, payloadLength);
}
else
{
LOG_SYSTEM_ERROR("Checksum FAILURE");
}
// reset for the next message
partialMessage = FALSE;
messageIndex = 0;
}
else if((messageIndex == payloadLength) && (END_BYTE != buffer[i]))
{
// malformed message - throw out and look for start of next message
LOG_SYSTEM_ERROR("Bytes read exceeded expected message length");
partialMessage = FALSE;
messageIndex = 0;
}
}
}
} // end for loop of bytes read on the port
}
else
{
LOG_SYSTEM_INFO("Read returned 0 bytes which is unexpected");
}
}
}
void MyClass::SendToParent(UINT8* pMsg, UINT8 size)
{
if ((pMsg != NULL) && (m_parentQueueHandle > 0))
{
// message is valid - pass up for processing
MsgQueueMessage msgToSend;
msgToSend.m_msgHeader = UART_MESSASGE;
bzero(msgToSend.m_msgData, sizeof(msgToSend.m_msgData));
for (UINT8 i = 0; i < size; i++)
{
msgToSend.m_msgData[i] = pMsg[i];
}
if (FAIL == msgsnd(m_parentQueueHandle, &msgToSend, sizeof(msgToSend), IPC_NOWAIT))
{
LOG_SYSTEM_ERROR("FAILED to send message on queue");
}
}
}
This acts like I am performing a buffer overflow but I just can't see it. When I set a breakpoint at the line UINT8 messageChecksum = message[messageIndex - CHKSUM_POS_FROM_END]; all data in the watch window appear valid. If I step over to the next line then the data, m_parentQueueHandle as an example, gets blown away.
This is my first time working with c++11 threads and particularly with c++. Any help or insights would be appreciated.
I think I found the issue. I added a bunch of printfs and found that the destructor for the class was being called. Much further upstreamI had the parent object being created as a local variable and it was going out of scope. This caused the child to go out of scope but the threads were still running. I certainly need to clean up the threads in the destructor.

CView::OnFilePrint crashing MFC application

We have an MFC application that has been used and maintained for many years. Recently we made some administrative changes to some computers that are running the application. Now the software occasionally crashes when printing from the application.
We are using pretty standard MFC code to initiate the printing. We added try/catch blocks around what we felt like are the pertinent areas of the code with no luck. Whatever is failing does not seem to throw.
We get the typical dialog stating that "____ MFC Application has stopped working". Closing the program is the only option.
The windows event logger shows that our application is the Faulting application.
The exception code is 0xc0000005, which appears to be an Access Denied error.
The application is in the CView::OnFilePrint() code when the crash occurs.
We have added some logging, and we know that we get through DoPreparePrinting, and OnBeginPrinting.
We believe that CDC::StartDoc would be the next thing called, then CView::OnPrepareDC. We don't get to OnPrepareDC when we fail.
We don't seem to find the source code for CView::OnFilePrint, so we are not sure what it looks like. From research online, we think that things happen in this order in OnFilePrint:
// what we think is in OnFilePrint:
CView::OnFilePrint()
{
OnPreparePrinting(); <- we get through our override of this
OnBeginPrinting(); <- we get through our override of this
// loop back to here on multiple docs
CDC::StartDoc();
CView::OnPrepareDC(); <- we do not reach our override of this
CView::OnPaint();
CDC::EndPage();
// loop back on multiple docs
...
// finish if last doc...
}
I would like to have the source for it so we could attempt to rewrite it and try to gracefully fail instead of failing by crashing.
I'm looking for:
1) any suggestions as to how to figure out why the process of printing causes our application to crash.
2) A location for where the CView::OnFilePrint code is located, if available.
(the only idea I have left to narrow down the problem is to call our own version of this so that we can step through it and add logging and/or see if we can at least fail gracefully when it the problem occurs.)
The printer is Xerox Phaser 3610, for what its worth.
source code for CView::OnFilePrint should be in C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\atlmfc\src\mfc\viewprnt.cpp, depending on VS version. There could also be a problem with printer initialization/access.
If there is any error it is most likely due to printer initialization. You can override OnFilePrint and add CPrintInfo printInfo for testing. Example:
//ON_COMMAND(ID_FILE_PRINT, &CView::OnFilePrint)
//ON_COMMAND(ID_FILE_PRINT_DIRECT, &CView::OnFilePrint)
ON_COMMAND(ID_FILE_PRINT, OnFilePrint)
ON_COMMAND(ID_FILE_PRINT_DIRECT, OnFilePrint)
void CMyView::OnFilePrint()
{
try
{
CPrintInfo printInfo;
}
catch(...)
{
//log error
AfxMessageBox(L"error");
}
CView::OnFilePrint();
}
As noted in comments, another possibility is that there is a bug somewhere else in the code, which may not necessarily be related to printing.
Inside of CView::OnFilePrint, this occurs:
CWnd * hwndTemp = AfxGetMainWnd();
It turns out that if you don't call OnFilePrint from the main thread, this returns NULL.
Due to slight timing changes when the computers were logged onto a domain, OnFilePrint was being called from another thread. This causes the above call to return null, then when this line gets executed:
hwndTemp->EnableWindow(FALSE);
The application crashes.
There are several ways to fix this. One is to use this:
CWnd * hwndTemp = AfxGetApp()->GetMainWnd();
In place of this:
CWnd * hwndTemp = AfxGetMainWnd();
Another way is to assure that OnFilePrint is only called from the main thread.
A cut to the chase version of the code in CView::OnFilePrint is here:
// disable main window while printing & init printing status dialog
// Store the Handle of the Window in a temp so that it can be enabled
// once the printing is finished
CWnd * hwndTemp = AfxGetMainWnd(); // <--- CAN RETURN NULL HERE
hwndTemp->EnableWindow(FALSE); // <--- CRASH WILL OCCUR HERE
CPrintingDialog dlgPrintStatus(this);
Full version of CView::OnFilePrint is below.
The OnFilePrint code, with the problem area noted:
void CView::OnFilePrint()
{
// get default print info
CPrintInfo printInfo;
ASSERT(printInfo.m_pPD != NULL); // must be set
if (LOWORD(GetCurrentMessage()->wParam) == ID_FILE_PRINT_DIRECT)
{
CCommandLineInfo* pCmdInfo = AfxGetApp()->m_pCmdInfo;
if (pCmdInfo != NULL)
{
if (pCmdInfo->m_nShellCommand == CCommandLineInfo::FilePrintTo)
{
printInfo.m_pPD->m_pd.hDC = ::CreateDC(pCmdInfo->m_strDriverName,
pCmdInfo->m_strPrinterName, pCmdInfo->m_strPortName, NULL);
if (printInfo.m_pPD->m_pd.hDC == NULL)
{
AfxMessageBox(AFX_IDP_FAILED_TO_START_PRINT);
return;
}
}
}
printInfo.m_bDirect = TRUE;
}
if (OnPreparePrinting(&printInfo))
{
// hDC must be set (did you remember to call DoPreparePrinting?)
ASSERT(printInfo.m_pPD->m_pd.hDC != NULL);
// gather file to print to if print-to-file selected
CString strOutput;
if (printInfo.m_pPD->m_pd.Flags & PD_PRINTTOFILE && !printInfo.m_bDocObject)
{
// construct CFileDialog for browsing
CString strDef(MAKEINTRESOURCE(AFX_IDS_PRINTDEFAULTEXT));
CString strPrintDef(MAKEINTRESOURCE(AFX_IDS_PRINTDEFAULT));
CString strFilter(MAKEINTRESOURCE(AFX_IDS_PRINTFILTER));
CString strCaption(MAKEINTRESOURCE(AFX_IDS_PRINTCAPTION));
CFileDialog dlg(FALSE, strDef, strPrintDef,
OFN_HIDEREADONLY|OFN_OVERWRITEPROMPT, strFilter, NULL, 0);
dlg.m_ofn.lpstrTitle = strCaption;
if (dlg.DoModal() != IDOK)
return;
// set output device to resulting path name
strOutput = dlg.GetPathName();
}
// set up document info and start the document printing process
CString strTitle;
CDocument* pDoc = GetDocument();
if (pDoc != NULL)
strTitle = pDoc->GetTitle();
else
EnsureParentFrame()->GetWindowText(strTitle);
DOCINFO docInfo;
memset(&docInfo, 0, sizeof(DOCINFO));
docInfo.cbSize = sizeof(DOCINFO);
docInfo.lpszDocName = strTitle;
CString strPortName;
if (strOutput.IsEmpty())
{
docInfo.lpszOutput = NULL;
strPortName = printInfo.m_pPD->GetPortName();
}
else
{
docInfo.lpszOutput = strOutput;
AfxGetFileTitle(strOutput,
strPortName.GetBuffer(_MAX_PATH), _MAX_PATH);
}
// setup the printing DC
CDC dcPrint;
if (!printInfo.m_bDocObject)
{
dcPrint.Attach(printInfo.m_pPD->m_pd.hDC); // attach printer dc
dcPrint.m_bPrinting = TRUE;
}
OnBeginPrinting(&dcPrint, &printInfo);
if (!printInfo.m_bDocObject)
dcPrint.SetAbortProc(_AfxAbortProc);
/**********************************************************************
Problem area.
If the calling thread is not the main thread, the call to AfxGetMainWnd
can return NULL. In this case, hwndTemp->EnableWindow(FALSE) will crash
the application.
**********************************************************************/
// disable main window while printing & init printing status dialog
// Store the Handle of the Window in a temp so that it can be enabled
// once the printing is finished
CWnd * hwndTemp = AfxGetMainWnd(); // <--- CAN RETURN NULL HERE
hwndTemp->EnableWindow(FALSE); // <--- CRASH WILL OCCUR HERE
CPrintingDialog dlgPrintStatus(this);
CString strTemp;
dlgPrintStatus.SetDlgItemText(AFX_IDC_PRINT_DOCNAME, strTitle);
dlgPrintStatus.SetDlgItemText(AFX_IDC_PRINT_PRINTERNAME,
printInfo.m_pPD->GetDeviceName());
dlgPrintStatus.SetDlgItemText(AFX_IDC_PRINT_PORTNAME, strPortName);
dlgPrintStatus.ShowWindow(SW_SHOW);
dlgPrintStatus.UpdateWindow();
// start document printing process
if (!printInfo.m_bDocObject)
{
printInfo.m_nJobNumber = dcPrint.StartDoc(&docInfo);
if (printInfo.m_nJobNumber == SP_ERROR)
{
// enable main window before proceeding
hwndTemp->EnableWindow(TRUE);
// cleanup and show error message
OnEndPrinting(&dcPrint, &printInfo);
dlgPrintStatus.DestroyWindow();
dcPrint.Detach(); // will be cleaned up by CPrintInfo destructor
AfxMessageBox(AFX_IDP_FAILED_TO_START_PRINT);
return;
}
}
// Guarantee values are in the valid range
UINT nEndPage = printInfo.GetToPage();
UINT nStartPage = printInfo.GetFromPage();
if (nEndPage < printInfo.GetMinPage())
nEndPage = printInfo.GetMinPage();
if (nEndPage > printInfo.GetMaxPage())
nEndPage = printInfo.GetMaxPage();
if (nStartPage < printInfo.GetMinPage())
nStartPage = printInfo.GetMinPage();
if (nStartPage > printInfo.GetMaxPage())
nStartPage = printInfo.GetMaxPage();
int nStep = (nEndPage >= nStartPage) ? 1 : -1;
nEndPage = (nEndPage == 0xffff) ? 0xffff : nEndPage + nStep;
VERIFY(strTemp.LoadString(AFX_IDS_PRINTPAGENUM));
// If it's a doc object, we don't loop page-by-page
// because doc objects don't support that kind of levity.
BOOL bError = FALSE;
if (printInfo.m_bDocObject)
{
OnPrepareDC(&dcPrint, &printInfo);
OnPrint(&dcPrint, &printInfo);
}
else
{
// begin page printing loop
for (printInfo.m_nCurPage = nStartPage;
printInfo.m_nCurPage != nEndPage; printInfo.m_nCurPage += nStep)
{
OnPrepareDC(&dcPrint, &printInfo);
// check for end of print
if (!printInfo.m_bContinuePrinting)
break;
// write current page
TCHAR szBuf[80];
ATL_CRT_ERRORCHECK_SPRINTF(_sntprintf_s(szBuf, _countof(szBuf), _countof(szBuf) - 1, strTemp, printInfo.m_nCurPage));
dlgPrintStatus.SetDlgItemText(AFX_IDC_PRINT_PAGENUM, szBuf);
// set up drawing rect to entire page (in logical coordinates)
printInfo.m_rectDraw.SetRect(0, 0,
dcPrint.GetDeviceCaps(HORZRES),
dcPrint.GetDeviceCaps(VERTRES));
dcPrint.DPtoLP(&printInfo.m_rectDraw);
// attempt to start the current page
if (dcPrint.StartPage() < 0)
{
bError = TRUE;
break;
}
// must call OnPrepareDC on newer versions of Windows because
// StartPage now resets the device attributes.
OnPrepareDC(&dcPrint, &printInfo);
ASSERT(printInfo.m_bContinuePrinting);
// page successfully started, so now render the page
OnPrint(&dcPrint, &printInfo);
if ((nStep > 0) && // pages are printed in ascending order
(nEndPage > printInfo.GetMaxPage() + nStep)) // out off pages
{
// OnPrint may have set the last page
// because the end of the document was reached.
// The loop must not continue with the next iteration.
nEndPage = printInfo.GetMaxPage() + nStep;
}
// If the user restarts the job when it's spooling, all
// subsequent calls to EndPage returns < 0. The first time
// GetLastError returns ERROR_PRINT_CANCELLED
if (dcPrint.EndPage() < 0 && (GetLastError()!= ERROR_SUCCESS))
{
HANDLE hPrinter;
if (!OpenPrinter(LPTSTR(printInfo.m_pPD->GetDeviceName().GetBuffer()), &hPrinter, NULL))
{
bError = TRUE;
break;
}
DWORD cBytesNeeded;
if(!GetJob(hPrinter,printInfo.m_nJobNumber,1,NULL,0,&cBytesNeeded))
{
if (GetLastError() != ERROR_INSUFFICIENT_BUFFER)
{
bError = TRUE;
break;
}
}
JOB_INFO_1 *pJobInfo;
if((pJobInfo = (JOB_INFO_1 *)malloc(cBytesNeeded))== NULL)
{
bError = TRUE;
break;
}
DWORD cBytesUsed;
BOOL bRet = GetJob(hPrinter,printInfo.m_nJobNumber,1,LPBYTE(pJobInfo),cBytesNeeded,&cBytesUsed);
DWORD dwJobStatus = pJobInfo->Status;
free(pJobInfo);
pJobInfo = NULL;
// if job status is restart, just continue
if(!bRet || !(dwJobStatus & JOB_STATUS_RESTART) )
{
bError = TRUE;
break;
}
}
if(!_AfxAbortProc(dcPrint.m_hDC, 0))
{
bError = TRUE;
break;
}
}
}
// cleanup document printing process
if (!printInfo.m_bDocObject)
{
if (!bError)
dcPrint.EndDoc();
else
dcPrint.AbortDoc();
}
hwndTemp->EnableWindow(); // enable main window
OnEndPrinting(&dcPrint, &printInfo); // clean up after printing
dlgPrintStatus.DestroyWindow();
dcPrint.Detach(); // will be cleaned up by CPrintInfo destructor
}
}

No sound output with WASAPI

I am having trouble with WASAPI. It do not output any sound and I have been checked the data that writing to the buffer.
Because of it does not output any sound, I haven't any idea to find out the problem.
It may have some problems in following code.
SoundStream::SoundStream() : writtenCursor(0), writeCursor(0), distroy(false)
{
IMMDeviceEnumerator * pEnumerator = nullptr;
HResult(CoCreateInstance(__uuidof(MMDeviceEnumerator), NULL, CLSCTX_ALL, IID_PPV_ARGS(&pEnumerator)));
IMMDevice * pDevice = nullptr;
HResult(pEnumerator->GetDefaultAudioEndpoint(eRender, eMultimedia, &pDevice));
SafeRelease(&pEnumerator);
HResult(pDevice->Activate(__uuidof(IAudioClient), CLSCTX_ALL, NULL, (void**)&pAudioClient));
SafeRelease(&pDevice);
WAVEFORMATEXTENSIBLE * pwfx = nullptr;
hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
REFERENCE_TIME hnsRequestedDuration = REFTIMES_PER_SEC * 2;
HResult(pAudioClient->GetMixFormat((WAVEFORMATEX**)&pwfx));
HResult(pAudioClient->Initialize(
AUDCLNT_SHAREMODE_SHARED,
AUDCLNT_STREAMFLAGS_EVENTCALLBACK,
hnsRequestedDuration,
0,
(WAVEFORMATEX*)pwfx,
NULL));
pAudioClient->SetEventHandle(hEvent);
channel = (size_t)pwfx->Format.nChannels;
bits = (size_t)pwfx->Format.wBitsPerSample;
validBits = (size_t)pwfx->Samples.wValidBitsPerSample;
frequency = (size_t)pwfx->Format.nSamplesPerSec;
buffer.reshape({ 0, channel, bits >> 3 });
CoTaskMemFree(pwfx);
HResult(pAudioClient->GetBufferSize(&bufferFrameCount));
HResult(pAudioClient->Start());
if (pAudioClient)
{
thread = std::thread([&]()
{
this->Sync();
});
}
}
You could look at my WASAPI.cpp code at http://jdmcox.com (which works fine).
You should also check if the expected wave format is float:
//SubFormat 00000003-0000-0010-8000-00aa00389b71 defines KSDATAFORMAT_SUBTYPE_IEEE_FLOAT
//SubFormat 00000001-0000-0010-8000-00aa00389b71 defines KSDATAFORMAT_SUBTYPE_PCM
GUID G;
WORD V;
WAVEFORMATEX *pwfx = NULL;
bool itsfloat;
pAudioClient->GetMixFormat(&pwfx);
// Do we received a WAVEFORMATEXTENSIBLE?
if(pwfx.cbSize >= 22) {
G = ((WAVEFORMATEXTENSIBLE*)pwfx)->SubFormat;
V = ((WAVEFORMATEXTENSIBLE*)pwfx)->Samples.wValidBitsPerSample;
if (G.Data1 == 3) itsfloat = true;
else if (G.Data1 == 1) itsfloat = false;
}
You know you received a WAVEFORMATEXTENSIBLE and not a simple WAVEFORMATEX because the "pwfx.cbSize >= 22".
See more at:
IAudioClient::GetMixFormat
https://learn.microsoft.com/en-us/windows/win32/api/audioclient/nf-audioclient-iaudioclient-getmixformat
WAVEFORMATEXTENSIBLE
https://learn.microsoft.com/en-us/windows/win32/api/mmreg/ns-mmreg-waveformatextensible
You could look at my WASAPI.cpp code at http://jdmcox.com AGAIN.
Now it works in shared mode as well as exclusive mode.
I should note that no conversion of wave format or wave is necessary in shared mode -- Windows takes care of both converting to and from their format used to mix waves.

Strange code behavior adding OpenCV to VC++ Kinect SDK samples

I'm new to C++ and trying to add OpenCV into Microsoft's Kinect samples. I was able to do it for the ColorBasics-D2D sample by modifying this function
void CColorBasics::ProcessColor()
{
HRESULT hr;
NUI_IMAGE_FRAME imageFrame;
// Attempt to get the color frame
hr = m_pNuiSensor->NuiImageStreamGetNextFrame(m_pColorStreamHandle, 0, &imageFrame);
if (FAILED(hr))
{
return;
}
INuiFrameTexture * pTexture = imageFrame.pFrameTexture;
NUI_LOCKED_RECT LockedRect;
// Lock the frame data so the Kinect knows not to modify it while we're reading it
pTexture->LockRect(0, &LockedRect, NULL, 0);
// Make sure we've received valid data
if (LockedRect.Pitch != 0)
{
BYTE * pBuffer = (BYTE*) LockedRect.pBits;
cvSetData(img,(BYTE*) pBuffer, img->widthStep);
Mat &m = Mat(img);
Mat &hsv = Mat();
vector<Mat> mv = vector<Mat>(3,Mat(cvSize(640,480),CV_8UC1));
cvtColor(m,hsv,CV_BGR2HSV);
cvtColor(hsv,m,CV_HSV2BGR);//*/
IplImage iplimg(m);
cvNamedWindow("rgb",1);
cvShowImage("rgb",&iplimg);
// Draw the data with Direct2D
m_pDrawColor->Draw(static_cast<BYTE *>(LockedRect.pBits), LockedRect.size);
// If the user pressed the screenshot button, save a screenshot
if (m_bSaveScreenshot)
{
WCHAR statusMessage[cStatusMessageMaxLen];
// Retrieve the path to My Photos
WCHAR screenshotPath[MAX_PATH];
GetScreenshotFileName(screenshotPath, _countof(screenshotPath));
// Write out the bitmap to disk
hr = SaveBitmapToFile(static_cast<BYTE *>(LockedRect.pBits), cColorWidth, cColorHeight, 32, screenshotPath);
if (SUCCEEDED(hr))
{
// Set the status bar to show where the screenshot was saved
StringCchPrintf( statusMessage, cStatusMessageMaxLen, L"Screenshot saved to %s", screenshotPath);
}
else
{
StringCchPrintf( statusMessage, cStatusMessageMaxLen, L"Failed to write screenshot to %s", screenshotPath);
}
SetStatusMessage(statusMessage);
// toggle off so we don't save a screenshot again next frame
m_bSaveScreenshot = false;
}
}
// We're done with the texture so unlock it
pTexture->UnlockRect(0);
// Release the frame
m_pNuiSensor->NuiImageStreamReleaseFrame(m_pColorStreamHandle, &imageFrame);
}
This works fine. However, when I wanted to add something like this to the SkeletalViewer example, it is just displaying an empty window.
/// <summary>
/// Handle new color data
/// </summary>
/// <returns>true if a frame was processed, false otherwise</returns>
bool CSkeletalViewerApp::Nui_GotColorAlert( )
{
NUI_IMAGE_FRAME imageFrame;
bool processedFrame = true;
HRESULT hr = m_pNuiSensor->NuiImageStreamGetNextFrame( m_pVideoStreamHandle, 0, &imageFrame );
if ( FAILED( hr ) )
{
return false;
}
INuiFrameTexture * pTexture = imageFrame.pFrameTexture;
NUI_LOCKED_RECT LockedRect;
pTexture->LockRect( 0, &LockedRect, NULL, 0 );
if ( LockedRect.Pitch != 0 )
{
BYTE * pBuffer = (BYTE*) LockedRect.pBits;
cvSetData(img,(BYTE*) pBuffer, img->widthStep);
Mat m(img);
IplImage iplimg(m);
cvNamedWindow("rgb",1);
cvShowImage("rgb",&iplimg);
m_pDrawColor->Draw( static_cast<BYTE *>(LockedRect.pBits), LockedRect.size );
}
else
{
OutputDebugString( L"Buffer length of received texture is bogus\r\n" );
processedFrame = false;
}
pTexture->UnlockRect( 0 );
m_pNuiSensor->NuiImageStreamReleaseFrame( m_pVideoStreamHandle, &imageFrame );
return processedFrame;
}
I'm not sure why the same code doesn't work in this example. I'm using Visual Studio 2010 and OpenCV 2.4.2.
Thanks
Figured it out. Changed it to this
if ( LockedRect.Pitch != 0 )
{
BYTE * pBuffer = static_cast<BYTE *>(LockedRect.pBits);
cvSetData(img,(BYTE*) pBuffer, img->widthStep);
Mat m(img);
IplImage iplimg(m);
cvNamedWindow("rgb",1);
cvShowImage("rgb",&iplimg);
waitKey(1);
m_pDrawColor->Draw( static_cast<BYTE *>(LockedRect.pBits), LockedRect.size );
}

Natively buffering audio

This probably is one of my own mistakes, but I can't seem to find what is wrong. After trying to improve performance of my application, I moved audio buffering from the Java layer to the native layer. Audio handling (recording/playing) is already done natively using the OpenSL ES API.
Yet the native buffering is causing my application to crash whenever I start the application. I use a simple Queue implementation as my buffer, where the first node is the oldest data (FIFO).
struct bufferNode{
struct bufferNode* next;
jbyte* data;
};
struct bufferQueue{
struct bufferNode* first;
struct bufferNode* last;
int size;
};
The audio data is referred to by the jbyte* in the bufferNode. Access to the Queue is done via these two methods, and is synchronized with a mutex.
void enqueueInBuffer(struct bufferQueue* queue, jbyte* data){
SLresult result;
if(queue != NULL){
if(data != NULL){
result = pthread_mutex_lock(&recMutex);
if(result != 0){
decodeMutexResult(result);
logErr("EnqueueInBuffer", "Unable to acquire recording mutex");
} else {
struct bufferNode* node = (struct bufferNode*)malloc(sizeof(struct bufferNode));
if(node == NULL){
logErr("EnqueueInBuffer", "Insufficient memory available to buffer new audio");
} else {
node->data = data;
if(queue->first == NULL){
queue->first = queue->last = node;
} else {
queue->last->next = node;
queue->last = node;
}
queue->size = queue->size + 1;
node->next = NULL;
}
}
result = pthread_mutex_unlock(&recMutex);
if(result != 0){
decodeMutexResult(result);
logErr("EnqueueInBuffer", "Unable to release recording mutex");
}
} else {
logErr("EnqueueInBuffer", "Data is NULL");
}
} else {
logErr("EnqueueInBuffer", "Queue is NULL");
}
}
void dequeueFromBuffer(struct bufferQueue* queue, jbyte* returnData){
SLresult result;
result = pthread_mutex_lock(&recMutex);
if(result != 0){
decodeMutexResult(result);
logErr("DequeueFromBuffer", "Unable to acquire recording mutex");
} else {
if(queue->first == NULL){
returnData = NULL;
} else {
returnData = queue->first->data;
struct bufferNode* tmp = queue->first;
if(queue->first == queue->last){
queue->first = queue->last = NULL;
} else {
queue->first = queue->first->next;
}
free(tmp);
queue->size = queue->size - 1;
}
}
result = pthread_mutex_unlock(&recMutex);
if(result != 0){
decodeMutexResult(result);
logErr("DequeueFromBuffer", "Unable to release recording mutex");
}
}
Where the log and decode methods are selfdeclared utility methods. Log just logs the message to the logcat, while the decode methods "decode" any error number possible from the previous method call.
Yet I keep getting an error when I try to enqueue audio data. Whenever I call the enqueueInBuffer method, I get a SIGSEGV native error, with code 1 (SEGV_MAPERR). Yet I can't seem to find what is causing the error. Both the Queue and the audio data exist when I try to make the enqueueInBuffer method call (which is done in an OpenSL ES Recorder callback, hence the synchronization).
Is there something else going on what causes the segmentation fault? Probably I am responsible for it, but I can't seem to find the error.
Apparently, this was caused by a line of code I have in my OpenSL ES Recorder callback.
The callback originally looked like this:
void recorderCallback(SLAndroidSimpleBufferQueueItf bq, void *context){
SLresult result;
enqueueInBuffer(&recordingQueue, (*recorderBuffers[queueIndex]));
result = (*bq)->Enqueue(bq, recorderBuffers[queueIndex], RECORDER_FRAMES * sizeof(jbyte));
if(checkError(result, "RecorderCallB", "Unable to enqueue new buffer on recorder") == -1){
return;
}
queueIndex = queueIndex++ % MAX_RECORDER_BUFFERS;
}
However, it seems that the last line of the callback didn't correctly create the new index. The buffers I use are in an array, which is 4 long.
Changing the last line to
queueIndex = (queueIndex + 1) % MAX_RECORDER_BUFFERS;
seems to have fixed the error.

Resources