Using a mutex to limit instance of application is not respected if one instance if started from Visual Studio - visual-c++

Question

In InitInstance of my dialog application I use this code to detect other running versions:
strOwner.LoadString(IDS_APP_MUTEX);
m_hMutex = ::CreateMutex(nullptr, FALSE, strOwner);
HWND hOtherInstance = nullptr;
if (DetectRunningInstance(hOtherInstance))
{
DetectFileToOpenFromFileExplorer();
TryToOpenFileInOtherInstance(hOtherInstance);
return FALSE; // Terminates the creation
}
The other functions referred to are:
bool CMeetingScheduleAssistantApp::DetectRunningInstance(HWND& rhOtherWnd) noexcept
{
HWND hOther = nullptr;
if (m_hMutex != nullptr) // indicates running instance
{
if (::GetLastError() == ERROR_ALREADY_EXISTS)
{
EnumWindows(searcher, reinterpret_cast<LPARAM>(&hOther));
if (hOther != nullptr)
{
::SetForegroundWindow(::GetLastActivePopup(hOther));
if (IsIconic(hOther))
{
::ShowWindow(hOther, SW_RESTORE);
}
rhOtherWnd = hOther;
return true; // terminates the creation
}
}
}
return false;
}
And:
void CMeetingScheduleAssistantApp::DetectFileToOpenFromFileExplorer()
{
CCommandLineInfo cmdInfo;
m_bOpenFileFromFileExplorer = false;
m_strFileToOpenFromFileExplorerPath;
ParseCommandLine(cmdInfo);
if (PathFileExists(cmdInfo.m_strFileName))
{
m_bOpenFileFromFileExplorer = true;
m_strFileToOpenFromFileExplorerPath = cmdInfo.m_strFileName;
}
}
And:
void CMeetingScheduleAssistantApp::TryToOpenFileInOtherInstance(HWND hOtherInstance)
{
CString strFile = GetFileToOpenFromFileExplorerPath();
S_COPY_PACKET sCopyDataPacket{};
_tcscpy_s(sCopyDataPacket.szFile, strFile);
sCopyDataPacket.guidSignature = CopyData_Signature;
COPYDATASTRUCT cds;
cds.dwData = COPYDATA_TYPE_MSA;
cds.cbData = sizeof(sCopyDataPacket);
cds.lpData = &sCopyDataPacket;
DWORD_PTR dwResult{};
if (SendMessageTimeout(hOtherInstance, WM_COPYDATA,
NULL, reinterpret_cast<LPARAM>(&cds), SMTO_BLOCK, 2000, &dwResult) != 0)
{
// The message was sent and processed
if (dwResult == FALSE)
{
// The other instance returned FALSE. This is probably because it
// has a pop-up window open so can't open the file
::OutputDebugString(_T("InitInstance::SendMessageTimeout [dwResult was FALSE].\n"));
}
}
else
{
const DWORD dwError = ::GetLastError();
if (dwError == ERROR_TIMEOUT)
{
// The message timed out for some reason
::OutputDebugString(_T("InitInstance::SendMessageTimeout [ERROR_TIMEOUT].\n"));
}
else
{
// Another unknown error
}
CString strError;
strError.Format(_T("InitInstance::SendMessageTimeout [%d: %s]\n"), dwError,
(LPCTSTR)GetLastErrorAsStringEx(dwError));
::OutputDebugString(strError);
}
}
And the seracher:
BOOL CALLBACK CMeetingScheduleAssistantApp::searcher(HWND hWnd, LPARAM lParam) noexcept
{
DWORD_PTR result{};
const LRESULT ok = ::SendMessageTimeout(hWnd,
theApp.UWM_ARE_YOU_ME_MSG,
0, 0,
SMTO_BLOCK |
SMTO_ABORTIFHUNG,
200,
&result);
if (ok == 0)
return TRUE; // ignore this and continue
if (result == theApp.UWM_ARE_YOU_ME_MSG)
{
// found it
HWND *target = reinterpret_cast<HWND *>(lParam);
*target = hWnd;
return FALSE; // stop search
}
return TRUE; // continue search
}
Under normal circumstances this is fully functional. But I have a specific scenario where my mutex is not honored. I think this started with Visual Studio 2022 but I no longer have 2019 edition to confirm.
The scenario
My PC has an instance of the software I am developing "installed" and it has a shortcut on the taskbar. This links to the executable in the installed folder. If I simply use this shortcut then no issues, no multiple instances.
But, if I start my application from inside VS2022 instead, and then accidently click my other shortcut on the taskbar, I end up with two instances running. Why does this happen? I appreciate the otherinstance is running inside VS2022 but it is still the same software.
Is this by design or a coding oversight?
Update
If I manually double-click the release build EXE in file explorer, and then try to invoke the application in my VS2022 environment, it will jump to the existing instance. But ...
If I invoke the application in my VS2022 environment first, and then double-click the same release build EXE in file explorer, I end up with a multiple instance.