Stack issues when calling a DLL compiled with Visual C++ in GCC - visual-c++

I'm trying to call some functions in a DLL compiled with (I believe) Visual C++ from my program, which is compiled using GCC.
To call the functions in the DLL, I do a LoadLibrary() on the DLL, and then a GetProcAddress() to get the address of a particular function, which I then call. This function returns a list of pointers to the functions in the DLL I'm to call.
Well, when I try to call those functions, they don't work properly. I ran my program through a debugger, and it looks like the DLL library function is looking for one of the passed arguments at ebp+8, even though GCC has put it at ebp-24.
It looks definitely like a stack issue. What's more, when the GCC program function which calls the DLL function returns, my program crashes -- so something screwey is going on with the stack. Does anyone know what I need to do in order to fix this? I'm not able to access the DLL code.
Also: I tried putting __cdecl and __stdcall before the DLL function definition in my program's source file, but this changes nothing.

Looks like a calling convention problem. Make sure you're putting the calling convention tag in the right place. With GCC, it should look like this:
typedef int (__stdcall *MyFunctionType)(int arg1, const char *arg2);
MyFunctionType myFunction = (MyFunctionType)GetProcAddress(myModule, "MyFunction");
// check for errors...
int x = myFunction(3, "hello, world!");
[EDIT]
Looks like your problem has nothing to do with calling conventions (although getting them right is important). You're misusing BSTRs -- a BSTR is not a simple char* pointer. It's a pointer to a Unicode string (wchar_t*), and furthermore, there is a 4-byte length prefix hidden before the first characters of the string. See MSDN for full details. So, the call to SetLicense() should look like this:
BSTR User = SysAllocString(L""); // Not sure if you can use the same object here,
BSTR Key = SysAllocString(L""); // that depends on if SetLicense() modifies its
// arguments; using separate objects to be safe
// check for errors, although it's pretty unlikely
(textCapLib.sdk)->lpVtbl->SetLicense((textCapLib.sdk), User, Key);
SysFreeString(User); // Hopefully the SDK doesn't hang on to pointers to these
SysFreeString(Key); // strings; if it does, you may have to wait until later to
// free them

Related

Why cannot I use CString as return types or parameters in a MFC DLL made for windows CE?

I have got a library which was made for a desktop windows project. It is done in MFC VC++ by somebody else, and it works correctly. I will use one particular function from the library as an example for explaining the situation.
The example function goes like this:
CString GetFulPath(); // .h file
In the cpp file,
CString CwFolderBrowser::GetFullPath()
{
CString path;
if(this->M_pIDLIST!=NULL)
{
LPTSTR fullPath=path.GetBuffer(MAX_PATH);
::SHGetPathFromIDList(this->M_pIDLIST, fullPath); //ITEMIDLISTからパスを得る
path.ReleaseBuffer();
}
return path;
}
Now, I can include this library in my project and do something like:
CwFolderBrowser cFolderBrowser;
if(cFolderBrowser.ShowDialog() == TRUE)
cPath = cFolderBrowser.GetFullPath();
This will show a folder browser dialog and let me choose a folder. It works fine on desktop windows.
Currently, I am working on Windows CE device. We have converted the library for use with Windows CE by removing unsupported functions and stuff. The library compiles and builds correctly without errors.
Next, I create an MFC Smart Device project, include the converted library, its h file and lib files and set the proper directories for dlls. The project builds fine. I can #include the library's h file properly too.
The problem arises as soon as I call the GetFullPath function:
cPath = cFolderBrowser.GetFullPath();
It gives me an unresolved external link error! The Intellisense does show this function in its list and I can choose it and everything. But in vain.
Strangely, If I modify the library and change GetFullPath()'s signature as below,
LPCTSTR GetFulPath(); // .h file LPCTSTR instead of CString
In the cpp file,
LPCTSTR CwFolderBrowser::GetFullPath() // Return type changed to LPCTSTR
{ // instead of CString
... // Body modified accordingly
}
then, the unresolved external Link error disappears and it works!
I am stumped about this strange behaviour, because, I can use CString normally in the MFC Smart Device project and there are no errors. The link error shows up only when I try to call functions from the library (and other such libraries) dll. At the same time, BOOL, int etc. seems to have no problems as function return types.
Ofcourse, I can go through each library and change every instanceof a CString return to LPCTSTR, but that would be a very big change. I would like to know why CString works fine in project as well as dll when on desktop, while, on Win CE, it works in the project but not i the DLL (At the same time, the DLL itself compiles fine without errors wether it uses CString or LPCTSTR!).
So, basically, I would like to keep the function CString if possible, and would like to know the reason why this happens. The exact same error also happenes in other libraries too.
Any help is appreciated. Thank you.
UPDATE:
I saw a page on ATL & MFC 7.0 which said about using the /Zc:wchat_t option. I have checked the Dll project as well as my application. Both use the same option of 'Treat wchar_t as Built-in type' as Yes. So, that option matches up.
Further, as I mentioned above, changing the function return to LPCTSTR works. The error disappears. Everything is going fine until I convert the returned LPCTSTR back to CString. The CString turns out as empty/Null. This happens both inside the dll code itself, as well as in my application code too.
UPDATE2:
Thanks to Michael and Cody, I changed the function to LPCTSTR and made sure that the values were not going out of scope before I could use them like they suggested. Now the empty/Null problem is solved and I can get the path values properly.
The problem that remains is that I have to convert all the CString functions to LPCTSTR, which is not exactly feasible. I would like to keep the functions as CString.
This is a classic problem and has been asked often here on SO.
This cannot work:
LPCTSTR CwFolderBrowser::GetFullPath()
{
CString path;
if(this->M_pIDLIST!=NULL)
{
LPTSTR fullPath=path.GetBuffer(MAX_PATH);
::SHGetPathFromIDList(this->M_pIDLIST, fullPath);
path.ReleaseBuffer();
}
return (LPCTSTR)path; // << here you return a pointer to the zero
// terminated string in the path object,
} // but path will be deleted as soon as it goes
// out of scope
Maybe in some cases the function appears to work because the memory of the deleted CString object has not yet been overwritten.
You should do this (no error treatment here for simplicity):
LPCTSTR CwFolderBrowser::GetFullPath(TCHAR *pathbuffer)
{
if(this->M_pIDLIST!=NULL)
{
::SHGetPathFromIDList(this->M_pIDLIST, pathbuffer);
}
return (LPCTSTR)pathbuffer;
}
...
// call like this
TCHAR pathbuffer[MAX_PATH];
GetFullPath(pathbuffer);

C++/CLI and GetLastError

I have created a C++ Test Project for my C++ library in Visual Studio 2010. The test project uses C++/CLI (/clr set) and I am having problems retrieving the last error set by my library functions; GetLastError always returns zero.
In the example below I want to test that the correct return value and last error is set by my Write function:
[TestMethod]
void Write_InvalidHandle_Error()
{
char buffer[] = "Hello";
DWORD actual = -1;
DWORD expected = ERROR_INVALID_HANDLE;
int actualRetVal = 0;
int expectedRetVal = -1;
HANDLE handle = INVALID_HANDLE_VALUE;
actualRetVal = Write(handle, buffer);
actual = GetLastError();
Assert::AreEqual(expectedRetVal, actualRetVal);
Assert::AreEqual(expected, actual);
}
I have checked my Write function and it does set the correct return value and last error but the latter is not retrieved in my test method. Even when I change the Write function to just set the error and return the problem occurs (and I call no other function before calling GetLastError in my test method):
int Write(HANDLE h, const char* buf)
{
SetLastError(ERROR_INVALID_HANDLE);
return -1;
}
Any idea how I can fix this? I assume there is a problem with C++/CLI because when I use my library outside of this testing scenario (pure C++) GetLastError works.
Relying on GetLastError()/SetLastError() across the managed/unmanaged boundary is problematic.
When using P/Invoke and the DllImport attribute you can (must) set the SetLastError property to get access to the native error code on the managed side.
When using C++/CLI, however, the compiler handles all marshalling for you, and explicitly does not set that flag.
You can read some more details about it in this blog post. The gist of it is:
If you use DllImport explicitly in C++, the same rules apply as with
C#. But when you call unmanaged APIs directly from managed C++ code,
neither GetLastError nor Marshal.GetLastWin32Error will work reliably.
This is also covered at length in Chapter 9 of "Expert Visual C++/CLI" by Marcus Heege which is available on Google Books:
As mentioned before, for these native local functions, C++/CLI
automatically generates P/Invoke metadata without the lasterror flag,
because it is very uncommon to use the GetLastError value to
communicate error codes within a project. However, the MSDN
documentation on GetLastError allows you to use SetLastError and
GetLastError for your own functions. Therefore, this optimization can
theoretically cause wrong GetLastError values.
Basically, don't do it!
I would recommend to use (native) C++ exceptions to communicate errors between managed and unmanaged code. C++/CLI supports these very nicely. If you can't modify your Write() function directly, you could create a wrapper function on the unmanaged side which uses GetLastError() and then throws an exception if necessary.

How do I use this type of typedef?

in my header api file there is this. (not my code)
typedef void (WINAPI *PIN_FUNC)(char*,LPVOID);
__PINLIB__ int WINAPI PIN_GetNumeric(int m_id,char * Message,PIN_FUNC func,LPVOID Param);
and at one point I had it working by doing this in my code
static void WINAPI Pinpad_Handle(char *buf, LPVOID pParam);
void WINAPI PinpadHelper::Pinpad_Handle( char *buf, LPVOID pParam){...}
but i get the distinct feeling that I'm doing it wrong. And being new to VC++ I don't know how to fix it. The tutorial that i read on typedef mainly talked about variables and abstraction and so forth (which I understood that side of it) I thought that I could do this
static PIN_FUNC PinpadEvent(char* buffer, LPVOID pParam);
but that throws a error in Visual Studio. How do i properly do this? or did I have it right the first time?
You were doing it right originally. PIN_FUNC is just a C function pointer declaration. You must write a C function with the exact same signature as the function pointer. Which you did, nothing wrong with your original code. The compiler would have generated an error if you tried to assign the function pointer and you got the function signature wrong. Lots of programmers make the mistake of casting the error away, that's a fatal mistake that bombs badly at runtime. So never do that.
Not so sure what you tried to do in the last snippet. If you want to declare your own variable that stores the function pointer then that needs to look like this:
static PIN_FUNC PinpadEvent;
Of course it still needs to be assigned. Check your favorite C language programming book about function pointers if you are still confused.

PinvokeStackImbalance calling comctl32 With CallingConvention.Cdecl

We've just upgraded our projects from .NET 3.5SP1 to .NET 4.5 and now we're getting the PInvokeStackImbalance MDA on a call to DllGetVersion from comctl32. I've read around and understand that the Framework is no longer resilient to incorrect calling conventions by default. What is strange is that we are already specifying CallingConvention.Cdecl which I believe is the correct one. If I remove the CallingConvention attribute altogether and let it default (CallingConvention.WinAPI), we don't get the PInvokeStackImbalance. Can anyone shed any light on this? OS is Win 7 64 bit and it's a 32 bit app if that makes any difference. I've tried looking at the header file for the DllGetVersion function but I can't figure out what the calling convention should be .
[DllImport("Comctl32.dll", EntryPoint = "DllGetVersion", CallingConvention = CallingConvention.Cdecl)]
private static extern int NativeDllGetVersion(ref DLLVERSIONINFO pdvi);
In general, any Windows API call should use CallingConvention.StdCall.
In this case, if you look at the declaration of DllGetVersion, you'll see it's defined as:
HRESULT CALLBACK DllGetVersion(DLLVERSIONINFO *pdvi);
CALLBACK is defined as:
#define CALLBACK __stdcall
With the notes:
CALLBACK, WINAPI, and APIENTRY are all used to define functions with the __stdcall calling convention. Most functions in the Windows API are declared using WINAPI. You may wish to use CALLBACK for the callback functions that you implement to help identify the function as a callback function.
In your case, this means switching to StdCall is required.

Starting a method in another thread C++

I'm having trouble finding out how to run a method in a seperate thread in C++ (using Visual C++ 2008), I've tried a number of ways to do this but none of them so far have been successful.
I'm very new to C++ but a fairly experienced programmer in Java, but have been given a task to fix some bugs in an old C++ application. The program uses an object called 'Mpeg' to control packetising and depackitising an Mpeg file. After setting up an Mpeg object properly, mpeg.Depacketise needs to be called, which then runs the method DepacketiseInputFile().
I've tried to make DepacketiseInputFile() run in a seperate thread by both using _beginthread and the System::Threading::Thread object using
Thread^ th = gcnew Thread(gcnew ThreadStart(DepacketiseInputFile));
however this returns the errors
using &Mpeg::Depacketise gives the error
when using _beginthread the code I tried was
However with this I constantly had trouble getting the arguments correct, with errors like
cropping up.
Is there any simple way to do this that anyone can reccomend? I've spent a few days playing around with this but seem to be getting nowhere :(
Any help would be greatly appreciated.
Cheers.
What kind of type is Mpeg? What kind of method is DepacketiseInputFile?
If it's a regular unmanaged, C++ class, then use _beginthread, but you have to make DepacketiseInputFile a static. It cannot take a member function.
Also, don't call DepacketiseInputFile with DepacketiseInputFile(), pass it in with
&Mpeg::DepacketiseInputFile
You should use the void* you get to pass it to pass in a pointer to the Mpeg object (and then cast it back).
If you want to use ThreadStart, then Mpeg needs to be a managed class.
EDIT: If you want to make DepacketiseInputFile, but it needs to access the object, then you use the void* argument to pass in a pointer.
So in the .h:
void DepacketiseInputFileMember();
static void DepacketiseInputFile(void *thisObj);
Your code goes in DepacketiseInputFileMember(), and write DepacketiseInputFile like this:
void Mpeg::DepacketiseInputFile(void *thisObj)
{
Mpeg* mpeg = reinterpret_cast<Mpeg*>(thisObj);
mpeg->DepacketiseInputFileMember();
}
When you call _beginthread, use this
_beginnthread(&Mpeg::DepacketiseInputFile, (unsigned)0, anMpegObjectPointer);
where anMpegObjectPointer is a pointer to an object of type Mpeg. You have to make sure the lifetime of the object is longer than it would be needed in the thread.
Forgive my syntax, I am writing this in a textarea, not Visual Studio
Change
_beginthread(DepacketiseInputFile(), (unsigned)0, (void *)NULL);
to
_beginthread(DepacketiseInputFile, (unsigned)0, (void *)NULL);
You wanna pass the address of the function to run (DepacketiseInputFile) and not its return value of that function (which is what you get from DepacketiseInputFile()).
I'm assuming DepacketiseInputFile is declared as void DepacketiseInputFile(void*), and is not a non-static member function of some class. Otherwise, the types won't match even when you do remove the brackets.

Resources