Call to MultiByteToWideChar() in 64-bit office gives wrong result - excel

Question

I have an Excel VBA project I'm in the process of adapting for 64-bit Office. In one part, I make calls to MultiByteToWideChar() using any of 20 or so different code pages. (So StrConv is not an alternative.)
This has been working for me for years in 32-bit Office using the following declare:
Declare Function MultiByteToWideChar Lib "kernel32" ( _
ByVal codepage As Long, _
ByVal dwFlags As Long, _
lpMultiByteStr As Any, _
ByVal cbMultiByte As Long, _
ByVal lpWideCharStr As Long, _
ByVal cchWideChar As Long _
) As Long
'params: UINT, DWORD, LPCSTR, int, LPWSTR, int
'return: int
But my adaptation to 64-bit is not: I get wrong results (e.g. an empty string where a non-empty string is expected), and frequent crashes. I'm using a declare that I got from the Microsoft-provided Win32API_PtrSafe.TXT file. (Of course, it could have bugs.)
So, I'm guessing something is not right in the declare statement or in how I'm making the call.
Here's a minimal sample that repro's:
'Windows API declarations
Public Const MB_PRECOMPOSED = &H1 'use precomposed chars
Declare PtrSafe Function MultiByteToWideChar Lib "kernel32" ( _
ByVal CodePage As Long, _
ByVal dwFlags As Long, _
ByVal lpMultiByteStr As String, _
ByVal cchMultiByte As Long, _
ByVal lpWideCharStr As String, _
ByVal cchWideChar As Long _
) As Long
'params: UINT, DWORD, LPCSTR, int, LPWSTR, int
'return: int
' My function that calls MultiByteToWideChar
Private Function EncodedStringByteArrayToString(abStringData() As Byte, lngArrLen As Long, CodePage As Long) As String
Dim lngStrLen As Long, str As String
lngStrLen = MultiByteToWideChar(CodePage, MB_PRECOMPOSED, ByVal VarPtr(abStringData(1)), lngArrLen, 0&, 0)
str = String(lngStrLen, " ")
lngStrLen = MultiByteToWideChar(CodePage, MB_PRECOMPOSED, ByVal VarPtr(abStringData(1)), lngArrLen, StrPtr(str), lngStrLen)
EncodedStringByteArrayToString = str
End Function
' Sample routine to produce repro
Private Sub TestMB2WCBug()
Dim abStringData(1 To 9) As Byte
Dim resultString As String
abStringData(1) = 67
abStringData(2) = 111
abStringData(3) = 112
abStringData(4) = 121
abStringData(5) = 114
abStringData(6) = 105
abStringData(7) = 103
abStringData(8) = 104
abStringData(9) = 116
resultString = EncodedStringByteArrayToString(abStringData(), 9, 10000)
End Sub

Answer 1

This has been working for me for years in 32-bit Office
It could not possibly work with the Declare that you have shown.
MultiByteToWideChar expects an LPWSTR as the output buffer. VB performs automatic conversion from Unicode to ANSI when passing strings into Declared functions, so there is no way that the function would receive a pointer to a wide string buffer when lpWideCharStr is declared As String. At best, it would receive a buffer that is large enough so no buffer overflow would occur, and then VB would perform conversion back to Unicode when returning from the function, so you will end up with a double-unicode string.
lpMultiByteStr is not a string either, it's an array of bytes in some encoding.
The code inside EncodedStringByteArrayToString seems to know all that, because it correctly passes a byte array for lpMultiByteStr and an StrPtr for lpWideCharStr. This could have not happened with the current declaration of MultiByteToWideChar.
The declaration that is assumed by the code in EncodedStringByteArrayToString is:
Declare PtrSafe Function MultiByteToWideChar Lib "kernel32" ( _
ByVal CodePage As Long, _
ByVal dwFlags As Long, _
ByVal lpMultiByteStr As LongPtr, _
ByVal cchMultiByte As Long, _
ByVal lpWideCharStr As LongPtr, _
ByVal cchWideChar As Long _
) As Long
Apparently you had that before, so just put it back.