Calling unmanaged C DLL from VB CLR UDF in SQL Server - multithreading

Question

I am trying to integrate an existing C DLL (unmanaged obviously), that implements fuzzy matching, into SQL Server as a user defined function (UDF). The UDF is implemented with a CLR VB project. I have used this C code for nearly 20 years to do string matching on text files without a hitch. It has been compiled on about every platform under the sun and has never crashed or given erroneous results. Ever. Until now.
The usage of this UDF in a SQL SELECT statement looks something like this:
SELECT Field FROM Table WHERE xudf_fuzzy('doppler effect', Field) = 1;
xudf_fuzzy(Param1, Param2) = 1 is where the magic happens. Param1 is the clue word we are trying to match while Param2 is the field from the table to be tested against. If the match is successful within a certain number of errors, the UDF returns 1, if not it returns 0. So far so good.
Here is the CLR code that defines the fuzzy UDF and calls the C DLL:
Imports System
Imports System.Data
Imports System.Data.SqlClient
Imports System.Data.SqlTypes
Imports System.Text
Imports Microsoft.SqlServer.Server
Imports System.Runtime.InteropServices
Partial Public Class fuzzy
<DllImport("C:\Users\Administrator\Desktop\fuzzy64.dll", _
CallingConvention:=CallingConvention.StdCall)> _
Public Shared Function setClue(ByRef clue As String, ByVal misses As Integer) As Integer
End Function
<DllImport("C:\Users\Administrator\Desktop\fuzzy64.dll", _
CallingConvention:=CallingConvention.StdCall)> _
Public Shared Function srchString(ByRef text1 As String) As Integer
End Function
<Microsoft.SqlServer.Server.SqlFunction()> Public Shared Function _
xudf_fuzzy(ByVal strSearchClue As SqlString, ByVal strStringtoSearch As SqlString) As Long
Dim intMiss As Integer = 0
Dim intRet As Integer
Static Dim sClue As String = ""
xudf_fuzzy = 0
' we only need to set the clue whenever it changes '
If (sClue <> strSearchClue.ToString) Then
sClue = strSearchClue.ToString
intMiss = (Len(sClue) \ 4) + 1
intRet = setClue(sClue, intMiss)
End If
' return the fuzzy match result (0 or 1) '
xudf_fuzzy = srchString(strStringtoSearch.ToString)
End Function
Here is the front end of the C code being called. STRCT is where all of the global storage resides.
fuzzy.h
typedef struct {
short int INVRT, AND, LOWER, COMPL, Misses;
long int num_of_matched;
int D_length;
unsigned long int endposition, D_endpos;
unsigned long int Init1, NOERRM;
unsigned long int Mask[SYMMAX];
unsigned long int Init[MaxError];
unsigned long int Bit[WORDSIZE+1];
unsigned char prevpat[MaxDelimit];
unsigned char _buffer[Max_record+Max_record+256];
unsigned char _myPatt[MAXPAT];
} SRCH_STRUCT;
fuzzy.c
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <wtypes.h>
#include <time.h>
#include "fuzzy.h"
// call exports
__declspec(dllexport) int CALLBACK setClue(char**, int*);
__declspec(dllexport) int CALLBACK srchString(char**);
SRCH_STRUCT STRCT = { 0 };
int cluePrep(unsigned char []);
int srchMin(unsigned char [], int);
BOOL APIENTRY DllMain( HANDLE hModule, DWORD ul_reason_for_call, LPVOID lpReserved )
{
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
break;
}
return TRUE;
}
int CALLBACK setClue(char **pattern, int *misses)
{
int i;
unsigned char *p;
// code to do initialization stuff, set flags etc. etc.
STRCT.Misses = (int)misses;
p = &(STRCT._myPatt[2]);
STRCT._myPatt[0] = '\n';
STRCT._myPatt[1] = SEPCHAR;
strcpy((char *)p, *pattern);
//blah blah
// end setup stuff
i = cluePrep(STRCT._myPatt);
return 0;
}
int CALLBACK srchString(char **textstr)
{
int res,i = Max_record;
unsigned char c;
char *textPtr = *textstr;
STRCT.matched = 0;
//clean out any non alphanumeric characters while we load the field to be tested
while ((c = *textPtr++)) if ( isalpha(c) || isdigit(c) || c == ' ' ) STRCT._buffer[i++] = c;
STRCT._buffer[i] = 0;
// do the search
res = srchMin(STRCT.pattern, STRCT.Misses);
if (res < 0) return res;
return STRCT.matched;
}
The runtime library it is linked against is: Multi-threaded DLL (/MD)
The calling convention is: __cdecl (/Gd)
This is where it gets weird. If I have a regular dot-net application (code not shown) that grabs the entire recordset from the test database and iterates through all of the records one at a time calling this DLL to invoke the fuzzy match, I get back the correct results every time.
If I use the CLR UDF application shown above against the test database using the SQL statement shown above while using only one thread (a single core VM) I get back correct results every time as well.
When this DLL is used in CLR UDF mode on a multi-core machine then some of the results are wrong. The results are always a little off, but not consistently.
292 records should match and do in the first two test cases.
In the CLR UDF multi-threaded case the results will come back with 273, 284, 298, 290 etc.
All of the storage in the C DLL is in character arrays. No memory allocs are being used. It is also my understanding that if SQL Server is using this CLR app in multi-threaded mode that the threads are all assigned their own data space.
Do I need to somehow "pin" the strings before I send them to the C DLL? I just can't figure out how to proceed.

Answer 1

Pinning is not your issue, your C code is not thread-safe. The C code uses a global variable STRCT. There is only one instance of that global variable that will be shared across all threads. Each thread will update STRCT variable with different values, which would cause incorrect result.
You will have to refactor the C code so that it does not rely on the shared state of a global variable.
You may be able to get it to work by declaring STRCT with __declspec(thread) which will use Thread Local Storage so each thread gets its own copy of the variable. However, that would be unique to each unmanaged thread and there is no guarantee that there is a one-to-one mapping between managed and un-managed threads.
The better solution would be to get rid of the shared state completely. You could do this by allocating a SRCH_STRUCT in setClue and return that pointer. Then each call to srchString would take that SRCH_STRUCT pointer as a parameter. The VB.Net code would only have to treat this struct as in IntPtr, it would not need to know anything about the definition of SRCH_STRUCT. Note you would also need to add a new function to the DLL to deallocate the allocated SRCH_STRUCT.