The short story
When compiling a C++ application for the X64 platform with Common Language Runtime support and using the native sqlite library inside it, the application crashes inside sqlite3MemRealloc, attempting to allocate a huge amount of memory (around 5GB).
When the same application is compiled without CLR support, the required functionality works and NO attempt to allocate this amount of memory is made. I put a break-point with a condition to verify this fact.
The database itself is a small 800KB file, and we are attempting to run a simple "select * from XYZ" query. Tried it both with the existing sqlite 3.7.11 that we have in our code base and the latest sqlite 3.7.14.
This problem is consistent. No matter how many times I rebuild the application or play with some settings - with CLR support it crashes, without CLR support it works.
The longer story
I was trying to develop an application that leverages code from an existing code-base written in C++ but also leverages the power of the .NET framework.
I created a C++/CLI application that linked against the existing code (that utilizes sqlite inside it). My code does not use sqlite directly. The existing code that does uses sqlite is a native C++ library that the rest of the code base depends on. So I cannot touch it easily and therefore cannot simply use System.Data.Sqlite.
I isolated this problem by removing all dependencies on the .NET framework and creating a simple application that only utilizes the existing native code without using any .NET framework code, and compiling it twice - with and without CLR support.
Eventually, the problem was solved by compiling SQLITE with the memsys5 memory allocator and configuring it to use it on startup. When it did not depend on the MSVCRT memory allocation but on its own internal allocation strategy, everything worked.
It seems that by some mysterious force, the existance of the CLR inteferes with MSVCRT's memory allocation functions.
More info can be found here.
Related
I develop a C++ DLL with Visual Studio 2017 and I have some memory issues. I would like to use Application Verifier to debug it but the application who runs my DLL (a third-party application) is also heap corrupted. So if I activate Application Verifier, the application crashes before my DLL is loaded.
Is it possible to say to Application Verifier to only take care of heap corruption of a DLL or a module? If not, is there a way to debug my DLL with another tool?
Why not write a simple wrapper app? I've had to do that for a third party DLL that caused heap corruptions that would later crash our app. In this case it was in the destructor so I only had to load and free it. But if that didn't find it for you you could add unit tests for each export.
As far as I kno Application verifier uses the global windows flags to detect leaks and heap corruption like this. So you can directly use the gflags tool to enable this functionality.
Please check the following discussion from Microsoft How to use gflags for finding heap corruptions.
I wonder if I need to distribute VC++ runtime when I only write simple console applications using Visual C++ 2015.
No DLLs etc... just static linking of the standard library (and my own libraries).
You can self-service this: use a clean VM OS installation to check.
But, the answer is: no. be sure to set c/c++ > code generation > runtime library to Multi-threaded or Multi-threaded Debug, as appropriate. Generally during development, it's convenient to use dependency walker periodically to verify your exe isn't importing any DLLs.
I want to know the benefit of pre-JIT compilation (ngen.exe). What is the role of the Native Image Generator (NGen) process and why is it required?
Please provide an example.
For code execution on the .NET platform, the Common Intermediate Language (CIL) representation needs to be translated into machine code. If this happens immediately before execution this is referred to as JIT (Just In Time) compilation. Output of JIT is not persisted so your managed application has to go through JIT for every launch.
Alternatively, you can use pre-compilation to reduce startup overheads related with JIT compilation. NGen performs pre-compilation and keeps the native images in a native image cache. Then applications can run with the native images and may experience faster startup time due to reduced JIT compilation overhead. Initially, NGen was an install-time technology, developers made application installers issue NGen commands to trigger pre-compilation during install time. For more details, check out NGen Revs Up Your Performance with Powerful New Features. This article provides an example application that leverages NGen.
With Windows 8 (.NET 4.5), a new NGen mode: "Auto NGen" has been introduced. Basically, the .NET runtime generates usage logs for managed applications. When the system is idle, an automatic maintenance task runs in the background and generates native images. This way developers no longer have to deal with NGen explicitly. Note that this feature is only enabled for .NET 4.5+ applications that target Window Store or use the GAC. Here's an MSDN page that may be helpful:
Creating Native Images
And this is high-level overview of NGen and related technologies:
Got a need for speed? .NET applications start faster
Lastly, .NET framework libraries themselves use NGen for better performance. When .NET framework is serviced, some of the native images get invalidated. Then NGen needs to run to re-generate the invalid native images. This is done automatically via the .NET Runtime Optimization service which runs during idle time.
When a .NET compiler compiles C# or VB.NET code it half compiles them and creates CIL code. When you run this half-compiled .NET EXE file the JIT runs in the background and compiles the half CIL code in to full machine language. This mode is termed as normal JIT.
You can also go the other way around saying you do not want runtime compilation by running a full compiled EXE file. This compilation is done by using negen.exe. In this scenario the JIT does not participate at runtime. This is termed as pre-JIT mode.
If you want to see how they affect performance you can see this YouTube video which demonstrates normal-JIT and pre-JIT mode of compilation:
Explain JIT, Ngen.exe, Pre-jit, Normal-Jit and Econo-Jit.? (.NET interview questions)
Per MSDN:
The Native Image Generator (Ngen.exe) is a tool that improves the performance of managed applications. Ngen.exe creates native images, which are files containing compiled processor-specific machine code, and installs them into the native image cache on the local computer. The runtime can use native images from the cache instead of using the just-in-time (JIT) compiler to compile the original assembly.
I have used NGEN in the past during installation so that the software would start up faster.
NGen (Native Image Generator) basically compiles .NET byte code (CIL) into native code for the computer it's running on. The benefit is that given that you're not compiling the code to native every time, you run it or need it, but you do it just once, the application starts and run faster. If you want more information there are plenty of resources out there about the benefits of JIT vs. Ahead of Time Compilation (which is what NGen does).
I'm working on WinRt version of my class library dll. Finally, after the huge "code cleanup" my project is on building step and I have two ways. To build the solution with Release|AnyCPU as usually or build it with Release|ARM (Which unclear for me). Which dependencies my dll will get or avoid in process of building, what will be different, will there a specific IL optimizations on a second way?
If you're only using managed code, there's no reason not to use Release|AnyCPU. This way the same package will be used for all three platforms (ARM, x86 and x64).
On the other hand, if your project references natively compiled library, you'll need to set a specific platform, like Release|ARM, that your native library is compiled for. If the native library is installed as an extension (e.g. SQLite for Windows Runtime), you'll be able to compile your app for all 3 target platforms, each one referencing the appropriate native library, though they will need to be individual packages instead of a single universal one.
You'll still be able to submit your app to the store as a single app even if it has 3 separate packages, one for each platform.
My Unmanaged VC++ MFC (No .NET used, No CLR support, Use MFC in shared DLL) application trying to deploy with visual C++ runtime files as private assemblies.
It is properly running on windows 7 fresh installed computer.
But I gives “This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.” error in fresh installed wondows XP sp3 computer.
I checked in application event logs. But there also no more details, just showing the same error.
Then I read these threads and surf around the internet.
Thread - 1
Thread - 2
Article -1
But couldn't find any solution clue or trouble shooting method. so here looking for some assist.
The easiest way to test is to install depends on the computer. Most likely, your application is built to use a later version of C++ runtime libraries, e.g. <assemblyIdentity type='win32' name='Microsoft.VC80.CRT' version='8.0.50727.4053' processorArchitecture='x86' publicKeyToken='1fc8b3b9a1e18e3b' />, but on the XP system it is an older version.
You would need to check what version of the runtime library used by analysing the program's manifest. Then check what depends is showing.
If the required version of runtime is missing, distribute it with the program's install.
On the side note, you could consider switching to the static link. The size of the binaries will be bigger, but these type of problems will be gone