How to get a working x64 THREADSAFE Ghostscript DLL - multithreading

Main context
We're actually trying to get a multi-threading version of Ghostscript x64 DLL, to make use of it through Ghostscript .NET. This component is supposed to "allow runing multiple Ghostscript instances simultaneously within a single process", but, as we have checked in our project, works fine until concurrent requests are made to the application. Same behavior can be replicated lauching same method using Tasks. The error description that raises in both cases, just when a call to the process is made until the last is being executed, is:
An error occured when call to 'gsapi_new_instance' is made: -100
Even it does no seem to be related with .NET directly, I will post a sample of our C# method code, just for contextualize.
// Define switches...
string[] switchesArray = switches.ToArray();
using (GhostscriptProcessor procesador = new GhostscriptProcessor())
{
try
{
procesador.StartProcessing(switchesArray, null);
byte[] destinationFile = System.IO.File.ReadAllBytes(destinationPath);
return destinationFile;
}
catch (Exception ex)
{
throw ex;
}
finally
{
System.IO.File.Delete(sourceFile);
}
}
THREADSAFE solution
Starting our investigation, we found this KenS's answer on this post, indicating that Ghostscript DLL must be generated with GS_THREADSAFE compiler definition.
To clarify, as we make use of Ghostscript 9.52 x64 to generate our PDFs, we need this x64 DLL compiled for Release configuration. After trying to compile Ghostscript sources on Windows 10 x64 machine, using Visual Studio Community 2017 and Visual Studio Community 2019, we finally managed to build and generate all items (only with VS Community 2019) without GS_THREADSAFE parameter, just to confirm that compilation is fine, and we check that the DLLs and executables are working. For this process we took in mind all we found in Ghostscript official documentation.
As we have no other guide to include this GS_THREADSAFE parameter, we followed the instructions given in this solution, including XCFLAGS="-DGS_THREADSAFE=1" on nmake build commands, usign this sentence for Rebuild all option:
cd .. && nmake -f psi\msvc32.mak WIN64= SBR=1 DEVSTUDIO= XCFLAGS=-DGS_THREADSAFE=1 && nmake -f psi\msvc32.mak WIN64= DEVSTUDIO= XCFLAGS=-DGS_THREADSAFE=1 bsc
This approach, rises an error during build:
Error LNK2019 unresolved external symbol errprintf_nomem referenced in
function gs_log_error File \mkromfs.obj 1
As it seems, the file mkromfs.c has a method called errprintf_nomem, which can't be found when GS_THREADSAFE is set.
Questions
1 - Is there any public release of Ghostscript that include x64 DLLs compiled to be THREADSAFE?
And, if not (that's what I'm guessing...)
2 - Is it possible to get this DLL to be THREADSAFE without changing the source code?
3- Could anyone provide, please, a step by step guide or walkthrough to build a x64 Ghostscript DLL using GS_THREADSAFE using Visual Studio (or even any other possible alternative) over Windows 10 x64?
4 - A few posts talk about people achive to manage multithreading using Ghostscript .NET. I assume this examples are all using a GS_THREADSAFE DLL... Is any other workaround we have passed?
Thank a lot in advance.

To summarize all this questions, and as a guide for future developers having this same trouble, these are the answers we've found until now:
AS #KenS mentions in his reply: "No, the Ghostscript developers don't actually build thread-safe versions of the binaries."
At this very moment, clearly not, as it has been reported on this opened bug.
As it seems to be a matter of commercial licensing support, we avoid comment on this point anymore.
Thanks again to #HABJAN. I absolutely take back what I've stated on my question, as it is possible to have Ghostscript .NET working on multi-threading scenarios. Below comes the solution we applied, in case it could be useful for someone.
Based on HABJAN example, what we have done to achieve this was to create a custom class to capture Ghostscript logging:
protected class ConsoleStdIO : Ghostscript.NET.GhostscriptStdIO
{
public ConsoleStdIO(bool handleStdIn, bool handleStdOut, bool handleStdErr) : base(handleStdIn, handleStdOut, handleStdErr)
{
}
public override void StdIn(out string input, int count)
{
char[] userInput = new char[count];
Console.In.ReadBlock(userInput, 0, count);
input = new string(userInput);
}
public override void StdOut(string output)
{
//log
}
public override void StdError(string error)
{
//log
}
}
For our previous method, we simple include a call to this class and this avoids errors when multiple tasks are executed at the same time:
// Define switches...
string[] switchesArray = switches.ToArray();
using (GhostscriptProcessor procesador = new GhostscriptProcessor())
{
try
{
procesador.StartProcessing(switchesArray, new ConsoleStdIO(true, true, true));
byte[] destinationFile = System.IO.File.ReadAllBytes(destinationPath);
return destinationFile;
}
catch (Exception ex)
{
throw ex;
}
finally
{
System.IO.File.Delete(sourceFile);
}
}

Well, it seems to me that you are asking here for technical support.
You clearly want to use Ghostscript in a commercial undertaking, indeed one might reasonably say you want an enterprise version of Ghostscript. Presumably you don't want to alter the source in order to permit you to use an open source license, because you don't want to pay for a commercial license.
With that in mind the answers to your questions are:
No, the Ghostscript developers don't actually build thread-safe versions of the binaries.
Currently, no. That's probably an oversight.
That would be a technical support question, there's no guarantee of technical support to free users, it's the one of the few areas of leverage for dual license vendors to persuade people to take up a commercial license. So I hope you will understand that I'm not going to provide that.
as far as I can see, no.

Related

Can't release the ID3DX11Effect pointer?

I use Effect framework in my demo.
here are my defination in header file
ID3D11Buffer* m_pVertexBuffer;
ID3D11Buffer* m_pIndexBuffer;
ID3D11InputLayout* m_pInputLayout;
ID3DX11Effect* m_pFx;
ID3DX11EffectTechnique* m_pTechnique;
ID3DX11EffectMatrixVariable* m_pFxWorldViewProj;
I release them as follow
void HillsDemo::UnLoadContent()
{
if (m_pVertexBuffer) m_pVertexBuffer->Release();
if (m_pIndexBuffer) m_pIndexBuffer->Release();
if (m_pInputLayout) m_pInputLayout->Release();
if (m_pTechnique) m_pTechnique->Release();
if (m_pFx) m_pFx->Release();
}
then I run the demo,when I close the demo window, there is an error which means HillsDemo has stopped working.Why?Then I delete the lineif (m_pFx) m_pFx->Release();,there is no error. So is releasing the m_pFx make the error?
I view the documents of Effect11 https://github.com/Microsoft/FX11/wiki/Effects-11, there are on the document:
The following types are now derived from IUnknown: ID3DX11EffectType, ID3DX11EffectVariable, I3DX11EffectPass, ID3DX11EffectTechnique, ID3DX11EffectGroup. Note that these objects do not follow standard COM reference-counting rules and they are released when the parent ID3DX11Effect is released. This is mostly to simplify use of Effects 11 from managed languages.
Does it means that I should only release the m_pFx rather than release both m_pFx and m_pTechnique? Any ideas?
I assume you are using the latest version of Effects 11 from GitHub rather than the legacy DirectX SDK copy based on your reference to the project wiki.
You should really look at using ComPtr instead of raw pointers to COM objects. In the case of Effects 11, you'd just use them for ID3DX11Effect instances. You should stick with 'raw pointers' for ID3DX11EffectType, ID3DX11EffectVariable, I3DX11EffectPass, ID3DX11EffectTechnique, and ID3DX11EffectGroup. Don't bother calling Release on these raw pointers as the lifetime controlled by the ID3DX11Effect.
Something like:
Microsoft::WRL::ComPtr<ID3D11Buffer> m_pVertexBuffer;
Microsoft::WRL::ComPtr<ID3D11Buffer> m_pIndexBuffer;
Microsoft::WRL::ComPtr<ID3D11InputLayout> m_pInputLayout;
Microsoft::WRL::ComPtr<ID3DX11Effect> m_pFx;
ID3DX11EffectTechnique* m_pTechnique;
ID3DX11EffectMatrixVariable* m_pFxWorldViewProj;
void HillsDemo::UnLoadContent()
{
m_pVertexBuffer.Reset();
m_pIndexBuffer.Reset();
m_pInputLayout.Reset();
m_pFx.Reset();
m_pTechnique = nullptr;
pFxWorldViewProj = nullptr;
}

Problems using MemoryMappedFile class on Mono

I'm trying to port a new versio of the Isis2 library from .NET on Windows to Mono/Linux. This new code uses MemoryMappedFile objects, and I suddenly am running into issues with the Mono.Posix.Helper library. I believe that my issues would vanish if I could successfully compile and run the following test program:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO.MemoryMappedFiles;
namespace foobar
{
class Program
{
static int CAPACITY = 100000;
static void Main(string[] args)
{
MemoryMappedFile mmf = MemoryMappedFile.CreateNew("test", CAPACITY);
MemoryMappedViewAccessor mva = mmf.CreateViewAccessor();
for (int n = 0; n < CAPACITY; n++)
{
byte b = (byte)(n & 0xFF);
mva.Write<byte>(n, ref b);
}
}
}
}
... at present, when I try to compile this on Mono I get a bewildering set of linker errors: it seems unable to find libMonoPosixHelper.so, although my LD_LIBRARY_PATH includes the directory containing that file, and then if I manage to get past that stage, I get "System.NotImplementedException: The requested feature is not implemented." at runtime. Yet I've looked at the Mono implementation of the CreateNew method; it seems fully implemented, and the same is true for the CreateViewAccessor method. Thus I have a sense that something is going badly wrong when linking to the Mono libraries.
Does anyone have experience with MemoryMappedFile objects under Mono? I see quite a few questions about this kind of issue here and on other sites, but all seem to be old threads...
OK, I figured at least part of this out by inspection of the Mono code implementing this API. In fact they implemented CreateNew in a way that departs pretty drastically from the .NET API, causing these methods to behave very differently from what you would expect.
For CreateNew, they actually require that the file name you specify be the name of an existing Linux file of size at least as large as the capacity you specify, and also do some other checks for access permissions (of course), exclusive access (which is at odds with sharing...) and to make sure the capacity you requested is > 0. So if you had the file previously open, or someone else does, this will fail -- in contrast to .NET, where you explicitly use memory-mapped files for sharing.
In contrast, CreateOrOpen appears to be "more or less" correctly implemented; switching to this version seems to solve the problem. To get the effect of CreateNew, do a Delete first, wrapping it in a try/catch to catch IOException if the file doesn't exist. Then use File.WriteAllBytes to create a file with your desired content. Then call CreateOrOpen. Now this sounds dumb, but it works. Obviously you can't guarantee atomicity this way (three operations rather than one), but at least you get the desired functionality.
I can live with these restrictions as it works out, but they may surprise others, and are totally different from the .NET API definition for MemoryMappedFile.
As for my linking issues, as far as I can tell there is a situation in which Mono doesn't use the LD_LIBRARY_PATH you specify correctly and hence can't find the .so file or .dll file you used. I'll post more on this if I can precisely pin down the circumstances -- on this one, I've worked around the issue by statically linking to the library.

C# for-loop loop-expression syntax

I have a nice question about the loop-expression of a for-loop in C#.
I always like to learn and extend my knowledge so I often use my reflector to look and learn from other. Lately I was looking around in an assembly which I assume is a C# assembly.
I came across this:
public virtual void DoRows(Parse rows)
{
for (; rows != null; {
Parse more;
rows = more;
}
)
{
more = rows.More;
this.DoRow(rows);
}
}
I never knew I could use a code-block in my loop-expression. So I fired up MSDN and went looking in the language reference of C# but found nothing.
The C++ reference shows some interesting things tho (http://msdn.microsoft.com/en-us/library/b80153d8.aspx), but I am not a C++ developper and I have a strong feeling the assmbly was not written in C++.
dotPeek says the assembly is a v4.0.30319 / msil assembly.
Now here are the questions:
- Is this a C++ or a C# code-construct?!
- Can we do things like this in C#?
I am not able to reproduce code that looks like that and compiles in VS2010.
#edit: changed the word assembly in code-construct.
#edit II: Thanks for all your answers. I think this is either a bug in the dotPeek reflector or we found some for-loop-expression easter eggs. This is what Red Gate says about the same method:
public virtual void DoRows(Parse rows)
{
while (rows != null)
{
Parse more = rows.More;
this.DoRow(rows);
rows = more;
}
}
There is no such thing as a C++ or C# assembly. Once the code is compiled, it is ILCode. The reflector takes this ILCode and decompiles it to the language you selected.
This can result in code that is valid in ILCode, but not valid in the target language.
Compilation to IL is a many-to-one operation and decompilers have rules regarding which of the many choices they choose when decompiling. Often, different languages produce IL that is not possible to reproduce directly in another language.
In this instance either the decompiler has chosen poorly and the produced C# is invalid (a bug in the decompiler), or it cannot decompile the code into equivalent C#.

Possible C# 4.0 compiler error, can others verify?

Since I don't know exactly what part of it alone that triggers the error, I'm not entirely sure how to better label it.
This question is a by-product of the SO question c# code seems to get optimized in an invalid way such that an object value becomes null, which I attempted to help Gary with yesterday evening. He was the one that found out that there was a problem, I've just reduced the problem to a simpler project, and want verification before I go further with it, hence this question here.
I'll post a note on Microsoft Connect if others can verify that they too get this problem, and of course I hope that either Jon, Mads or Eric will take a look at it as well :)
It involves:
3 projects, 2 of which are class libraries, one of which is a console program (this last one isn't needed to reproduce the problem, but just executing this shows the problem, whereas you need to use reflector and look at the compiled code if you don't add it)
Incomplete references and type inference
Generics
The code is available here: code repository.
I'll post a description below of how to make the projects if you rather want to get your hands dirty.
The problem exhibits itself by producing an invalid cast in a method call, before returning a simple generic list, casting it to something strange before returning it. The original code ended up with a cast to a boolean, yes, a boolean. The compiler added a cast from a List<SomeEntityObject> to a boolean, before returning the result, and the method signature said that it would return a List<SomeEntityObject>. This in turn leads to odd problems at runtime, everything from the result of the method call being considered "optimized away" (the original question), or a crash with either BadImageFormatException or InvalidProgramException or one of the similar exceptions.
During my work to reproduce this, I've seen a cast to void[], and the current version of my code now casts to a TypedReference. In one case, Reflector crashes so most likely the code was beyond hope in that case. Your mileage might vary.
Here's what to do to reproduce it:
Note: There is likely that there are more minimal forms that will reproduce the problem, but moving all the code to just one project made it go away. Removing the generics from the classes also makes the problem go away. The code below reproduces the problem each time for me, so I'm leaving it as is.
I apologize for the escaped html characters in the code below, this is Markdown playing a trick on me, if anyone knows how I can rectify it, please let me know, or just edit the question
Create a new Visual Studio 2010 solution containing a console application, for .NET 4.0
Add two new projects, both class libraries, also .NET 4.0 (I'm going to assume they're named ClassLibrary1 and ClassLibrary2)
Adjust all the projects to use the full .NET 4.0 runtime, not just the client profile
Add a reference in the console project to ClassLibrary2
Add a reference in ClassLibrary2 to ClassLibrary 1
Remove the two Class1.cs files that was added by default to the class libraries
In ClassLibrary1, add a reference to System.Runtime.Caching
Add a new file to ClassLibrary1, call it DummyCache.cs, and paste in the following code:
using System;
using System.Collections.Generic;
using System.Runtime.Caching;
namespace ClassLibrary1
{
public class DummyCache<TModel> where TModel : new()
{
public void TriggerMethod<T>()
{
}
// Try commenting this out, note that it is never called!
public void TriggerMethod<T>(T value, CacheItemPolicy policy)
{
}
public CacheItemPolicy GetDefaultCacheItemPolicy()
{
return null;
}
public CacheItemPolicy GetDefaultCacheItemPolicy(IEnumerable<string> dependentKeys, bool createInsertDependency = false)
{
return null;
}
}
}
Add a new file to ClassLibrary2, call it Dummy.cs and paste in the following code:
using System;
using System.Collections.Generic;
using ClassLibrary1;
namespace ClassLibrary2
{
public class Dummy
{
private DummyCache<Dummy> Cache { get; set; }
public void TryCommentingMeOut()
{
Cache.TriggerMethod<Dummy>();
}
public List<Dummy> GetDummies()
{
var policy = Cache.GetDefaultCacheItemPolicy();
return new List<Dummy>();
}
}
}
Paste in the following code in Program.cs in the console project:
using System;
using System.Collections.Generic;
using ClassLibrary2;
namespace ConsoleApplication23
{
class Program
{
static void Main(string[] args)
{
Dummy dummy = new Dummy();
// This will crash with InvalidProgramException
// or BadImageFormatException, or a similar exception
List<Dummy> dummies = dummy.GetDummies();
}
}
}
Build, and ensure there are no compiler errors
Now try running the program. This should crash with one of the more horrible exceptions. I've seen both InvalidProgramException and BadImageFormatException, depending on what the cast ended up as
Look at the generated code of Dummy.GetDummies in Reflector. The source code looks like this:
public List<Dummy> GetDummies()
{
var policy = Cache.GetDefaultCacheItemPolicy();
return new List<Dummy>();
}
however reflector says (for me, it might differ in which cast it chose for you, and in one case Reflector even crashed):
public List<Dummy> GetDummies()
{
List<Dummy> policy = (List<Dummy>)this.Cache.GetDefaultCacheItemPolicy();
TypedReference CS$1$0000 = (TypedReference) new List<Dummy>();
return (List<Dummy>) CS$1$0000;
}
Now, here's a couple of odd things, the above crash/invalid code aside:
Library2, which has Dummy.GetDummies, performs a call to get the default cache policy on the class from Library1. It uses type inference var policy = ..., and the result is an CacheItemPolicy object (null in the code, but type is important).
However, ClassLibrary2 does not have a reference to System.Runtime.Caching, so it should not compile.
And indeed, if you comment out the method in Dummy that is named TryCommentingMeOut, you get:
The type 'System.Runtime.Caching.CacheItemPolicy' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Runtime.Caching, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Why having this method present makes the compiler happy I don't know, and I don't even know if this is linked to the current problem or not. Perhaps it is a second bug.
There is a similar method in DummyCache, if you restore the method in Dummy, so that the code again compiles, and then comment out the method in DummyCache that has the "Try commenting this out" comment above it, you get the same compiler error
OK, I downloaded your code and can confirm the problem as described.
I have not done any extensive tinkering with this, but when I run & reflector a Release build all seems OK (= null ref exception and clean disassembly).
Reflector (6.10.11) crashed on the Debug builds.
One more experiment: I wondered about the use of CacheItemPolicies so I replaced it with my own MyCacheItemPolicy (in a 3rd classlib) and the same BadImageFormat exception pops up.
The exception mentions : {"Bad binary signature. (Exception from HRESULT: 0x80131192)"}

how to use dll injecting?

i was looking how to inject a dll into a program (exe, or dll, etc). i have been googleing dll injecting but i have not found anything that is very helpful :(. i have not worked with dlls very much so im not sure on what to do, i really could use some help on this.
uhh the only thing i have really found is setwindowshookex but i can't find any examples for it and i don't how to use it. any questions just ask and i'll try to help.
EDIT hey i was googling and this looks like something about dll injecting that is worth looking at but i can't get the code to run :\ (How to hook external process with SetWindowsHookEx and WH_KEYBOARD)
The method I'm most familiar with was is was described by Jefferey Richter in Programming Applications for Microsoft Windows. I mention this because even if you don't get your hands on the book itself there is probably sample code floating around. I think he may have also written some journal articles. He, also mentions a couple of alternative approaches, of which I will describe only one, from memory. He also may have written some MSJ / MSDN articles that are relevant.
Anyway, the basic idea is to cause the process that you want to load your DLL to issue a call to LoadLibrary. This is done using CreateRemoteThread with the address of LoadLibary for lpStartAddress and the address of a string naming your DLL in for lpParameter. Arranging for and locating the string is done using VirtualAllocEx to allocate some memory in the remote process, and WriteProcessMemory to fill it with the string.
PSEUDO CODE:
void InjectDllIntoProcess(DWORD processId, char *dllName)
{
HANDLE hRemoteProcess = OpenProcess(
// Assumes that dll and function addresses are the same in different processes
// on the same system. I think that this is true even with ASLR, only issue I
// can think of is to make sure that the source and target process are both 32
// or both 64 bit, not a mixture.
// Note that it is asking for the ASCII version
HMODULE hDll = LoadLibrary(_T("Kernel32.dll"));
void *loadLibAddr = GetProcAddress(hDll, _T("LoadLibraryA"));
// Inject the DLL name
char * remoteAddr =
(char *)VirtualAllocEx(hRemoteProcess, NULL, strlen(dllName) + 1, ...
WriteProcessMemory(hRemoteProcess, remoteAddr, dllName, strlen(dllName) + 1 ...
CreateRemoteThread(hRemoteProcess, ??, 0, loadLibAddr, remoteAddr, ...
}

Resources