I'm trying to port a new versio of the Isis2 library from .NET on Windows to Mono/Linux. This new code uses MemoryMappedFile objects, and I suddenly am running into issues with the Mono.Posix.Helper library. I believe that my issues would vanish if I could successfully compile and run the following test program:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO.MemoryMappedFiles;
namespace foobar
{
class Program
{
static int CAPACITY = 100000;
static void Main(string[] args)
{
MemoryMappedFile mmf = MemoryMappedFile.CreateNew("test", CAPACITY);
MemoryMappedViewAccessor mva = mmf.CreateViewAccessor();
for (int n = 0; n < CAPACITY; n++)
{
byte b = (byte)(n & 0xFF);
mva.Write<byte>(n, ref b);
}
}
}
}
... at present, when I try to compile this on Mono I get a bewildering set of linker errors: it seems unable to find libMonoPosixHelper.so, although my LD_LIBRARY_PATH includes the directory containing that file, and then if I manage to get past that stage, I get "System.NotImplementedException: The requested feature is not implemented." at runtime. Yet I've looked at the Mono implementation of the CreateNew method; it seems fully implemented, and the same is true for the CreateViewAccessor method. Thus I have a sense that something is going badly wrong when linking to the Mono libraries.
Does anyone have experience with MemoryMappedFile objects under Mono? I see quite a few questions about this kind of issue here and on other sites, but all seem to be old threads...
OK, I figured at least part of this out by inspection of the Mono code implementing this API. In fact they implemented CreateNew in a way that departs pretty drastically from the .NET API, causing these methods to behave very differently from what you would expect.
For CreateNew, they actually require that the file name you specify be the name of an existing Linux file of size at least as large as the capacity you specify, and also do some other checks for access permissions (of course), exclusive access (which is at odds with sharing...) and to make sure the capacity you requested is > 0. So if you had the file previously open, or someone else does, this will fail -- in contrast to .NET, where you explicitly use memory-mapped files for sharing.
In contrast, CreateOrOpen appears to be "more or less" correctly implemented; switching to this version seems to solve the problem. To get the effect of CreateNew, do a Delete first, wrapping it in a try/catch to catch IOException if the file doesn't exist. Then use File.WriteAllBytes to create a file with your desired content. Then call CreateOrOpen. Now this sounds dumb, but it works. Obviously you can't guarantee atomicity this way (three operations rather than one), but at least you get the desired functionality.
I can live with these restrictions as it works out, but they may surprise others, and are totally different from the .NET API definition for MemoryMappedFile.
As for my linking issues, as far as I can tell there is a situation in which Mono doesn't use the LD_LIBRARY_PATH you specify correctly and hence can't find the .so file or .dll file you used. I'll post more on this if I can precisely pin down the circumstances -- on this one, I've worked around the issue by statically linking to the library.
Related
Main context
We're actually trying to get a multi-threading version of Ghostscript x64 DLL, to make use of it through Ghostscript .NET. This component is supposed to "allow runing multiple Ghostscript instances simultaneously within a single process", but, as we have checked in our project, works fine until concurrent requests are made to the application. Same behavior can be replicated lauching same method using Tasks. The error description that raises in both cases, just when a call to the process is made until the last is being executed, is:
An error occured when call to 'gsapi_new_instance' is made: -100
Even it does no seem to be related with .NET directly, I will post a sample of our C# method code, just for contextualize.
// Define switches...
string[] switchesArray = switches.ToArray();
using (GhostscriptProcessor procesador = new GhostscriptProcessor())
{
try
{
procesador.StartProcessing(switchesArray, null);
byte[] destinationFile = System.IO.File.ReadAllBytes(destinationPath);
return destinationFile;
}
catch (Exception ex)
{
throw ex;
}
finally
{
System.IO.File.Delete(sourceFile);
}
}
THREADSAFE solution
Starting our investigation, we found this KenS's answer on this post, indicating that Ghostscript DLL must be generated with GS_THREADSAFE compiler definition.
To clarify, as we make use of Ghostscript 9.52 x64 to generate our PDFs, we need this x64 DLL compiled for Release configuration. After trying to compile Ghostscript sources on Windows 10 x64 machine, using Visual Studio Community 2017 and Visual Studio Community 2019, we finally managed to build and generate all items (only with VS Community 2019) without GS_THREADSAFE parameter, just to confirm that compilation is fine, and we check that the DLLs and executables are working. For this process we took in mind all we found in Ghostscript official documentation.
As we have no other guide to include this GS_THREADSAFE parameter, we followed the instructions given in this solution, including XCFLAGS="-DGS_THREADSAFE=1" on nmake build commands, usign this sentence for Rebuild all option:
cd .. && nmake -f psi\msvc32.mak WIN64= SBR=1 DEVSTUDIO= XCFLAGS=-DGS_THREADSAFE=1 && nmake -f psi\msvc32.mak WIN64= DEVSTUDIO= XCFLAGS=-DGS_THREADSAFE=1 bsc
This approach, rises an error during build:
Error LNK2019 unresolved external symbol errprintf_nomem referenced in
function gs_log_error File \mkromfs.obj 1
As it seems, the file mkromfs.c has a method called errprintf_nomem, which can't be found when GS_THREADSAFE is set.
Questions
1 - Is there any public release of Ghostscript that include x64 DLLs compiled to be THREADSAFE?
And, if not (that's what I'm guessing...)
2 - Is it possible to get this DLL to be THREADSAFE without changing the source code?
3- Could anyone provide, please, a step by step guide or walkthrough to build a x64 Ghostscript DLL using GS_THREADSAFE using Visual Studio (or even any other possible alternative) over Windows 10 x64?
4 - A few posts talk about people achive to manage multithreading using Ghostscript .NET. I assume this examples are all using a GS_THREADSAFE DLL... Is any other workaround we have passed?
Thank a lot in advance.
To summarize all this questions, and as a guide for future developers having this same trouble, these are the answers we've found until now:
AS #KenS mentions in his reply: "No, the Ghostscript developers don't actually build thread-safe versions of the binaries."
At this very moment, clearly not, as it has been reported on this opened bug.
As it seems to be a matter of commercial licensing support, we avoid comment on this point anymore.
Thanks again to #HABJAN. I absolutely take back what I've stated on my question, as it is possible to have Ghostscript .NET working on multi-threading scenarios. Below comes the solution we applied, in case it could be useful for someone.
Based on HABJAN example, what we have done to achieve this was to create a custom class to capture Ghostscript logging:
protected class ConsoleStdIO : Ghostscript.NET.GhostscriptStdIO
{
public ConsoleStdIO(bool handleStdIn, bool handleStdOut, bool handleStdErr) : base(handleStdIn, handleStdOut, handleStdErr)
{
}
public override void StdIn(out string input, int count)
{
char[] userInput = new char[count];
Console.In.ReadBlock(userInput, 0, count);
input = new string(userInput);
}
public override void StdOut(string output)
{
//log
}
public override void StdError(string error)
{
//log
}
}
For our previous method, we simple include a call to this class and this avoids errors when multiple tasks are executed at the same time:
// Define switches...
string[] switchesArray = switches.ToArray();
using (GhostscriptProcessor procesador = new GhostscriptProcessor())
{
try
{
procesador.StartProcessing(switchesArray, new ConsoleStdIO(true, true, true));
byte[] destinationFile = System.IO.File.ReadAllBytes(destinationPath);
return destinationFile;
}
catch (Exception ex)
{
throw ex;
}
finally
{
System.IO.File.Delete(sourceFile);
}
}
Well, it seems to me that you are asking here for technical support.
You clearly want to use Ghostscript in a commercial undertaking, indeed one might reasonably say you want an enterprise version of Ghostscript. Presumably you don't want to alter the source in order to permit you to use an open source license, because you don't want to pay for a commercial license.
With that in mind the answers to your questions are:
No, the Ghostscript developers don't actually build thread-safe versions of the binaries.
Currently, no. That's probably an oversight.
That would be a technical support question, there's no guarantee of technical support to free users, it's the one of the few areas of leverage for dual license vendors to persuade people to take up a commercial license. So I hope you will understand that I'm not going to provide that.
as far as I can see, no.
I'm in the process of trying to migrate a R# extension project from R# 6 to R# 8. (I've taken over a project that someone wrote, and I'm new to writing extensions.)
In the existing v6 project there is a class that derives from RenameWorkflow, and the constructor used to look like this;
public class RenameStepWorkflow : RenameWorkflow
{
public RenameStepWorkflow(ISolution Solution, string ActionId)
: base(Solution, ActionId)
{
}
This used to work in R# SDK v 6, but now in V8, RenameWorkflow no longer has a constructor that takes Solution and actionId. The new constructor signature now looks like this;
public RenameWorkflow(
IShellLocks locks,
SearchDomainFactory searchDomainFactory,
RenameRefactoringService renameRefactoringService,
ISolution solution,
string actionId);
now heres my problem that I need help with (I think)
I've copied the constructor, and now the constructor of this class has to satisfy these new dependancies. Through some digging I've managed to find a way to satisfy all the dependencies, except for 'SearchDomainFactory'. The closest I can come to instantiating via the updated constructor is as follows;
new RenameStepWorkflow(Solution.Locks, JetBrains.ReSharper.Psi.Search.SearchDomainFactory.Instance, RenameRefactoringService.Instance, this.Solution, null)
All looks good, except that JetBrains.ReSharper.Psi.Search.SearchDomainFactory.Instance is marked as Obsolete, and gives me a compile error that I cannot work around, even using #pragma does not allow me to compile the code. The exact error message I get when I compile is Error 16 'JetBrains.ReSharper.Psi.Search.SearchDomainFactory.Instance' is obsolete: 'Inject me!'
Obvious next question..ok, how? How do I 'inject you'? I cannot find any documentation over this new breaking change, in fact, I cannot find any documentation (or sample projects) that even mentions DrivenRefactoringWorkflow or RenameWorkflow, (the classes that now require the new SearchDomainFactory), or any information on SearchDomainFactory.Instance suddenly now obsolete and how to satisfy the need to 'inject' it.
Any help would be most appreciated! Thank you,
regards
Alan
ReSharper has its own IoC container, which is responsible for creating instances of classes, and "injecting" dependencies as constructor parameters. Classes marked with attributes such as [ShellComponent] or [SolutionComponent] are handled by the container, created when the application starts or a solution is loaded, respectively.
Dependencies should be injected as constructor parameters, rather than using methods like GetComponent<TDependency> or static Instance properties, as this allows the container to control dependency lifetime, and ensure you're depending on appropriate components, and not creating leaks - a shell component cannot depend on a solution component for instance, it won't exist when the shell component is being created.
ReSharper introduced the IoC container a few releases ago, and a large proportion of the codebase has been updated to use it correctly, but there are a few hold-outs, where things are still done in a less than ideal manner - static Instance properties and calls to GetComponent. This is what you've encountered. You should be able to get an instance of SearchDomainFactory by putting it as a constructor parameter in your component.
You can find out more about the Component Model (the IoC container and related functionality) in the devguide: https://www.jetbrains.com/resharper/devguide/Platform/ComponentModel.html
When I enabled code contracts on my WPF control project I ran into a problem with an auto generated file which was created at compile time (XamlNamespace.GeneratedInternalTypeHelper). Note, the generated file is called GeneratedInternalTypeHelper.g.cs and is not the same as the GeneratedInternalTypeHelper.g.i.cs which there are several obsolete blog posts about.
I'm not exactly sure what its purpose is, but I am assuming it is important for some internal reflection to resolve XAML. The problem is that it does not have code contracts, nor is the code contract system smart enough to recognize it as an auto generated file. This leads to a bunch of errors from the static checker.
I tried searching for a solution to this problem, but it seems like nobody is developing WPF controls and using code contracts. I did come across an interesting attribute, ContractVerificationAttribute, which takes a boolean value to set whether the assembly or class is to be verified. This allows you to decorate a class as not verified. Sadly the GeneratedInternalTypeHelper is regenerated with every compile, so it is not possible to exclude just this one class. The inverse scenario is possible though, decorate the assembly as not verified and then opt in for every class.
To mitigate the obvious hack I wanted to create a test that would at least verify that the exposed classes have code contract verification with a test like the following to ensure that own classes were at least being verified:
[Fact]
public void AllAssemblyTypesAreDecoratedWithContractVerificationTrue()
{
var assembly = typeof(someType).Assembly;
var exposedTypes = assembly.GetTypes().Where(t=>!string.IsNullOrWhiteSpace(t.Namespace) && t.Namespace.StartsWith("MyNamespace") && !t.Name.StartsWith("<>"));
var areAnyNotContractVerified = exposedTypes.Any(t =>
{
var verificationAttribute = t.GetCustomAttributes(typeof(ContractVerificationAttribute), true).OfType<ContractVerificationAttribute>();
return verificationAttribute.Any() && verificationAttribute.First().Value;
});
Assert.False(areAnyNotContractVerified);
}
As you can see it takes all classes in the controls assembly and finds the one from the company namespace which are not also auto generated anonymous types (<>WeirdClassName).
(I also need to exclude Resources and settings, but I hope you get the idea).
I'm not loving the solution since there are ways of avoiding contract verification, but currently it's the best I can come up with. If anyone has a better solution, please let me know.
So you can treat this class exactly like you would treat any other "3rd party" class or library. I'm sure certain assumptions would hold with the interaction with this generated class so at the interaction points, decorate your own code with Contract.Assume(result != null) or similar.
var result = new GennedClass().GetSomeValue();
Contract.Assume(result != null);
What this does is translate into an assertion that is checked at run time, but it allows the static analyzer to reason about the rest of the code that you do control.
i was looking how to inject a dll into a program (exe, or dll, etc). i have been googleing dll injecting but i have not found anything that is very helpful :(. i have not worked with dlls very much so im not sure on what to do, i really could use some help on this.
uhh the only thing i have really found is setwindowshookex but i can't find any examples for it and i don't how to use it. any questions just ask and i'll try to help.
EDIT hey i was googling and this looks like something about dll injecting that is worth looking at but i can't get the code to run :\ (How to hook external process with SetWindowsHookEx and WH_KEYBOARD)
The method I'm most familiar with was is was described by Jefferey Richter in Programming Applications for Microsoft Windows. I mention this because even if you don't get your hands on the book itself there is probably sample code floating around. I think he may have also written some journal articles. He, also mentions a couple of alternative approaches, of which I will describe only one, from memory. He also may have written some MSJ / MSDN articles that are relevant.
Anyway, the basic idea is to cause the process that you want to load your DLL to issue a call to LoadLibrary. This is done using CreateRemoteThread with the address of LoadLibary for lpStartAddress and the address of a string naming your DLL in for lpParameter. Arranging for and locating the string is done using VirtualAllocEx to allocate some memory in the remote process, and WriteProcessMemory to fill it with the string.
PSEUDO CODE:
void InjectDllIntoProcess(DWORD processId, char *dllName)
{
HANDLE hRemoteProcess = OpenProcess(
// Assumes that dll and function addresses are the same in different processes
// on the same system. I think that this is true even with ASLR, only issue I
// can think of is to make sure that the source and target process are both 32
// or both 64 bit, not a mixture.
// Note that it is asking for the ASCII version
HMODULE hDll = LoadLibrary(_T("Kernel32.dll"));
void *loadLibAddr = GetProcAddress(hDll, _T("LoadLibraryA"));
// Inject the DLL name
char * remoteAddr =
(char *)VirtualAllocEx(hRemoteProcess, NULL, strlen(dllName) + 1, ...
WriteProcessMemory(hRemoteProcess, remoteAddr, dllName, strlen(dllName) + 1 ...
CreateRemoteThread(hRemoteProcess, ??, 0, loadLibAddr, remoteAddr, ...
}
Edit: I've figured out the constructor for the singleton is getting called multiple times so it appears the classes are getting loaded more than once by separate class loaders. How can I make a global singleton in Tomcat? I've been googling, but no luck so far.
I have a singleton object that I construct like thus:
private static volatile KeyMapper mapper = null;
public static KeyMapper getMapper()
{
if(mapper == null)
{
synchronized(Utils.class)
{
if(mapper == null)
{
mapper = new LocalMemoryMapper();
}
}
}
return mapper;
}
The class KeyMapper is basically a synchronized wrapper to HashMap with only two functions, one to add a mapping and one to remove a mapping. When running in Tomcat 6.24 on my 32bit Windows machine everything works fine. However when running on a 64 bit Linux machine (CentOS 5.4 with OpenJDK 1.6.0-b09) I add one mapping and print out the size of the HashMap used by KeyMapper to verify the mapping got added (i.e. verify size = 1). Then I try to retrieve the mapping with another request and I keep getting null and when I checked the size of the HashMap it was 0. I'm confident the mapping isn't accidentally being removed since I've commented out all calls to remove (and I don't use clear or any other mutators, just get and put).
The requests are going through Tomcat 6.24 (configured to use 200 threads with a minimum of 4 threads) and I passed -Xnoclassgc to the jvm to ensure the class isn't inadvertently getting garbage collected (jvm is also running in -server mode). I also added a finalize method to KeyMapper to print to stderr if it ever gets garbage collected to verify that it wasn't being garbage collected.
I'm at my wits end and I can't figure out why one minute the entry in HashMap is there and the next it isn't :(
Another wild guess: is it possible the two requests are being served by different copies of your web app? Each would be in its own ClassLoader and thus have a different copy of the singleton.
Have you tried removing the outer check
if(mapper == null)
{
Thereby always hitting the Synchronized point, it's subtle stuff but possibly you're hitting the double-checked locking idiom problem. Described here and in many other articles.
Must admit I've never seen the problem actually bite someone before, but this sure sounds like it.
With this solution, the JVM guarantees that it's only one mapper and that's it's initialized before use.
public enum KeyMapperFactory {
;
private static KeyMapper mapper = new LocalMemoryMapper();
public static KeyMapper getMapper() {
return mapper;
}
}
This may not be the cause of your problem but you are using the faulty double-checked locking. See this,
http://en.wikipedia.org/wiki/Double-checked_locking#Usage_in_Java
I found a rather poor fix. I exported my code as a JAR and put it in $TOMCAT/lib and that worked. This is clearly a class loader issue.
Edit: Figured out the solution
Ok, I finally figured out the problem.
I had made my application the default application for the server by adding a to server.xml and setting the path to "". However, when I was accessing it through the URL http://localhost/somepage.jsp for somethings, but also the URL http://localhost/appname/anotherpage.jsp for other things.
Once I changed all the URLs to use http://localhost/ instead of http://localhost/appname the problem was fixed.