I'm working on an open source Haskell package and I want to use a proprietary package of mine as a dependency without leaking its source code.
One way would be to compile it to a binary and call it via System.Process.callCommand, but this would be unnecessarily inefficient.
Is there another way to distribute the package in a binary format or at least strongly obfuscated?
Related
I am trying to build a shared Haskell library that is used by a C project afterwards. I am on a linux platform so my question is from that context.
Suppose I have a haskell package foo with a library named foo, say version 0.1 which exports some functions via ffi.
I can easily generate a shared library (.so) that I can then link with, but my issue is that the generated library is named libHSfoo-0.1-$COMPONENT_ID.so which makes it quite cumbersome to link with since $COMPONENT_ID is unpredictable as far as I can tell.
The $COMPONENT_ID comes, to the best of my knowledge from the following Cabal structure and it looks like I could write cabal hooks to at least copy the generated shared library, or create a symbolic link to it from a fixed location.
I am wondering whether there is a better way to specify the component-id to get an easily predictable name of the shared library without post-processing?
It seems like I can achieve this if in the configure hook I set the configArgs to just the library component, and the configCID to my desired name of the library, but that seems like a fragile solution and I am thinking there is a better way for this.
The name of the library also affects linking when there are other Haskell packages dependent on this one, which would make it even more convenient to specify/override the name.
I am using stack to drive cabal, if that is relevant.
In order to reduce the executable size of a Rust program (called runtime in my code), I am trying to compress it and then include it in a second program (called szl) that decompresses it and executes it.
I have done that by using a Cargo build script in szl that opens the output binary from runtime, compresses it, and then generates a file that is ready for use by include_bytes!.
The issue with this approach is the dependencies are not handled properly. For example, Cargo may try to build szl before runtime (and fail), and when the source code of runtime is modified, szl is not rebuilt.
Is there a way to tell Cargo that szl depends on the binary from runtime (and transitively on the source code of runtime), or should I use another approach such as an external Makefile?
While not exactly your use case, you might get it to work with the links manifest key. It would allow you to express a dependency between the two programs and you can pass more information with DEP_FOO_KEY variables.
Before you go to such drastic measures, it might be worth it to try other known strategies for reducing rust binary size (such as calling strip, remove debug symbols, LTO, panic=abort) etc.
How does one find and understand excess data dependencies in a Haskell program so that one is able to eliminate them?
I once used ghc-vis to investigate data dependencies in a Haskell program but since Stack has moved on such that ghc-vis no longer installs in unison with most current development it's no longer an option and I wonder what do people use these days instead.
Try to fix ghc-vis (or actually, its dependencies).
From the logs you reported on the ghc-vis issue tracker https://github.com/def-/ghc-vis/issues/24, the errors all belong to these two categories, neither of which requires expertise specific to the broken packages, so you should be able to fix them yourself, that's the beauty of open source:
Failed to load interface... There are files missing: this might be related to your Haskell distribution. How did you install Haskell? For example Haskell packages on Arch are dynamically linked: https://wiki.archlinux.org/index.php/Haskell
Ambiguous occurence: at least one package you depend on exports a name which clashes with the actually intended name. Look at the broken package and fix its version bounds or fix its imports.
At this point, the problems you are encountering have little to do with ghc-vis, but with wl-pprint-text, polyparse, and cairo.
Suppose one is using the stackbuild tool to make a Haskell library (importing packages from Hackage, and so forth) to be used with a C/C++ project in which main is located in C/C++.
Supposing your project is named Lib.hs (which uses external libraries from hackage), is there a way to use stack to export your Lib.o, Lib.hi, and Lib_stub.h to be consumed by a C/C++ compiler like gcc or g++?
EDIT: A related question might be: "how can one use Stack as a build tool to be used with a Haskell & C/C++ project in which main is located in C/C++?
EDIT2: Upon reflection, one way to solve this problem would be to use Stack as usual, but migrate your C/C++ main function to Haskell. Is this the best way to do it? Are there huge performance costs to this or anything I should be aware of?
Stack can't really do this on its own.
There's support for generating so called "foreign libraries" added to Cabal, but it's not in a released version, yet. See commit 382143 This will produce a shared library that dynamically links against the dynamic versions of each Haskell package used.
You can build your package with stack and then after the fact you can assemble a single native library. In the Galua project we do this with a custom Setup.hs and a separate linking script.
The result of this linking process is that you get a standalone statically linked library suitable for inclusion in a C project: libgalua.a.
Do note that for creating standalone libraries on Linux suitable for being linked into a shared library that you'll need to recompile GHC to generate PIC static libraries (macOS does this by default).
Here is my desired use case:
I have a package with a single module that reads HDF5 files and writes some of their data to Haskell records. To do the work, the library uses the bindings-hdf5 package. Here is my cabal's build-depends. reader-types is a module I wrote that defines the types of the Haskell records that contain the read-in data.
build-depends: base >=4.7 && <4.8
, text
, vector
, containers
, bindings-hdf5
, reader-types
Note that my cabal file does not currently use extra-libraries or ghc-options. I can load my module, src/Mabel.hs in ghci as long as I specify the required hdf5_hl library:
ghci src/Mabel.hs -lhdf5_hl -L/long/nixos/path/lib
and within ghci, I can run my function perfectly fine.
Now, what I want to do is compile this library/module into a single, compiled file that I can later load with the GHC API in a different Haskell program. By single file, I mean that it needs to run even if the hdf5_hl library does not exist on the system. Preferably, it would also run even if text, vector, and/or containers are missing, but this is not essential because reader-types requires those types anyway. When loading the module with the GHC API, I want it to load in already compiled form, and not run interpreted.
My purpose for doing this is that I want the self-contained file to act as a single, pre-compiled plugin file that is later loaded and executed by a different Haskell executable. Other plugins might not use hdf5 at all, and the only package they are guaranteed to use is reader-types, which essentially defines the plugin interface types.
The hdf5 library on my system contains the following files: libhdf5_la.la, libhdf5_hl.so, libhdf5.la, libhdf5.so, and similar files that have the version number in the file name.
I have done a lot of googling, but am getting confused by all the edge cases I am finding. Here are some examples that I'm either sure don't fit my case, or I can't tell.
I do not want to compile a Haskell library to use from C or Python, only a Haskell program using GHC API.
I do not want to compile C wrappers for a C++ library into a Haskell module because the bindings already exist and the library is already a C library.
I do not to want compile a library that is entirely self-contained because, since I am loading it with the GHC API, I don't need the GHC runtime included in the library. (My understanding is that the plugins must be compiled with the same ghc version they will be loaded with in the GHC API).
I do not want to compile C bindings and the C library at the same time because the C library is already compiled and the bindings are specified in separate package (bindings-hdf5).
The closest resource for what I want to do is this exchange on the mailing list from 2009. However, I added extra-libraries: hdf5_hl or extra-libraries: hdf5 to my cabal file, and in both cases the resulting .a, .so, .dyn_hi, .dyn_o, .hi, and .o files in dist/build are all the exact same size as without using extra-libraries, so I'm confident it is not working correctly.
What changes to my cabal file do I need to make to create a self-contained, standalone file that I can later load with the GHC API? If this is not possible, what are the alternatives?
Instead of using the GHC API, I am also open to using the plugins library to load the plugin, but the self-contained requirements are still the same.
EDIT: I do not care what form the compiled "plugin" must take (I assume object file is the right way), but I want to load it dynamically from an separate executable at run time and execute functions it defines with known names and known types. The reason I want a single file is that there will eventually be other different plugins, and I want them all to behave the same way without having to worry about lib paths and dependencies for each one. A compiled, single file is a simpler interface for doing this than zipping/unzipping archives that include Haskell object code and their dependencies.