I have a .Net Core application which is to run on Ubuntu. With it I'm looking to distribute 2 .so files that the application depends upon. The app would load those libraries via a [DllImport("mylibrary")] attribute.
If I try to run my application as-is, it complains that it cannot find mylibrary. This is because the library doesn't exist in the search path. If I install the libraries via apt-get instead of distributing them, everything works (the libraries end up in /usr/lib/x86_64-linux-gnu/ which is part of my search path for libraries).
This isn't a very scalable solution. I want something that just "works" when running dotnet myapplication.dll without having to pre-install via apt-get (or otherwise) my .so files.
How can I get the library loader to work with my local .so files without requiring some extra step by the end user before running my application?
The idea of having the .Net Core application running some installation / bash commands upon startup to setup the environment sounded like a decent idea except it requires sudo. This isn't a deal-killer, but isn't terribly clean either.
Any ideas?
Related
I have completed a small project that uses several libraries. the CMakeList.txt looks like this:
cmake_minimum_required(VERSION 3.5)
project(tf_ct_log C)
set(CMAKE_C_STANDARD 99)
include_directories(include /usr/local/include/hiredis /usr/include/openssl)
link_directories(/usr/lib/x86_64-linux-gnu)
set(HDR include/ct_logger.h)
add_executable(tf_ct_logger src/main.c src/ct_logger.c ${HDR})
find_package(OpenSSL REQUIRED)
find_package(Threads REQUIRED)
find_library(PostgreSQL REQUIRED)
find_library(jansson REQUIRED)
target_link_libraries(tf_ct_logger OpenSSL::SSL jansson pthread pq)
I would like to be able to build a package that can be installed in another machine, without downloading any dependencies. with ldd, I 've got all dependencies of the application and copied those files (libxyz.so...) into a subdirectory deps in my project. How can I create that package using those dependencies so that the end user will just use the object files of my project along with the dependencies libraries to create the executable?
It gets really hairy real quick when you need to create a native package for multiple flavors of installers (Debian, RH, Arch, etc.), especially if customization is involved.
If you just need a clean reproducible to get it on a box and run -- I would strongly suggest looking towards packaging it as a Docker container.
You start from some lightweight Linux distro container (Alpine is the latest trend), derive it into one with C and C++ runtime and anything else you depend on and call this "my prod container". From that you derive one with C++ compiler and debugger installed and call it "my dev container".
We actually wrote a little memo a while back, while making our open source hobby usable for others.
You will probably still need to clean up your CMake file to an extent that the "install" target works (mine is here).
I may have not expressed myself correctly. I just wanted to compile and build executables from my project, then find a way to copy it on another machine without having to install all dependencies before running the app.
The solution I found (with the help of a more experienced developer) was as follows:
1- Get all dependencies using ldd
2- Copy dependencies in a directory dependencies
3- On the target environment, copy content of dependencies into /usr/local/lib/myapp/
4- On the target environment, goto /etc/ld.so.conf.d/
5- create the file myapp.conf with one line in it: /usr/local/lib/myapp
6- run ldconfig
Then, the executable I created on my development machine runs here smoothly!
Of course all the steps I described up there must be listed in a script for automation
In short: This question is basically about telling Linux to load the development version of the .so file for executables in the dev directory and the installed .so file for others.
In long: Imagine a shared library, let's call it libasdf.so. And imagine the following directories:
/home/user/asdf/lib: libasdf.so
/home/user/asdf/test: ... perform_test
/opt/asdf/lib: libasdf.so
/home/user/jkl: ... use_asdf
In other words, you have a development directory for your library (/home/user/asdf) and you have an installed copy of its previous stable version (/opt/asdf) and some other programs using it (/home/user/jkl).
My question is, how can I tell Linux, to load /home/user/asdf/lib/libasdf.so when executing /home/user/asdf/test/perform_test and to load /opt/asdf/lib/libasdf.so when executing /home/user/jkl/use_asdf? Note that, even though I specify the directory by -L during link, Linux uses other methods (for example /ect/ld.so.conf and $LD_LIBRARY_PATH) to find the .so file.
The reason I need such a thing is that, of course the executables in the development directory need to link with the latest version of the library, while the other programs, would want to use the stable version.
Putting ../lib in the library path doesn't seem like a secure idea, not to mention not completely correct since you can't run the test from a different directory.
One solution I thought about is to have perform_test link with libasdf-dev.so and upon install, copy libasdf-dev.so as libasdf.so and have others link with that. This solution has one problem though. Imagine the following additional directory:
/home/user/asdf/tool: ... use_asdf_too
Which gets installed to:
/opt/asdf/bin: use_asdf_too
In my solution, it is unknown what use_asdf_too should be linked against. If linked against libasdf.so, it wouldn't work properly if invoked from the dev directory and if linked against libasdf-dev.so, it wouldn't work properly if invoked from the installed location.
What can I do? How is this managed by other people?
Installed shared objects usually don't just end with ".so". Usually they also include their soname, such as libadsf.so.42.1. The .so file for development is typically a symlink to a fully-versioned filename. The linker will look for the .so file and resolve it to the full filename, and the loader will then load the fully-versioned library instead.
This question already has answers here:
How to make exe files from a node.js app?
(20 answers)
Closed 7 years ago.
Supposed I have written a Node.js application, and I now would like to distribute it. Of course, I want to make it easy for the user, hence I do not want him to install Node.js, run npm install and then manually type node app.js.
What I'd prefer was a single executable file, e.g. an .exe file on Windows.
How could I approach this?
I am aware of this thread, anyway this is only about Windows. How could I achieve this in a platform-independent manner? Any ideas? Best practices? ...?
The perfect solution was a "compiler" I can give a source folder to. The source folder contains the app itself in various .js files, the node_modules folder and some metadata, such as the package.json. The output should be binaries for various platforms, such as Windows, OS X and Linux.
Oh, and what's important: I do not want to make any changes to the source code, so calls to require with relative paths should still work, even if this relative path is now inside the packaged app.
Any ideas?
PS: I do not want the user to install Node.js independently, it should be included inside the executable as well.
Meanwhile I have found the (for me) perfect solution: nexe, which creates a single executable from a Node.js application including all of its modules.
It's the next best thing to an ideal solution.
First, we're talking about packaging a Node.js app for workshops, demos, etc. where it can be handy to have an app "just running" without the need for the end user to care about installation and dependencies.
You can try the following setup:
Get your apps source code
npm install all dependencies (via package.json) to the local node_modules directory. It is important to perform this step on each platform you want to support separately, in case of binary dependencies.
Copy the Node.js binary – node.exe on Windows, (probably) /usr/local/bin/node on OS X/Linux to your project's root folder. On OS X/Linux you can find the location of the Node.js binary with which node.
For Windows:
Create a self extracting archive, 7zip_extra supports a way to execute a command right after extraction, see: http://www.msfn.org/board/topic/39048-how-to-make-a-7-zip-switchless-installer/.
For OS X/Linux:
You can use tools like makeself or unzipsfx (I don't know if this is compiled with CHEAP_SFX_AUTORUN defined by default).
These tools will extract the archive to a temporary directory, execute the given command (e.g. node app.js) and remove all files when finished.
Not to beat a dead horse, but the solution you're describing sounds a lot like Node-Webkit.
From the Git Page:
node-webkit is an app runtime based on Chromium and node.js. You can write native apps in HTML and JavaScript with node-webkit. It also lets you call Node.js modules directly from the DOM and enables a new way of writing native applications with all Web technologies.
These instructions specifically detail the creation of a single file app that a user can execute, and this portion describes the external dependencies.
I'm not sure if it's the exact solution, but it seems pretty close.
Hope it helps!
JXcore will allow you to turn any nodejs application into a single executable, including all dependencies, in either Windows, Linux, or Mac OS X.
Here is a link to the installer:
https://github.com/jxcore/jxcore-release
And here is a link to how to set it up:
http://jxcore.com/turn-node-applications-into-executables/
It is very easy to use and I have tested it in both Windows 8.1 and Ubuntu 14.04.
FYI: JXcore is a fork of NodeJS so it is 100% NodeJS compatible, with some extra features.
In addition to nexe, browserify can be used to bundle up all your dependencies as a single .js file. This does not bundle the actual node executable, just handles the javascript side. It too does not handle native modules. The command line options for pure node compilation would be browserify --output bundle.js --bare --dg false input.js.
There are a number of steps you have to go through to create an installer and it varies for each Operating System. For Example:
on Mac OS X you need to create a .pkg, there are instructions on how to do that here: https://coolaj86.com/articles/how-to-create-an-osx-pkg-installer.html
on Ubuntu Linux you need to create a .deb, there are instruction on how to do that here: https://coolaj86.com/articles/how-to-create-a-debian-installer.html
on Microsoft Windows you need to create a .exe or .msi, there are instruction on how do that using the innosetup installer here: https://coolaj86.com/articles/how-to-create-an-innosetup-installer.html
You could create a git repo and setup a link to the node git repo as a dependency. Then any user who clones the repo could also install node.
#git submodule [--quiet] add [-b branch] [-f|--force]
git submodule add /var/Node-repo.git common
You could easily package a script up to automatically clone the git repo you have hosted somewhere and "install" from one that one script file.
#!/bin/sh
#clone git repo
git clone your-repo.git
I have an application that relies on Qt, GDCM, and VTK, with the main build environment being Qt. All of these libraries are cross-platform and compile on Windows, Mac, and Linux. I need to deploy the application to Linux after deploying on Windows. The versions of vtk and gdcm I'm using are trunk versions from git (about a month old), more recent than what I can get apt-get on Ubuntu 11.04, which is my current (and only) Linux deployment target.
What is the accepted method for deploying an application that relies on these kinds of libraries?
Should I be statically linking here, to avoid LD_LIBRARY_PATH? I see conflicting reports on LD_LIBRARY_PATH; tutorials like this one suggest that it's the 'right way' to modify the library path to use shared libraries through system reboots. Others suggest that I should never set LD_LIBRARY_PATH. In the default version of GDCM, the installation already puts libraries into the /usr/local/lib directory, so those libraries get seen when I run ldd <my program>. VTK, on the other hand, puts its libraries into /usr/local/lib/vtk-5.9, which is not part of the LD_LIBRARY_PATH on most user's machines, and so is not found unless some change is made to the system. Copying the VTK files into '/usr/local/lib' does not allow 'ldd' to see the files.
So, how can I make my application see VTK to use the libraries?
On windows, deploying the dlls is very straightforward, because I can just include them in the installer, and the application finds them because they are in the local directory. That approach does not work in Linux, so I was going to have the users install Qt, GDCM, and VTK from whatever appropriate source and use the default locations, and then have the application point to those default locations. However, since VTK is putting things into a non-standard location, should I also expect users to modify LD_LIBRARY_PATH? Should I include the specific versions of the libraries that I want and then figure out how to make the executable look in the local directory for those libraries and ignore the ones it finds in the library path?
Every "serious" commercial application I have ever seen uses LD_LIBRARY_PATH. They invariably include a shell script that looks something like this:
#!/bin/sh
here="${0%/*}" # or you can use `dirname "$0"`
LD_LIBRARY_PATH="$here"/lib:"$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
exec "$0".bin "$#"
They name this script something like .wrapper and create a directory tree that looks like this:
.wrapper
lib/ (directory full of .so files)
app1 -> .wrapper (symlink)
app1.bin (executable)
app2 -> .wrapper (symlink)
app2.bin (executable)
Now you can copy this whole tree to wherever you want, and you can run "/path/to/tree/app1" or "/path/to/tree/app2 --with --some --arguments" and it will work. So will putting /path/to/tree in your PATH.
Incidentally, this is also how Firefox and Chrome do it, more or less.
Whoever told you not to use LD_LIBRARY_PATH is full of it, IMHO.
Which system libraries you want to put in lib depends on which Linux versions you want to officially support.
Do not even think about static linking. The glibc developers do not like it, they do not care about supporting it, and they somehow manage to break it a little harder with every release.
Good luck.
In general, you're best off depending on the 'normal' versions of the libraries for whatever distribution you're targetting (and saying you don't support dists that don't support recent enough versions of the lib), but if you REALLY need to depend on a bleeding edge version of some shared lib, you can link your app with -Wl,-rpath,'$ORIGIN' and then install a copy of the exact version you want in the same directory as your executable.
Note that if you use make, you'll need $$ in the makefile to get a single $ into the argument that is actually sent to the linker. The single qutoes are needed so the shell doesn't munge things...
Well, there are two options for deploying Linux application.
The correct way:
make a package for your app and for the libraries, if they are so special, that they can't be installed from standard repositories
There are two major package formats. RPM and DEB.
The easy way:
make a self-extract file that will install the "windows way" into /opt.
You can have libraries in the same directory as the executable, it's just not the preferred way.
I'm trying to build a Win32 DLL from an audio-DSP related Linux library (http://breakfastquay.com/rubberband/). There are makefiles and config scripts for Linux, but no help for Windows. The author provides a Win32 binary of a sample app using the library, and I see a number of "#ifdef MSVC" and "#ifdef WIN32" scattered around, so I don't think I'm starting completely from scratch but I'm stuck nevertheless.
As my programming knowledge in either platform is rather limited, I'd appreciate any help.
First of all, what is the right way to get started here? Visual Studio? Cygwin? Initially I started off creating a Win32 DLL project in Visual Studio, adding the source files, thinking about adding a .def file, etc, but at some point I felt like this was going nowhere.
As for Cygwin, this was the first time using it, and I don't even know if this is the sort of thing that Cygwin is designed for. Is it?
On Cygwin, I ran ./configure and got stuck at something like this:
"checking for SRC... configure: error: Package requirements (samplerate) were not met: No package 'samplerate' found"
After looking through the log, it appears that pkg-config is looking for samplerate.pc. How do I handle packages in Windows? libsamplerate is just an open source library, and I have source and a DLL for this. But I'm not sure how to use them to satisfy the dependency requirements for librubberband (which is what I'm trying to build)
I'm completely lost at this point and if anyone can give me a nudge in the right direction... and, is there an easier way to do this?
Many thanks in advance.
If you're still stuck on this I can throw a little light.
You may have to build everything from sources (or have the libraries installed in your environment). You're using Cygwin, I would recommend MinGW and MSYS too, but sometimes it's just not possible to use this combination to build the program or library.
So if using Cygwin, first ensure that you have a proper environment installed. This is that you have the correct development headers installed.
Then download libsndfile. Extract the sources to a directory and from the Cygwin bash shell navigate to that directory. There perform:
./configure
make
make install prefix=/cygdrive/c/cygwin
Notice that I use a prefix, that prefix should point to the directory Cygwin is installed in order to correctly install the libraries (the same happens to MinGW and MSYS, the prefix should point to the MinGW installation directory). Maybe using the usr directory in the prefix works too, I've never tried it.
Now download FFTW, as it will be needed for libsamplerate and rubberband. Same procedure as with libsndfile: extract, configure, make & make install using the prefix. Now copy the header files of FFTW (in the example they'd be in /cygdrive/c/cygwin/include) to the include directory in the usr directory (in the example /cygdrive/c/cygwin/usr/include).
Next SRC (libsamplerate), same procedure.
Then the Vamp plugin SDK. In order to compile the it you may need to edit the file src\vamp-hostsdk\PluginLoader.cpp, deleting RTLD_LOCAL from a dlopen() call (it's safe, it's already the default behaviour).
Also, you may need to install it by hand (in my experiences it didn't like the prefix). Or set the environmental variable PKG_CONFIG_PATH pointing to the paths of pkgconfig, e.g.:
set PKG_CONFIG_PATH=/cygdrive/c/cygwin/lib/pkgconfig:/usr/local/lib/pkgconfig
Now, create a file called ladspa.h in the include directory with the contents of the LADSPA header
Finally, configure and build rubberband, it should find everything it needs.
To build in MSYS using MinGW follow the same procedure, using the according prefix. Using Visual Studio is another alternative, but you may need to use some of the pre-built libraries (for example for libsndfile) as building Linux libraries natively in Windows may be complicated or even impossible (without hacking the source code) in VS.
Anyway, the autor of rubberband provides binaries; I think you should consider use them instead of going through all of this.
Linux to w32 is mostly a tricky thing.
For each of your dependencies, download the source and:
./configure
make
sudo make install
Also, I recommend you to use MinGW + msys in place of CygWin (as the latter produces executables that depend on its libraries). However in your situtation, use the VS approach -- 't will save you a lot of time.