I would like to have a specific Cygwin installation as one zip-file archive in order to deploy then on different Windows PCs. After investigating the problem the suggested way to perform this is "trial-and-error" Cygwin installation and packages choosing (What packages should I install with Cygwin to make it not bloated but also have everything I would need as a developer?)
May be more concrete requirement will help: I need minimal Cygwin installation + Perl + famous Perl libraries (Win32::API, etc) + GNU make + Gcc compilation tools. Which packages should I select when installing Cygwin?
I have been experimenting with Cygwin attempting to get a "bare-bones", minimal install. I do find that installing utilities like grep, gawk, sed and similar tools has dependencies on cygwin, base-Cygwin and sometimes unwanted tools like bash, coreutils etc.,
I wanted to get only the tools and their required dlls installed and started examining the Cygwin package. I discovered that not using the setup.exe supplied by Cygwin is an alternative way to accomplish minimal Cygwin installs.
And this is how I got it done.
Download only the packages you want from any of the Cygwin mirror
sites using ftp or http. Alternatively you can use the setup.exe
supplied by Cygwin to download all the packages - download only and
no install.
Once the download is successfully completed, individual
packages like zlib, gawk, grep, libiconv are found under the
x86/release or x86_64/release directory Each package is 'tar'red and
compressed using tool 'xz' or bzip and stored in respective
directories.
To install a specific tool like sed or gawk, all that
needs to be done, is to extract the tool executable and its
dependencies (.dll)
Before you attempt the following, please ensure you have a tool like 7z.exe, xz.exe, bzip2 or other that is capable of uncompressing an .xz or bzip archive
Installing gawk example below :
Extract gawk.exe from gawk-4.1.3-1.tar.xz archive using the command - 7z.exe e -so gawk-4.1.3-1.tar.xz | tar xvf -
Once that is done, you should find gawk.exe in a subfolder usually, usr/bin under the release/gawk folder
Find the dependencies for gawk - you can do this in a couple of ways.
Examine the Cygwin setup.ini file found in x86 or x86_64 folder.
Look the string '# gawk' and in the lines after this line you should find a "requires:" line that lists the dependencies.
Mine reads like this - "requires: bash cygwin libgmp10 libintl8 libmpfr4 libreadline7"
For gawk to run, bash is not a must since we have the windows command shell. (bash is included to get a few other dlls required by gawk. However, that causes a lot more unecessary files to be installed). The other dependencies contain files that gawk needs to run.
Extract each of the above packages using tools like 7z or xz into individual files.
After all the dependencies are extracted, copy your needed tool(s) (grep/sed/gawk) to a folder and all the dependent .dlls
You should now be able to run your tool with the minimum set of .dlls required in a bare-bones cygwin installation.
Caution : It may not be sufficient to just extract the dependencies listed in setup.ini for each tool. Sometimes, you may need to execute/run the tool to discover that there are more dlls required.
There are other means of finding out the dlls required by an .exe - you can use dumpbin from MS or dependency walker, ndepends or similar tools to find the list of dependent dlls
Consult - How do I detect the DLLs required by an application?
How do I find out which dlls an executable will load?
I also brute forced this dll info by just running the tool and installing the missing dlls listed one by one by extracting from the required packages.
When you run a tool and it errors out with a missing .dll message, search for the package that contains the dll here - https://cygwin.com/cgi-bin2/package-grep.cgi . Enter the full/partial name of the missing dll to find the name of the package containing the dll.
Eventually, I have ended up with a bare-bones cygwin install with only the tools and dlls that I need.
Example : gawk - gawk.exe and the following dlls - cygwin1.dll, cyggmp-10.dll, cygiconv-2.dll, cygintl-8.dll, cygmpfr-4.dll, cyggcc_s-seh-1.dll, cygncursesw-10.dll, cygreadline7.dll
sed - sed.exe and dlls - cygwin1.dll, cygintl-8.dll
Hope this is found useful. The Cywin installer also does dll re-basing, which I will not venture into here.
With Cygwin's dependence system, you should just make a new installation, unselect every package, and then select Perl, make and gcc, and it will install only the required packages for that. I think some of the dependent packages are just recommended and not required, but the Cygwin installer conveniently marks them as recommended.
Related
Consider these 4 pieces of software:
COBRA 2.05
LibSBML 5.10
MATLAB R2013a (Also known as 8.1, 64-bit; MATLAB no longer supports 32-bit Linux anyway)
A 64-bit Linux OS (such as Ubuntu 14.04 or the latest Mint but not restricted to them)
Intro
The COBRA toolbox is an optimization suite that runs on top of MATLAB aimed at the development of MATLAB code for metabolic network modelling. Such a "network" is a system of equations that can have a very large number of equations and variables (such as thousands). Therefore, routines to read and write those large models according to some format specification are a must-have, and COBRA uses the standard SBML for that.
Problem
Unlike the Windows versions, the Linux binary packages do not integrate well out-of-the-box: to begin with, the pre-compiled Linux binary of libSBML (open-source) available for download does not come with MATLAB support. If one tries to use the pre-compiled libSBML, COBRA won't find the "MATLAB bindings" and therefore won't be able to, for example, read and write SBML XML files from the disk in a m-script.
The question
What needs to be done to make COBRA 2.05 running on top of MATLAB R2013a under Linux (Ubuntu 14.04 or the latest Mint, but this is not likely distro-specific) able to read and write SBML XML files? In other words, what needs to be done system-wide to make COBRA pass its own testSBML test?
This is how I got it working and what I could learn from all the hassle regarding how my Linux box works. I hope I am not forgetting/missing/mistaking anything.
1. MATLAB
1.1. Install MATLAB
Install it in its default location (you will need root access for this), don't be stubborn like I tried to be. Why:
1.1.1. Integration
It is very likely you will want to install some other software that uses the MATLAB framework at some point, and in the real world software doesn't always find other software even if you know how to tell it where to look for.
1.1.2. Make your life easier
Even though it seems like a good idea to install a big software in a place where you have lots of available space and that you can use in multiple machines (specially in Linux, which doesn't have that abomination called Registry, and has symbolic links), that would only perhaps be a good idea - apart from item 1.1.1 - if that place is a partition with a Linux filesystem, since at some point, some executable/script will need execution permission, and mounting the entire partition with execution permission for all its files is rather unsafe. Therefore, do not put MATLAB in an NTFS partition of an external HD; perhaps creating a Linux partition in the external HD just for Linux-specific stuff could work for this matter, but how much hassle is that?
1.2. Setup MATLAB so people and other software can launch it
Even though I have seen somewhere that the MATLAB installer eventually shows an option to create symbolic links in the system path for convenience, it didn't in my case. But that is OK, since I would have to replace the symbolic link /usr/local/bin/matlab by the following shell-script (same path, same filename) anyway:
#!/bin/sh
export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH
exec /usr/local/MATLAB/R2013a/bin/matlab $*
OBS: That LD_LIBRARY_PATH is needed for MATLAB to find SBML bindings later and to be able to use them. Also, if you install some third-party solver such as TOMLAB, you will most likely need to add some more paths in this little launcher script.
OBS 2: In my case, the installation script didn't automatically create any launchers or shortcuts, but I have found an iconless and extension-less Matlab 8.01 file and a matlab icon as a png file, and that first file was a template .desktop file that I could edit to fit my needs and put in /usr/share/applications so the Unity Dash would find it. The contents of this Matlab.desktop file are:
[Desktop Entry]
Type=Application
Icon=/usr/local/MATLAB/R2013a/Matlab.png
Name=Matlab 8.01
Comment="Start Matlab 8.01"
Exec=/usr/local/MATLAB/R2013a/bin/matlab -desktop
Categories=Development;
Name=Matlab 8.01
GenericName=Matlab 8.01
Comment="Start Matlab 8.01"
2. libSBML
2.1. Install libSBML
libSBML is provided by a deb package specific for Ubuntu (and for CentOS), and also versions for several flavours of Windows and MacOSX (their home page: http://sbml.org/Software/libSBML). Guess which is the only platform whose binaries weren't compiled with MATLAB support? Linux, of course. That means you will need to compile from source (and that the deb package is therefore useless to you). To compile:
2.1.1. Install dependencies
The dependency libxml2-dev (from software manager or from a terminal):
sudo apt-get install libxml2-dev
2.1.2. Save yourself some time in the future
Usually, one would do configure, make and then make install. But this is not recommended for the same reason as installing anything that doesn't come in a pretty little package: you will loose control of which files went where, and will need to keep the source-code to be able to run make uninstall if you need to uninstall it later. So, install checkinstall and use it to replace the step make install, since checkinstall creates a package for your system that can be later uninstalled or reinstalled just as any regular packaged software (from software manager or from a terminal):
sudo apt-get install checkinstall
2.1.3. Configure the compiling-process
Get LibSBML source code and extract it to some folder. From a terminal, navigate to that folder and configure the compilation:
./configure --with-matlab
OBS: with the with-matlab flag, the configure script will fail it it cannot find an executable whose filename is matlab. If it fails, it outputs that the matlab file could not be found, but the test it performs is actually both for the existence of the file and whether it is executable. So, if the file is in an NTFS partition, configure will fail even if it finds the file, but will tell you the file couldn't be found. You can enforce it to look for the executable in /path/to/matlab/root by passing (it will look for a bin folder inside that path, and for the executable inside that bin folder):
./configure --with-matlab=/path/to/matlab/root
OBS: This will install libSBML in the default location: /usr/local/lib. Again, it is a good idea to just let it install in its default location, but if you need to change it, you can pass the path with the flag: --prefix=/your/installation/path
OBS 2: You might ask why libSBML needs to find and execute matlab to be compiled with support for it: it needs to fire up MATLAB later to build MEX-files (compiled MATLAB code), so I would speculate you wouldn't be able to install libSBML after all if your MATLAB has no compiler to generate MEX-files.
2.1.4. Build and install libSBML
make
checkinstall
VERY IMPORTANT OBS:
I) checkinstall asks for confirmation of the metadata of the package it is about to create. In my case, the string for the version field came by default as "Source" (without the quotes), which caused checkinstall to fail because dpkg (the system tool that actually builds the deb file) failed complaining the version number must start with, well, a number. So, save yourself some time and make sure the string in the version field starts with a number (i.e. "5.10", without the quotes obviously)
II) checkinstall asks if you want to exclude from the future package files that the make install command would put in your home folder and says it is a good idea to exclude them. LibSBML has a test.xml file that it needs to be in the $HOME folder later, or else it won't let you integrate with MATLAB. And even though it tells you a test.xml is missing, it doesn't tell you where that file should be or if that file was something that came with the library. I only noticed it because checkinstall had found a $HOME/test.xml reference earlier (that I excluded from the package, of course) and I had found that odd. So, save yourself some time and exclude $HOME/test.xml from the package generated by checkinstall, and then search for test.xml inside the source-code folder and copy it to $HOME as soon as libSBML finishes being installed by checkinstall.
2.2. Integrate libSBML to MATLAB
Fire up MATLAB, navigate to where the SBML MATLAB-bindings were installed in step 2.1.5 (in my case: /usr/local/lib) and run the file installSBML.m that should be there.
2.2.1. Shared libraries problems
In my case, I had an error due to an old unresolved issue: libstdc++.so.6 not having a reference to GLIBCXX_3.4.15. Turns out that MATLAB was trying to use a libstdc++.so.6.0.13 (libstdc++.so.6 was a symbolic link pointing to this file) that came with it in /usr/local/MATLAB/R2013a/sys/os/glnxa64, which indeed didn't have that reference (one could verify that by issuing:
strings /usr/local/MATLAB/R2013a/sys/os/glnxa64/libstdc++.so.6.0.13 | grep GLIBC
). My system has a libstdc++.so.6.0.19 located in /usr/lib/x86_64-linux-gnu that has that reference, so I enforced MATLAB to use 6.0.19 one by setting the LD_LIBRARY_PATH properly (refer to step 1.2) and also by renaming the libstdc++.so.6 that came with MATLAB to something else so it would not find it and would keep looking until it found my system's. A friend of mine running Linux Mint didn't need to rename anything: for him, setting the LD_LIBRARY_PATH was enough.
2.2.2. Other problems
installSBML.m will fail if it doesn't find that $HOME/test.xml file mentioned in step 2.1.5, regardless of whether the library actually works. It assumes that if it could not test itself using a file that it assumes to be in $HOME, the user shouldn't have the option to install it anyway.
3. COBRA / SBML toolbox
3.1. Setup COBRA
In MATLAB, navigate to <YOUR_COBRA_ROOT_FOLDER_HERE>/external/toolboxes/SBMLToolbox-4.1.0/toolbox and run the file install.m there. You should have all set so it finds the MATLAB-bindings you set up in step 2.2.
3.2. MATLAB setpaths problems
I had to manually edit the file /usr/local/MATLAB/R2013a/toolbox/local/pathdef.m as root to include the folder /usr/local/lib (where libSBML and its MATLAB-bindings are) to make that setting persistent. Every time I restarted MATLAB, its setpath had gone back to the default, no matter if I started MATLAB as root when setting its setpath via the MATLAB GUI.
3.3. Test
Now you have hopefully connected all the dots. Try it: in MATLAB, navigate to <YOUR_COBRA_ROOT_FOLDER_HERE> and issue:
initCobraToolbox
testAll
If you haven't got any third-party solvers installed and configured, it should pass 14 of the 19 tests, including the SBML test (testSBML). Now you can load SBML files into MATLAB and play with them.
I also needed to add a symbolic link from /usr/local/lib/libsbml.so.5 to the MATLAB sys folder by:
sudo ln -s /usr/local/lib/libsbml.so.5 /usr/local/MATLAB/R2014a/sys/os/glnxa64/
This finally made the installation possible.
I installed using Cmake. To do this it is necessary to find the FindMatlab.cmake in the source package and insert the MATLAB path manually!
.............
elseif(EXISTS "/Applications/MATLAB_R2008a.app/")
set(MATLAB_ROOT_PATH "/Applications/MATLAB_R2008a.app/")
endif()
else()
if (EXISTS "/usr/local/MATLAB/R2014a/")
set(MATLAB_ROOT_PATH "/usr/local/MATLAB/R2014a/")
endif()
endif()
..........
FYI, to resolve the shared library issue at point 2.2.1 I needed to install the package matlab-support (in Ubuntu repositories)
I'm zipping a pre-built (no source/object files) binary application for distribution. The binary application requires a couple of libraries not included by default. The only way I seem to be able to get the application to start on the end-user is by including a run.sh that sets the library path to the current directory:
export LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH
./MyApp.out
However, I'd really like to allow the user to just unzip the zip and doubleclick MyApp.out (without the shell script). Can I edit MyApp.out to search the current directory for the library? I've done something similar on OSX using install_name_tool, but that tool isn't available here.
You want to set the rpath. See this answer. So link using
gcc yourobjects*.o -L/some/lib/dir/ -lsome -Wl,-rpath,.
But you might want even to use -Wl,-rpath,$PWD or perhaps -Wl,-rpath,'$ORIGIN'. See this.
You could also (and this should work for a pre-built executable) configure your /etc/ld.so.conf by adding a line there with an absolute path (of the directory containing the lib), then running ldconfig -v ... See ldconfig(8)
I would suggest adding /usr/local/lib into /etc/ld.so.conf and making a symlink from /usr/local/lib/libfoo.so to e.g. $HOME/libfoo.so etc... (then run ldconfig ...). I don't think adding a user specific directory to /etc/ld.so.conf is reasonable ...
PS. What you really want is to package your application (e.g. as a *.deb package for Debian or Ubuntu, or an *.rpm for Fedora or Redhat). Package management systems handle dependencies!
I have been trying to do build and use OpenCV 2.3.0 on my Fedora15 Lovelock 64bit machine.
Background:
First, on my 64bit Fedora15, OpenCV2.2.0 seems to be in the locations namely
/usr/share/opencv
/usr/doc
/usr/lib64 &
/usr/bin
I do not find the include files though (in /usr/include). This means that the development package was n t installed. My package manager does not list the development packages when i try to Add/remove software.
I have a need to create applications, some of which just link to 2.2 and others which link to 2.3.O of the OpenCV library.So, I thought the best solution would be to have a separate location for 3rd party libraries that i use for my development . So I created a directory in /local named soft and created an OpenCV directory. A directory structure like this one.
/local/soft/
OpenCV/
OpenCV2.2.0/
source-files
build
OpenCV2.3.0/
source-files
build
installation
share/opencv
doc
include
lib
Now, i tried building OpenCV2.3.0 and i succeeded. I configure CMake to use CMAKE_INSTALL_PREFIX to the directory named "installation" (see above), instead of the default /usr/local/. Clean. huh?
I tried building and installing OpenCv 2.2.0 in the same way. Alas 2.2.0 complains something during the build. So i thought i ll link to the already existing version in the standard locations. BUT, when i try to install the dev packages for 2.2 using my package manager,the development files for x86_64 are not found :-) which means i dont have the headers to link to the libraries in the standard location.
I cant build my executable since linker ld would not find the OpenCV that i have installed in the non-standard location.(although i point it to the exact location using the -L and -l options with gcc in my Eclipse).
Question 1: Am i doing the right thing in maintaining installations in non-standard locations? Is /usr/ the standard location where the package manager will always do the installation?
Question2 : What is the right way of linking to these libraries installed in non-standard locations? Why would not ld recognize my .so files in the lib folder?
sudo g++ logpolar.cpp -o logpolar.o -I /local/soft/OpenCV/opencv2.3.1/installation/include/ -l/local/soft/OpenCV/opencv2.3.1/build/lib/libopencv_core.so
But ld canot find -l/local/soft/OpenCV/opencv2.3.1/build/lib/libopencv_core.so
I checked the lib folder and there sure is a beautiful symbolic link to libopencv_core.so.2.3
The standard approach is to use /usr/local directory structure that already has predefined paths like /usr/local/bin, /usr/local/sbin, /usr/local/include, /usr/local/lib.
You put your software here and everything will JustWork(TM). Every Linux distro (incl. Fedora) is set up so it will load programs (libraries, headers) from this libraries.
If you would use GNU toolchain (autoconf, automake => autotools) you would be fine. With CMake you probably need to setup paths for /usr/local/include and /usr/local/lib.
On the other hand this approach wont let you use multiple versions. You can only have one. The one in /usr/local overrides the system one (installed in /usr/bin) because these paths goes first.
You can keep your approach, it is nothing incorrect. We usually put such a software in the /opt folder, so you would go for /opt/opencv/X.Y where X.Y are the version numbers.
Q2: Read the gcc man page and search for the -L option. You need something like:
gcc ... -I/opt/opencv/2.0/include -lsystem_lib -L/opt/opencv/2.0/lib -lopencv ... ...
Do not forget to set LD_LIBRARY_PATH when running programs in multiple versions to properly load correct version:
LD_LIBRARY_PATH=/opt/opencv/2.0/lib /opt/opencv/2.0/bin/opencv
I have a libfoo-devel rpm that I can create, using the trick to override _topdir. Now I want to build a package "bar" which has a BuildRequires 'libfoo-devel". I can't seem to find the Right Way to get access to the contents of libfoo-devel without having to install it on the build host. How should I be doing it?
EDIT:
My build and target distros are both SuSE.
I prefer solutions that don't require mock, since I believe SuSE does not include it in its stock repo.
Subsequent EDIT:
I believe that the answer I seek is in the build package. Perhaps it's SuSE's answer to mock? Or it's the distributed version of the oBS service?
DESCRIPTION
build is a tool to build SuSE Linux
RPMs in a safe and clean way. build
will install a minimal SuSE Linux as
build system into some directory and
will chroot to this system to compile
the package. This way you don't risk
to corrupt your working system (due to
a broken spec file for example), even
if the package does not use BuildRoot.
build searches the spec file for a
BuildRequires: line; if such a line is
found, all the specified rpms are
installed. Otherwise a selection of
default packages are used. Note that
build doesn't automatically resolve
missing dependencies, so the specified
rpms have to be sufficient for the
build.
Note that if you really don't need libfoo-devel installed to build package bar the most sensible alternative would be to remove libfoo-devel from the BuildRequires directive (and maybe put the requirement where it belongs).
However, if you cannot do that for some reason, create a "development" rpm database. Basically it involves using rpm --initdb --root /path/to/fake/root. Then populate it with all of the "target packages" of your standard distro installation.
That's a lot of rpm --install --root /path/to/fake/root --justdb package-name.rpm commands, but maybe you can figure out a way to copy over your /var/lib/rpm/* database files and use those as a starting point. Once you have the alternative rpm database, you can fake the installation of the libfoo-devel package with a --justdb option. Then you'll be home free on the actual rpm build.
If neither mock nor the openSUSE Build Service are a viable choice then you will have to buckle down and install the package, either directly or in a chroot; the package provides files that the SRPM packager has decided are required to build, and hence is in the BuildRequires tag.
I'm trying to build a Win32 DLL from an audio-DSP related Linux library (http://breakfastquay.com/rubberband/). There are makefiles and config scripts for Linux, but no help for Windows. The author provides a Win32 binary of a sample app using the library, and I see a number of "#ifdef MSVC" and "#ifdef WIN32" scattered around, so I don't think I'm starting completely from scratch but I'm stuck nevertheless.
As my programming knowledge in either platform is rather limited, I'd appreciate any help.
First of all, what is the right way to get started here? Visual Studio? Cygwin? Initially I started off creating a Win32 DLL project in Visual Studio, adding the source files, thinking about adding a .def file, etc, but at some point I felt like this was going nowhere.
As for Cygwin, this was the first time using it, and I don't even know if this is the sort of thing that Cygwin is designed for. Is it?
On Cygwin, I ran ./configure and got stuck at something like this:
"checking for SRC... configure: error: Package requirements (samplerate) were not met: No package 'samplerate' found"
After looking through the log, it appears that pkg-config is looking for samplerate.pc. How do I handle packages in Windows? libsamplerate is just an open source library, and I have source and a DLL for this. But I'm not sure how to use them to satisfy the dependency requirements for librubberband (which is what I'm trying to build)
I'm completely lost at this point and if anyone can give me a nudge in the right direction... and, is there an easier way to do this?
Many thanks in advance.
If you're still stuck on this I can throw a little light.
You may have to build everything from sources (or have the libraries installed in your environment). You're using Cygwin, I would recommend MinGW and MSYS too, but sometimes it's just not possible to use this combination to build the program or library.
So if using Cygwin, first ensure that you have a proper environment installed. This is that you have the correct development headers installed.
Then download libsndfile. Extract the sources to a directory and from the Cygwin bash shell navigate to that directory. There perform:
./configure
make
make install prefix=/cygdrive/c/cygwin
Notice that I use a prefix, that prefix should point to the directory Cygwin is installed in order to correctly install the libraries (the same happens to MinGW and MSYS, the prefix should point to the MinGW installation directory). Maybe using the usr directory in the prefix works too, I've never tried it.
Now download FFTW, as it will be needed for libsamplerate and rubberband. Same procedure as with libsndfile: extract, configure, make & make install using the prefix. Now copy the header files of FFTW (in the example they'd be in /cygdrive/c/cygwin/include) to the include directory in the usr directory (in the example /cygdrive/c/cygwin/usr/include).
Next SRC (libsamplerate), same procedure.
Then the Vamp plugin SDK. In order to compile the it you may need to edit the file src\vamp-hostsdk\PluginLoader.cpp, deleting RTLD_LOCAL from a dlopen() call (it's safe, it's already the default behaviour).
Also, you may need to install it by hand (in my experiences it didn't like the prefix). Or set the environmental variable PKG_CONFIG_PATH pointing to the paths of pkgconfig, e.g.:
set PKG_CONFIG_PATH=/cygdrive/c/cygwin/lib/pkgconfig:/usr/local/lib/pkgconfig
Now, create a file called ladspa.h in the include directory with the contents of the LADSPA header
Finally, configure and build rubberband, it should find everything it needs.
To build in MSYS using MinGW follow the same procedure, using the according prefix. Using Visual Studio is another alternative, but you may need to use some of the pre-built libraries (for example for libsndfile) as building Linux libraries natively in Windows may be complicated or even impossible (without hacking the source code) in VS.
Anyway, the autor of rubberband provides binaries; I think you should consider use them instead of going through all of this.
Linux to w32 is mostly a tricky thing.
For each of your dependencies, download the source and:
./configure
make
sudo make install
Also, I recommend you to use MinGW + msys in place of CygWin (as the latter produces executables that depend on its libraries). However in your situtation, use the VS approach -- 't will save you a lot of time.