We have a Linux server application that is comprised of a number of open-source tools as well as programs we've written ourselves. Ideally we would like to be able to install this application on any common Linux distribution.
In the past, we've written perl scripts to automate installs of this application. Unfortunately, due to idiosyncrasies of different Linux distros, the logic inside these install scripts gets horribly complex, and can change as new versions of each supported distro are released. Maintaining the installer thus becomes one of the most time-intensive parts of the project!
I'm looking for assistance, be it a framework, documentation, code samples, that can make this process less painful. Here are the types of things our installer needs to do:
Create user/group accounts
Create directory trees with specific ownership and permissions
Install open-source applications, potentially compiling them from source during install
Insert pre-compiled binaries, scripts, config files, and docs into specific directories
Register init-type startup and shutdown scripts
Generate encryption keys
Verify connectivity to a central server
Instead of the installer approach, I think a better way than having a single script that does it at install time is to have a build system which generates .deb or .rpm files suitable for installation on each system you have to support.
A poor man's way of going at that might be to use checkinstall, which creates packages from the files installed via 'make install'. So you'd build your app on each system and have the package magically created in the distro's native format.
I believe that most of the tasks which you describe are fairly standardized between Linux distros. In my experience, the following should work the Debian family (including Ubuntu) and the Red Hat family (including Fedora and CentOS):
Create user / group accounts - adduser command
Create directory trees - mkdir or install, or just expand a tarball
Install open source applications - Unless you have particularly esoteric needs, this should probably be left to the distro's package manager.
Install files - install, or just expand a tarball
Startup and shutdown scripts - install to /etc/init.d then symlink to /etc/rc*.d
VMware Server is freely available for Linux and does most of the tasks which you describe. It uses Perl and maybe shell for its installation and configuration, so you might see the approach that it takes.
However, speaking as a Linux admin, I strongly prefer applications that integrate with my package management system. In other words, create .deb and .rpm files, as Vinko Vrsalovic suggested. Building packages is extremely well documented:
Building RPMs for Fedora (or Red Hat or CentOS): draft documentation, RPM Guide
Building .debs for Debian (or Ubuntu): Debian Maintainer's Guide
I tried Autopackage a few years ago, don't know how universal it is but worked quite well (was the only truly universal way back then). Surely you have to provide some LSB-compatible ways of setting up proper directories on your own, but this piece of software should help you.
Though there's probably still too much diversity among linux distributions to do everything in a completely platform-agnostic way but I may be wrong.
You may want to try BitRock InstallBuilder. It is a cross platform installation tool that allows you to do exactly what you are looking for (adding users, installing services, install pre-compiled binaries, etc). Although some of the other posts mention a number of tools that you could use in your scripts, the problem is that every Linux distribution is a bit different and simple tasks like adding an user or installer a service are suddenly non-trivial when you need to do them across Debian, Ubuntu, Mandriva, RedHat, Gentoo, etc. A good cross platform installer should isolate you from all that. Many commercial open source companies like MySQL, SugarCRM, Zenoss, Jaspersoft, Groundwork etc. have built installers based on our technology exactly because of that (in addition to their regular source code tarballs, etc.) We also provide free licenses for open source projects.
Autopackage now merged with Listaller project. Documentation's not really thorough yet but seems to be working.
Related
I started to learn wxWidgets some days ago and I found it really good, but when I tried to "export" it to another platform, it didn't work. I expected that, but when I searched how to distribute wxWidgets apps, I didn't find that much, except to install all the wxWidgets library on that platform.
I use Debian 10, while the other platform where I need to export my app uses Lubuntu. It says that it needs a wxgtk3.0 library, but I guess that library will need even more libraries.
How do I export my wxWidgets application without asking the user to manually install and build all the library on its platform? Maybe even using .deb package (if really needed). Thanks in advance.
Please understand that every single distribution uses it own format comparing to Windows where there is an Installation Wizard or OSX where there is a Bundle.
Now, you can create a deb file where you set everything up.
So you can create an rpm (which is basically the same thing as deb, just for different distribution).
This is the best way as it ensures that all dependencies and their versions are satisfied.
Hope this helps!
Many Linux distributions have wxWidgets packages, so one possibility would be to simply ask users to install these packages when they need to use your application. This is not really different from installing GTK libraries or even X11 (or Wayland) that your application also depends on -- the only difference is that these libraries are almost surely already installed on any Linux desktop system, while wxWidgets ones might not be.
Another alternative is to link your application statically with wxWidgets libraries. This will make it much bigger and will prevent the users from upgrading the libraries on their system to improve the application behaviour, but can be simpler for the users to install. Note that if you choose the static linking route, you typically need to build your application on the oldest distribution you want to support (which is probably not Debian 10, which is relatively recent), as this is a simple way to ensure that it doesn't require newer versions of the (other, non-wx) libraries than the ones already present on the user's system.
I have developed a small application in Qt Creator on Ubuntu 12.04 which I want should run on any other linux distro (mostly different versions of CentOS and ubuntu), just like any portable application on windows does.
I want to be able to simply share the binary file of the Application, and run the application.
I am able to successfully do this in windows, by just building the project in QT Creator and then putting the required libraries in the Application directory and then transfering them to other windows systems.
I searched all over and found out that I should be trying to build the project using LSB(Linux Standard Base) Compatibility, so that it runs on other linux distros. Is that the right way to do this?
I am very new to Qt and also to Linux (dont know much of Shell Scripting).
Thus, I dont know how I should proceed to make the Application LSB Compliant.
I have refered to, the following links:
Distributing Qt-based binaries on Linux and
Deploying Qt applications on Linux but have not beem able to understand what I am suposed to do.
I also found this question here which states a very similar situation as mine, but because I am a novice, I dont know how I should do this.
Moreover, considering that the first two articles were written 6 years back, shouldn't there be a simpler way to deploy Qt apps on the linux platform now?
I also saw something about static linking, is that the way to go?
Isn't there a way by which all of this can be done through Qt Creator itself?
If there is no hope of creating a portable Qt Application for Linux, then is there a way, say a shell script or something that would combine all the steps required to compile the Qt project on another computer and run it. Say, download Qt-SDK if not present, run qmake and make and then the newly compiled application, if not already there, so that the user can run the program just by running one script.
Your problem here is not the Linux Standard Base, but rather the presence or not of the specific version of Qt you need (or a later one).
Exactly like on a Windows machine, a user may have any of Qt installed, or they may not have it at all. On Windows it is easier to check for the presence of a certain version of Qt than it is on Linux, thus it is easier to write install tools that automate the experience.
To solve your problem there are a few ways:
Inform the user that your program requires a certain version of Qt or higher, and let the user handle the problem
Learn how to create packages for every distribution you want to target and create specific packages
Use a program like 0Install or Elf Statifier to create a package/executable containing all the necessary libraries.
The latter is similar to what many Windows and Mac programs do (they include every library they need within the installer), but it is not the preferred way on Linux, which relies heavily on shared libraries.
Making a binary application compatible with any other Linux distro is practically impossible since you will never know in advance which libraries are available in distro X, or what version of that library is available. Even among a single distro (e.g. Ubuntu), binary application are almost never backward-compatible, since anything built on Ubuntu 12.04 will have dependencies on versions libraries which are installed on that version of Ubuntu, and trying to run that binary on Ubuntu 10.04 will most probably fail simply because it doesn't have a recent enough version of glibc or some other necessary library.
However, the idea can be much more implementable if you limit yourself to a finite list of distros and versions of those distros. You can then know which libraries are available for those distros, and aim for the lowest common denominator. I used to maintain a binary application which had to support several distros (Ubuntu, Fedora, OpenSUSE, SLED, Mandriva), and the way I would do it is install the oldest distro I was targeting on my build machine. That way, the binary application would be linked to the oldest versions of the libraries available on those distros. Unless there's a new major version of such a library (which happens quite rarely, and even then, distros usually distribute the previous major version for a while for compatibility purposes), your compiled binary will then be compatible with all your targeted distros.
Therefore, the quick piece of advice I would give for your situation, use the oldest LTS version of Ubuntu which is still supported (10.04 at the moment) for your development, and you should be pretty safe for most recent popular distros. For the application you already developped on Ubuntu 12.04, you should have no problem simply recompiling the same source on 10.04. Understand that you will never however achieve 100% compatibility with a compiled C++ Qt application.
If Qt is not all that important to you, you could use a higher-level or interpreted language such as Python, Java, Perl or Ruby. With such languages, you can usually count on the language implementation already being installed on the target distro.
Deploy an application in Linux is a nightmare, luckily there are some solutions. Check this projects to build a portable binary with all their dependencies bundled:
http://statifier.sourceforge.net/statifier/main.html
http://www.magicermine.com/index.html
http://www.pgbovine.net/cde.html
Another solution is make a portable 0install package:
http://0install.net/
I recomend this solution. Personally I have been problems with the 3 first packagers.
I am writing a (closed-source) application and will provide binaries for all three major platforms (MacOS X, Linux and Windows). It uses Qt under the LGPL license so I am required to dynamically link with the Qt libraries. I understand how to bundle the application for MacOS and Windows by simply providing the Qt dynamic library with the application, but I am unsure how to do this properly for Linux.
It's traditional to provide the application as a package (.deb and .rpm) and allow the package system to resolve the dependencies. Should use this method? If so which distros should I be aiming for (I am assuming Ubuntu and Fedora)? If anyone has any experience with this, I'd be interested in hearing it.
deb, rpm and tarballs are the right ways to go. There are a few non-standard ways that I know of which you might want to consider as well which are more distro agnostic.
Makeself - Self extracting shell scripts that contain your application. This is quite widely used by closed source software distributors.
ZeroInstall - useful for non-root installations.
Listaller - Merged with autoinstall and it seems to be unstable as of now.
You need to provide at least 3 packages for Linux; a .deb for Debian-based systems, a .rpm for RPM-based systems, and a tarball for everything else. Some find it necessary to refine it to provide three or four different .rpm packages, for Fedora, SuSE, Mandriva, and RHEL, depending on the exact library requirements the software has.
There are many ways to do so... but if you want a distribution generic way to bundle an application for Linux, you may check this ( http://codevarium.gameka.com.br/deploying-cc-linux-applications-supporting-both-32-bit-and-64-bit/ ) tutorial. It shows how to copy all the dependencies to a library folder and make a shell script to properly execute the bundle.
I've written an application which as of yet is not open source and I'd like to distribute the executable across various linux distros. What's the best way to do this, I've looked a little bit at .rpm and .deb packaging but I can't find if that can be used for binaries or not. Ideally I'd like something like the PackageMaker on OS X or a regular installer on windows that will have it automatically copy into /usr/bin. Is that what .rpm and .deb packages are for or do I have to bundle a shell script that will do it automatically?
RPM and DEB packages are the two primary mechanisms for distributing binary packages in Linux. RPM is used by RedHat and its derivatives (Fedora, CentOS), while DEB is used in Debian and Ubuntu.
The .rpm and .deb files themselves are generally "dumb" archives, and are installed to the correct locations in the filesystem by pre-installed helper applications. You don't have to worry about writing scripts to install files, unless it's a very complicated application which needs special per-system configuration.
The usual patterns I see for distributing binaries are:
Release a compessed tarball (.tar.gz or .tar.bz2), and let distribution packagers worry about the details. This works well for popular applications, but if it's newly released, nobody will care enough about your application to package it.
Release as a tarball, plus RPM and/or DEB packages (depending on customer needs). Customers with a supported distribution may install the pre-made package. Anybody who's using an unusual distribution is probably happy to install from a tarball anyway.
MojoSetup is a user-friendly, perfectly cross-distro solution and nicely-licensed (zlib, very permissive). All it requires is the standard sh shell which comes with any Linux distribution. It also allows for desktop shortcuts the easy way by creating freedesktop.org spec shortcuts, which are supported by just about all graphical environments for Linux (so just dump in a PNG at different resolutions and fill in the blanks of the .desktop file).
Installers are scripted using the very simple Lua programming language and there are several example installer scripts in the Mercurial repository as well as a lengthy tutorial. There are also many years to back up its development into a lightweight cross-distro installer.
The rpm and deb will store the binaries. You'll need to have a different binaries for each distro or distro variant most likely, just because on different distros things are different like paths.
I recommend starting with the two you have rpm and deb and nail those two distro. Then maybe do a tarball for misc distros have people can extract and directory structure and copy and handle permissions on their own.
Also, for things like deb you can setup a site as a repository. That makes it easy for people to add the repo and get/install the deb in ubuntu very easily. A lot of 3rd party closed source devs do that.
That's what .rpm and .deb files do BUT you have to be sure that the installee distro has the ability to deal with the .rpm and .deb files. If you want something that's sure to run across multiple distros, where you can't be sure that they will have the right package manager, then you pretty much have to resort to the shell script method. I would advise, if you can get away with it, building your binary for both .rpm and .deb - this way, you get most of the distros covered, and you allow users to install in a way they are comfortable and familiar with, and you don't have to try to roll your own installer / uninstaller shell scripts.
You should probably provide a native package for each Linux distribution that you officially support (as you officially support them, you'll be testing on them so doing this should be trivial), and provide a .tar.gz which people can drop in for other ones.
Users can always make their own .rpm etc for some alien distribution which you don't support; but they can't complain to you unless it doesn't work on an officially supported OS.
Which OSs do you officially support? You'll obviously need to test on them all (at the very least, you'll need to pass all your regression test suite on each OS on each release).
This is of course complicated if you support multiple architectures.
I'd like to install Gentoo. I need it to develop GUI C++ applications using wxWidgets, so I need:
build tools: make, automake, autoconf, etc.
C++ compiler (GCC)
X Window System for testing (Fluxbox or something minimal would be enough)
Now, I have two options:
download the small network installer (57MB) do a network install
download the 600MB CD
I'd like to download as less as possible and still have all the tools above.
I also don't understand whether the network installer will first prompt me for the packages I want, or it will fetch 600 MB of data anyway?
I might want to install it on other computers later, so I'd go with 'full' install from CD if the network install does not save me anything.
Gentoo is ultraminimalist by default.
The install CD gets you a basic working system, a basic compile environment ( Some version of the GCC suite ), and package management.
Its up to you then to install what you want to use.
Its not like many other distributions where theres a big set of "default" packages to have installed.
You have to know what you want, and install what you want.
The "Live" Cd will make things a bit quicker by having a few precompiled binaries available, but besides that, you still have to choose what you want to install.
I also don't understand whether the
network installer will first prompt me
for the packages I want, or it will
fetch 600 MB of data anyway?
it will only install what you want to. If you use NetInstall and install nothing except GCC, it will only download enough to have GCC.
Welcome to gentoo.
It can be a little daunting for first timers, but once you've gotten past the steep learning curve you'll love it :)
You're not missing anything. Furthermore, if you actually want any useful applications, you're going to have to do a lot more downloading than that even. The point is, the small network install CD lets you download whichever version of those components that you want, and the latest version of portage, etc, instead of providing you with (likely) outdated copies on the full CD.
Gentoo is fundamentally a network based distro. The minimal CD is really minimal, it contains just enough to have a functional system booting as livecd so one can install the distribution basically from the network. The livecd (there are also livedvd's around, just not as regularly released (they eat diskspace and bandwith). contains a full graphical environment (and being a compile-yourself distro) obviously gcc as C++ compiler, and can be used to install a binary version of the packages to disk (actually from the livecd environment using some clever hackery).
However, gentoo is a continuously updated distribution. If you want to update your system you need to get the packages from the network (there are ways to find what to download etc, but that is not for beginners) and update. In general, if you don't update every couple of months, your updates can become painful or really painful.
After you have installed Gentoo — emerge vim wxGTK and read
and you are ready to go!
Happy coding.
Use network install. It does save you something and even if you do multiple installations you'll probably want the newest packages anyway.
And no the network installer will not download the 600MB without asking you that would make no sense.
Gentoo doesn't exactly "prompt" for packages. The network CD will get you a small base system, then it's up to you to set everything else you need up yourself. This is one of the positive and negative things about Gentoo.
Update: looks like network install is not that minimal. There's 57MB .iso image, but you also need to download stage3 which is about 120MB and portage with is 29MB. And later on, you need the Linux kernel, which is about 46MB. This totals about 250MB. Or am I missing something?
Update: I have installed the Kernel, X Window system, mc, Lilo, etc. and have a working system :) Summing all up, it downloaded about 580 MB. Well, it is still less than install CD!