I'm packaging a PyQt application for Linux as a .deb package, following the Debian maintenance guide.
The manual does a good job describing how to build the python binaries with debuild -b, and install global data files in /usr/share/<package>/ through the debian/install file. However, I don't see any mention of installing user settings files - cache files or files for configuration changes that the current user running the program might want to save.
As far as I understand, other programs usually save these in a hidden directory on the users home path - eg atom's user data in /home/<username>/.atom/.
The manual does mention conffiles. However these seem to be globally installed. I'm also not sure if they're suitable for config files that change frequently as a result of user actions, since package updates will attempt to solve conflicts between new and existing conffiles.
Some other documentation mentions postinstall scripts, but this seems potentially too complicated for something that should be common to many debian packages?
Related
Internet research indicates that one can remove the Cygwin package files to save space. Within the Cygwin installation folder, I have a subfolder (say) http%3a%2f%2fcygwin.mirror.constant.com%2f reflecting the mirror site. The file http%3a%2f%2fcygwin.mirror.constant.com%2f/x86_64/setup.ini seems to contain all the information about installed packages. The only other folders/files are in the following two folder trees:
http%3a%2f%2fcygwin.mirror.constant.com%2f/noarch/release/*
http%3a%2f%2fcygwin.mirror.constant.com%2f/x86_64/release/*
Can I remove everything in the http%3a%2f%2fcygwin.mirror.constant.com%2f folder tree except http%3a%2f%2fcygwin.mirror.constant.com%2f/x86_64/setup.ini and still have the installer know what I have installed, what needs updating, and what needs downloading for a re-install?
The information about installed package is stored in
/etc/setup/installed.db
the cache in http%3a%2f%2fcygwin.mirror.constant.com%2f
is only needed during installation or re-installation of any package.
See info in the Cygwin User Guide
https://cygwin.com/cygwin-ug-net/setup-net.html#setup-localdir
After installing Cygwin, the cache is no longer necessary, but you may
want to retain the packages as backups, for installing Cygwin to
another system, or in case you need to reinstall a package.
My knowledge on Linux administration is limited and hence wanted to check here about the pros and cons of installing any RHEL/CentOS Linux software using rpm packages over installing through tar/zip files.
Thanks
a non-exhaustive list of pros and contras:
rpm
intelligent dependency managment
conflict checking
allow easy and clean uninstall
allow for upgrades / downgrades
list all files owned by a package
a central database with all packages installed, which files they own, their interdependencies
from source
you choose yourself all compiler flags
you can choose a custom installation path
I have tried to explain the diff, pros and cons,
Tar
Basically tar is old way of dealing with in Linux. We can say its existence when the Linux was created.
Usually the tar consists of Source Code and needs to be compiled in binary format for us to use.
Pros:
Using tar packages you gain more control over the programs that you install.
If you want certain portions that avoided, you could do that on the go. Which give you the upper hand.
Cons:
The main issue comes in the maintainability of the packages installed.
They are hard to manage. Once you install, there was no way to manage the software unless and until its well documented. It also hard to version them and you are left blank on the software version you have. The possible reason for this because of the non-indexing nature of files. The files could be spread across your file system, which makes it difficult to remove or upgrade it.
Hard to automate.
It is also hard to automate because of the complexities in maintaining the packages.
Below I tried explaining how tar file are compiled to get better understanding,
Prepare(setup) environment for building
./configure
This script has lots of options that you should change. Like --prefix or --with-dir=/foo. That means every system has a different configuration. Also ./configure checks for missing libraries that should be installed. Anything wrong here causes not to build your application. That's why distros have packages that are installed on different places, because every distro thinks it's better to install certain libraries and files to certain directories. It is said to run ./configure, but in fact you should change it always.
Building the system
make
This is actually make all by default. And every make has different actions to do. Some do building, some do tests after building, some do checkout from external SCM repositories. Usually you don't have to give any parameters, but again some packages execute them differently.
Install to the system
make install
This installs the package in the place specified with configure. If you want you can specify ./configure to point to your home directory. However, lots of configure options are pointing to /usr or /usr/local. That means then you have to use actually sudo make install because only root can copy files to /usr and /usr/local.
Please go through the below link for more information on the above commands
Why always ./configure; make; make install; as 3 separate steps?
RPM
The RPM Package Manager (RPM) is an open packaging system,
RPM packages pre-compiled binary packages (as well as source packages) for an easy one-click installation experience. RPM by itself does not manage dependency and resolve conflicts. When combined with Yum or PackageKit it will resolve all the dependency for the package.
RPM makes system updates easy. Installing, uninstalling and upgrading RPM packages can be accomplished with short commands. RPM maintains a database of installed packages and their files, so you can invoke powerful queries and verification on your system. During upgrades, RPM handles configuration files carefully, so that you never lose your customisation, that you cannot accomplish with regular .tar files.
RPM feature has the ability to verify packages. If you deleted an important file for some package, you can verify the package. You will notified of changes, if any—at which point you can reinstall the package, if necessary. Any configuration files that you modified are preserved during re installation.
Pros:
Install, reinstall, remove, upgrade and verify packages
Use a database of installed packages to query and verify packages
Use metadata to describe packages, their installation instructions, and so on
Package pristine software sources into source and binary packages
Add packages to Yum repositories
Digitally sign your packages
Querying a package (if the package is on your local file system or after the package is installed)
Validating a package (checking a package has not been tampered with, before or after installation).
Cons
Not as customisable as tar.
eg on usability: We will see how to install package using Tar or rpm:
in Tar:
$ tar xvf package.tar
$ cd package
$ ./configure --prefix=PREFIX
$ make
$ make install
in RPM:
rpm -U package-2.4.x-1.i686.rpm
That simple!!.
It basically depends on the usability and the purpose of your use.
Each of them has its on pros and cons depends on how and for what we use it.
I know it a long explanation,how this will give you clear picture. I know there are more untouched such as architecture and execution. I am not pretty confident to explain those here.
In simple words you can say that rpm are prepackaged binaries. They're just ready to go, it does everything for you. But to install rpm and deb you need to be root to have some write permissions. That leaves some serious security hole in the system. You may be unknowingly installing a Torjan horse. Also if the packages are screwed up they may cause the installation to fail altogether.
I personally recommend using tar as you are in more control. It is old school, I know, that's why a bit difficult but, in my opinion best way to go.
You can further refer to the link:
https://tldp.org/HOWTO/Software-Building-HOWTO-4.html
Alright so after trying to chase down the dependencies for various pieces of software for the n-th time and replicating work that various people do for all the different linux distributions I would like to know if there is a better way of bundling various pieces of software into one .rpm or .deb file for easier distribution.
My current set up for doing this is a frankenstein monster of various tools but mainly Vagrant and libguestfs (built from source running in Fedora because none of the distributions actually ship it with virt-diff). Here are the steps I currently follow:
Spin up a base OS using either a Vagrant box or by create one from live CDs.
Export the .vmdk and call it base-image.
Spin up an exact replica of the previous image and go to town: use the package manager,
or some other means, to download, compile, and install all the pieces that I need. Once again, export the .vmdk and call it non-base-image.
Make both base images available to the Fedora guest OS that has libguestfs.
Use virt-diff to diff the two images and dump that data to file called diff.
Run several ruby scripts to massage diff into another format that contains the information I need and none of the stuff I don't like things in /var.
Run another script to generate a command script for guestfish with a bunch of copy-out commands.
Run the guestfish script.
Run another script to regenerate the symlinks from diff because guestfish can't do it.
Turn the resulting folder structure into a .deb or .rpm file and ship it.
I would like to know if there is a better way to do this. You'd think there would be but I haven't figured it out.
I would definitely consider something along the lines of:
A)
yum list (select your packages/dependencies whatever)
use yumdownloader on the previous list (or use th pkgs you have already downloaded)
createrepo
ship on media with install script that adds the cd repo to repolist, etc.
or B)
first two steps as above, then pack the rpms into an archive build a package that contains all of the above and kicks off the actual install of the rpms (along the lines of rpm -Uvh /tmp/repo/*) as a late script (in the cleanup phase, maybe). Dunno if this can be done avoiding locks on the rpm database.
I think you reached the point of complexity - indeed a frankenstein monster - where you should stop fearing of making proper packages with dependencies. We did this in my previous work - we had a set of fabricated rpm packages - and it was very easy and straightforward, including:
pre/post install scripts
uninstall scripts
dependencies
We never had to do anything you just described. And for the customer, installing even a set of packages was very easy!
You can follow a reference manual of how to build RPM package for more info.
EDIT: If you need a single installation package, then create this master packge, that would contain all the other packages (with dependencies set properly) and installed them in the post-install script (and uninstalled them in the uninstall script).
There are mainly 3 steps to make a package with all dependencies (let it be A, B & C).
A. Gather required files.
There are many way to gather files of the main software and its dependencies. In order to get all the dependices and for error free run you need to use a base OS (i.e live system)
1. Using AppDirAssistant
This app is used by www.portablelinuxapps.org to create portable app directory. They scan and watch for the files accessed by the app to find required.
2. Using chroot & overlayfs
In this method you don't need to boot into live cd instead chroot into it.
a. mount the .iso # /cdrom and
b. mount the filesystem(filesystem.squashfs) # another place, say # /tmp/union/root
c. Bind mount /proc # /tmp/union/root/proc
d. Overlay on it
mount -t overlayfs overlayfs /tmp/union/root -o lowerdir=/tmp/union/root,upperdir=/tmp/union/rw
e. Chroot
chroot /tmp/union/root
Now you can install packages using apt-get or another method (only from the chrooted terminal). All the changed files are stored # /tmp/union/rw. Take files from there.
3. Using manually collected packages
Use package manager to collect dependencies. For example
apt-get install package --print-uris will print download uris for dep packages. Using this uris download packages and extract all (dpkg -x 1.deb ./extracted).
B. Clean garbage files
After gathering files remove unwanted files
C. Pack files
1. Using appImageAssistance
If you manually gathered files then you need to copy appname.desktop file from ./usr/share/applications to root of directory tree. Also copy file named AppRun from another app or extract it from AppDirAssistance.
2. Make a .deb or .rpm using gathered files.
Is the problem primarily that of ensuring that your customers have installed all the standard upstream distro packages necessary for your package to run?
If that's the case, then I believe the most straightforward solution would be to leverage the yum and apt infrastructure to have those tools track down and install the necessary prerequisite packages.
If you supply a native yum/apt repository with complete pre-req specs (the hard work you've apparently already completed). Then the standard system install tool takes care of the rest. See link below for more on creating a personal repository for yum/apt.
For off-line customers, you can supply media with your software, and a mirror - or mirror subset - of the upstream distro, and instructions for adding them to yum config/apt config.
Yum
Creating a Yum Repository in the Fedora Deployment Guide
Apt
How To Setup A Debian Repository on the Debian Wiki
So your customers aren't ever going to install any other software that might specify a different version of those dependencies that you are walking all over, right?
Why not just create your own distro if you're going to go that far?
Or you can just give them a bunch of packages and a single script that does rpm -i dep1 dep2 yourpackage
I have a question about the structure of the source code from a cygport package.
Here is the contents of a Cygports source file:
the actual source bundle for the project (tar.gz, tar.bz2, etc.)
the any number of *.patch files.
a .cygport file
I am trying to build gedit-3.4.2 from cygports repository.
How does the .cygport file help me run the proper options in the ./configure ?
For instance, in gedit if i don't specify --disable-spell it won't proceed due to error. How do I get the list of ./configure options that were used to build the project when the cygport was built?
Is there some way we can use the cygport executable to build the cygport and change the prefix too?
Here is the contents of gedit-3.4.2-1.cygport:
inherit python gnome2
DESCRIPTION="GNOME text editor"
PATCH_URI="3.4.2-cygwin.patch"
DEPEND="gnome-common gtk-doc
girepository(Gtk-3.0)
pkgconfig(enchant)
pkgconfig(gtksourceview-3.0)
pkgconfig(libpeas-gtk-1.0)"
PKG_NAMES="${PN} ${PN}-devel"
PKG_HINTS="setup devel"
gedit_CONTENTS="--exclude=gtk-doc --exclude=libgedit* etc/ usr/bin/ usr/lib/gedit/ ${PYTHON_SITELIB#/} usr/share/"
gedit_devel_CONTENTS="usr/include/ usr/lib/gedit/libgedit* usr/lib/pkgconfig/ usr/share/gtk-doc/"
DIFF_EXCLUDES="*.desktop.in *.schemas.in *-marshal.h"
CYGCONF_ARGS="--libexecdir=/usr/lib --enable-python"
KEEP_LA_FILES="none"
EDIT Someone from Cygwin Ports mailing list said:
"The configure options are
--libexecdir=/usr/lib --enable-python
Which is from CYGCONF_ARGS."
Here is the contents of a Cygports source file:
You'd do better to think of it as a Cygwin package source file.
cygport is simply a tool for automating the creation of Cygwin binary and source packages. It is the primary tool available, but unlike with some other packaging systems, there's really nothing forcing you to use it. It is quite possible to build a Cygwin package entirely by hand, since it is really nothing more than a tarball that Cygwin's setup.exe can blindly unpack into the Cygwin root directory (typically c:\cygwin) with the expectation that this will put the package's files in sensible locations.
Before cygport existed, people did build their own ad hoc packaging systems. Many Cygwin package maintainers still use these tools they created. (Yours truly included; two of my three packages use cygport, but the third still uses a custom build system.)
Ultimately, you want to read the cygport manual, in /usr/share/doc/cygport/manual.html.
(Yes, I know, "RTFM" answers are frowned on here. But, as one who currently maintains two cygport based packages in the official Cygwin package repository, please believe me when I tell you that the manual is still the single best resource available on this topic.)
How does the .cygport file help me run the proper options in the ./configure ?
As you found out through other resources, you'd first need to edit the CYGCONF_ARGS value in the .cygport file.
The simplest possible step after that is cygport gedit-3.4.2-1.cygport all. That attempts to rebuild all the binary packages in a single step. It also builds a new source package containing updated .cygport and patch files.
If something breaks in the all build process, it is usually faster to switch to using the sub-commands contained by all instead of completely restarting the process. The all step just runs prep, compile, install, package, and finish for you, in that order. For instance, if all fails during the compilation step, there's probably no need to repeat the prep step.
(It is exceptionally uncommon for cygport or a sane build system to wreck the build tree, forcing you to re-run prep. Far more commonly, you end up needing to re-do prep when you manually wreck the build tree while trying to get a new package to build for the first time and need to start over.)
For instance, in gedit if i don't specify --disable-spell it won't proceed due to error.
You can probably fix that by installing the libaspell-devel package from the official Cygwin package repository with setup.exe.
Personally, I wouldn't disable any feature unless it meant installing unofficial packages, such as those from the Cygwin Ports project.[*] It is nice to have Cygwin Ports repository, but because it contains so many packages, installing one can end up creating an "install the world" situation: package A depends on packages B, C and D, and C depends on E, F, G, H, and G depends on I, J, K, and... Dependency hierarchies within the Cygwin package repo tend to be flatter and narrower than those in the Cygports repo.
Is there some way we can use the cygport executable to build the cygport and change the prefix too?
You have guessed that you just add --prefix=/my/private/program/tree to CYGCONF_ARGS, I trust.
[*] If you are feeling confused about "Cygwin Ports" and cygport, the naming similarity is no coincidence. cygport is a tool created by Yaakov Selkowitz for himself when creating the Cygwin Ports package repository. Later, it became popular enough among other Cygwin package maintainers that it pushed out most of the competing build systems.
I have created an Ubuntu package to install my RCP app. The installed files are owned by root. Is it possible for a user to subsequently install updates through P2? Documentation about Eclipse multi-user installs suggests that it is possible, along with the answer to this question.
However, when I start up the application, it does not automatically check for updates as usual, and the Update Site that I had specified in p2.inf is not listed in the "Install New Software..." dialog.
Using the -configuration or -data runtime options did not help.
I can make it work with a hack by running sudo chown -R <my_username> /opt/<my_app_installation>. When I subsequently launch the application, it does properly check for updates on startup, and my update site is properly listed in the "Install New Software..." dialog. Certainly I would prefer that whatever data it is writing to that installation directory be instead written to the user's home directory.
Supplementary info:
Here is a list of files and folders that showed up in my installation directory only after the directory was given ownership by <my_username> and the program was run.
/opt/<my_app_installation>/configuration/org.eclipse.core.runtime
/opt/<my_app_installation>/configuration/org.eclipse.equinox.app
/opt/<my_app_installation>/configuration/org.eclipse.osgi
/opt/<my_app_installation>/p2/org.eclipse.equinox.p2.engine/profileRegistry/profile.profile/1339896994308.profile.gz
/opt/<my_app_installation>/p2/org.eclipse.equinox.p2.engine/profileRegistry/profile.profile/.data/.settings/org.eclipse.equinox.p2.ui.sdk.prefs
/opt/<my_app_installation>/p2/org.eclipse.equinox.p2.engine/profileRegistry/profile.profile/.data/.settings/org.eclipse.equinox.p2.ui.sdk.scheduler.prefs
/opt/<my_app_installation>/p2/org.eclipse.equinox.p2.repository
More experimental results:
Even with a writable (chown'd) installation directory, no files are placed there when the
-configuration $HOME/.my_app_files runtime option is supplied.
There are lots of limitation of p2 itself for share install. AFAIK there is no significant improvement in latest release Juno as well.
But a guy from Redhat is working on p2 install with RPM package, you can read his progress in his blog post. The work and idea could be shared with Debian package.