Trying to make Host sFlow for XenServer 7.0 - linux

I spent all day trying to make Host sFlow 2.0.6-1 from sources (https://github.com/sflow/host-sflow/releases) for XenServer 7.0 using the XenServer DDK from this site: http://xenserver.org/overview-xenserver-open-source-virtualization/download.html
First I had to make 2 changes to the file hsflowd-xen.spec:
Changed line 3 to: "Version: 2.0.6" (it was still 2.0.1)
Changed line 20 to: "%setup -n hsflowd-2.0.6-1" (added the name because the default one was wrong).
Now my problem is that I dont have the xenstore.h file. After long searches I found that it's in the package libxen-dev (or libxen-devel) but I couldn't find it with its dependecies anywhere.
The four most probable solutions I think are :
1. (The lazy one) Get the iso file for Host sFlow already built for XenServer 7.0 (the official site stopped building at 6.5)
Set up a proper yum repository that will contain libxen-dev and its dependencies. I can't even connect to the official CentOS repositories because the files in /etc/yum.repos.d/ have a bad URL.
This is the content of /etc/centos-release: "XenServer DDK release 7.0.0-125770c (xenenterprise)"
Somehow manage to use 'xenstore.a' instead of 'xenstore.h'. I changed the code in src/Linux/mod_xen.c to include 'xenstore.a' instead of 'xenstore.h' but when I build it, it creates a new file with the old code and ignores my changes. I probably changed the wrong files because there are different copies of the whole code. I'm not even sure it would work though even if I did manage to include 'xenstore.a'.
Make xenstore from sources. I didn't try it because I only found old sources and I figured I'd miss the dependencies too.
PS: I'm n00b at CentOS and Makefiles in general so the solution might be obvious and I just don't know it.

With gratitude to lagange, I updated the host-sflow project with a XenServer 7 build. I also added a Docker recipe so you can replace all these steps with just "./docker_build_on xenserver". Please raise issues on https://github.com/sflow/host-sflow.

I finally succeeded in building it. That's what I had to do step by step:
Import the XenServer DDK 7.0.0 into XenCenter.
Extend xvda1 following these steps: https://support.citrix.com/article/CTX125405
Make these changes to hsflowd-xen.spec:
3rd line: Version: 2.0.6
20th line: %setup -n hsflowd-2.0.6-1
Add these two lines before %description:
%define debug_package %{nil}
%define _unpackaged_files_terminate_build 0
Change file /etc/yum.repos.d/CentOS-Base
Change all occurrences of "$releasever" to "7".
Change all occurrences of "$basearch" to "x86_64".
Change "enabled=0" to "enabled=1" for each repository.
Uncomment baseurl lines for each repository.
Mount the Development packages (binpkg.iso available on the xenserver.org download page) and add a file for it in /etc/yum.repos.d/
Mine looks like it:
[binpkg]
name=CitrixXenServer7
enabled=1
baseurl=file:///mnt/binpkg/
gpgcheck=0
Install the two following packages with Yum (dependencies should install correctly now):
xen-libs-devel.x86_64
xen-dom0-libs-devel.x86_64
Make the file and install it using this tutorial: https://raw.githubusercontent.com/sflow/host-sflow/v2.0.4/INSTALL.XenServer

Related

Simple guide for installing Eclipse multi-user on Linux

Is there a simple guide available anywhere on how to install a recent version of Eclipse on Linux as a "normal" application?
The version in the Debian repositories, even for Debian 9, is a 3.X version of Eclipse, which must be very old, as they are now at V4.7.
The default method for the "eclipse.org" versions to install themselves is to put everything in the home directory, which is not a great approach for a system where the home directory is supposed to only contain user data (e.g. the whole home directory is backed up daily)
I can't believe that something so widely used does not have any good instructions for how to install it as a normal application would be - where all the application files are in a shared folder under /opt, and only user data is stored in the home folder.
I have found this thread, which may be promising:
https://www.eclipse.org/forums/index.php/t/1080525/
And a previous question seemed reasonably good, but on trying to follow it, the installer screens have changed, and everything about "bundle pools" became very confusing!
https://askubuntu.com/questions/695382/how-to-install-eclipse-using-its-installer
Not sure if you are still looking for an answer. This is how I have been doing for a few years now.
Download it from website and extract eclipse folder under /opt
Change ownership of the folder to your user and make it readonly recursively.
Create a desktop file to launch it as a regular user.
You can make the folder writable once in a month to make sure latest updates are installed then make the folder readonly again.
TL;DR give up now
I found this when Googling for how to do a multi-user install on Linux Mint 20.
Although Eclipse publish specific documentation on how to do a multi-user installation this little correlation with the current reality.
I first tried installing the package version of Eclipse. That installed the flatpak package, not a native package. It turns out that flatpak has a bug (opened in 2017) whereby it creates a new image each time you run the binary - and at 1.2G that was eating my enterprise storage very quickly. Although the package installed from the command line use apt it did not add any record to the installed package list - so I couldn't uninstall it with apt (using flatpak uninstall eclipse seems to have worked).
I then downloaded the tarball eclipse.
This has a GUI installer which would not run via [x]sudo - so I had to subvert the usual security model on Mint/Ubuntu and make the root user a logon account.
I specified the install location as /usr/local/eclipse. The installer did its thing. Sadly it did not provision a menu entry for the program. Since it actually installs in a sub-directory of the target named with versioning information (/usr/local/eclipse/jee-2019/12/) I created a shell script in /usr/local/bin/ to point to the binary (so when I upgrade, I just need to change the shell script to point to the new location). I then created a .desktop entry to run the shell script and ran:
xdg-desktop-menu install --novendor eclipse.desktop
I also copied the eclipse.desktop file to /etc/skel/Desktop and to /root/Desktop
From my root session, opening the .desktop file by double clicking, or running the shell script started eclipse.
Sadly, this did not create a start menu entry (it has worked for me with other applications).
Logging on a different user, copying the .desktop file to ~/Desktop and running it resulted in....
(Of course it would have been far too easy to display the failure reason in the dialog, or even put the path to the log file in a widget where I could copy and paste it)
Looking at /usr/local/eclipse/jee-2019/12//eclipse.ini, I find this:
-startup
plugins/org.eclipse.equinox.launcher_1.5.600.v20191014-2022.jar
--launcher.library
/root/.p2/pool/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.1100.v20190907-0426
-product
org.eclipse.epp.package.jee.product
-showsplash
/root/.p2/pool/plugins/org.eclipse.epp.package.common_4.14.0.20191212-1200
--launcher.defaultAction
openFile
--launcher.appendVmargs
-vm
/usr/lib/jvm/java-11-openjdk-amd64/bin
-vmargs
-Dosgi.requiredJavaVersion=1.8
-Dosgi.instance.area.default=#user.home/eclipse-workspace
-XX:+UseG1GC
-XX:+UseStringDeduplication
--add-modules=ALL-SYSTEM
-Dosgi.requiredJavaVersion=1.8
-Dosgi.dataAreaRequiresExplicitInit=true
-Xms256m
-Xmx1024m
--add-modules=ALL-SYSTEM
-Declipse.p2.max.threads=10
-Doomph.update.url=http://download.eclipse.org/oomph/updates/milestone/latest
-Doomph.redirection.index.redirection=index:/->http://git.eclipse.org/c/oomph/org.eclipse.oomph.git/plain/setups/
There is a whole catalog of issues here.
I replaced the relative path for -startup with an absolute one, copied /root/.p2/pool to /usr/local/eclipse/pool and updated the -startup and -showsplash entries in the file.
I got a whole new error message:
!SESSION Fri Feb 19 11:11:27 GMT 2021 ------------------------------------------
!ENTRY org.eclipse.equinox.launcher 4 0 2021-02-19 11:11:27.893
!MESSAGE Could not find extension: reference:file:org.eclipse.osgi.compatibility.state_1.1.600.v20190814-1451.jar
!ENTRY org.eclipse.equinox.launcher 4 0 2021-02-19 11:11:28.070
!MESSAGE Exception launching the Eclipse Platform:
!STACK
java.lang.ClassNotFoundException: org.eclipse.core.runtime.adaptor.EclipseStarter
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
...
I get exactly the same error after converting the Fedora 33 RPM to a .deb and installing that.
I'm going to go check the price of intellij licences.

RPM create a trigger that will use version information from the new (triggering) RPM

A little background - Since Redhat stopped updating gcc versions for RHEL6, we had to implement our own gcc5 deployment.
We can't upgrade to RHEL7 at this time, nor will it solve this problem as Redhat hadn't deployed gcc5 for rhel7 yet.
Our current issue is with /usr/src/kernels/$(uname -r)/include/linux/compiler-gcc5.h
That path and the compiler-gcc*.h files are owned by kernel-devel, and a gcc5 one exists only for fedora (only for distros where gcc5 has been introduced officialy).
I can easily create an rpm that will own the file (the reason is mainly so we will receive a conflict if RH ever deploys kerenl-devel with that file, so we can stop deploying our own) for the current kernel, but I'd like to use %triggerin to copy the file to any new kernel-devel installed on the system.
using uname -r wouldn't work for 2 reasons:
it's evaluated at rpmbuild time, not install time.
it evaluates the current running kernel, not the future one just installed.
I can overcome 1 by using a %post script, I think, but can't figure out how to overcome 2, nor how to own the file by my rpm in it's final location (can I %ghost a path I don't know in advance, or use something like dkms?)
Is there a way to know the exact version of the rpm that triggers the %triggerin and incorporate it into the path?
Is there an alternate solution?

How to create debian bundle?

i am new to Debian. so can anybody send a link to Debian source code of configuring and installation. i know all linux commands, but i need step by step explanation to create Debian package of my project. which includes some jar files and some config files
thanks in advance
there is plenty of documentation on how to create a Debian package:
https://wiki.debian.org/HowToPackageForDebian
if you want to see a real-world example, you can get the code for any (official) package, by simply running apt-get source <mypackage>. E.g. the following command gets everything needed to create the hello Debian package (with the exception of build-dependencies):
apt-get source hello
This contains both
the code needed to create the deb with dpkg-buildpackage (everything contained in the debian/ subfolder)
and the original source code (everything outside of the debian/) subfolder).

Build environment isolation and file system diffing

Alright so after trying to chase down the dependencies for various pieces of software for the n-th time and replicating work that various people do for all the different linux distributions I would like to know if there is a better way of bundling various pieces of software into one .rpm or .deb file for easier distribution.
My current set up for doing this is a frankenstein monster of various tools but mainly Vagrant and libguestfs (built from source running in Fedora because none of the distributions actually ship it with virt-diff). Here are the steps I currently follow:
Spin up a base OS using either a Vagrant box or by create one from live CDs.
Export the .vmdk and call it base-image.
Spin up an exact replica of the previous image and go to town: use the package manager,
or some other means, to download, compile, and install all the pieces that I need. Once again, export the .vmdk and call it non-base-image.
Make both base images available to the Fedora guest OS that has libguestfs.
Use virt-diff to diff the two images and dump that data to file called diff.
Run several ruby scripts to massage diff into another format that contains the information I need and none of the stuff I don't like things in /var.
Run another script to generate a command script for guestfish with a bunch of copy-out commands.
Run the guestfish script.
Run another script to regenerate the symlinks from diff because guestfish can't do it.
Turn the resulting folder structure into a .deb or .rpm file and ship it.
I would like to know if there is a better way to do this. You'd think there would be but I haven't figured it out.
I would definitely consider something along the lines of:
A)
yum list (select your packages/dependencies whatever)
use yumdownloader on the previous list (or use th pkgs you have already downloaded)
createrepo
ship on media with install script that adds the cd repo to repolist, etc.
or B)
first two steps as above, then pack the rpms into an archive build a package that contains all of the above and kicks off the actual install of the rpms (along the lines of rpm -Uvh /tmp/repo/*) as a late script (in the cleanup phase, maybe). Dunno if this can be done avoiding locks on the rpm database.
I think you reached the point of complexity - indeed a frankenstein monster - where you should stop fearing of making proper packages with dependencies. We did this in my previous work - we had a set of fabricated rpm packages - and it was very easy and straightforward, including:
pre/post install scripts
uninstall scripts
dependencies
We never had to do anything you just described. And for the customer, installing even a set of packages was very easy!
You can follow a reference manual of how to build RPM package for more info.
EDIT: If you need a single installation package, then create this master packge, that would contain all the other packages (with dependencies set properly) and installed them in the post-install script (and uninstalled them in the uninstall script).
There are mainly 3 steps to make a package with all dependencies (let it be A, B & C).
A. Gather required files.
There are many way to gather files of the main software and its dependencies. In order to get all the dependices and for error free run you need to use a base OS (i.e live system)
1. Using AppDirAssistant
This app is used by www.portablelinuxapps.org to create portable app directory. They scan and watch for the files accessed by the app to find required.
2. Using chroot & overlayfs
In this method you don't need to boot into live cd instead chroot into it.
a. mount the .iso # /cdrom and
b. mount the filesystem(filesystem.squashfs) # another place, say # /tmp/union/root
c. Bind mount /proc # /tmp/union/root/proc
d. Overlay on it
mount -t overlayfs overlayfs /tmp/union/root -o lowerdir=/tmp/union/root,upperdir=/tmp/union/rw
e. Chroot
chroot /tmp/union/root
Now you can install packages using apt-get or another method (only from the chrooted terminal). All the changed files are stored # /tmp/union/rw. Take files from there.
3. Using manually collected packages
Use package manager to collect dependencies. For example
apt-get install package --print-uris will print download uris for dep packages. Using this uris download packages and extract all (dpkg -x 1.deb ./extracted).
B. Clean garbage files
After gathering files remove unwanted files
C. Pack files
1. Using appImageAssistance
If you manually gathered files then you need to copy appname.desktop file from ./usr/share/applications to root of directory tree. Also copy file named AppRun from another app or extract it from AppDirAssistance.
2. Make a .deb or .rpm using gathered files.
Is the problem primarily that of ensuring that your customers have installed all the standard upstream distro packages necessary for your package to run?
If that's the case, then I believe the most straightforward solution would be to leverage the yum and apt infrastructure to have those tools track down and install the necessary prerequisite packages.
If you supply a native yum/apt repository with complete pre-req specs (the hard work you've apparently already completed). Then the standard system install tool takes care of the rest. See link below for more on creating a personal repository for yum/apt.
For off-line customers, you can supply media with your software, and a mirror - or mirror subset - of the upstream distro, and instructions for adding them to yum config/apt config.
Yum
Creating a Yum Repository in the Fedora Deployment Guide
Apt
How To Setup A Debian Repository on the Debian Wiki
So your customers aren't ever going to install any other software that might specify a different version of those dependencies that you are walking all over, right?
Why not just create your own distro if you're going to go that far?
Or you can just give them a bunch of packages and a single script that does rpm -i dep1 dep2 yourpackage

Run time installation directory of debian package contents

I have a debian package that I built that contains a tar ball of the files, a control file, and a postinst file. Its built using dpkg-deb and it installs properly using dpkg.
The modification I would like to make is to have the installation directory of the files be determined at runtime based on an environment variable that will be set when dpkg -i is run on the deb file. I echo out the environment variable in the postinst script and I can see that its set properly.
My questions:
1) Is it possible to dynamically determine the installation directory at runtime?
2) If its possible how would I go about this? I have read about the rules file and the mypackage.install files but I don't know if either of these would allow me to accomplish this.
I could hack it by copying the files to the target location in the posinst script but I would prefer to do it the right way if possible.
Thanks in advance!
So this is what I found out about this problem over the past couple of weeks.
With prepackaged binaries you can't build a debian package with a destination directory dynamicall determined at runtime. I believe that this might be possible if installing a package that is built from source where you can set the install directory using configure. But in this case since these are embedded Ubuntu machines they don't have make so I didn't pursue such an option. I did work out a non traditional method (hack) for installing that did work. Since debian packages simply contain a tar ball relative to / simply build your package relative to a directory under /tmp. In the postinst script you can then determine where to copy the files from the archive into a permanent location.
I expected that after rebooting and the automatic deletion of the subdirectory under /tmp that dpkg might not know that the file package existed. This wasn't a problem. When I ran 'dpkg -l myapp' it showed as still installed. Updating the package using dpkg/apt-get also worked without a hitch.
What I did find is that if you attempted to remove the package using 'dpkg -r myapp' that dpkg would try and remove /tmp which wasn't good. However /tmp isn't easily removed so it never succeeded. Plus in our situation we never remove packages but instead simply upgrade them.
I eventually had to abandon the universal package due to code differences in the sources resulting in having to recompile per platform but I would have left it this way and it did work.
I tried using --instdir to change the install directory of the package and it does relocate the files but dpkg fails since the dpkg file can't be found relative to the new instdir. Using --instdir is sort of like a chroot. I also tried --admindir and --root in various combinations to see if I could use the dpkg system relative to / but install relocate the files but they didn't work. I guess rpm has a relocate option that works but not Ubuntu.
You can also write a script that runs dpkg-deb with a different environment for 6 times, generating 6 different packages. When you make a modification, you simply have to run your script, and all 6 packages gets generated and you can install them on your machines avoiding postinst hacking!
Why not install to a standard location, and simply use a postinst script to create symbolic links to the desired location? This is much cleaner, and shouldn't break anything in dpk -I.

Resources