When I am using cpan to install module, there is a download source which is very unstable and slow, and I might want to install that module on many machines?
Is that possible to change the perl module repository to other address, or copy the repos to local disk.
Or, can we save the local module files to local when we use cpan. It seems cpan would delete it after installation.
Is that possible to change the perl module repository to other address
You can reconfigure cpan (and change your mirror) by running a CPAN shell:
perl -MCPAN -eshell
and then typing:
o conf init
or copy the repos to local disk.
CPAN Mini is the usual tool for this.
cpanm (called "cpanminus") is a very popular alternative installer for Perl modules. With cpanm, you can use the --mirror option to point it at a different repository URL. I believe cpanm can also create a local cache of all the modules you install, for quicker access later.
Pinto is another option for creating a CPAN-like repository on local disk. However, it aspires to be a more "complete" solution for managing Perl modules. So it might be more than you are looking for.
You could set up a CPAN::Mini repository. You need to install CPAN::Mini, and provide a config file .minicpanrc. Mine is quite short:
# contents of .minicpanrc in home directory
local: /home/ebaudrez/mirrors/minicpan
remote: http://cpan.cu.be/
Obviously, you'll need to tune that to your installation. You can find a list of CPAN mirrors here. Then you have to create the local mirror by calling
minicpan
Beware: the first time, this will take quite some time. Expect a couple of gigabytes of storage to be consumed. Subsequent calls will be much faster. It found it made no sense to call it more than once daily. Also note that only the last release of a distribution is kept, and old or removed distributions will be removed from your local repository.
And then finally, you can tell cpanm to use your mirror preferentially, and to fail back to real CPAN only when the particular distribution or version you're after is not available in your local mirror:
PERL_CPANM_OPT="--mirror $HOME/mirrors/minicpan --mirror http://search.cpan.org/CPAN"
It's possible to configure CPAN or CPANPLUS to use your mirror, too, but I found it slightly less convenient to do so, and prefer cpanminus anyway.
Now you can install any distribution/version you want using cpanm, e.g.,
cpanm Dist::Zilla; # enjoy as the dependencies fly by ;-)
or, say, PDL version 2.006 (note: latest release at the time of writing is 2.007 ;-))
cpanm PDL#2.006 # note: that is new syntax since cpanminus 1.6
Stuff that is in your local mirror will be used if available. Global CPAN will be used for fallback.
I'm surprised nobody has added how to just edit the default CPAN configuration. Using the CPAN shell to do this is just crazy, in comparison to editing a small text file.
Here's how to do it:
Choose some CPAN mirrors from the Official CPAN mirror list.
Locate your CPAN configuration, usually in:
$HOME/.cpan/CPAN/MyConfig.pm
(If not there, then search for the file: MyConfig.pm.)
Edit the list-item called urllist, and add your URLs to it.
'urllist' => [
q[http://httpupdate3.cpanel.net/CPAN/],
q[http://mirrors.servercentral.net/CPAN/],
q[ftp://cpan.cse.msu.edu/]
],
Related
I have a remote CentOS server (Release 6.10) set up by someone else. I have quite a few perl modules installed on the machine.
I have set up a local CentOS server (Release 7.7.1908). I would like to have the EXACT same set of perl modules on my local machine. Installing them one by one via cpan is an option but I can run into issues as some of the perl modules are older (very) versions.
I was wondering, if I can copy modules from the remote server to my local server. Can I do that? Are there other options?
It's not safe to copy modules from one machine to another because things may not be set up identically. It's best to reinstall them.
You can use the autobundle command in the cpan shell to create a dump of all the modules you have installed on the old machine. You can then use that dump to tell the cpan shell on the new machine what modules to install.
Thanks to Polar Bear, here's a link to an article that explains how to reinstall the autobundle.
The solution suggested by Andy Lester is perhaps the best way to do it. I found a documentation here in addition to ones suggested.
In my case however this was not straight forward because the source server environment is very old and there were many dependencies that I will need to manually resolve. In general if you have similar environments and clean installs, the auto bundle approach will make it easy.
Perl's cpan command is a powerful way to manage Perl modules. However, when maintaining modules system-wide under UNIX, Michal Ingeli notes that another possible option is
yum install 'perl(PerlModuleName)'. If available, should yum be my first resort in this case?
For example, the command cpanm CGI installs the CGI module under my ~/perl5 directory, which may be best if the CGI module is only needed by scripts run under my account. But this won't provide the CGI module to scripts run by other accounts.
I can use cpanm -l <directory> to force the cpanm command to load modules to a specific directory (e.g., cpanm -l /usr/local CGI to install CGI to /usr/local/lib/perl5), or I can edit ~/cpan/CPAN/MyConfig.pm to change the default install location cpan uses.
But on nearly all systems, multiple Perl system library locations exist (/usr/local/share/perl5, /usr/share/perl5/vendor_perl, /usr/lib64/perl5, etc.), and choosing the correct one is somewhat arbitrary since these are not generated by the cpan command.
With this in mind, should I turn to yum (if available) before cpan for system-wide UNIX Perl module management? It's easy enough to test with a command like:
yum install 'perl(LWP::Simple)'
If yum failed in this instance, I would fall back to:
cpanm -l <directory> LWP::Simple
What do you recommend in this type of case, and why?
(Note that nxadm has answered a more general question about this.)
To summarize answers so far:
If at all possible, use the system package manager to update CPAN modules. E.g., for LWP::Simple:
yum install 'perl(LWP::Simple)', or
apt-get install liblwp-simple-perl
If the preceding fails, try to implement a separate Perl environment in which to use CPAN modules not present in the system-wide libraries. Consider local::lib or Perlbrew for this;
Only if the above options don't apply, use cpanm -l <directory> to load the module to a system-wide directory.
I can't speak from experience with RPM/yum systems, but I have done a lot of work with Perl applications on Debian systems and I do highly recommend going with the system packaged versions of CPAN modules if you can. I know a lot of people disagree and historically they may have had good reason but I've been doing it for a long time and find it works very well.
In the Debian world there are an enormous number of Perl modules in pre-packaged form and if you happen to need one that isn't packaged you can build your own package with dh-make-perl and put it in your local apt repository. Being able to run apt-get install your-application and have it pull in all the required dependencies is a real time saver when your code is moving through Dev -> Staging/UAT -> Production workflows. It also gives you confidence that the version of a particular module you're deploying to production is the same as the one you tested in UAT.
One thing you absolutely should not do is use cpanm or the cpan shell as root to install modules into the system directories. If you decide to install direct from CPAN, then use local::lib to install the modules in an application-specific lib directory.
[Edit] Some sample commands as requested:
On a Debian-based system, you would first install the dh-make-perl tool:
sudo apt-get-install dh-make-perl
Then to download a package from CPAN and build it into a .deb file you would run a command like this*:
dh-make-perl --build --cpan Algorithm::CouponCode
You could install the resulting .deb file with:
sudo dpkg -i libalgorithm-couponcode-perl_1.005-1_all.deb
Managing your own apt repository is a whole other topic. In my case I'd copy the .deb to an appropriate directory on the local apt server and run a script to update the index (I think our script uses dpkg-scanpackages).
Note in my opening paragraph above I recommend using systems packages "if you can". To be clear, I meant in the case where most of the modules you want are already packaged by Debain. The example above did not build packages for any dependencies. If your app involves installing modules which have long dependency chains that are not in Debian already, then using cpanm and local::lib will simplify the install. But then you shoulder the burden of repeating that as your code advances through staging to production servers. And you may need to use cpanfile or carton to make sure you're getting the same versions at each step.
* one gotcha: if you have previously set up local::lib so that cpan installs go into a private directory (e.g.: /home/user/perl5) then that will affect the pathnames used in the .deb produced by dh-make-perl. To avoid that, run this before dh-make-perl:
unset PERL5LIB PERL_LOCAL_LIB_ROOT PERL_MB_OPT PERL_MM_OPT
Your system's perl was put there for your system's use. The folks that maintain your distribution will update it when they see fit to another version that suits the needs of your system. Using your system's Package Manager to manage it is really your best idea.
Feel free to use it, but if you need a different version, for whatever reason, you are best rolling your own into a separate location. When maintaining your own perl install, use CPAN.
Alright so after trying to chase down the dependencies for various pieces of software for the n-th time and replicating work that various people do for all the different linux distributions I would like to know if there is a better way of bundling various pieces of software into one .rpm or .deb file for easier distribution.
My current set up for doing this is a frankenstein monster of various tools but mainly Vagrant and libguestfs (built from source running in Fedora because none of the distributions actually ship it with virt-diff). Here are the steps I currently follow:
Spin up a base OS using either a Vagrant box or by create one from live CDs.
Export the .vmdk and call it base-image.
Spin up an exact replica of the previous image and go to town: use the package manager,
or some other means, to download, compile, and install all the pieces that I need. Once again, export the .vmdk and call it non-base-image.
Make both base images available to the Fedora guest OS that has libguestfs.
Use virt-diff to diff the two images and dump that data to file called diff.
Run several ruby scripts to massage diff into another format that contains the information I need and none of the stuff I don't like things in /var.
Run another script to generate a command script for guestfish with a bunch of copy-out commands.
Run the guestfish script.
Run another script to regenerate the symlinks from diff because guestfish can't do it.
Turn the resulting folder structure into a .deb or .rpm file and ship it.
I would like to know if there is a better way to do this. You'd think there would be but I haven't figured it out.
I would definitely consider something along the lines of:
A)
yum list (select your packages/dependencies whatever)
use yumdownloader on the previous list (or use th pkgs you have already downloaded)
createrepo
ship on media with install script that adds the cd repo to repolist, etc.
or B)
first two steps as above, then pack the rpms into an archive build a package that contains all of the above and kicks off the actual install of the rpms (along the lines of rpm -Uvh /tmp/repo/*) as a late script (in the cleanup phase, maybe). Dunno if this can be done avoiding locks on the rpm database.
I think you reached the point of complexity - indeed a frankenstein monster - where you should stop fearing of making proper packages with dependencies. We did this in my previous work - we had a set of fabricated rpm packages - and it was very easy and straightforward, including:
pre/post install scripts
uninstall scripts
dependencies
We never had to do anything you just described. And for the customer, installing even a set of packages was very easy!
You can follow a reference manual of how to build RPM package for more info.
EDIT: If you need a single installation package, then create this master packge, that would contain all the other packages (with dependencies set properly) and installed them in the post-install script (and uninstalled them in the uninstall script).
There are mainly 3 steps to make a package with all dependencies (let it be A, B & C).
A. Gather required files.
There are many way to gather files of the main software and its dependencies. In order to get all the dependices and for error free run you need to use a base OS (i.e live system)
1. Using AppDirAssistant
This app is used by www.portablelinuxapps.org to create portable app directory. They scan and watch for the files accessed by the app to find required.
2. Using chroot & overlayfs
In this method you don't need to boot into live cd instead chroot into it.
a. mount the .iso # /cdrom and
b. mount the filesystem(filesystem.squashfs) # another place, say # /tmp/union/root
c. Bind mount /proc # /tmp/union/root/proc
d. Overlay on it
mount -t overlayfs overlayfs /tmp/union/root -o lowerdir=/tmp/union/root,upperdir=/tmp/union/rw
e. Chroot
chroot /tmp/union/root
Now you can install packages using apt-get or another method (only from the chrooted terminal). All the changed files are stored # /tmp/union/rw. Take files from there.
3. Using manually collected packages
Use package manager to collect dependencies. For example
apt-get install package --print-uris will print download uris for dep packages. Using this uris download packages and extract all (dpkg -x 1.deb ./extracted).
B. Clean garbage files
After gathering files remove unwanted files
C. Pack files
1. Using appImageAssistance
If you manually gathered files then you need to copy appname.desktop file from ./usr/share/applications to root of directory tree. Also copy file named AppRun from another app or extract it from AppDirAssistance.
2. Make a .deb or .rpm using gathered files.
Is the problem primarily that of ensuring that your customers have installed all the standard upstream distro packages necessary for your package to run?
If that's the case, then I believe the most straightforward solution would be to leverage the yum and apt infrastructure to have those tools track down and install the necessary prerequisite packages.
If you supply a native yum/apt repository with complete pre-req specs (the hard work you've apparently already completed). Then the standard system install tool takes care of the rest. See link below for more on creating a personal repository for yum/apt.
For off-line customers, you can supply media with your software, and a mirror - or mirror subset - of the upstream distro, and instructions for adding them to yum config/apt config.
Yum
Creating a Yum Repository in the Fedora Deployment Guide
Apt
How To Setup A Debian Repository on the Debian Wiki
So your customers aren't ever going to install any other software that might specify a different version of those dependencies that you are walking all over, right?
Why not just create your own distro if you're going to go that far?
Or you can just give them a bunch of packages and a single script that does rpm -i dep1 dep2 yourpackage
I have heard that it is best to not install modules from CPAN where your system's version of Perl is. I know how to install modules using the command line, I was just wondering if there is a way to keep CPAN separate from the system's core Perl.
Should I:
Download the source and make a directory specifically for these modules?
Anybody have any other ideas or implementations they have used successfully?
I am using Arch Linux with Perl 5.16.2.
Are you looking for something like local::lib
local::lib - create and use a local lib/ for perl modules with PERL5LIB
Download and extract the latest version of local::lib:
curl -LO http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.008004.tar.gz
tar xzf local-lib-1.008004.tar.gz
cd local-lib-1.008004/
Deploy it:
perl Makefile.PL --bootstrap=$HOME/perl5
make
make test
make install
Save persistent configuration:
cat << PROFILE >> $HOME/.profile
eval \$(perl -I\$HOME/perl5/lib/perl5/ -Mlocal::lib)
PROFILE
Now, you can logoff/logon your session or simply source ~/.profile.
After that steps, CPAN modules will be installed locally.
You don't need to install the module manually. You just need to have somewhere to install it to, and your environment configured to install it there. Then you can use cpan/cpanp/cpanm/etc as normal. (cpan minus wins for me)
Setting up that environment manually is a bit of a pain, so most people use an application to set up the configuration for them.
The two main choices for this are:
local::lib — This sets up your environment variables so you can install modules away from the system perl, but continues to use the system perl.
Perlbrew — this installs a complete perl for you so lets you avoid your system perl entirely, and use a more up to date version of perl itself then might come with your system. It also manages multiple perl installs side by side (so you can test your modules against different versions of perl).
Personally, I prefer Perlbrew (as it makes it easy to play with shiny new features like the yada yada operator and smart match (not that smart match is all that new now) but it takes longer to set up (as you have to compile perl).
I have heard that it is best to not install modules from CPAN where your system's version of Perl is.
The idea is to avoid breaking your distro's tools by upgrading a module they use.
Installing the modules to a fresh directory and telling Perl about it using PERL5LIB (which is what aforementioned install::lib does) is not going to help at all in that case, since Perl sees exactly the same thing as if you had installed the module in the usual site directory.
(One would mainly use PERL5LIB to install modules when one doesn't have permission to install to the default directories.)
The other problem with using the system Perl is that you are prevented from upgrading it.
The solution to both is to install your own build of Perl. This is very easy to do using perlbrew.
What's about cpanminus?
CPAN minus module
Why don't you pack the modules into real packages, rpm or dep style? That way you keep control over the installed software, you can remove and update the packages as required and as you are used to. So instead of bypassing the management, which rarely is a good idea, you stay in control.
If you are using an rpm based distribution I really recommend OBS for this task. You can create your own project, configure sources, test them and have packages created for all sorts of distributions and architectures. And when you import your home projects repository into your software management then installing the packages comes down to a single click.
I'm looking into trying to find an easy way to manage packages compiled from source so that when it comes time to upgrade, I'm not in a huge mess trying to uninstall/install the new package.
I found a utility called CheckInstall, but it seems to be quite old, and I was wondering if this a reliable solution before I begin using it?
http://www.asic-linux.com.mx/~izto/checkinstall/
Also would simply likely to know any other methods/utilities that you use to handle these installations from source?
Whatever you do, make sure that you eventually go through your distribution's package management system (e.g. rpm for Fedora/Mandriva/RH/SuSE, dpkg for Debian/Ubuntu etc). Otherwise your package manager will not know anything about the packages you installed by hand and you will have unsatisfied dependencies at best, or the mother of all messes at worst.
If you don't have a package manager, then get one and stick with it!
I would suggest that you learn to make your own packages. You can start by having a look at the source packages of your distribution. In fact, if all you want to do is upgrade to version 1.2.3 of MyPackage, your distribution's source package for 1.2.2 can usually be adapted with a simple version change (unless there are patches, but that's another story...).
Unless you want distribution-quality packages (e.g. split library/application/debugging packages, multiple-architecture support etc) it is usually easy to convert your typical configure & make & make install scenario into a proper source package. If you can convince your package to install into a directory rather than /, you are usually done.
As for checkinstall, I have used it in the past, and it worked for a couple of simple packages, but I did not like the fact that it actually let the package install itself onto my system before creating the rpm/deb package. It just tracked which files got installed so that it would package them, which did not protect against unwelcome changes. Oh, and it needed root prilileges to work, which is another main sticking point for me. And lets not go into what happens with statically linked core utilities...
Most tools of the kind seem to work that way, so I simply learnt to build my own packages The Right Way (TM) and let checkinstall and friends mess around elsewhere. If you are still interested, however, there is a list of similar programs here:
http://www.dwheeler.com/essays/automating-destdir.html
PS: BTW checkinstall was updated at the end of 2009, which probably means that it's still adequately current.
EDIT:
In my opinion, the easiest way to perform an upgrade to the latest version of a package if it is not readily available in a repository is to alter the source package of the latest version in your distribution. E.g. for Centos the source packages for the latest version are here:
http://mirror.centos.org/centos/5.5/os/SRPMS/
http://mirror.centos.org/centos/5.5/updates/SRPMS/
...
If you want to upgrade e.g. php, you get the latest SRPM for your distrbution e.g. php-5.1.6-27.el5.src.rpm. Then you do:
rpm -hiv php-5.1.6-27.el5.src.rpm
which installs the source package (just the sources - it does not compile anything). Then you go to the rpm build directory (on my mandriva system its /usr/src/rpm), you copy the latest php source tarball to the SOURCES subdirectory and you make sure it's compressed in the same way as the tarball that just got installed there. Afterwards you edit the php.spec file in the SPECS directory to change the package version and build the binary package with something like:
rpmbuild -ba php.spec
In many cases that's all it will take for a new package. In others things might get a bit more complicated - if there are patches or if there are some major changes in the package you might have to do more.
I suggest you read up on the rpm and rpmbuild commands (their manpages are quite good, in a bit extensive) and check up the documentation on writing spec files. Even if you decide to rely on official backport repositories, it is useful to know how to build your own packages. See also:
http://www.rpm.org/wiki/Docs
EDIT 2:
If you are already installing packages from source, using rpm will actually simplify the building process in the long term, apart from maintaining the integrity of your system. The reason for this is that you won't have to remember the quirks of each package on your own ("oooh, right, now I remember, foo needs me to add -lbar to its CFLAGS"), as the build process will be in the .spec file, which you could imagine as a somewhat structured build script.
As far as upgrading goes, if you already have a .spec file for a previous version of the package, there are two main issues that you may encounter, but both exist whether you use rpm to build your package or not:
A patch that was applied to the previous version by the distribution does not apply any more. In many cases the patch has already been applied to the upstream package, so you can simply drop it. In others you may have to edit it - or I suppose if you deem it unimportant you can drop it too.
The package changed in some major way which affected e.g. the layout of the files it installs. You do read the release notes notes for each new version, don't you?
Other than these two issues, upgrading often boils down to just changing a version number in the spec file and running rpmbuild - even easier than installing from a tarball.
I would suggest that you have a look at the tutorials or at the source package for some simple piece of software such as:
http://mirror.centos.org/centos/5.5/os/SRPMS/ipv6calc-0.61-1.src.rpm
http://mirror.centos.org/centos/5.5/os/SRPMS/libevent-1.4.13-1.src.rpm
If you have experience in buildling packages from a tarball, using rpm to build software is not much of a leap really. It will never be as simple as installing a premade binary package, however.
I use checkinstall on Debian. It should not be so different on CentOS. I use it like that:
./configure
make
sudo checkinstall make install # fakeroot in place of sudo works usally for more security
# install the package generated