Other package managers have a lock down option. For example, on a dev machine you might have certain packages and certain versions. The goal would be to install those same packages and versions on a staging or production environment. Sometimes this is called shrinkwrap or lockdown.
Does luarocks have something similar?
As of LuaRocks 2.4.2, this is unfortunately not available as a single command.
But here is a sequence of operations that produces a similar effect.
In the dev machine:
mkdir packages
cd packages
luarocks list --porcelain | awk '{print $1}' | uniq > packages.txt
for p in $(cat packages.txt); do luarocks pack $p; done
luarocks-admin make-manifest .
cd ..
tar czvpf packages.tar.gz packages
Copy packages.tar.gz to the target machine then run:
tar zxvpf packages.tar.gz
luarocks --only-server=./packages install foo
This will install package "foo" picking dependencies and sub-dependencies only from the packages/ directory (and not from the network), so all dependencies are guaranteed to be the ones you packaged in the dev machine.
Related
Is it possible to use RPM or YUM or any other package manager in Linux, specifically CentOS, to install a package either already downloaded or from repo to a custom location without admin/root access?
I tried building from sources, using cmake, configure, make, make install etc, but, it ended up having so many dependencies one after other.
Or are there any better alternatives?
It is possible to use yum and rpm to install any package in the repository of the distribution. Here is the recipe:
Find the package name
Use yum search.
Download
Download the package and all of its dependencies using yumdownloader (which is available on CentOS by default). You'll need to pass it --resolve to get dependency resolution. yumdownloader downloads to the current directory unless you specify a --destdir.
mkdir -p ~/rpm
yumdownloader --destdir ~/rpm --resolve vim-common
Choose a prefix location
It might be ~, ~/centos, or ~/y. If your home is slow because it is on a network file system, you can put it in /var/tmp/....
mkdir ~/centos
Extract all .rpm packages
Extract all .rpm packages to your chosen prefix location.
cd ~/centos && rpm2cpio ~/rpm/x.rpm | cpio -id
rpm2cpio outputs the .rpm file as a .cpio archive on stdout.
cpio reads it from from stdin
-i means extract (to the current directory)
-d means create missing directory
You can optionally use -v: verbose
Configure the environment
You will need to configure the environment variable PATH and LD_LIBRARY_PATH for the installed packages to work correctly. Here is the corresponding sample from my ~/.bashrc:
export PATH="$HOME/centos/usr/sbin:$HOME/centos/usr/bin:$HOME/centos/bin:$PATH"
export MANPATH="$HOME/centos/usr/share/man:$MANPATH"
L='/lib:/lib64:/usr/lib:/usr/lib64'
export LD_LIBRARY_PATH="$HOME/centos/usr/lib:$HOME/centos/usr/lib64:$L"
Edited note (thanks to #AmitNaidu for pointing out my mistake):
According to bash documentation about startup files, when connecting to a server via ssh, only .bashrc is sourced:
Invoked by remote shell daemon
Bash attempts to determine when it is being run with its standard input connected to a network connection, as when executed by the remote shell daemon, usually rshd, or the secure shell daemon sshd. If Bash determines it is being run in this fashion, it reads and executes commands from ~/.bashrc, if that file exists and is readable.
Now if you want to install a lot of packages that way, you might want to automate the process. If so, have a look at this repository.
Extra note: if you are trying to install any of gcc, zlib, make, cmake, git, fish, zsh or tmux , you should really consider using conda, see my other answer.
TL;DR Use Miniconda, conda-forge is amazing.
curl "https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh" | sh
Or, alternatively:
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh > Miniconda.sh
bash Miniconda.sh -b -p ~/conda
# -b is used to specify that this is done "in batch", so skip the EULA prompt
# -p lets you specify where you want conda installed
Commonly wanted packages:
gcc conda install gcc
zlib conda install zlib
make conda install make
cmake conda install cmake
git conda install git
fish conda install -c conda-forge fish
zsh conda install -c ActivisionGameScience zsh
tmux conda install -c conda-forge tmux
This tmux has a bug with the name of the ncurse library it uses. You can work around it by going to your da/lib folder and symlinking ln -sT libtinfow.so.6.1 libtinfo.so.6
For the rest, you can try https://anaconda.org/search?q=.
I've tried for a long time to get a package manager to work well on CentOS/RedHat but without success. The best I could do was to install a Gentoo Prefix at the correct location on another CentOS with root access, then scp a .tar.xz of the whole installation to the target server (only way to get a proper gcc for Gentoo Prefix). I could emerge (build & install) packages on the target server but kept hitting problems with locals and permissions.
I recently achieved a user installation of some interesting packages using conda. Here is how to install it from the command line:
curl "https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh" | sh
If like me, your home folder is hosted on a remote drive (a network file system), you might not want to install it in your home folder, so you might want to use something like mkdir /var/tmp/lo then specify an installation folder like /var/tmp/lo/da during the installation.
You'll then be able to install quite a lot of packages, though maybe not all those you wanted. Most of the time, if it is not in the default channel, it will be in conda-forge. You can check for existing packages at https://anaconda.org/search?q=
Other package managers I've tried to use after conda:
Linuxbrew
I thought that with that it would be easy to install homebrew (linuxbrew) but their sources are messy and use hard-coded absolute path to ruby interpreter, which fails because it isn't the last version and so on and so on and I gave up.
Nix
Nix still requires you to use the /nix folder. They hard-coded it too and it's hard to sed it correctly from every download it has to do during the installation (let alone updates).
Gentoo Prefix
I expect Gentoo Prefix to be easier to install directly now that we gcc can be used on the target server. -- Ok, I tried but met permissions bugs during installation (2018-09-28):
portage.exception.OperationNotPermitted: chown(b'~/gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.12/image/var', 2000, 2000)
PkgSrc
I'm going to try pkgsrc now. -- Use (older) version 64-bit EL 6.x if on CentOS 6 or if encountering (G)LibC version issues with the 7.x one. -- No luck, pkgsrc hard codes /usr/pkg/sbin and /usr/pkg/bin. So it can't be used as user, unless maybe setting up a fakechroot environment. But I've never done that and I expect usability issues.
Please comment/answer if you succeed in installing any other package manager.
Download the packages, and indicate to include dependencies with the --resolve flag.
yumdownloader --resolve openslide-tools
Iterate over all downloaded rpm files.
for i in *.rpm; do rpm2cpio $i | cpio -idv; done
the output will be stored in your present working directory $PWD/usr/*
This answer by goldilocks sounds like what you are looking for.
https://unix.stackexchange.com/a/61295
It's still not a pretty process, but seems easier than building from source.
Otherwise you might want to look into non-root package managers as an alternative to yum.
Yes it is. If the software is packaged in repos. And admin installed
PackageKit-command-not-found package.
See:
https://fedoraproject.org/wiki/Features/PackageKitCommandNotFound
which node
returns
/usr/sbin/node
which nvm
returns nothing
nvm ls returns nothing
express no longer works
If I try curl https://raw.githubusercontent.com/creationix/nvm/v0.11.1/install.sh | bash
It says
nvm is already installed in /home/adam/.nvm, trying to update
=> HEAD is now at 7a423b7... v0.11.1
error: branch 'master' not found.
and it is true that .nvm is located there.
Any idea what I should do to either uninstall everything and reinstall it or fix my install?
The problem is that apt-get install nodejs installs node as "nodejs" on your path. Grunt, express, etc. all expect node to be node on your path. The reason for this is there was a package named node already in the Ubuntu apt repository before node.js came into existence, so node.js couldn't claim "node". The simplest solution is to do which nodejs then create a symlink targeting that file, name it node, and put that on your path (you can just put it in the same directory).
For you I recommend uninstalling everything first (apt-get remove).
I fixed this by deleting everything (purge Ubuntu packages and delete everything created by node and npm in /usr/*) and compiling the node package from their website.
I copied the commands from https://www.digitalocean.com/community/tutorials/how-to-install-an-upstream-version-of-node-js-on-ubuntu-12-04 instead of linking, in case the site goes offline:
echo 'export PATH=$HOME/local/bin:$PATH' >> ~/.bashrc
. ~/.bashrc
mkdir ~/local
mkdir ~/node-latest-install
cd ~/node-latest-install
curl http://nodejs.org/dist/node-latest.tar.gz | tar xz --strip-components=1
./configure --prefix=~/local
make install
curl -L https://npmjs.org/install.sh | sh
node -v
The unofficial install scripts are buggy and it's about time they make an official repository or pre-compile some .deb files to make the installation easier. Hope this helps.
Many sites (including various SO articles) talk about using "rename" using Perl expressions to rename files.
This would be perfect, but apparently this is not the rename utility I have, and none of these articles seem to comprehend that there are multiple versions of "rename" and I can't seem to find where to get version that accepts Perl expressions.
How can I get my hands on the more powerful rename utility mentioned here, here, and here?
I'm running Fedora 20. My current rename command is from the util-linux package and apparently I need the Perl version, which is better.
I can only speak for Debian. The two programs are called
/usr/bin/rename.ul from the util-linux package (hence the .ul suffix)
/usr/bin/prename from the perl package
The actual rename command works via the /etc/alternatives mechanism, whereby
/usr/bin/rename is a symlink to /etc/alternatives/rename
/etc/alternatives/rename is a symlink to /usr/bin/prename
The same problem has been bugging me on Cygwin, which is a Red Hat product, so should be more similar to Fedora. I'll have a look on my company laptop on Monday. And I remember the Perl-rename having worked there sometimes. Probably before I installed util-linux.
If you install the Perl-rename to /usr/local/bin it will have precedence over rename from util-linux. Same goes for the manpage when installed to /usr/local/share/man/man1/.
I've just created a separate Perl-rename package on Github: https://github.com/subogero/rename
You can install it using cpan, which is the perl repository similar to pip for python.
Here is a tutorial on using cpan.
If you try to run rename it it looks like this
rename --help
call: rename from to files...
To install the perl rename you can do the following. You might need to install a few dependencies, you can generally just push enter
cpan
cpan1> install File::Rename
CPAN: Storable loaded ok (v2.20)
Going to read '/root/.cpan/Metadata'
Database was generated on Wed, 30 Sep 2015 08:17:02 GMT
Running install for module 'File::Rename'
....
Running Build install
Installing /usr/local/share/man/man1/rename.1
Installing /usr/local/share/perl5/File/Rename.pm
Installing /usr/local/share/man/man3/File::Rename.3pm
Installing /usr/local/bin/rename
Writing /usr/local/lib64/perl5/auto/File/Rename/.packlist
RMBARKER/File-Rename-0.20.tar.gz
./Build install -- OK
That is how you would install the rename from cpan.
Next is to get it working on your system. As you might have more then one rename installed.
which rename
/usr/bin/rename
When you actually want this one.
/usr/local/bin/rename --help
Usage:
rename [ -h|-m|-V ] [ -v ] [ -n ] [ -f ] [ -e|-E *perlexpr*]*|*perlexpr*
[ *files* ]
Options:
-v, -verbose
Verbose: print names of files successfully renamed.
-n, -nono
No action: print names of files to be renamed, but don't rename.
-f, -force
Over write: allow existing files to be over-written.
-h, -help
Help: print SYNOPSIS and OPTIONS.
-m, -man
Manual: print manual page.
-V, -version
Version: show version number.
-e Expression: code to act on files name.
May be repeated to build up code (like "perl -e"). If no -e, the
first argument is used as code.
-E Statement: code to act on files name, as -e but terminated by
';'.
I just put it into /usr/bin/ but with a slight different name to make sure I did not break any existing scripts / programs the depend on the old one.
ln -s /usr/local/bin/rename /usr/bin/rename.pl
I had to do the following:
# In bash
sudo yum install perl-CPAN
sudo cpan
# In CPAN shell
install Module::Build
install File::Rename
On RedHat 8.4
sudo yum install perl-CPAN
sudo cpan
install module::Build
install File::Rename
than you can create an alias:
alias prename='/usr/local/bin/rename'
an use:
touch pic.jpeg
prename 's/\.jpeg$/.jpg/' *.jpeg
For Debian-family (.deb) distros, I recommend #SzG's answer.
For RedHat-family (.rpm) distros (e.g. Fedora), if your time is precious (like mine), you can download, compile, and install, from source via cpan in one, terse command:
# Install (replace `rename-1.9` below with another version if desired)
curl -L "http://search.cpan.org/CPAN/authors/id/P/PE/PEDERST/rename-1.9.tar.gz" | tar -xz && ( cd "rename-1.9"; perl "Makefile.PL"; make && make install )
# Cleanup
rm -rf "rename-1.9"
Note:
INSTALL_BASE can be set to modify the base installation directory.
e.g. perl "Makefile.PL" INSTALL_BASE=/usr/local
source
For Arch Linux, its
sudo pacman -S perl-rename
I created a post about Perl's rename for many distro:
https://unix.stackexchange.com/a/727288/12574
rpm based distros:
dnf install prename
archlinux:
pacman -S perl-rename
*BSD:
pkg install p5-File-Rename
Debian like/Ubuntu
apt install rename
slackware:
slackbuild
I recently had to install the glorious Perl rename package to Alpine Linux in a Docker container for a Gitlab CI/CD operation:
apk update
apk add --no-cache make perl-utils
cpan File::Rename
In our development environment, another team is using default Perl. So we shouldn't touch it. How do I install another Perl? How do I install Perl modules using CPAN?
anyenv is a great platform to install local versions of all the great open environments, Perl included:
$ git clone https://github.com/riywo/anyenv ~/.anyenv
$ echo 'export PATH="$HOME/.anyenv/bin:$PATH"' >> ~/.bash_profile
$ echo 'eval "$(anyenv init -)"' >> ~/.bash_profile # change profile if needed
$ exec $SHELL -l
This will set up anyenv. From here, you will install plenv, the Perl environment tool. Each of the environment tools allows you to manage that languages different installed versions.
$ anyenv install plenv
Now we can work with the plenv tool...
List available Perl versions:
$ plenv install --list
Install the Perl 5.18.2 binary:
$ plenv install 5.18.2 -Dusethreads
Change global default Perl to 5.18.2:
$ plenv global 5.18.2
Change local project Perl to 5.18.2:
$ plenv local 5.18.2
Run this command after installing a CPAN module, containing an executable script:
$ plenv rehash
Install cpanm to the current Perl:
$ plenv install-cpanm
Install any modules you need from CPAN with
$ cpanm JSON
I use Carton to manage dependencies within a project and recommend you take a look at it.
Now that you have anyenv, remember you can explore different versions of other languages too. anyenv is a priceless tool.
$ anyenv install --list
Available **envs:
denv
jenv
luaenv
ndenv
phpenv
plenv
pyenv
rbenv
That's what perlbrew is about.
After installing perlbrew, e.g. via
$ curl -L http://install.perlbrew.pl | bash
(or App::perlbrew from CPAN), you can use
$ perlbrew install perl-5.18.2
$ perlbrew switch perl-5.18.2
You need to download and install Perl from source. You may download Perl from http://www.perl.org/get.html.
In order to use another cpan from another Perl version you may not type "cpan" due to the fact that your Linux user will execute the default locations. Instead you have to execute your "alternate" cpan with the full alternate path. Execute with root and clear the hidden cpan folder from ".cpan" from user home.
I want to upgrade all packages, not just a specific one with cabal install --upgrade-dependencies.
This bit of shell hackery works for me on OS X:
cabal list --simple-output --installed | awk '{print $1}' | uniq | xargs -I {} cabal install {} --reinstall
EDIT: Now forces a reinstall, and avoids installing a package more than once when more than one version is present. Thanks for the comments!
EDIT YEARS LATER: Now that Cabal sandboxes and Stack exist, I strongly recommend against trying to upgrade packages in place. You'll end up with far fewer headaches if you instead can just wipe out an existing sandbox and reinstall up-to-date dependencies.
The .cabal/world file contains a list of every package you installed explicitly (listed in a cabal install command, as opposed to pulled through dependencies). Trim it to remove packages that are only useful as dependencies, packages that are deprecated, and version-locked packages that you'd rather upgrade.
Cabal doesn't know how to clean-up after itself, but you can remove almost everything. The next command will reinstall from .cabal/packages (a tarball cache):
cp -t bin .cabal/bin/cabal
rm -rf .cabal/{bin,lib,share} .ghc/*-*-*/
ghc-pkg check |&egrep -- '^[A-Za-z0-9-]+-[0-9]' |xargs -n1 --no-run-if-empty ghc-pkg unregister
Now reinstall everything that was manually installed:
cabal install world --upgrade-dependencies --force-reinstalls