Many sites (including various SO articles) talk about using "rename" using Perl expressions to rename files.
This would be perfect, but apparently this is not the rename utility I have, and none of these articles seem to comprehend that there are multiple versions of "rename" and I can't seem to find where to get version that accepts Perl expressions.
How can I get my hands on the more powerful rename utility mentioned here, here, and here?
I'm running Fedora 20. My current rename command is from the util-linux package and apparently I need the Perl version, which is better.
I can only speak for Debian. The two programs are called
/usr/bin/rename.ul from the util-linux package (hence the .ul suffix)
/usr/bin/prename from the perl package
The actual rename command works via the /etc/alternatives mechanism, whereby
/usr/bin/rename is a symlink to /etc/alternatives/rename
/etc/alternatives/rename is a symlink to /usr/bin/prename
The same problem has been bugging me on Cygwin, which is a Red Hat product, so should be more similar to Fedora. I'll have a look on my company laptop on Monday. And I remember the Perl-rename having worked there sometimes. Probably before I installed util-linux.
If you install the Perl-rename to /usr/local/bin it will have precedence over rename from util-linux. Same goes for the manpage when installed to /usr/local/share/man/man1/.
I've just created a separate Perl-rename package on Github: https://github.com/subogero/rename
You can install it using cpan, which is the perl repository similar to pip for python.
Here is a tutorial on using cpan.
If you try to run rename it it looks like this
rename --help
call: rename from to files...
To install the perl rename you can do the following. You might need to install a few dependencies, you can generally just push enter
cpan
cpan1> install File::Rename
CPAN: Storable loaded ok (v2.20)
Going to read '/root/.cpan/Metadata'
Database was generated on Wed, 30 Sep 2015 08:17:02 GMT
Running install for module 'File::Rename'
....
Running Build install
Installing /usr/local/share/man/man1/rename.1
Installing /usr/local/share/perl5/File/Rename.pm
Installing /usr/local/share/man/man3/File::Rename.3pm
Installing /usr/local/bin/rename
Writing /usr/local/lib64/perl5/auto/File/Rename/.packlist
RMBARKER/File-Rename-0.20.tar.gz
./Build install -- OK
That is how you would install the rename from cpan.
Next is to get it working on your system. As you might have more then one rename installed.
which rename
/usr/bin/rename
When you actually want this one.
/usr/local/bin/rename --help
Usage:
rename [ -h|-m|-V ] [ -v ] [ -n ] [ -f ] [ -e|-E *perlexpr*]*|*perlexpr*
[ *files* ]
Options:
-v, -verbose
Verbose: print names of files successfully renamed.
-n, -nono
No action: print names of files to be renamed, but don't rename.
-f, -force
Over write: allow existing files to be over-written.
-h, -help
Help: print SYNOPSIS and OPTIONS.
-m, -man
Manual: print manual page.
-V, -version
Version: show version number.
-e Expression: code to act on files name.
May be repeated to build up code (like "perl -e"). If no -e, the
first argument is used as code.
-E Statement: code to act on files name, as -e but terminated by
';'.
I just put it into /usr/bin/ but with a slight different name to make sure I did not break any existing scripts / programs the depend on the old one.
ln -s /usr/local/bin/rename /usr/bin/rename.pl
I had to do the following:
# In bash
sudo yum install perl-CPAN
sudo cpan
# In CPAN shell
install Module::Build
install File::Rename
On RedHat 8.4
sudo yum install perl-CPAN
sudo cpan
install module::Build
install File::Rename
than you can create an alias:
alias prename='/usr/local/bin/rename'
an use:
touch pic.jpeg
prename 's/\.jpeg$/.jpg/' *.jpeg
For Debian-family (.deb) distros, I recommend #SzG's answer.
For RedHat-family (.rpm) distros (e.g. Fedora), if your time is precious (like mine), you can download, compile, and install, from source via cpan in one, terse command:
# Install (replace `rename-1.9` below with another version if desired)
curl -L "http://search.cpan.org/CPAN/authors/id/P/PE/PEDERST/rename-1.9.tar.gz" | tar -xz && ( cd "rename-1.9"; perl "Makefile.PL"; make && make install )
# Cleanup
rm -rf "rename-1.9"
Note:
INSTALL_BASE can be set to modify the base installation directory.
e.g. perl "Makefile.PL" INSTALL_BASE=/usr/local
source
For Arch Linux, its
sudo pacman -S perl-rename
I created a post about Perl's rename for many distro:
https://unix.stackexchange.com/a/727288/12574
rpm based distros:
dnf install prename
archlinux:
pacman -S perl-rename
*BSD:
pkg install p5-File-Rename
Debian like/Ubuntu
apt install rename
slackware:
slackbuild
I recently had to install the glorious Perl rename package to Alpine Linux in a Docker container for a Gitlab CI/CD operation:
apk update
apk add --no-cache make perl-utils
cpan File::Rename
Related
Whenever i install a package using the sudo apt-get install command, I would like to know where in the file system the package is palced. Does it get placed in a single folder like in the windows file system, or do the files get placed across different directories?
list package files:
$ dpkg -L <package-name>
get package name: (apt install , so you should already know it), you can also list all package and find someone by grep command:
$ dpkg -l | grep <package-name-keyword>
Normally you could use
which program
To get the location of the executable. If you refer the program data each manufactor has a different location for data. Normally is under /usr/bin of /home/youruser
Trying using
ls -al
To show hidden folders
Good luck
Is it possible to use RPM or YUM or any other package manager in Linux, specifically CentOS, to install a package either already downloaded or from repo to a custom location without admin/root access?
I tried building from sources, using cmake, configure, make, make install etc, but, it ended up having so many dependencies one after other.
Or are there any better alternatives?
It is possible to use yum and rpm to install any package in the repository of the distribution. Here is the recipe:
Find the package name
Use yum search.
Download
Download the package and all of its dependencies using yumdownloader (which is available on CentOS by default). You'll need to pass it --resolve to get dependency resolution. yumdownloader downloads to the current directory unless you specify a --destdir.
mkdir -p ~/rpm
yumdownloader --destdir ~/rpm --resolve vim-common
Choose a prefix location
It might be ~, ~/centos, or ~/y. If your home is slow because it is on a network file system, you can put it in /var/tmp/....
mkdir ~/centos
Extract all .rpm packages
Extract all .rpm packages to your chosen prefix location.
cd ~/centos && rpm2cpio ~/rpm/x.rpm | cpio -id
rpm2cpio outputs the .rpm file as a .cpio archive on stdout.
cpio reads it from from stdin
-i means extract (to the current directory)
-d means create missing directory
You can optionally use -v: verbose
Configure the environment
You will need to configure the environment variable PATH and LD_LIBRARY_PATH for the installed packages to work correctly. Here is the corresponding sample from my ~/.bashrc:
export PATH="$HOME/centos/usr/sbin:$HOME/centos/usr/bin:$HOME/centos/bin:$PATH"
export MANPATH="$HOME/centos/usr/share/man:$MANPATH"
L='/lib:/lib64:/usr/lib:/usr/lib64'
export LD_LIBRARY_PATH="$HOME/centos/usr/lib:$HOME/centos/usr/lib64:$L"
Edited note (thanks to #AmitNaidu for pointing out my mistake):
According to bash documentation about startup files, when connecting to a server via ssh, only .bashrc is sourced:
Invoked by remote shell daemon
Bash attempts to determine when it is being run with its standard input connected to a network connection, as when executed by the remote shell daemon, usually rshd, or the secure shell daemon sshd. If Bash determines it is being run in this fashion, it reads and executes commands from ~/.bashrc, if that file exists and is readable.
Now if you want to install a lot of packages that way, you might want to automate the process. If so, have a look at this repository.
Extra note: if you are trying to install any of gcc, zlib, make, cmake, git, fish, zsh or tmux , you should really consider using conda, see my other answer.
TL;DR Use Miniconda, conda-forge is amazing.
curl "https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh" | sh
Or, alternatively:
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh > Miniconda.sh
bash Miniconda.sh -b -p ~/conda
# -b is used to specify that this is done "in batch", so skip the EULA prompt
# -p lets you specify where you want conda installed
Commonly wanted packages:
gcc conda install gcc
zlib conda install zlib
make conda install make
cmake conda install cmake
git conda install git
fish conda install -c conda-forge fish
zsh conda install -c ActivisionGameScience zsh
tmux conda install -c conda-forge tmux
This tmux has a bug with the name of the ncurse library it uses. You can work around it by going to your da/lib folder and symlinking ln -sT libtinfow.so.6.1 libtinfo.so.6
For the rest, you can try https://anaconda.org/search?q=.
I've tried for a long time to get a package manager to work well on CentOS/RedHat but without success. The best I could do was to install a Gentoo Prefix at the correct location on another CentOS with root access, then scp a .tar.xz of the whole installation to the target server (only way to get a proper gcc for Gentoo Prefix). I could emerge (build & install) packages on the target server but kept hitting problems with locals and permissions.
I recently achieved a user installation of some interesting packages using conda. Here is how to install it from the command line:
curl "https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh" | sh
If like me, your home folder is hosted on a remote drive (a network file system), you might not want to install it in your home folder, so you might want to use something like mkdir /var/tmp/lo then specify an installation folder like /var/tmp/lo/da during the installation.
You'll then be able to install quite a lot of packages, though maybe not all those you wanted. Most of the time, if it is not in the default channel, it will be in conda-forge. You can check for existing packages at https://anaconda.org/search?q=
Other package managers I've tried to use after conda:
Linuxbrew
I thought that with that it would be easy to install homebrew (linuxbrew) but their sources are messy and use hard-coded absolute path to ruby interpreter, which fails because it isn't the last version and so on and so on and I gave up.
Nix
Nix still requires you to use the /nix folder. They hard-coded it too and it's hard to sed it correctly from every download it has to do during the installation (let alone updates).
Gentoo Prefix
I expect Gentoo Prefix to be easier to install directly now that we gcc can be used on the target server. -- Ok, I tried but met permissions bugs during installation (2018-09-28):
portage.exception.OperationNotPermitted: chown(b'~/gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.12/image/var', 2000, 2000)
PkgSrc
I'm going to try pkgsrc now. -- Use (older) version 64-bit EL 6.x if on CentOS 6 or if encountering (G)LibC version issues with the 7.x one. -- No luck, pkgsrc hard codes /usr/pkg/sbin and /usr/pkg/bin. So it can't be used as user, unless maybe setting up a fakechroot environment. But I've never done that and I expect usability issues.
Please comment/answer if you succeed in installing any other package manager.
Download the packages, and indicate to include dependencies with the --resolve flag.
yumdownloader --resolve openslide-tools
Iterate over all downloaded rpm files.
for i in *.rpm; do rpm2cpio $i | cpio -idv; done
the output will be stored in your present working directory $PWD/usr/*
This answer by goldilocks sounds like what you are looking for.
https://unix.stackexchange.com/a/61295
It's still not a pretty process, but seems easier than building from source.
Otherwise you might want to look into non-root package managers as an alternative to yum.
Yes it is. If the software is packaged in repos. And admin installed
PackageKit-command-not-found package.
See:
https://fedoraproject.org/wiki/Features/PackageKitCommandNotFound
In our development environment, another team is using default Perl. So we shouldn't touch it. How do I install another Perl? How do I install Perl modules using CPAN?
anyenv is a great platform to install local versions of all the great open environments, Perl included:
$ git clone https://github.com/riywo/anyenv ~/.anyenv
$ echo 'export PATH="$HOME/.anyenv/bin:$PATH"' >> ~/.bash_profile
$ echo 'eval "$(anyenv init -)"' >> ~/.bash_profile # change profile if needed
$ exec $SHELL -l
This will set up anyenv. From here, you will install plenv, the Perl environment tool. Each of the environment tools allows you to manage that languages different installed versions.
$ anyenv install plenv
Now we can work with the plenv tool...
List available Perl versions:
$ plenv install --list
Install the Perl 5.18.2 binary:
$ plenv install 5.18.2 -Dusethreads
Change global default Perl to 5.18.2:
$ plenv global 5.18.2
Change local project Perl to 5.18.2:
$ plenv local 5.18.2
Run this command after installing a CPAN module, containing an executable script:
$ plenv rehash
Install cpanm to the current Perl:
$ plenv install-cpanm
Install any modules you need from CPAN with
$ cpanm JSON
I use Carton to manage dependencies within a project and recommend you take a look at it.
Now that you have anyenv, remember you can explore different versions of other languages too. anyenv is a priceless tool.
$ anyenv install --list
Available **envs:
denv
jenv
luaenv
ndenv
phpenv
plenv
pyenv
rbenv
That's what perlbrew is about.
After installing perlbrew, e.g. via
$ curl -L http://install.perlbrew.pl | bash
(or App::perlbrew from CPAN), you can use
$ perlbrew install perl-5.18.2
$ perlbrew switch perl-5.18.2
You need to download and install Perl from source. You may download Perl from http://www.perl.org/get.html.
In order to use another cpan from another Perl version you may not type "cpan" due to the fact that your Linux user will execute the default locations. Instead you have to execute your "alternate" cpan with the full alternate path. Execute with root and clear the hidden cpan folder from ".cpan" from user home.
I'm trying to install some perl module but everytime this message is shown
New CPAN.pm version (v2.00) available.
[Currently running version is v1.960001]
You might want to try
install CPAN
reload cpan
to both upgrade CPAN.pm and run the new version without leaving
the current session.
I've tried to do
install CPAN
reload cpan
With
pi#raspbmc:~$ sudo perl -MCPAN -e shell
Terminal does not support AddHistory.
cpan shell -- CPAN exploration and modules installation (v1.960001)
Enter 'h' for help.
cpan[1]> install CPAN
But the result is
Going to read '/root/.cpan/sources/authors/01mailrc.txt.gz'
............................................................................DONE
Going to read '/root/.cpan/sources/modules/02packages.details.txt.gz'
Database was generated on Mon, 28 Oct 2013 23:41:06 GMT
HTTP::Date not available
..............
New CPAN.pm version (v2.00) available.
[Currently running version is v1.960001]
You might want to try
install CPAN
reload cpan
to both upgrade CPAN.pm and run the new version without leaving
the current session.
...............pi#raspbmc:~$
The version is still 1.960001.
The following procedure
kuz1#banana:~$ sudo perl -MCPAN -e shell
cpan[1]> install CPAN
cpan[2]> reload cpan
works for me on the BANANA Pi Single-Board Computer with Ubuntu 14.04.
download CPAN module from this page http://metacpan.org/pod/CPAN
untar it (tar zxf CPAN-2.00.tar.gz), cd into the directory and run
perl Makefile.PL
make test
sudo make install
edit: if it breaks making a path with :: in it then perhaps your filesystem type does not support these characters in a filename
To see the filesystem type, run mount without any parameters and all the mounted filesystems with their types will be shown
The error Can't write-open blib/man3/CPAN::Admin.3pm seems to suggest a filesystem problem, but ext4 in rw mode sounds alright. Try this from the CPAN-2.00 directory
perl -e 'open($f,">blib/man3/CPAN::thisisatest") || die $!;'
Is there any apt-get-like program for use with Cygwin?
I already tried cyg-apt but when I try I get this error:
cyg-apt: downloading: http://cygwin.mirrors.pair.com/setup-2.bz2
cyg-apt: downloading: http://cygwin.mirrors.pair.com/setup-2.ini
cyg-apt: bad URL http://cygwin.mirrors.pair.com/setup-2.ini, exiting.
Best I have ever used:
apt-cyg package manager
You can do this using Cygwin’s setup.exe from Windows command line. Example:
cd C:\cygwin64
setup-x86_64 -q -P wget,tar,gawk,bzip2,subversion,vim
For a more convenient installer, you may want to use the
apt-cyg package manager. Its syntax is
similar to apt-get, which is a plus. For this, follow the above steps and then
use Cygwin Bash for the following steps:
wget rawgit.com/transcode-open/apt-cyg/master/apt-cyg
install apt-cyg /bin
Now that apt-cyg is installed. Here are a few examples of installing some
packages:
apt-cyg install nano
apt-cyg install git
apt-cyg install ca-certificates
Update: you can read the more complex answer, which contains more methods and information.
There exists a couple of scripts, which can be used as simple package managers. But as far as I know, none of them allows you to upgrade packages, because it’s not an easy task on Windows since there is not possible to overwrite files in use. So you have to close all Cygwin instances first and then you can use Cygwin’s native setup.exe (which itself does the upgrade via “replace after reboot” method, when files are in use).
apt-cyg
The best one for me. Simply because it’s one of the most recent. It works correctly for both platforms - x86 and x86_64. There exists a lot of forks with some additional features. For example the kou1okada fork is one of improved versions.
Cygwin’s setup.exe
It has also command line mode. Moreover it allows you to upgrade all installed packages at once.
setup.exe-x86_64.exe -q --packages=bash,vim
Example use:
setup.exe-x86_64.exe -q --packages="bash,vim"
You can create an alias for easier use, for example:
alias cyg-get="/cygdrive/d/path/to/cygwin/setup-x86_64.exe -q -P"
Then you can for example install the Vim package with:
cyg-get vim
you can always make a bash alias to setup*.exe files in $home/.bashrc
cygwin 32bit
alias cyg-get="/cygdrive/c/cygwin/setup-x86.exe -q -P"
cygwin 64bit
alias cyg-get="/cygdrive/c/cygwin64/setup-x86_64.exe -q -P"
now you can install packages with
cyg-get <package>
This got it working for me:
curl https://raw.githubusercontent.com/transcode-open/apt-cyg/master/apt-cyg > \
apt-cyg && install apt-cyg /bin
No. The only officially supported tool for downloading and updating Cygwin packages is the setup.exe file you used for the initial install, although that can be invoked with command line arguments to help the process.
From that same page:
The basic reason for not having a more full-featured package manager is that such a program would need full access to all of Cygwin's POSIX functionality. That is, however, difficult to provide in a Cygwin-free environment, such as exists on first installation. Additionally, Windows does not easily allow overwriting of in-use executables so installing a new version of the Cygwin DLL while a package manager is using the DLL is problematic.
You can use Chocolatey to install cyg-get and then install your packages with it.
For example:
choco install cyg-get
Then:
cyg-get install my-package