I'm trying to understand the Dockerfile https://hub.docker.com/r/rdsubhas/tor-privoxy-alpine/~/dockerfile/, which contains a RUN executive with
apk --update add privoxy tor#testing runit#testing
I wanted to check my understanding of how the apk command is being used, so I tried opening a terminal in the Alpine environment as follows:
docker run -it --rm alpine:latest /bin/ash
after which I simply ran apk to see its usage:
/ # apk
apk-tools 2.6.8, compiled for x86_64.
usage: apk COMMAND [-h|--help] [-p|--root DIR] [-X|--repository REPO]
[-q|--quiet] [-v|--verbose] [-i|--interactive] [-V|--version]
[-f|--force] [-U|--update-cache] [--progress] [--progress-fd FD]
[--no-progress] [--purge] [--allow-untrusted] [--wait TIME]
[--keys-dir KEYSDIR] [--repositories-file REPOFILE] [--no-network]
[--no-cache] [--arch ARCH] [--print-arch] [ARGS]...
The following commands are available:
add Add PACKAGEs to 'world' and install (or upgrade) them, while
ensuring that all dependencies are met
del Remove PACKAGEs from 'world' and uninstall them
fix Repair package or upgrade it without modifying main dependencies
update Update repository indexes from all remote repositories
info Give detailed information about PACKAGEs or repositores
search Search package by PATTERNs or by indexed dependencies
upgrade Upgrade currently installed packages to match repositories
cache Download missing PACKAGEs to cache and/or delete unneeded files
from cache
version Compare package versions (in installed database vs. available) or
do tests on literal version strings
index Create repository index file from FILEs
fetch Download PACKAGEs from global repositories to a local directory
audit Audit the directories for changes
verify Verify package integrity and signature
dot Generate graphviz graphs
policy Show repository policy for packages
stats Show statistics about repositories and installations
Global options:
-h, --help Show generic help or applet specific help
-p, --root DIR Install packages to DIR
-X, --repository REPO Use packages from REPO
-q, --quiet Print less information
-v, --verbose Print more information (can be doubled)
-i, --interactive Ask confirmation for certain operations
-V, --version Print program version and exit
-f, --force Do what was asked even if it looks dangerous
-U, --update-cache Update the repository cache
--progress Show a progress bar
--progress-fd FD Write progress to fd
--no-progress Disable progress bar even for TTYs
--purge Delete also modified configuration files (pkg
removal) and uninstalled packages from cache (cache
clean)
--allow-untrusted Install packages with untrusted signature or no
signature
--wait TIME Wait for TIME seconds to get an exclusive repository
lock before failing
--keys-dir KEYSDIR Override directory of trusted keys
--repositories-file REPOFILE Override repositories file
--no-network Do not use network (cache is still used)
--no-cache Read uncached index from network
--arch ARCH Use architecture with --root
--print-arch Print default arch and exit
This apk has coffee making abilities.
The thing is, I don't see the --update option in this documentation (only --update-cache).
What I suspect is that apk --update add [package] is simply shorthand for apk update followed by apk add [package]. Can anyone confirm this?
See https://github.com/gliderlabs/docker-alpine/pull/503
apk --update flag is really --update-cache.
Apk uses getopt_long (3),
https://github.com/alpinelinux/apk-tools/blob/v2.10.3/src/apk.c#L574
So, --update flag is only abbreviated from --update-cache by getopt_long.
Long option names may be abbreviated if the abbreviation is unique or
is an exact match for some defined option.
In short:
To get the latest list of available packages, use the update command.
it is similar to the Debian apt-get update that you do before apt-get install my_package.
from https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management#Update_the_Package_list:
extract
Update the Package list
Remote repositories change as packages are added and upgraded. To get the latest list of available packages, use the update command. The command downloads the APKINDEX.tar.gz from each repository and stores it in the local cache, typically /var/cache/apk/, /var/lib/apk/ or /etc/apk/cache/.
apk update
Tip: If using remote repositories, it is a good idea to do an update just before doing an add or upgrade command. That way you know you are using the latest software available.
Related
I have GCC v9. But I'm trying to install a GCC 4.8.1 version to test a library compilation on that very old version of GCC.
The version is not available in the official Ubuntu repos,it is deprecated, but I've found it in other mirrors as told by the official GCC website. This one seems like popular one:
https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test
I have very little knowledge of linux package systems except for the basic. I want to keep both versions. So I should do this:
sudo apt -y install gcc-4.8.1 gcc-9
The reason why I want to use this command and not install it from the file, apart from the difficulty of doing that for me, is that I'm following a guide in order to have several GCCs on my system:
https://www.fosslinux.com/39386/how-to-install-multiple-versions-of-gcc-and-g-on-ubuntu-20-04.htm
When I add the url to the sources.list file seems like it is working.
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt update -q
But when I try to call the install with gcc-4.8.1 or gcc-4.8 , or even gcc-4 the package doesn't exist.
Package gcc-4.8 is not available, but is referred to by another
package. This may mean that the package is missing, has been
obsoleted, or is only available from another source E: Package
'gcc-4.8' has no installation candidate
Also, I don't know if websites like these can be added to the repos list in order to find the package using APT:
http://www.netgull.com/gcc/releases/gcc-4.8.1/
[EDIT]
I downloaded the package from the website I linked. I have no idea how to install this by hand. If only I could find a repository that could help me with this... I have no idea how to make APT help me with the installation.
But I'm trying to install a GCC 4.8.1 version to test a library compilation on that very old version of GCC.
Developers have tools up their sleeve so they don't have to install dependencies and bloating their systems for every library (and every configuration of that library!) they want to try out and test.
Use docker. You could write for example a testing script, assuming your project uses make:
# test_my_lib_in_gcc-4.8.sh
#!/bin/sh
docker run -ti --rm -v $PWD:/project -w /project gcc:4.8 -u $UID:$GID sh <<EOF
make && make test
EOF
that will compile and test your application in using 4.8 gcc. Consider how easy it is to change gcc version - just change the number. You could test your library in gcc, in different versions, and using other compilers and on different distributions to make sure it works for others. If you're a developer of the library, write an automatized CI pipeline that would automatically test your application each commit in specific docker environment, using ex. https://docs.gitlab.com/ee/ci/README.html or https://travis-ci.org/ .
I am trying to build a centos image, then run basic yum commands from a company network with no internet access. After successfully grabbing the centos artifact in step 1, next comes RUN yum update where the container tries to load plugins using http://mirrorlist.centos.org, and that obviously will not work. It cannot resolve that host because no web access. So, I get the error:
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
..."Could not resolve host http://mirrorlist.centos.org; Unknown error"
One of the configured repositories failed (Unknown),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled
yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum
will then just ignore the repository until you permanently enable it
again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid>
or
subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot find a valid baseurl for repo: base/7/x86_64
The command '/bin/sh -c yum update' returned a non-zero code: 1
I have a repo file in /etc/yum.repos.d that contains content described here. In that file, I have multiple local repo URLs. An [updates] entry has a baseurl for /updates. Is this entry supposed to be used by the container when I do a RUN yum update in my Dockerfile? How does the container know where to look for a local mirror repo or other repo?
Is there also an issue regarding localhost on the host vs. localhost in the container?
I have researched a dozen S.O. entries with no luck.
UPDATE: Dockerfile so far...
FROM path.to.repo/centos
RUN yum update
So, it errors upon yum update.
When you're creating images that can't reach the web but only internal network, you must change tools configuration before trying to use them.
With yum, you have to remove existing repos and replacing them with yours before RUN yum update, something like that :
FROM path.to.repo/centos
RUN rm -rf /etc/yum.repos.d/*.repo
COPY myprivate.repo /etc/yum.repos.d/
RUN yum update
File myprivate.repo must be defined in the same folder as your Dockerfile and must declares your repos.
Furthermore, this created image can now be used as a base image for all others images you need to create.
I'm trying to install aws-cli from edge repository but I cannot
https://pkgs.alpinelinux.org/package/edge/community/x86_64/aws-cli
Is it an issue with OS version? ( 3.11 / 3.12 )
If so is there a workaround?
root#6f97c6559fe9:/ # echo http://dl-cdn.alpinelinux.org/alpine/edge/main >> /etc/apk/repositories
root#6f97c6559fe9:/ # apk update
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
v3.11.6-71-gb45d3b45cc [http://dl-cdn.alpinelinux.org/alpine/v3.11/main]
v3.11.6-68-gf6abc2afac [http://dl-cdn.alpinelinux.org/alpine/v3.11/community]
v3.12.0-442-g76e377ea0b [http://dl-cdn.alpinelinux.org/alpine/edge/main]
OK: 16123 distinct packages available
root#6f97c6559fe9:/ # apk add aws-cli
ERROR: unsatisfiable constraints:
aws-cli (missing):
required by: world[aws-cli]
root#6f97c6559fe9:/ # apk add --update aws-cli
ERROR: unsatisfiable constraints:
aws-cli (missing):
required by: world[aws-cli]
root#6f97c6559fe9:/ # cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.11.3
PRETTY_NAME="Alpine Linux v3.11"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
So the package page of Alpine seems to confirm that aws-cli is indeed not part of Alpine 3.11 package repository.
This said, you can install it using AWS own set of instructions, you will just need both curl and python in order to do so.
For AWS CLI v1:
apk add python curl
curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Source: https://docs.aws.amazon.com/cli/latest/userguide/install-linux.html#install-linux-bundled
For AWS CLI v2, sadly, it seems Alpine is not yet supported:
nateprewitt commented on 24 Feb
Hi #firstval, it looks like you found
a response on this behavior in #4685. We're currently tracking Docker
support in #3553 which would be a prerequisite for this to work.
That said, we can definitely do better with the exceptions being
returned. We're working on getting a warning in our install script to
will alert you when the platform isn't supported.
We'll track the remaining piece for alpine support in #3553. Thanks!
Source: https://github.com/aws/aws-cli/issues/4971
Further down in #3553:
There is an official docker image for aws-cli: https://hub.docker.com/r/amazon/aws-cli
Some people got it working, but with quite a huge amount of dependancies needed (you actually need a c compiler, as it seems): https://github.com/aws/aws-cli/issues/3553#issuecomment-615149941
This would also explain why, even on Alpine 3.12, the actual package install the version 1.xx and not a 2.xx version.
Running on Alpine Linux 3.10, I've installed the distribution's git package using apk.
In an existing checkout directory, when I try to launch git add's interactive mode:
$ git add -i
I get the error:
git: 'add--interactive' is not a git command. See 'git --help'.
The git add help indicates that -i is a valid option.
What is happening?
The interactive mode feature of git uses perl, and in many Linux distributions the perl-based parts of git are separated out into another package, so that the core functionality of git can be used without needing to install perl.
On Alpine, the git package just has this core functionality.
To get the missing functionality on Alpine, install the git-perl package.
$ sudo apk add git-perl
On RedHat Linux, you may need to add the perl-Git package:
$ sudo dnf install -y perl-Git
Is it possible to use RPM or YUM or any other package manager in Linux, specifically CentOS, to install a package either already downloaded or from repo to a custom location without admin/root access?
I tried building from sources, using cmake, configure, make, make install etc, but, it ended up having so many dependencies one after other.
Or are there any better alternatives?
It is possible to use yum and rpm to install any package in the repository of the distribution. Here is the recipe:
Find the package name
Use yum search.
Download
Download the package and all of its dependencies using yumdownloader (which is available on CentOS by default). You'll need to pass it --resolve to get dependency resolution. yumdownloader downloads to the current directory unless you specify a --destdir.
mkdir -p ~/rpm
yumdownloader --destdir ~/rpm --resolve vim-common
Choose a prefix location
It might be ~, ~/centos, or ~/y. If your home is slow because it is on a network file system, you can put it in /var/tmp/....
mkdir ~/centos
Extract all .rpm packages
Extract all .rpm packages to your chosen prefix location.
cd ~/centos && rpm2cpio ~/rpm/x.rpm | cpio -id
rpm2cpio outputs the .rpm file as a .cpio archive on stdout.
cpio reads it from from stdin
-i means extract (to the current directory)
-d means create missing directory
You can optionally use -v: verbose
Configure the environment
You will need to configure the environment variable PATH and LD_LIBRARY_PATH for the installed packages to work correctly. Here is the corresponding sample from my ~/.bashrc:
export PATH="$HOME/centos/usr/sbin:$HOME/centos/usr/bin:$HOME/centos/bin:$PATH"
export MANPATH="$HOME/centos/usr/share/man:$MANPATH"
L='/lib:/lib64:/usr/lib:/usr/lib64'
export LD_LIBRARY_PATH="$HOME/centos/usr/lib:$HOME/centos/usr/lib64:$L"
Edited note (thanks to #AmitNaidu for pointing out my mistake):
According to bash documentation about startup files, when connecting to a server via ssh, only .bashrc is sourced:
Invoked by remote shell daemon
Bash attempts to determine when it is being run with its standard input connected to a network connection, as when executed by the remote shell daemon, usually rshd, or the secure shell daemon sshd. If Bash determines it is being run in this fashion, it reads and executes commands from ~/.bashrc, if that file exists and is readable.
Now if you want to install a lot of packages that way, you might want to automate the process. If so, have a look at this repository.
Extra note: if you are trying to install any of gcc, zlib, make, cmake, git, fish, zsh or tmux , you should really consider using conda, see my other answer.
TL;DR Use Miniconda, conda-forge is amazing.
curl "https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh" | sh
Or, alternatively:
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh > Miniconda.sh
bash Miniconda.sh -b -p ~/conda
# -b is used to specify that this is done "in batch", so skip the EULA prompt
# -p lets you specify where you want conda installed
Commonly wanted packages:
gcc conda install gcc
zlib conda install zlib
make conda install make
cmake conda install cmake
git conda install git
fish conda install -c conda-forge fish
zsh conda install -c ActivisionGameScience zsh
tmux conda install -c conda-forge tmux
This tmux has a bug with the name of the ncurse library it uses. You can work around it by going to your da/lib folder and symlinking ln -sT libtinfow.so.6.1 libtinfo.so.6
For the rest, you can try https://anaconda.org/search?q=.
I've tried for a long time to get a package manager to work well on CentOS/RedHat but without success. The best I could do was to install a Gentoo Prefix at the correct location on another CentOS with root access, then scp a .tar.xz of the whole installation to the target server (only way to get a proper gcc for Gentoo Prefix). I could emerge (build & install) packages on the target server but kept hitting problems with locals and permissions.
I recently achieved a user installation of some interesting packages using conda. Here is how to install it from the command line:
curl "https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh" | sh
If like me, your home folder is hosted on a remote drive (a network file system), you might not want to install it in your home folder, so you might want to use something like mkdir /var/tmp/lo then specify an installation folder like /var/tmp/lo/da during the installation.
You'll then be able to install quite a lot of packages, though maybe not all those you wanted. Most of the time, if it is not in the default channel, it will be in conda-forge. You can check for existing packages at https://anaconda.org/search?q=
Other package managers I've tried to use after conda:
Linuxbrew
I thought that with that it would be easy to install homebrew (linuxbrew) but their sources are messy and use hard-coded absolute path to ruby interpreter, which fails because it isn't the last version and so on and so on and I gave up.
Nix
Nix still requires you to use the /nix folder. They hard-coded it too and it's hard to sed it correctly from every download it has to do during the installation (let alone updates).
Gentoo Prefix
I expect Gentoo Prefix to be easier to install directly now that we gcc can be used on the target server. -- Ok, I tried but met permissions bugs during installation (2018-09-28):
portage.exception.OperationNotPermitted: chown(b'~/gentoo/tmp/var/tmp/portage/sys-apps/gentoo-functions-0.12/image/var', 2000, 2000)
PkgSrc
I'm going to try pkgsrc now. -- Use (older) version 64-bit EL 6.x if on CentOS 6 or if encountering (G)LibC version issues with the 7.x one. -- No luck, pkgsrc hard codes /usr/pkg/sbin and /usr/pkg/bin. So it can't be used as user, unless maybe setting up a fakechroot environment. But I've never done that and I expect usability issues.
Please comment/answer if you succeed in installing any other package manager.
Download the packages, and indicate to include dependencies with the --resolve flag.
yumdownloader --resolve openslide-tools
Iterate over all downloaded rpm files.
for i in *.rpm; do rpm2cpio $i | cpio -idv; done
the output will be stored in your present working directory $PWD/usr/*
This answer by goldilocks sounds like what you are looking for.
https://unix.stackexchange.com/a/61295
It's still not a pretty process, but seems easier than building from source.
Otherwise you might want to look into non-root package managers as an alternative to yum.
Yes it is. If the software is packaged in repos. And admin installed
PackageKit-command-not-found package.
See:
https://fedoraproject.org/wiki/Features/PackageKitCommandNotFound