Docker: container cannot find local repo - linux

I am trying to build a centos image, then run basic yum commands from a company network with no internet access. After successfully grabbing the centos artifact in step 1, next comes RUN yum update where the container tries to load plugins using http://mirrorlist.centos.org, and that obviously will not work. It cannot resolve that host because no web access. So, I get the error:
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
..."Could not resolve host http://mirrorlist.centos.org; Unknown error"
One of the configured repositories failed (Unknown),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled
yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum
will then just ignore the repository until you permanently enable it
again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid>
or
subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot find a valid baseurl for repo: base/7/x86_64
The command '/bin/sh -c yum update' returned a non-zero code: 1
I have a repo file in /etc/yum.repos.d that contains content described here. In that file, I have multiple local repo URLs. An [updates] entry has a baseurl for /updates. Is this entry supposed to be used by the container when I do a RUN yum update in my Dockerfile? How does the container know where to look for a local mirror repo or other repo?
Is there also an issue regarding localhost on the host vs. localhost in the container?
I have researched a dozen S.O. entries with no luck.
UPDATE: Dockerfile so far...
FROM path.to.repo/centos
RUN yum update
So, it errors upon yum update.

When you're creating images that can't reach the web but only internal network, you must change tools configuration before trying to use them.
With yum, you have to remove existing repos and replacing them with yours before RUN yum update, something like that :
FROM path.to.repo/centos
RUN rm -rf /etc/yum.repos.d/*.repo
COPY myprivate.repo /etc/yum.repos.d/
RUN yum update
File myprivate.repo must be defined in the same folder as your Dockerfile and must declares your repos.
Furthermore, this created image can now be used as a base image for all others images you need to create.

Related

Using Gitlab as a yum repository?

My customer is migrating off of Nexus (which has a yum repository), and they want to use Gitlab. I know Docker can hold docker images and JAR files via its maven feature. But does Gitlab allow you to host yum repositories as well? I wasn't able to find anything after some googling.
You can store rpm packages (or .deb, etc.) in the Gitlab Registry, but there isn't official support for that package type so you'd have to use the "Generic" version. The downside to this is that you wouldn't be able to use the Gitlab Registry as a yum repo, however you could do something like:
#this downloads the package with filename `:filename:`
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/:project_id:/packages/generic/:package_name:/:package_version:/:filename:"
# Use rpm to install a package from a local file instead of a yum repo:
rpm -i :filename:.rpm
# For this use case, the file will have to be a .rpm file
The -i flag tells rpm to install the package. Another option is to yum localinstall :filename.rpm:.
Generic Packages must be enabled on your Gitlab instance (if you're using a self-hosted version).
Generic Packages docs are here: https://docs.gitlab.com/ee/user/packages/generic_packages/#download-package-file
An example .gitlab-ci.yml file using Generic Packages is here: https://gitlab.com/guided-explorations/cfg-data/write-ci-cd-variables-in-pipeline/-/blob/master/.gitlab-ci.yml
Check out OpenRepo: https://github.com/openkilt/openrepo
This is an open source package hosting server that can make packages available for both Debian (APT) and Red Hat (RPM) files.
In this case, you would configure your GitLab CI build to push your rpm files to the OpenRepo server.

Yum Repo Issues

My CentOS-Base.repo has the following mirrors:
[base]
name=CentOS-$releasever - Base
baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled=1
#released updates
[updates]
name=CentOS Server updates
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
baseurl=http://custom-url/centOS-updates/
enabled=1
When I say yum install yum-utils, it fails with the error saying
http://custom-url/centOS-updates/Packages/yum-utils-1.1.31-46.el7_5.noarch.rpm: [Errno 14] HTTP Error 500 - Internal Server Error
Trying other mirror.
Error downloading packages:
yum-utils-1.1.31-46.el7_5.noarch: [Errno 256] No more mirrors to try.
I have two questions here:
Why is it looking at the `updates` repo instead of base. `yum info yum-utils` has the repo field set to `updates`. When I disable the `updates` repo in the repo file and do a `yum info yum-utils` it shows the repo field as `base` correctly. What decides the repo to look at given a package?
Why is 'Package' appended to the baseUrl when it tries to find yum-utils in the updates repo? This is causing a `500 Internal Server Error` as the path with Packages appended to it is invalid
A first-aid to internal server error from Yum is most likely a corrupt repodata on the remote server and/or mismatch cache between the client and server.
To wipe-clean the cache, use
$ yum --enablerepo=* clean all
or just this, if all the repos are enabled by default:
$ yum clean all
This does all the clean up available in yum. Here are what they do:
CLEAN OPTIONS
The following are the ways which you can invoke yum in clean mode. Note that "all files" in the commands below means "all files in currently enabled reposito‐
ries". If you want to also clean any (temporarily) disabled repositories you need to use --enablerepo='*' option.
yum clean expire-cache
Eliminate the local data saying when the metadata and mirrorlists were downloaded for each repo. This means yum will revalidate the cache for each repo.
next time it is used. However if the cache is still valid, nothing significant was deleted.
yum clean packages
Eliminate any cached packages from the system. Note that packages are not automatically deleted after they are downloaded.
yum clean headers
Eliminate all of the header files, which old versions of yum used for dependency resolution.
yum clean metadata
Eliminate all of the files which yum uses to determine the remote availability of packages. Using this option will force yum to download all the metadata
the next time it is run.
yum clean dbcache
Eliminate the sqlite cache used for faster access to metadata. Using this option will force yum to download the sqlite metadata the next time it is run,
or recreate the sqlite metadata if using an older repo.
yum clean rpmdb
Eliminate any cached data from the local rpmdb.
yum clean plugins
Tell any enabled plugins to eliminate their cached data.
yum clean all
Does all of the above.

yum error - centos 7.1 x86_64

I'm facing an issue with the yum command on a dedicated server (hosted by OVH):
[root#mail-server ~]# yum clean all
[root#mail-server ~]# yum update
Modules complémentaires chargés : fastestmirror
One of the configured repositories failed (Inconnu),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).
3. Disable the repository, so yum won't use it by default. Yum will then
just ignore the repository until you permanently enable it again or use
--enablerepo for temporary usage:
yum-config-manager --disable <repoid>
4. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot find a valid baseurl for repo: base/7/x86_64
After investigation, i found that the error bellow (the url return not ok from the dedicated server) :
KO From dedicated server :
[root#mail-server ~]# curl "http://mirrorlist.centos.org/?release=7.1.1503&arch=x86_64&repo=os"
Invalid release
[root#mail-server ~]#
OK From home (ADSL)
MacBook-Air-de-John:~ jjohn$ curl "http://mirrorlist.centos.org/?release=7.1.1503&arch=x86_64&repo=os"
http://mirror.ate.info/ftp.centos.org/7.1.1503/os/x86_64/
http://ftp.rezopole.net/centos/7.1.1503/os/x86_64/
http://distrib-coffee.ipsl.jussieu.fr/pub/linux/centos/7.1.1503/os/x86_64/
http://mirror0.babylon.network/centos/7.1.1503/os/x86_64/
http://centos.crazyfrogs.org/7.1.1503/os/x86_64/
http://centos.mirrors.ovh.net/ftp.centos.org/7.1.1503/os/x86_64/
http://mirrors.ircam.fr/pub/CentOS/7.1.1503/os/x86_64/
ftp://ftp.free.fr/mirrors/ftp.centos.org/7.1.1503/os/x86_64/
http://mirror.ibcp.fr/pub/Centos/7.1.1503/os/x86_64/
http://ftp.ciril.fr/pub/linux/centos/7.1.1503/os/x86_64/
MacBook-Air-de-John:~ jjohn$
I got the same issue after a fresh install and on a friend's dedicated server (hosted by OVH too).
I don't know if the error is coming from OVH network (DNS, ...) or mirrorlist.centos.org server.
I solved the problem by typing
# dhclient
Your DNS doesn't seem to resolve the centos mirror list
Use
# dhclient
or Add "nameserver 8.8.8.8" in "/etc/resolv.conf" file
# vi /etc/resolv.conf [press i] now you are in insert mode
nameserver 8.8.8.8 [press: Esc][press :wq]
#
hope it work's..!
first you have to go to the /etc/yum.repos.d directory and edit the CentOS-Base.repo using vi editor. look for following lines;
mirrorlist=
baseurl=
remove the # sign in the #baseurl if you have difficulties in accessing the baseurl/mirrorlist web address using yum then save and exit the vi editor.
goodluck and enjoy the yum package-management utility in linux.
Best Regards-KB

Set proxy for terminal in linux

In CentOS 7 how can i connect terminal through a "proxy with username and password" and use some command like:
yum update
For now when i use this command i got this error:
Loaded plugins: fastestmirror, langpacks
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os error was
14: curl#7 - "Failed to connect to 2a02:2498:1:3d:5054:ff:fed3:e91a: Network is unreachable"
One of the configured repositories failed (Unknown),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).
3. Disable the repository, so yum won't use it by default. Yum will then
just ignore the repository until you permanently enable it again or use
--enablerepo for temporary usage:
yum-config-manager --disable <repoid>
4. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot find a valid baseurl for repo: base/7/x86_64
yum uses HTTP so you need to set a http proxy for your environment.
Check this out: https://www.centos.org/docs/5/html/yum/sn-yum-proxy-server.html
try to run the command and you will be connected to do upload or whatever you want
Just add : dhclient and press enter the this command : yum update

apt-get update and apt-get upgrade in Chef

If package "nginx" in Chef gets translated into apt-get install nginx on an Ubuntu node, what can be written in a Chef recipe that would translate into:
apt-get -y update
apt-get -y upgrade
Couldn't figure out from the apt cookbook.
The Opscode "apt" cookbook's default recipe will run apt-get update to ensure that the package cache is updated. We recommend putting that early in your node's run list so later on packages can be installed with the correct versions.
We generally don't recommend that users use "apt-get upgrade" in a recipe, for a couple reasons.
apt-get may upgrade a package that has conflicting configuration or other issues that cannot be resolved without running the command again, or running other apt/dpkg commands manually.
Automated upgrades of all packages on the system can have unintended side effects on the running system (the edge cases are many and possibly thorny, so I can't cover them all).
Instead, use the "upgrade" action for packages that should always update to the latest version.
package "nginx" do
action :upgrade
end
If you're reusing a cookbook that defines the cookbook, you can write a recipe that modifies the action of the existing resource, like this:
resources("package[nginx]").action(:upgrade)
The #resources method in a recipe will look up in the Resource Collection the specified resource (package nginx). Then sending the #action method with the argument :upgrade will tell Chef that the action should be to upgrade.
Edit Update: Do be careful when choosing packages that would be upgraded automatically in this way. An upstream change in a package can cause detrimental effects on the system. This is especially true if such a package does a restart of services it manages during the post installation scripts. Know your infrastructure, and if in doubt run your own package repository that has the critical packages you need for the application stack.
The Apt chef recipe will not update with every chef run. The attribute which controls this is called periodic_update_min_delay and is set to 86400 (The attribute should be called sec_delay). If the following file exists and is older than 24 hours apt will update the cache.
/var/lib/apt/periodic/update-success-stamp
It also appears that the apt recipe (default.rb) includes a directive to force an update which your recipe could call.
# For other recipes to call to force an update
execute 'apt-get update' do
If you're doing that though, you'll want a not_if to avoid running it too often at which point you might as well call it manually yourself. I got sick of messing with this and ended up just calling apt-get update in a stanza before my install.
execute "apt-get-update" do
command "apt-get update"
end
I suspect the long-term solution for security updates is to set update delay to a few hours.

Resources