Using mount command while Docker build - linux

So this is not about seeking workarounds to -v.
I have a Dockerfile whose intent is to install a cross-compiler in /usr/local/<cross-compiler-path>, inside the container. Later during a build process, a file would be mounted to this cross-compiler, like this:
root#5bee5daf8165:/# mount <blah.img.gz> /usr/local/<cross-compiler-path>
I get mount: /usr/local/<cross-compiler-path>: mount failed: Operation not permitted.
Although if I skip this step, finish build, run a --privileged container and mount, it works fine.
I understand the reason for not giving privileged mode in the build since it breaks the 'portability' of containers as they depend on host volumes. But in my case, I am attempting to mount it inside the Container's own file system. Why is that not allowed?
For the record, I tried installing the cross-compiler on a different path, like this:
root#5bee5daf8165:/# mount <blah.img.gz> /home/<cross-compiler-path>
But that doesn't work either. I want to attempt the build inside the Dockerfile and discard the build cache which bloat up my container once I no longer need them. What options do I have?

As mentioned in "Can You Mount a Volume While Building Your Docker Image to Cache Dependencies?" from Vladislav Supalov
Although there’s no functionality in Docker to have volumes at build-time, you can use multi-stage builds, benefit from Docker caching and save time by copying data from other images - be it multi-stage or tagged ones.
When building an image, you can’t mount a volume. However, you can copy (COPY) data from another image! By combining this, with a multi-stage build, you can pre-compute an expensive operation once, and re-use the resulting state as a starting point for future iterations.
Example:
FROM ubuntu as intermediate
RUN apt-get install -yqq python-dev python-virtualenv
RUN virtualenv /venv/
RUN mkdir -p /src
# those don't change often
ADD code/basic-requirements.txt /src/basic-requirements.txt
RUN /venv/bin/pip install -r /src/basic-requirements.txt
FROM ubuntu
RUN apt-get install -yqq python-dev python-virtualenv
# the data comes from the above container
COPY --from=intermediate /venv /venv
ADD code/requirements.txt /src/requirements.txt
# this command, starts from an almost-finished state every time
RUN /venv/bin/pip install -r /app/requirements.txt
The OP add in the comments:
I want to mount a volume internally to the container fs using the mount command while build, which currently doesn't work.
Just wanted to know if 'mount' operation, in general is tied to the kernel?
Kernel or not, using mount directly (outside of the sanctioned volumes) is not allowed for security reason, as described here by BMitch.
Docker removes the mount privilege from containers because using this you could mount the host filesystem and escape the container.
If you really need to mount something during the build process, you might consider buildah, which can build without running a container for each layer (like docker build does), and can do so without being root.
Use ONBUILD to read your existing Dockerfile.
Note that with "buildah mount, you can do the reverse: Mounts the specified container's root file system in a location which can be accessed from the host, and returns its location.
That is another alternative.

Related

File not found in Alpine Container, despite existing

I have an Alpine container that I copy a binary to (in this case it is spar). The entry point is dumb-init /usr/bin/spar but it results in a No such file or directory. When I run sh inside of the container, /usr/bin/spar exists. Trying to run it in
dumb-init ...
/usr/bin/spar / spar from /
./spar / spar from usr/bin/
All result in the same error. I tried changing the permissions with chmod 777 /usr/bin/spar giving everyone full access, still no luck.
Am I missing something that is specific to alpine? In another SO issue someone mentioned that switching from Alpine to Ubuntu solved their issue, but no further info was provided.
Here is the dockerfile that creates the image(s)
ARG intermediate=quay.io/wire/alpine-intermediate
ARG deps=quay.io/wire/alpine-deps
#--- Intermediate stage ---
FROM ${intermediate} as intermediate
#--- Minified stage ---
FROM ${deps}
ARG executable
ENV CACHEDEXECUTABLE ${executable}
COPY --from=intermediate /dist/${executable} /usr/bin/${executable}
# TODO: only if executable=brig, also copy templates. Docker image conditionals seem hacky:
# https://stackoverflow.com/questions/31528384/conditional-copy-add-in-dockerfile
# For now, adds ~2 MB of additional files into every container
COPY --from=intermediate /dist/templates/ /usr/share/wire/templates/
# ARGs are not available at runtime, create symlink at build time
# more info: https://stackoverflow.com/questions/40902445/using-variable-interpolation-in-string-in-docker
RUN ln -s /usr/bin/${executable} /usr/bin/service
ENTRYPOINT /usr/bin/dumb-init /usr/bin/${CACHEDEXECUTABLE}
If spar is a binary, that binary exists in the container, and you call it directly or it's in the containers path, then the two likely reasons for a file not found are:
Dynamically linked libraries that don't exist inside the container. E.g. of you run ldd spar it will show you those links, and there's a good chance you'll see libc despite trying to run on Alpine where it uses musl.
The binary is for another platform/architecture, and you have binfmt_misc setup, but without the --fix-binary option so it's looking for the interpreter path in the container filesystem rather than the host. This will be visible by the lack of an F flag in the /proc file for that platform.

Determine the source of read-only docker violation

I have a container with a complex, multi-threaded software.
I am now in a process of trying to make it run in read-only mode (with rw mounted /tmp directory).
The problem is that the software fails to start as it attempts to modify several files.
Is there a way to determine which files are being modified?
For example:
docker run --name alpine --read-only --rm --tty alpine
docker exec alpine touch hello.txt // access violation
I want to know that something inside the container tried to modify hello.txt.

create an app in a docker container (confused about tasks order)

I have to build a simple app which reads a text file and process it's content (like remove multiple spaces, process words etc) but my I am confused about the first part of my homework.
"Initialize a git repository in a docker container then implement an app...."
I use Debian, I installed docker and git and I studied about it. From what I read I have to create a Dockerfile which will contain some instructions then I build the image and then run the container, run?
But I am still confused, what is the order of these thigs? Can I go firstly and write the app in Intelij and then to create that Dockerfiler? Or I have to create first the container then to code the app? But how I build the container? I read a lot about this, can you give me some advice? I mention that after every app "task" (read text file, process text etc) I have to execute git add, git commit and git push (if it helps for answer)
If the instruction says to "Initialize a Git repository in a docker container" then you are expected to:
run e.g. a Debian container
if Git is not present install it
initialize the repo
write your app
submit homework
You could:
docker run \
--interactive --tty --rm \
--name=homework \
--volume=${PWD}/homework:/homework \
--workdir=/homework \
debian:buster-slim
This will run a Debian "buster" image as a container and should (!) give you a shell prompt in the container.
A directory /homework in the container will be mapped to your host machine's ${PWD}/homework and you will be in the /homework directory when the container starts. This means that you won't lose your work if you exit the container.
From within the container's prompt:
# pwd
/homework
# git
bash: git: command not found
# apt update && apt install -y git
...
done.
# git
usage: git [--version] [--help] [-C <path>] [-c <name>=<value>]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
# git init
Initialized empty Git repository in /homework/.git/
Notes
If you exit the container, you can rerun the docker run ... command to return to it.
When you exist the container, you can ls -la ${PWD}/homework to see the .git directory (at least) persisted on your host.
Ensure you run it from the same directory where it created ${PWD}/homework. Or revise the --volume=...
I'd recommend an overall workflow of
Build the application, without Docker; then
Package it in a Docker image (if it makes sense to).
You should be able to build the application totally normally. Whatever language you're using to build the application, make sure to use its normal packaging tools. For example, your package.json/Gemfile/requirements.txt/go.mod should list out all of the library dependencies your application needs to run. Run it locally, write appropriate unit tests for it, and generally build something that works.
Once it works, then push it into Docker. You'll need to write a Dockerfile that builds the image. A generic recipe for this is
FROM language-base-image # python:3.9, node:14, ...
WORKDIR /app
COPY dependencies-file . # requirements.txt, package.json, ...
RUN install the dependencies # pip install, npm install, ...
COPY . .
RUN build the application # npm run build, ...
CMD ./the_application # npm run start, ...
You should then be able to docker build an image, and docker run a container from the resulting image. The Docker documentation includes a sample application that runs through this sequence.
Note in particular that the problem task of "read a text file" is substantially harder in Docker than without. You need to use a bind mount to give access to the host filesystem to the container, and then refer to the container-side path. For example,
docker run --rm -v $PWD/data:/data my-image \
./the_application --input /data/file.txt
I would not bother trying to use Docker as my primary development environment, especially for an introductory project. Docker is designed as an isolation system and it's intentionally tricky to work with host files from a container, and vice versa. Especially if you can use a fairly routine programming language that you can easily install with apt-get or brew, and you don't have tricky host-library dependencies, it's substantially easier to do most of your development in an ordinary host build environment use Docker only at a late stage.

Docker - accessing files inside container from host

I am new to docker.
I ran a node-10 images and inside the running container I cloned a repository, ran the app which started a server with file watcher. I need to access the codebase inside the container, open it up in an IDE running on the windows host. If that is done, then I also want that as I change the files in the IDE these changes induce the filewatcher in the container.
Any help is appreciated. Thanks,
The concept you are looking for is called volumes. You need to start a container and mount a host directory inside it. For the container, it will be a regular folder, and it will create files in it. For you, it will also be a regular folder. Changes made by either side will be visible to another.
docker run -v /a/local/dir:/a/dir/in/your/container
Note though that you can run into permission issues that you will need to figure out separately.
It depends on what you want to do with the files.
There is the docker cp command that you can use to copy files to/from a container.
However, it sounds to me like you are using docker for development, so you should mount a volume instead, that is, you mount a directory on the host as a volume in docker, so anything written to that directory will show up in the container, and vice versa.
For instance if you have the code base that you develop against in C:\src on your windows machine, then you run docker like docker run -v c:\src:/app where /app is the location that node is looking in. However, for Windows there are a few things to consider since Docker is not native in Windows, so have a look at the documentation first.
Hi I think you should use mount volumes for the source code and edit your code from your IDE normally:
docker run -it -v "$PWD":/app -w /app -u node node:10 yarn dev
here docker will create an image setting the working dir to "/app", mount the current dir to "/app" and run "yarn dev" at start up with the "node" user (none root user)
Hope this is helpfull.

Within lxc/docker container - what happens if apt-get upgrade includes kernel update?

I am reading a lot of Docker guides where the will often use some Ubuntu base image and in the Dockerfile directly or in a bash script that gets copy to container and run on start, it has things like 'apt-get upgrade'
As i understand it, the container still uses the hosts kernel. So what happens when the apt-get upgrade includes a kernel upgrade? Does it create a /boot and install the files as usual but the underlying LXC has some pass-through/whitelist mechanism for specific directories that always come from host... so it ignores those files in guest container ?
Thanks
fLo
The host's /boot is not visible to a Docker container, and the kernel image package should not be installed in such a container, since it's not needed. (Even if it is, though, it's entirely inert.)

Resources