Container 1 : I have a docker container which is spawned by a Dockerfile image that i built serving Python dependencies to a specific use-case. This container is mount to my file system project directory to /source directory in the Docker container.
Container 2 : Another container contains Jupyter notebook and it is only configured to use Jupyter notebook.
I don't use Jupyter notebook all the time for Python developement, i use my code editor instead. I just want an easier way to mount Jupyter notebook when i want to and edit my project files.
Now, can i mount Container-2 to Container-1 such that contents in the /source directory in Container-1 is persisted to my project directory while enabling Container-1 and Conatiner-2 to take up a shared space? In my case i want Container-1 to be mount to my file system to /source and I want the Jupyter Notebook, running in Container-2 to be able to edit files inside the /source directory in Container-1.
if I'm understanding correctly, what you are trying to do is using one container installed libs in another container
I think that this approach is bad, since there might be some problems with OS level dependencies that are missing. The first possible solution is installing the dependencies in both containers, or at least using the 1st docker image as base to the 2nd
If you yet rather doing it your way, you can try mounting volume between the 1st container virtual env / site-packages with the 2nd
Related
I am developing a nodejs app using windows 10 WSL with remote container in visual studio code.
What are the best practices for Dockerfile and docker-compose.yml at this time?
Since we are in the development phase, we don't want to COPY or ADD the program source code in the Dockerfile (it's not practical to recreate the image every time we change one line).
I use docker compose to bind the folder with the source code on the windows side with volume, but in that case, the source code folder and the set of files from the Docker container will all have Root permission.
In the Docker container, node.js runs as node general user.
For the above reasons, node.js will not have write permission to the folders you bind.
Please let me know how to solve this problem.
I found a way to specify UID or GUID, but I could not specify UID or GID because I am binding from windows.
You can optionally mount Node code using NFS in Docker-compose
So this is not about seeking workarounds to -v.
I have a Dockerfile whose intent is to install a cross-compiler in /usr/local/<cross-compiler-path>, inside the container. Later during a build process, a file would be mounted to this cross-compiler, like this:
root#5bee5daf8165:/# mount <blah.img.gz> /usr/local/<cross-compiler-path>
I get mount: /usr/local/<cross-compiler-path>: mount failed: Operation not permitted.
Although if I skip this step, finish build, run a --privileged container and mount, it works fine.
I understand the reason for not giving privileged mode in the build since it breaks the 'portability' of containers as they depend on host volumes. But in my case, I am attempting to mount it inside the Container's own file system. Why is that not allowed?
For the record, I tried installing the cross-compiler on a different path, like this:
root#5bee5daf8165:/# mount <blah.img.gz> /home/<cross-compiler-path>
But that doesn't work either. I want to attempt the build inside the Dockerfile and discard the build cache which bloat up my container once I no longer need them. What options do I have?
As mentioned in "Can You Mount a Volume While Building Your Docker Image to Cache Dependencies?" from Vladislav Supalov
Although there’s no functionality in Docker to have volumes at build-time, you can use multi-stage builds, benefit from Docker caching and save time by copying data from other images - be it multi-stage or tagged ones.
When building an image, you can’t mount a volume. However, you can copy (COPY) data from another image! By combining this, with a multi-stage build, you can pre-compute an expensive operation once, and re-use the resulting state as a starting point for future iterations.
Example:
FROM ubuntu as intermediate
RUN apt-get install -yqq python-dev python-virtualenv
RUN virtualenv /venv/
RUN mkdir -p /src
# those don't change often
ADD code/basic-requirements.txt /src/basic-requirements.txt
RUN /venv/bin/pip install -r /src/basic-requirements.txt
FROM ubuntu
RUN apt-get install -yqq python-dev python-virtualenv
# the data comes from the above container
COPY --from=intermediate /venv /venv
ADD code/requirements.txt /src/requirements.txt
# this command, starts from an almost-finished state every time
RUN /venv/bin/pip install -r /app/requirements.txt
The OP add in the comments:
I want to mount a volume internally to the container fs using the mount command while build, which currently doesn't work.
Just wanted to know if 'mount' operation, in general is tied to the kernel?
Kernel or not, using mount directly (outside of the sanctioned volumes) is not allowed for security reason, as described here by BMitch.
Docker removes the mount privilege from containers because using this you could mount the host filesystem and escape the container.
If you really need to mount something during the build process, you might consider buildah, which can build without running a container for each layer (like docker build does), and can do so without being root.
Use ONBUILD to read your existing Dockerfile.
Note that with "buildah mount, you can do the reverse: Mounts the specified container's root file system in a location which can be accessed from the host, and returns its location.
That is another alternative.
I am new to docker.
I ran a node-10 images and inside the running container I cloned a repository, ran the app which started a server with file watcher. I need to access the codebase inside the container, open it up in an IDE running on the windows host. If that is done, then I also want that as I change the files in the IDE these changes induce the filewatcher in the container.
Any help is appreciated. Thanks,
The concept you are looking for is called volumes. You need to start a container and mount a host directory inside it. For the container, it will be a regular folder, and it will create files in it. For you, it will also be a regular folder. Changes made by either side will be visible to another.
docker run -v /a/local/dir:/a/dir/in/your/container
Note though that you can run into permission issues that you will need to figure out separately.
It depends on what you want to do with the files.
There is the docker cp command that you can use to copy files to/from a container.
However, it sounds to me like you are using docker for development, so you should mount a volume instead, that is, you mount a directory on the host as a volume in docker, so anything written to that directory will show up in the container, and vice versa.
For instance if you have the code base that you develop against in C:\src on your windows machine, then you run docker like docker run -v c:\src:/app where /app is the location that node is looking in. However, for Windows there are a few things to consider since Docker is not native in Windows, so have a look at the documentation first.
Hi I think you should use mount volumes for the source code and edit your code from your IDE normally:
docker run -it -v "$PWD":/app -w /app -u node node:10 yarn dev
here docker will create an image setting the working dir to "/app", mount the current dir to "/app" and run "yarn dev" at start up with the "node" user (none root user)
Hope this is helpfull.
Basically, I am running Docker on my Windows 10 machine. I have mounted a windows directory inside this container to access the files on my windows machine, on which a couple of tasks are to be performed.
Which file system runs inside a docker container?
Is it the same as that of the OS on which this container is based? For instance, if I run a container having ubuntu as base OS, will it be the one that the current version of ubuntu (inside this container is running)?
Or is it the one that is running on the docker daemon?
Also, I am running an application inside this container, which accesses files in my windows directory as well as creates a couple of files. Now, these files are being written to Windows, hence follow the file system of Windows(NTFS).
So, how does it work? (Different file system inside the docker container and the windows file system; both in conjunctuion?)
Which file system runs inside a docker container?
The one from the docker host (Windows NTFS or Ubuntu FS).
$ docker run -d -P --name web -v /src/webapp:/opt/webapp training/webapp python app.py
This command mounts the host directory, /src/webapp, into the container at /opt/webapp.
If the path /opt/webapp already exists inside the container’s image, the /src/webapp mount overlays but does not remove the pre-existing content.
Once the mount is removed, the content is accessible again.
Now, these files are being written to Windows, hence follow the file system of Windows(NTFS).
Yes, and that filesystem is case-sensitive (as illustrated in 18756).
I am using: https://github.com/MLstate/PEPS for mail by installing it on our ubuntu server. It uses Docker containers, I tried to figure out how to access application files like css/js in those containers, but have not been successful. Furthest I got was going to /var/lib/docker/containers/CONTAINERID but once I view contents of the containers they are all the same and css/js files are no where to be seen.
The easiest way to access those files is running an interactive shell in the container. To do that you can run docker run -i -t <CONTAINER_ID> /bin/bash.
Regarding the files, docker images and containers are composed by layers and volumes. The files you are looking for will be located probably in /var/lib/docker/aufs/layers (depending of your layer filesystem), but you can omit accessing to the files directly and get them in a interactive session.