Im new, very new to this whole docker thing and also programming thing.
I built a simple script that outputs a series of files (namely a .tsv and a .txt) from an api.
Now, this works fine running the script on the regular linux terminal (I know, thats not the same thing).
However, with docker, it never seems to output my file to my host machine.
Interesting thing is, when running 'ls' in the docker terminal, all the files show up!
sudo docker run -it -v /home/USER/DESKTOP/PROJECT:/CONTAINERWORKINGDIR IMAGENAME
also, tried mounting it via the dockerfile, which also seems to not do anything.
I don't get an error, the host path is just empty, always...
running 'ls' will show my files, sitting neatly in the docker directory, but when verifying on the host path, its empty. Also, verified that all the files are accessible.
Docker File:
# Use an official Python runtime as a base image
FROM python:3.11.1-alpine
# Set the working directory to /AudiblePy
WORKDIR /AudiblePy
# Copy the current directory contents into the container at /AudiblePy
COPY main.py /AudiblePy
COPY additional.py /AudiblePy
COPY requirements.txt /AudiblePy
# Install any needed packages specified in requirements.txt
RUN pip install --trusted-host pypi.python.org -r requirements.txt
# Run app.py when the container launches
CMD ["python", "main.py"]
Would love some help, Im stumped and it seems that I can't find any information on why this is...
Related
Apologies I am very new to Docker. I have the following Docker file which contains the following commands (see below). I am not sure I understand all commands and I would appreciate some explanation. I commented all the lines I understood but put a question mark in others. Please see below
#That this line means that python will be our base. Can some comment here please and explain this line more?
FROM python:3.9 as base
#create a working directory in the virtual machine (VM)
WORKDIR /code
# copy all the python requirements stored in requirements.txt into the new directoy (in the VM)
COPY ./requirements.txt /code/requirements.txt
# activate the package manager pip. But why use no-cache-dir?
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
# copy all files to the new directory (in the VM)
COPY ./ /code/
# I don't understand the line below. Please explain? why uvicorn? app.main:app is the
#location of the fastapi
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "180"]
Thank you
A Docker file states all steps that Docker will execute when creating your image. From that image, a container can be created.
#That this line means that python will be our base. Can some comment here please and explain this line more?
FROM python:3.9 as base
This is very basic docker stuff, follow a (beginners) tutorial and you will learn a lot more than just someone spoon-feeding little bits of knowledge.
#create a working directory in the virtual machine (VM)
WORKDIR /code
You are creating a container image, not a VM. That is a similar but very different concept and should not be mixed.
# copy all the python requirements stored in requirements.txt into the new directoy (in the VM)
COPY ./requirements.txt /code/requirements.txt
This copies all files to the image.
# activate the package manager pip. But why use no-cache-dir?
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
RUN is an image build step, and the outcome will be committed to the Docker image. So, in this step, you are telling docker that you want an image that has everything installed as outlined in requirements.txt with pip. No cache, by default PIP saves the whl's of the packages you are installing, but that only would increase the image and are no longer required. So, no cache.
# copy all files to the new directory (in the VM)
COPY ./ /code/
Again, not VM but image, an image that will later be used to create a container.
# I don't understand the line below. Please explain? why uvicorn? app.main:app is the
#location of the fastapi
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "180"]
Because you are trying to run a FastAPI project, and FastAPI is just the app; you need a server to actually be able to fire request at FastAPI. This is explained on the very first page of the FastAPI documentation actually.
"app.main:app" express your project has such python file:
<Project Root Dir>
app - folder
main.py -- python file
In the main.py, you init a FastAPI instance and named app, like this:
# main.py
....
app = FastAPI()
...
unicorn use above rules to get the FastAPI instance app, then load it.
I'm interested in running a NodeJS script inside a Docker container, because that seems to be the easiest way to run stuff in unRAID (small scripts at least).
My current Dockerfile looks like this:
FROM node:12.9.1
COPY app.js /home/node/
COPY package*.json /home/node/
RUN mkdir /home/node/saves
WORKDIR /home/node
RUN npm install
CMD ["node", "app.js"]
Not perfect, but works well enough. What my script does, is that it scrapes certain websites for data, then places it in a folder called saves.
But because I do COPY app.js /home/node/, every time I want to make a tiny change to my app.js file, I need to rebuild the whole image, delete the container, and start a new one. Kind of irritating, but it has worked for me.. for now.
When I start my container, I want that volume to stay persistent, so I do this:
docker run --net=bridge -h scraper --name scraper -d -v /mnt/user/scripts/scraper/saves:/home/node/saves scraper
This works, but as I said, if I want to change my app.js (like add a new site to scrape), I have to rebuild the image and run the above command again. Every single time.
What's a better approach than this? I could solve this by not copying the files, but instead run npm install and then node app.js every time, but this script runs every 3 minutes, so that would be a huge waste of resources.
I could also store the appropriate data inside my /saves/ folder, then read that in the NodeJS script every time, but I feel like that's kind of a hack.
Use Environment variables.
If the type of changes to app.js file already known, you could
use environment variables to supply such changes to docker run, and
an entrypoint script file to make such modification using
environment variable.
The content of entrypoint script will depend on what you want to do.
# Add to your Dockerfile
ENV SITE_TO_SCRAP=example.com
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
# Run docker with
-e SITE_TO_SCRAP=abc.com
Entry Point script may looks like
#!/bin/bash
# Modify app.js
# Assuming changing the line SCRAP_URL="someurl"
sed -i -e "s#SCRAP_URL=\"someurl\"#SCRAP_URL=\"${SITE_TO_SCRAP}\"#" /home/node/app.js
# This will execute the CMD
exec "$#"
Make node cache folder persistent
To run npm install do it in entry point script not in docker file. make node.js cache folder persistent by mounting -v host/path/to/cache:/root/.npm. this way node install use cached files whenever possible. use docker run --rm node:{version} npm config get cache to get container cache directory.
Manually mount modified files.
# To your docker run command add required file/directory mount(s)
-v /path/to/modified/app.js:/home/node/app.js
I'm trying to run tacotron2 on docker within Ubuntu WSL2 (v.20.04) on Win10 2004 build. Docker is installed and running and I can run hello world successfully.
(There's a nearly identical question here, but nobody has answered it.)
When I try to run docker build -t tacotron-2_image docker/ I get the error:
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /home/nate/docker/Dockerfile: no such file or directory
So then I navigated in bash to where docker is installed (/var/lib/docker) and tried to run it there, and got the same error. In both cases I created a docker directory, but kept getting that error in all cases.
How can I get this to work?
As mentioned here, the error might have nothing to do with "symlinks", and everything with the lack of Dockerfile, which should be in the Tacotron-2/docker folder.
docker build does mention:
The docker build command builds Docker images from a Dockerfile and a “context”.
A build’s context is the set of files located in the specified PATH or URL.
In your case, docker build -t tacotron-2_image docker/ is supposed to be executed in the path you have cloned the Rayhane-mamah/Tacotron-2 repository.
To be sure, you could specify said Dockerfile, but that should not be needed:
docker build -t tacotron-2_image -f docker/Dockerfile docker/
Or:
cd
git clone https://github.com/Rayhane-mamah/Tacotron-2
cd Tacotron-2
cd docker
docker build -t tacotron-2_image .
I thought these commands I'm executing are for the purpose of installing it
To build the image, you need the sources (the repository to clone).
If the name of you Dockerfile is with capital F rename it
For others like me who somehow couldn't get it works because of symlink
just copy out your files out to a new directory that hasn't been symlink and build your image from there
if ony if you've confirm that your Dockerfile isn't dockerfile .Dockerfile ,DockerFile or dockerfile.txt .
My OS elementary which is base on ubuntu.
I have an application which I have dockerize it. The application recieves a configuration file which can change per user. I used the following command to pass config file to the image.
docker run --rm --name $(PROJECT_NAME) -v $(pwd)/config.yaml:/root/config.yaml
I don't want to build the image again for a new config file. but it doesn't seems to work as when I inspected the contents of the file on the container it showed me the same file. It seems that when I change the file on my local system and pass it to the container in run command, the change is not reflected and it loads previous config file. Any ideas what I might be doing wrong or any best practices to deal with such a scenario.
Dockerfile
# base image
FROM amazonlinux:2
# copy requirements file to working directory
COPY requirements.txt ./
RUN yum -y install python3 \
&& pip3 install --no-cache-dir -r requirements.txt
COPY app/. ./
ENTRYPOINT python3 run.py config.yaml
I was able to resolve it. Here is my updated Docker file : I have put my config file in config/ folder and mount it. Now I can build the image once and can change the config file inside the config/ folder and changes will be reflected inside the container.
# base image
FROM amazonlinux:2
# copy requirements file to working directory
COPY requirements.txt ./
COPY app/. ./
RUN yum -y install python3 \
&& pip3 install --no-cache-dir -r requirements.txt
ENTRYPOINT python3 run.py config/config.yaml
This is the run command:
docker run --rm --name $(PROJECT_NAME) -v $(CONFIG_PATH):/config $(DOCKER_REPO)
CONFIG_PATH = $(PWD)/config
What you see is "normal".
When you mount a file or directory, you actually mount a pointer (inode) inside the container. It's the way docker work with volume.
Some/most editor use a "safe write" method that consist to write a new file and replace the old file with the new one. When the editor do that, the new updated file have a new pointer (inode) and the old one still exist pointing to the old content.
On linux, a file is only deleted when no one use it, it's why you can't see it but the container still have it.
You will have the same problem if you mount a directory and you delete it and recreate one with the same name, the container will not see the new directory.
It's a know limitation of the bind mount and the best you can do is to mount a directory with your file inside. The other way is to restart your container, Docker will find the new inode and use it.
I'm a newbie with Docker and I'm trying to start with NodeJS so here is my question..
I have this Dockerfile inside my project:
FROM node:argon
# Create app directory
RUN mkdir -p /home/Documents/node-app
WORKDIR /home/Documents/node-app
# Install app dependencies
COPY package.json /home/Documents/node-app
RUN npm install
# Bundle app source
COPY . /home/Documents/node-app
EXPOSE 8080
CMD ["npm", "start"]
When I run a container with docker run -d -p 49160:8080 node-container it works fine..
But when I try to map my host project with the container directory (docker run -p 49160:8080 -v ~/Documentos/nodeApp:/home/Documents/node-app node-cont) it doesn't work.
The error I get is: Error: Cannot find module 'express'
I've tried with other solutions from related questions but nothing seems to work for me (or I know.. I'm just too rookie with this)
Thank you !!
When you run your container with -v flag, which mean mount a directory from your Docker engine’s host into a container, will overwrite what you do in /home/Documents/node-app,such as npm install.
So you cannot see the node_modules directory in the container.
$ docker run -d -P --name web -v /src/webapp:/webapp training/webapp python app.py
This command mounts the host directory, /src/webapp, into the container at /webapp. If the path /webapp already exists inside the container’s image, the /src/webapp mount overlays but does not remove the pre-existing content. Once the mount is removed, the content is accessible again. This is consistent with the expected behavior of the mount command.
mount a host directory as a data volume.As what the docs said,the pre-existing content of host directory will not be removed, but no information about what's going on the exist directory of the container.
There is a example to support my opinion.
Dockerfile
FROM alpine:latest
WORKDIR /usr/src/app
COPY . .
I create a test.t file in the same directory of Dockerfile.
Proving
Run command docker build -t test-1 .
Run command docker run --name test-c-1 -it test-1 /bin/sh,then your container will open bash.
Run command ls -l in your container bash,it will show test.t file.
Just use the same image.
Run command docker run --name test-c-2 -v /home:/usr/src/app -it test-1 /bin/sh. You cannot find the file test.t in your test-c-2 container.
That's all.I hope it will help you.
I recently faced the similar issue.
Upon digging into docker docs I discovered that when you run the command
docker run -p 49160:8080 -v ~/Documentos/nodeApp:/home/Documents/node-app node-cont
the directory on your host machine ( left side of the ':' in the -v option argument ) will be mounted on the target directory ( in the container ) ##/home/Documents/node-app##
and since your target directory is working directory and so non-empty, therefore
"the directory’s existing contents are obscured by the bind mount."
I faced an alike problem recently. Turns out the problem was my package-lock.json, it was outdated in relation to the package.json and that was causing my packages not being downloaded while running npm install.
I just deleted it and the build went ok.