this is my dockerfile:
FROM public.ecr.aws/lambda/python:3.8-arm64
COPY requirements.txt ./
RUN yum update -y && \
yum install -y gifsicle && \
pip install -r requirements.txt
COPY . .
CMD ["app.handler"]
I'm getting the following error:
#8 200.6 No package gifsicle available.
#8 200.7 Error: Nothing to do
I finally managed to do it by building the package from source. My Dockerfile:
FROM public.ecr.aws/lambda/python:3.8-arm64
RUN yum -y install install make gcc wget gzip
RUN wget https://www.lcdf.org/gifsicle/gifsicle-1.93.tar.gz
RUN tar -xzf gifsicle-1.93.tar.gz
RUN cd gifsicle-1.93 && \
./configure && \
make && \
make install
COPY requirements.txt ./
RUN yum update -y && \
pip install -r requirements.txt
COPY . .
CMD ["app.handler"]
Related
I am fairly new to Docker and containerisation. I am wanting to decrease the size of my_proj docker container in production.
I prefer installing packages and managing dependencies via Poetry.
How can I specify using CPU-only PyTorch in a Dockerfile?
To do this via. bash terminal, it would be:
poetry add pytorch-cpu torchvision-cpu -c pytorch
(or conda install...)
My existing Dockerfile:
FROM python:3.7-slim as base
RUN apt-get update -y \
&& apt-get -y --no-install-recommends install curl wget\
&& rm -rf /var/lib/apt/lists/*
ENV ROOT /home/worker/python/my_proj
WORKDIR $ROOT
ARG ATLASSIAN_TOKEN
ARG POETRY_HTTP_BASIC_AZURE_PASSWORD
ARG ACCESS_KEY
ENV AWS_ACCESS_KEY_ID=$ACCESS_KEY
ARG SECRET_KEY
ENV AWS_SECRET_ACCESS_KEY=$SECRET_KEY
ARG REPO
ENV REPO_URL=$REPO
ENV PYPIRC_PATH=$ROOT/.pypirc
ENV \
PYTHONFAULTHANDLER=1 \
POETRY_VERSION=1.1.4 \
POETRY_HOME=/etc/poetry \
XDG_CACHE_HOME=/home/worker/.cache \
POETRY_VIRTUALENVS_IN_PROJECT=true \
MPLCONFIGDIR=/home/worker/matplotlib \
PATH=/home/worker/python/my_proj/.venv/bin:/usr/local/bin:/etc/poetry/bin:$PATH
ADD https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py ./
RUN python get-poetry.py && chmod +x /etc/poetry/bin/poetry
RUN --mount=type=cache,target=/root/.cache pip install twine keyring artifacts-keyring
RUN --mount=type=cache,target=/root/.cache apt update && apt install gcc -y
FROM base as ws
ARG WS_APIKEY
ARG WS_PROJECTVERSION=
ARG WS_PROJECTNAME=workers-python-my_proj
ARG WS_PRODUCTNAME=HALO
COPY --chown=worker:worker . .
RUN --mount=type=cache,uid=1000,target=/home/worker/.cache poetry install --no-dev
COPY --from=openjdk:15-slim-buster /usr/local/openjdk-15 /usr/local/openjdk-15
ENV JAVA_HOME /usr/local/openjdk-15
ENV PATH $JAVA_HOME/bin:$PATH
RUN --mount=type=cache,uid=1000,target=/home/worker/.cache ./wss_agent.sh
FROM base as test
COPY . .
RUN poetry config experimental.new-installer false
RUN poetry install
RUN cd my_proj && poetry run invoke deployconfluence_server_pass=$ATLASSIAN_TOKEN
FROM base as package
COPY . .
RUN poetry build
RUN python -m pip install --upgrade pip && \
pip install twine keyring artifacts-keyring && \
twine upload -r $REPO_URL --config-file $PYPIRC_PATH dist/* --skip-existing
FROM base as build
COPY . .
RUN poetry config experimental.new-installer false
RUN poetry install --no-dev
RUN pip3 --no-cache-dir install --upgrade awscli
RUN aws s3 cp s3://....tar.gz $ROOT/my_proj # censored url
RUN mkdir $ROOT/my_proj/bert-base-cased && cd $ROOT/my_proj/bert-base-cased && \
wget https://huggingface.co/bert-base-cased/resolve/main/config.json && \
wget https://huggingface.co/bert-base-cased/resolve/main/tokenizer.json && \
wget https://huggingface.co/bert-base-cased/resolve/main/tokenizer_config.json
FROM python:3.7-slim as production
ENV ROOT=/home/worker/python/my_proj \
VIRTUAL_ENV=/home/worker/python/my_proj/.venv\
PATH=/home/worker/python/my_proj/.venv/bin:/home/worker/python/my_proj:$PATH
COPY --from=build /home/worker/python/my_proj/pyproject.toml /home/worker/python/
COPY --from=build /home/worker/python/my_proj/.venv /home/worker/python/my_proj/.venv
COPY --from=build /home/worker/python/my_proj/my_proj /home/worker/python/my_proj
WORKDIR $ROOT
ENV PYTHONPATH=$ROOT:/home/worker/python/
ENTRYPOINT [ "primary_worker", "--mongo" ]
Installing it via pip should work:
RUN pip3 install torch==1.9.0+cpu torchvision==0.10.0+cpu torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
I am trying to install python 3.5 inside docker with a base image centos7. This is our Dockerfile
FROM base-centos7:0.0.8
# Install basic tools
RUN yum install -y which vim wget git gcc
# Install python 3.5
RUN yum install -y https://repo.ius.io/ius-release-el7.rpm \
&& yum update -y \
&& yum install -y python35u python35u-libs python35u-devel python35u-pip
RUN python3.5 -m pip install --upgrade pip
But during the build, docker build image is failing with the following errors
executor failed running [/bin/sh -c yum install -y https://repo.ius.io/ius-release-el7.rpm
&& yum update -y
&& sudo yum install -y python35u python35u-libs python35u-devel python35u-pip]: exit code: 127.
Can anyone guide me in resolving this issue. and why am I seeing this issue in very first place.
You can use python image from docker hub
https://hub.docker.com/_/python
Example of dockerfile :
FROM python:3.6
RUN mkdir /code
WORKDIR /code
ADD . /code/
RUN pip install -r requirements.txt
EXPOSE 5000
CMD ["python", "/code/app.py"]
i think it's easy , isn't ?
the centos repo uses:
FROM centos/s2i-base-centos7
EXPOSE 8080
ENV PYTHON_VERSION=3.5 \
PATH=$HOME/.local/bin/:$PATH \
PYTHONUNBUFFERED=1 \
PYTHONIOENCODING=UTF-8 \
LC_ALL=en_US.UTF-8 \
LANG=en_US.UTF-8 \
PIP_NO_CACHE_DIR=off
RUN INSTALL_PKGS="rh-python35 rh-python35-python-devel rh-python35-python-setuptools rh-python35-python-pip nss_wrapper \
httpd24 httpd24-httpd-devel httpd24-mod_ssl httpd24-mod_auth_kerb httpd24-mod_ldap \
httpd24-mod_session atlas-devel gcc-gfortran libffi-devel libtool-ltdl enchant" && \
yum install -y centos-release-scl && \
yum -y --setopt=tsflags=nodocs install --enablerepo=centosplus $INSTALL_PKGS && \
rpm -V $INSTALL_PKGS && \
# Remove centos-logos (httpd dependency) to keep image size smaller.
rpm -e --nodeps centos-logos && \
yum -y clean all --enablerepo='*'
source here
The problem is not difficult, I build the image changing
FROM base-centos7:0.0.8 ====> FROM centos:7
You can consult the images version of centos in https://hub.docker.com/_/centos
PD: The container showed: errro exited(1), you should focus on the main process.
I want to create a docker image with specifically python 3.5 on a specific base image which is the nvidia/cuda (9.0-base image) the latter has no python environment.
The reason I need specific versions is to support running cuda10.0 python3.5 and a gcc version<7 to compile the driver all together on the same box
When I try and build the docker environments (see below) I always end up with the system update files which load python3.6
The first version I run (below) runs a system update dependencies which installs python 3.6 I have tried many variants to avoid this but always end up 3.6 in the final image.
Any suggestions for getting this running with python3.5 are welcome
Thanks
FROM nvidia/cuda
RUN apt-get update && apt-get install -y libsm6 libxext6 libxrender-dev python3.5 python3-pip
COPY . /app
WORKDIR /app
RUN pip3 install -r requirements.txt
ENTRYPOINT [ "python3" ]
CMD [ "app.py" ]
Another variant (below) I have tried is with virtualenv and here again I can't seem to force a python 3.5 environment
FROM nvidia/cuda
RUN apt-get update && apt-get install -y --no-install-recommends libsm6 libxext6 libxrender-dev python3.5 python3-pip python3-virtualenv
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m virtualenv --python=/usr/bin/python3 $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
COPY . /app
WORKDIR /app
RUN pip3 install -r requirements.txt
ENTRYPOINT [ "python3" ]
CMD [ "app.py" ]
You can try using conda. I used several stages to minimize final container and to speedup/cache local builds.
# first stage
FROM nvidia/cuda:11.1-base-ubuntu18.04 as builder
RUN apt-get update && apt-get install -y curl wget gcc build-essential
# install conda
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-4.5.12-Linux-x86_64.sh -O ~/miniconda.sh && \
/bin/bash ~/miniconda.sh -b -p /opt/conda
# create env with python 3.5
RUN /opt/conda/bin/conda create -y -n myenv python=3.5
# install requirements
WORKDIR /app
COPY requirements.txt /app
ENV PATH=/opt/conda/envs/myenv/bin:$PATH
RUN pip install -r requirements.txt
RUN pip uninstall -y pip
####################
# second stage (note: FROM container must be the same as builder)
FROM nvidia/cuda:11.1-base-ubuntu18.04 as runner
# copy environment data including python
COPY --from=builder /opt/conda/envs/myenv/bin /opt/conda/envs/myenv/bin
COPY --from=builder /opt/conda/envs/myenv/lib /opt/conda/envs/myenv/lib
# do some env settings
ENV PATH=/opt/conda/envs/myenv/bin:$PATH
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
####################
# final image
from runner
WORKDIR /app
COPY ./run.py /app
CMD [ "python", "run.py"]
You can install from PPA and use it as usual:
FROM nvidia/cuda
RUN apt-get update && apt-get install -y --no-install-recommends software-properties-common \
libsm6 libxext6 libxrender-dev curl \
&& rm -rf /var/lib/apt/lists/*
RUN echo "**** Installing Python ****" && \
add-apt-repository ppa:deadsnakes/ppa && \
apt-get install -y build-essential python3.5 python3.5-dev python3-pip && \
curl -O https://bootstrap.pypa.io/get-pip.py && \
python3.5 get-pip.py && \
rm -rf /var/lib/apt/lists/*
COPY requirements.txt requirements.txt
RUN pip3.5 install -r requirements.txt
CMD ["python3.5", "app.py"]
I am trying to create a docker image for python 3.5 and Ubuntu 16.04.
I have created the docker file but at the time of requirement installation using pip the package "psycopg2" is giving the error.
psycopg2 needs Postgres installed in.
Dockerfile
FROM ubuntu:16.04
# bring system up-to-date
RUN apt-get update -qq && \
apt-get upgrade -qqy
RUN apt-get update && apt-get install software-properties-common python-software-properties
RUN add-apt-repository ppa:jonathonf/python-3.6
RUN apt-get update
RUN apt-get install -y libpq-dev build-essential python3.6 python3.6-dev python3-pip python3.6-venv
RUN apt-get install -y git
# update pip
RUN python3.6 -m pip install pip --upgrade
RUN python3.6 -m pip install wheel
RUN apt-get install -y wget
RUN apt-get install bzip2
RUN rm -rf /usr/share/fonts/truetype/*
RUN apt-get update && apt-get install -y --force-yes gettext libgettextpo-dev fontconfig
# install phantom js
RUN mkdir -p /usr/share && \
cd /usr/share \
&& wget -O phantomjs.tar.bz2 https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 && tar xvfj phantomjs.tar.bz2 && mv phantomjs-2.1.1-linux-x86_64 phantomjs && rm -rf phantomjs.tar.bz2
RUN echo "ttf-mscorefonts-installer msttcorefonts/accepted-mscorefonts-eula select true" | debconf-set-selections \
&& apt-get update \
&& apt-get install -y ttf-mscorefonts-installer
COPY ./helvetica /usr/share/fonts/truetype/msttcorefonts
RUN fc-cache -f -v
RUN fc-list | grep -i 'helvetica'
# Requirements have to be pulled and installed here, otherwise caching won't work
COPY ./requirements /requirements
RUN pip3 install -r /requirements/local.txt
COPY ./compose/production/django/entrypoint.sh /entrypoint.sh
RUN sed -i 's/\r//' /entrypoint.sh
RUN chmod +x /entrypoint.sh
COPY ./compose/local/django/start.sh /start.sh
RUN sed -i 's/\r//' /start.sh
RUN chmod +x /start.sh
COPY ./compose/local/django/celery/worker/start.sh /start-celeryworker.sh
RUN sed -i 's/\r//' /start-celeryworker.sh
RUN chmod +x /start-celeryworker.sh
COPY ./compose/local/django/celery/beat/start.sh /start-celerybeat.sh
RUN sed -i 's/\r//' /start-celerybeat.sh
RUN chmod +x /start-celerybeat.sh
WORKDIR /app
ENTRYPOINT ["/entrypoint.sh"]
Is there an available docker image for ubuntu 16.04 and python3.5?
I used the official nodejs image to create a docker container that run a nodejs app.
But now I want to create the same docker but my own base on ubuntu:14.04 but it doesn't work
Dockerfile mynode
FROM ubuntu:14.04
RUN apt-get update -y
RUN apt-get upgrade -y
RUN apt-get install nodejs -y
RUN apt-get install nodejs-legacy -y
RUN apt-get install npm -y
RUN npm install -g nodemon
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY ./prj/package.json /usr/src/app/
RUN npm install
ADD ./prj /usr/src/app
EXPOSE 9977
# Run app using nodemon
CMD ["nodemon", "/usr/src/app/app.js"]
If I change the line from ubuntu:14.04 to node it works well.
But now if I use a docker-compose to build and run the container it doesn't work.
node:
restart: always
build: ./mynodeFolder
ports:
- "9977:9977"
I have the error :
Now if I use the image build with the mynode DockerFile and use docker run -it --rm myNewContainer bash and then start my app using nodemon it works perfectly !
So everything is well installed and packed within the my dockerimage, so why it doesn't work when build with the docker-compose
It works now when i use this DockerFile
# Set the base image to Ubuntu
FROM ubuntu:14.04
#FOR DEBUGGING
RUN apt-get update -y
RUN apt-get upgrade -y
RUN apt-get install nodejs -y
RUN apt-get install nodejs-legacy -y
RUN apt-get install npm -y
# Install nodemon
RUN npm install -g nodemon
# Provides cached layer for node_modules
RUN mkdir -p /usr/src/app
# Define working directory
WORKDIR /usr/src/app
ADD ./prj /usr/src/app
RUN npm install
# Expose port
EXPOSE 9977
# Run app using nodemon
CMD ["nodemon", "/usr/src/app/app.js"]
Here is the Dockerfile for the official node image:
https://github.com/nodejs/docker-node/blob/5d433ece4d221fac7e38efbec25ffea2dea56286/5.2/Dockerfile
RUN set -ex && for key in 9554F04D7259F04124DE6B476D5A82AC7E37093B 94AE36675C464D64BAFA68DD7434390BDBE9B9C5 0034A06D9D9B0064CE8ADF6BF1747F4AD2306D93 FD3A5288F042B6850C66B31F09FE44734EB7990E 71DCFD284A79C3B38668286BC97EC7A07EDE3FC1 DD8F2338BAE7501E3DD5AC78C273792F7D83545D ; do gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; done
ENV NPM_CONFIG_LOGLEVEL=info
ENV NODE_VERSION=5.2.0
RUN curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.gz" && curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" && gpg --verify SHASUMS256.txt.asc && grep " node-v$NODE_VERSION-linux-x64.tar.gz\$" SHASUMS256.txt.asc | sha256sum -c - && tar -xzf "node-v$NODE_VERSION-linux-x64.tar.gz" -C /usr/local --strip-components=1 && rm "node-v$NODE_VERSION-linux-x64.tar.gz" SHASUMS256.txt.asc
CMD "node"
If you go to http://imagelayers.io, you can see the combined Dockerfile for the image, including the buildpack-deps:jessie base image:
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl wget && rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --no-install-recommends bzr git mercurial openssh-client subversion procps && rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --no-install-recommends autoconf automake bzip2 file g++ gcc imagemagick libbz2-dev libc6-dev libcurl4-openssl-dev libevent-dev libffi-dev libgeoip-dev libglib2.0-dev libjpeg-dev liblzma-dev libmagickcore-dev libmagickwand-dev libmysqlclient-dev libncurses-dev libpng-dev libpq-dev libreadline-dev libsqlite3-dev libssl-dev libtool libwebp-dev libxml2-dev libxslt-dev libyaml-dev make patch xz-utils zlib1g-dev && rm -rf /var/lib/apt/lists/*
RUN set -ex && for key in 9554F04D7259F04124DE6B476D5A82AC7E37093B 94AE36675C464D64BAFA68DD7434390BDBE9B9C5 0034A06D9D9B0064CE8ADF6BF1747F4AD2306D93 FD3A5288F042B6850C66B31F09FE44734EB7990E 71DCFD284A79C3B38668286BC97EC7A07EDE3FC1 DD8F2338BAE7501E3DD5AC78C273792F7D83545D ; do gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; done
ENV NPM_CONFIG_LOGLEVEL=info
ENV NODE_VERSION=5.2.0
RUN curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.gz" && curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" && gpg --verify SHASUMS256.txt.asc && grep " node-v$NODE_VERSION-linux-x64.tar.gz\$" SHASUMS256.txt.asc | sha256sum -c - && tar -xzf "node-v$NODE_VERSION-linux-x64.tar.gz" -C /usr/local --strip-components=1 && rm "node-v$NODE_VERSION-linux-x64.tar.gz" SHASUMS256.txt.asc
CMD "node"