So, here's the problem:
I am trying to build and run a jupyter notebook inside the docker container. At some point I need to install a few python packages with pip3 install. The build looks fine, all the modules are being installed, but when I import any of those modules inside the jupyter notebook, it says "No module named <module_name>", as if it wasn't installed.
Here's the Dockerfile:
FROM jupyter/base-notebook
LABEL maintainer="Jupyter Project <jupyter#googlegroups.com>"
USER root
RUN apt-get update && apt-get install -yq --no-install-recommends \
build-essential \
python3-pip
RUN sudo -H pip3 install matplotlib \
pillow \
numpy
USER $NB_UID
I can't figure out why this doesn't work.
Related
I can install apt packages using one single Docker RUN command like:
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
build-essential \
zlib1g-dev \
glade \
python3-dev \
python3-pip \
python3-setuptools \
pkg-config \
libglib2.0-dev \
libgirepository1.0-dev \
libcairo2-dev \
gir1.2-gtk-3.0 \
However I cannot do the same using Python pip within the Docker RUN command like:
RUN pip3 install \
wheel \
pycairo \
PyGObject \
requests \
pyinstaller
Ending with error:
Collecting pyinstaller (from -r /tmp/requirements.txt (line 5))
Downloading https://files.pythonhosted.org/packages/b0/e6/e5760666896739115b0e4538a42cdd895215581618ec885ad043dd35ee57/pyinstaller-4.10.tar.gz (2.7MB)
Complete output from command python setup.py egg_info:
Error: Building wheels requires the 'wheel' package. Please `pip install wheel` then try again.
Why?
Despite I can do this:
RUN pip3 install wheel
RUN pip3 install pycairo
RUN pip3 install PyGObject
RUN pip3 install requests
RUN pip3 install pyinstaller
Still wish to understand why it cannot be done within ONE Docker RUN pip install command
The problem with combined install
RUN pip3 install \
wheel \
pycairo \
PyGObject \
requests \
pyinstaller
is that pip doesn't fully install packages until all listed packages are processed and there're packages that require wheel to be already fully installed. First install wheel separately, then everything else:
RUN pip3 install wheel
RUN pip3 install \
pycairo \
PyGObject \
requests \
pyinstaller
wheel is a special package — it's required to install other packages.
I have created my own image containing my project in an ubuntu system. I was able to successfully build the image. No error showed up build-time. However, at run-time, a python import error is thrown up. The package is "requests". I have tried running a separate install statement from Dockerfile. I have also tried just including it in requirements.txt. But both return the same error. What is the fix to this?
ModuleNotFoundError: No module named 'requests'
Dockerfile:
FROM ubuntu:latest
RUN apt-get update && apt-get install -y software-properties-common
RUN add-apt-repository -y ppa:deadsnakes/ppa
RUN apt-get update && apt-get install -y python3.7
RUN apt-get install -y python3.7 curl python3-distutils && curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && python3.7 get-pip.py
RUN apt-get install -y build-essential libssl-dev libffi-dev python3.7-dev
RUN pip3 install requests
RUN rm -rf /var/lib/apt/lists/*
# We copy just the requirements.txt first to leverage Docker cache
COPY ./requirements.txt /dummy/requirements.txt
WORKDIR /dummy
RUN pip install -r requirements.txt
COPY . /dummy
ENTRYPOINT [ "python3" ]
CMD [ "main.py" ]
I already tried
sudo apt-get install build-essential python-dev python-setuptools
sudo apt-get install python-numpy python-scipy
sudo apt-get install libatlas-dev libatlas3gf-base
It was showing Unable to locate package libatlas3gf-base
So I tried
pip install --user --install-option="--prefix=" -U scikit-learn
But it failed. Failure is in the image as shown in this drive link "https://drive.google.com/open?id=1_YZlQYpP5aGGbbEDKzIzsYeiEVIgEmGe".
Try installing with pip
sudo apt-get install python3-pip
sudo pip install pandas or sudo pip3 install pandas
sudo pip install numpy or sudo pip3 install numpy
Try also using a virtual enviroment just in case
apt-get install python-virtualenv
virtualenv testVirtualEnv
cd testVirtualEnv
source bin/activate
Now install dependencies
Virtual enviroments are also a good way of making projects in a more managable way
I am quite new to Docker/Python and trying to update an existing AWX's Docker Image's Dockerfile to ensure that i have the latest versions of the following packages in /usr/lib/python3.6/site-packages
/usr/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/connectionpool.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/poolmanager.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/util/retry.py
Following is my environment info:
Docker version 19.03.3, build a872fc2f86
Docker Base Image: centos:latest
The Dockerfile in which i am making changes for this is available here.
From what i have read from multiple posts/blogs so far, this should normally be updated by a simple:
pip3 install --upgrade pip
but when i run this, i am getting the following (and it shows a different path which includes local in the path.
bash-4.4# pip3 install --upgrade pip
Requirement already up-to-date: pip in /usr/local/lib/python3.6/site-packages (19.3.1)
I have even tried the following but this didn't help:
bash-4.4# python3 -m ensurepip --upgrade
Requirement already up-to-date: setuptools in /usr/lib/python3.6/site-packages
Requirement already up-to-date: pip in /usr/local/lib/python3.6/site-packages
bash-4.4# pip3 install --upgrade setuptools
Requirement already up-to-date: setuptools in /usr/local/lib/python3.6/site-packages (42.0.2)
This is how the above changes were applied in the Dockerfile
In this Dockerfile, i have added the following lines after line 124 in the Dockerfile to update pip packages:
RUN yum update -y
RUN pip2 install --upgrade pip
RUN pip3 install --upgrade pip
Snapshot of required sections from Dockerfile
FROM centos:latest
USER root
# sync with installer/roles/image_build/templates/Dockerfile.j2
RUN dnf -y update && \
dnf -y install epel-release 'dnf-command(config-manager)' && \
dnf module -y enable 'postgresql:10' && \
dnf config-manager --set-enabled PowerTools && \
.
.
.
ansible \
python3-devel \
python3-libselinux \
python3-pip \
python3-psycopg2 \
python3-setuptools \
dnf-utils
ADD https://github.com/krallin/tini/releases/download/v0.14.0/tini /tini
RUN chmod +x /tini
RUN python3 -m ensurepip && pip3 install virtualenv
RUN pip3 install supervisor
ADD Makefile /tmp/Makefile
RUN mkdir /tmp/requirements
ADD requirements/requirements_ansible.txt \
requirements/requirements_ansible_uninstall.txt \
requirements/requirements_ansible_git.txt \
requirements/requirements.txt \
requirements/requirements_tower_uninstall.txt \
requirements/requirements_git.txt \
/tmp/requirements/
RUN cd /tmp && VENV_BASE="/var/lib/awx/venv" make requirements_awx requirements_ansible_py3
.
.
.
RUN echo "{{ awx_version }}" > /var/lib/awx/.tower_version
COPY {{ awx_sdist_file }} /tmp/{{ awx_sdist_file }}
RUN OFFICIAL=yes /var/lib/awx/venv/awx/bin/pip install /tmp/{{ awx_sdist_file }}
Somehow the changes are not getting reflected in the correct path. Can someone please suggest how to update the packages inside: /usr/lib/python3.6/site-packages
I have a Python file, in which I tried to import Spark libraries.
When I built it with the Docker File it is giving me error as 'JAVA_HOME' is not set.
I tried to install Java through Docker file, but it is giving error as well.
Below is the Dockerfile I tried to execute.
FROM python:3.6.4
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y software-properties-common && \
add-apt-repository ppa:webupd8team/java -y && \
apt-get update && \
echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \
apt-get install -y oracle-java8-installer && \
apt-get clean
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
ADD Samplespark.py /
COPY Samplespark.py /opt/ml/Samplespark.py
RUN pip install pandas
RUN pip install numpy
RUN pip install pyspark
RUN pip install sklearn
RUN pip install sagemaker_pyspark
RUN pip install sagemaker
CMD [ "python", "./Samplespark.py" ]
ENTRYPOINT ["python","/opt/ml/Samplespark.py"]
Please help me to install the Java dependencies for PySpark in Docker.
You have Debian os, not ubuntu os. These ppas are for ubuntu os. According to this, article oracle java8 is not available in Debian due to licensing issues.
You have following options-
1. Use an Ubuntu docker image which comes with preinstalled oracle java8 like this one
2. Follow this tutorial on how to install Oracle java8 on Debian Jessie
3. Install open_jdk sudo apt-get install openjdk-8-jre