Dockerfile for Python3 and OpenCV - python-3.x

I need to have a Dockerfile with Python3 and the latest version of OpenCV. The Dockerfile I have written is described below:
FROM ubuntu
#Install OpenCV
RUN apt-get update &&\
apt-get install -y cmake
RUN apt-get install -y gcc g++
RUN apt-get install -y python3-dev python3-numpy
# To support GUI features
RUN apt-get install -y libavcodec-dev libavformat-dev libswscale-dev
# To support GTK 2
RUN apt-get install -y libgtk2.0-dev
# To support GTK 3
RUN apt-get install -y libgtk-3-dev
#Optional dependencies
RUN apt-get install -y libpng-dev
RUN apt-get install -y libjpeg-dev
RUN apt-get install -y libopenexr-dev
RUN apt-get install -y libtiff-dev
RUN apt-get install -y libwebp-dev
# Clone OpenCV repo
RUN apt-get install -y git
RUN git clone https://github.com/opencv/opencv.git
#Compile
RUN mkdir /opencv/build && \
cd /opencv/build
RUN cmake ..
RUN make
However, when i run it, it gives me the following error with cmake.
Step 17/27 : RUN cmake ..
---> Running in 3dca32df2036
CMake Error: The source directory "/" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
The command '/bin/sh -c cmake ..' returned a non-zero code: 1
Do you know what am i doing wrong?

You just need to chain the last RUN instructions:
#Compile
RUN mkdir /opencv/build && \
cd /opencv/build && \
cmake .. && \
make
Explanation:
A RUN statement creates a new image layer and executes the specified shell command. This means that each RUN statement will run the command in a separate shell, so the current directory from the previous RUN will not be preserved (you can see that the CMake current directory is / by looking closely at the error message).
You can find more info about RUN statement in the Docker documentation and I also recommend reading Best practices for writing Dockerfiles.

Related

Docker - Can i have two FROM commands? [duplicate]

This question already exists:
Docker - install node as part of the ubuntu image
Closed 2 years ago.
I want to be able to set up an image that has both the Ubuntu and the node images.
I started with the Ubuntu and downloaded all the necessary programs, including node.js.
Here is the latest Dockerfile i created, out of the many i tried:
FROM ubuntu
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y software-properties-common
#install freecad
RUN add-apt-repository ppa:freecad-maintainers/freecad-daily
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y freecad-daily
#install utils
RUN DEBIAN_FRONTEND=noninteractive apt install -y git
RUN apt install -y python3-distutils
#install node
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y git-core curl build-essential openssl libssl-dev \
&& git clone https://github.com/nodejs/node.git \
&& cd node \
&& ./configure \
&& make \
&& sudo make install
COPY . .
RUN npm install
CMD ["node","server.js"]
Can i use two different FROM commands, for building my image?
The one to download ubuntu, and the other to download node.js?

yum in dockerfile - There are no enabled repos

I am having issues with installing python3 with yum in dockerfile. I did look on internet, I did try few things, not working. Its just small thing but not able to figure it out. When I try to build below docker file I do get error . I get error at line -
RUN yum install -y oracle-epel-release-el7
There are no enabled repos.
Run "yum repolist all" to see the repos you have.
You can enable repos with yum-config-manager --enable <repo>
The docker file is below.
FROM openjdk:13-jdk-slim
ARG MAVEN_VERSION=3.6.3
ARG USER_HOME_DIR="/root"
ARG SHA=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
# Install prerequisites
RUN apt-get update && apt-get install -y curl
RUN apt-get update && apt-get install -y yum
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
&& echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c - \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
# Install Python 3.6
RUN yum install -y oracle-epel-release-el7
RUN yum install -y python36
# Install AWS CLI
RUN pip3 install awscli
CMD ["/bin/bash"]
Why mix apt and yum? You're already using apt-get, just use it to install your python too:
apt-get install python3.6

Cannot install pyodbc in docker and getting error command 'gcc' failed with exit status 1

I'm trying install pyodbc in docker running inside linux container
But I'm getting the following error
Click here to view the image
src/pyodbc.h:56:10: fatal error: sql.h: No such file or directory
#include <sql.h>
^~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for pyodbc
Here is my dockerfile
FROM mcr.microsoft.com/azure-functions/python:2.0
ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
AzureFunctionsJobHost__Logging__Console__IsEnabled=true
COPY . /home/site/wwwroot
FROM ubuntu
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && \
apt-get -y install gcc mono-mcs && \
rm -rf /var/lib/apt/lists/*
FROM python
RUN apt-get update && apt-get install -y python3-pip
# RUN /usr/bin/pip -r /home/site/wwwroot/requirements.txt
# WORKDIR /home/site/wwwroot
COPY --from=0 /home/site/wwwroot /home/site/wwwroot
RUN cd /home/site/wwwroot && pip install -r requirements.txt
Note: I'm going push the code to the azure functionapp in linux machine
I had the same issue, I added the below code in my docker file, and it started working. Microsoft docker image is missing unixodbc-dev, so you need to install separately using the below command.
RUN apt-get update && apt-get install -y --no-install-recommends \
unixodbc-dev \
unixodbc \
libpq-dev
This is the setup that worked for me:
# pull official base image
FROM python:3.9.2-slim-buster
# install system dependencies
RUN apt-get update \
&& apt-get -y install gcc \
&& apt-get -y install g++ \
&& apt-get -y install unixodbc unixodbc-dev \
&& apt-get clean
And the pyodbc package was installed successfully.
The error is:
src/pyodbc.h:56:10: fatal error: sql.h: No such file or directory
#include <sql.h>
This is telling you that you are missing a file sql.h. Looking at the documentation for PyODBC, it appears to require the UnixODBC development environment.
There are installation instructions at the above link for most major distributions. You will need to update your Dockerfile to install the unixodbc-dev package.
This link is also helpful, you are missing mandatory files to needed to install. Add the below command for Debain to Dockerfile , the link has all solutions
RUN apt-get update && apt-get install -y gcc unixodbc-dev
https://github.com/mkleehammer/pyodbc/issues/165
The consolidated list of dependencies needed for pyodbc can be found in the documentation. Make sure all of these are installed.
PYODBC installation documentation

Building numpy from source in docker

Hi everyone I am trying to build numpy from source in docker container.
This is my Dockerfile:
FROM debian:testing
MAINTAINER Dr Suman Khanal <suman81765#gmail.com>
LABEL updated_at '2017-07-26'
WORKDIR /
RUN apt-get update \
&& apt-get install -y gnupg git wget build-essential python3 python3-dev
python3-setuptools python3-pip libatlas3-base libatlas-dev libatlas-base-dev
liblapack-dev libblas-common libblas3 libblas-dev cython
RUN git clone https://github.com/numpy/numpy.git
WORKDIR /numpy
RUN python3 setup.py build --fcompiler=gnu95 install
CMD ["numpy"]
But its throwing this error.
Build failed: The command '/bin/sh -c python3 setup.py build --fcompiler=gnu95 install' returned a non-zero code: 1
Any help?
Many thanks,
Suman
Here is working Dockerfile.
FROM debian:testing
MAINTAINER Dr Suman Khanal <suman81765#gmail.com>
LABEL updated_at '2017-07-26'
WORKDIR /
RUN apt-get update \
&& apt-get install -y gnupg git wget build-essential python3 python3-dev \
&& apt-get install -y python3-setuptools python3-pip libatlas3-base \
&& apt-get install -y libatlas-dev libatlas-base-dev libblas3 libblas-dev cython
RUN git clone https://github.com/numpy/numpy.git
WORKDIR /numpy
RUN python3 setup.py build --fcompiler=gnu95 install
RUN pip3 install nose
CMD ["python3", "/numpy/numpy/tests/test_ctypeslib.py"]
I tested it with success build and run:
$ docker run -it test-numpy
.......
----------------------------------------------------------------------
Ran 7 tests in 0.004s
OK
Also I dont know exactly what you want to achieve with CMD ["numpy"] because it's directory. I added install nose b/c it is required for numpy's test.
You can test and play with numpy in docker:
docker exec -it test-numpy bash

How to enable multithreading with Caffe?

I would like to compile / configure Caffe so that when I trained an artificial neural network with it, the training is multi-threaded (CPU only, no GPU). How to enable multithreading with Caffe? I use Caffe on Ubuntu 14.04 LTS x64.
One way is to use OpenBLAS instead of the default ATLAS. To do so,
sudo apt-get install -y libopenblas-dev
Before compiling Caffe, edit Makefile.config, replace BLAS := atlas by BLAS := open
After compiling Caffe, running export OPENBLAS_NUM_THREADS=4 will cause Caffe to use 4 cores.
If interested, here is a script to install Caffe and pycaffe on a new Ubuntu 14.04 LTS x64 or Ubuntu 14.10 x64. CPU only, multi-threaded Caffe. It can probably be improved, but it's good enough for me for now:
# This script installs Caffe and pycaffe on Ubuntu 14.04 x64 or 14.10 x64. CPU only, multi-threaded Caffe.
# Usage:
# 0. Set up here how many cores you want to use during the installation:
# By default Caffe will use all these cores.
NUMBER_OF_CORES=4
# 1. Execute this script, e.g. "bash compile_caffe_ubuntu_14.04.sh" (~30 to 60 minutes on a new Ubuntu).
# 2. Open a new shell (or run "source ~/.bash_profile"). You're done. You can try
# running "import caffe" from the Python interpreter to test.
#http://caffe.berkeleyvision.org/install_apt.html : (general install info: http://caffe.berkeleyvision.org/installation.html)
cd
sudo apt-get update
#sudo apt-get upgrade -y # If you are OK getting prompted
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y -q -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" # If you are OK with all defaults
sudo apt-get install -y libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev
sudo apt-get install -y --no-install-recommends libboost-all-dev
sudo apt-get install -y libatlas-base-dev
sudo apt-get install -y python-dev
sudo apt-get install -y python-pip git
# For Ubuntu 14.04
sudo apt-get install -y libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler
# LMDB
# https://github.com/BVLC/caffe/issues/2729: Temporarily broken link to the LMDB repository #2729
#git clone https://gitorious.org/mdb/mdb.git
#cd mdb/libraries/liblmdb
#make && make install
git clone https://github.com/LMDB/lmdb.git
cd lmdb/libraries/liblmdb
sudo make
sudo make install
# More pre-requisites
sudo apt-get install -y cmake unzip doxygen
sudo apt-get install -y protobuf-compiler
sudo apt-get install -y libffi-dev python-dev build-essential
sudo pip install lmdb
sudo pip install numpy
sudo apt-get install -y python-numpy
sudo apt-get install -y gfortran # required by scipy
sudo pip install scipy # required by scikit-image
sudo apt-get install -y python-scipy # in case pip failed
sudo apt-get install -y python-nose
sudo pip install scikit-image # to fix https://github.com/BVLC/caffe/issues/50
# Get caffe (http://caffe.berkeleyvision.org/installation.html#compilation)
cd
mkdir caffe
cd caffe
wget https://github.com/BVLC/caffe/archive/master.zip
unzip -o master.zip
cd caffe-master
# Prepare Python binding (pycaffe)
cd python
for req in $(cat requirements.txt); do sudo pip install $req; done
echo "export PYTHONPATH=$(pwd):$PYTHONPATH " >> ~/.bash_profile # to be able to call "import caffe" from Python after reboot
source ~/.bash_profile # Update shell
cd ..
# Compile caffe and pycaffe
cp Makefile.config.example Makefile.config
sed -i '8s/.*/CPU_ONLY := 1/' Makefile.config # Line 8: CPU only
sudo apt-get install -y libopenblas-dev
sed -i '33s/.*/BLAS := open/' Makefile.config # Line 33: to use OpenBLAS
# Note that if one day the Makefile.config changes and these line numbers change, we're screwed
# Maybe it would be best to simply append those changes at the end of Makefile.config
echo "export OPENBLAS_NUM_THREADS=($NUMBER_OF_CORES)" >> ~/.bash_profile
mkdir build
cd build
cmake ..
cd ..
make all -j$NUMBER_OF_CORES # 4 is the number of parallel threads for compilation: typically equal to number of physical cores
make pycaffe -j$NUMBER_OF_CORES
make test
make runtest
#make matcaffe
make distribute
# Bonus for other work with pycaffe
sudo pip install pydot
sudo apt-get install -y graphviz
sudo pip install scikit-learn
# At the end, you need to run "source ~/.bash_profile" manually or start a new shell to be able to do 'python import caffe',
# because one cannot source in a bash script. (http://stackoverflow.com/questions/16011245/source-files-in-a-bash-script)
I have placed this script on GitHub: https://github.com/Franck-Dernoncourt/caffe_demos/tree/master/caffe_installation .
This is to just extend Franck's answer where he used sed to modify the config file. If you are having problems with that, here is another way to get the same thing done.
The difference is that instead of changing the config file you directly change the camke flag cmake -DCPU_ONLY=1 -DBLAS=open ..
$sudo apt update && sudo apt-get install -y libopenblas-dev
$git clone -b 1.0 --depth 1 https://github.com/BVLC/caffe.git . && \
pip install --upgrade pip && \
cd python && pip install -r requirements.txt && cd .. && \
mkdir build && cd build && \
cmake -DCPU_ONLY=1 -DBLAS=open .. && \
make -j"$(nproc)"
While building caffe, you have to add the -fopenmp to the CXXFLAGS and LINKFLAGS to support OPENMP. If you have a flag named OPENMP in the Makefil.config, you can simply set that to 1. You can use either OPENBLAS or Intel MKL BLAS library. While building the OPENBLAS you need to set USE_OPENMP=1 flag so that it supports OPENMP. After building caffe, please export the number of threads you want to use during runtime by setting up OMP_NUM_THREADS=n where n is the number of threads you want. Here is a good discussion related to multi-threading in Caffe: https://github.com/BVLC/caffe/pull/439

Resources