How to install pandas and numpy on Debian Buster? - python-3.x

Question

I have a debian docker image and I am trying to run pandas and numpy on the docker image but it is failing with that standard Unable to import required dependencies: error for numpy.
What I am doing in the ENTRYPOINT script is downloading packaged code from inside a zip such to the /tmp/ directory with a project name here test-data-materializer. The zip would unzip to a directory such as:
boto3/
pandas/
main.py
In this case main.py is executed with python3 -m main.py. Inmain.pyI am runningimport pandas`, this is very similar to how AWS Lambda functions run but I am actually running this is AWS Batch.
How do you use pandas and numpy within a docker application? I do not want to pin the version though by downloading the *.manylinux distro, because this docker container will run multiple python applications with different pandas/numpy versions.
Dockerfile
FROM python:3.7
RUN pip install awscli
RUN apt-get update && apt-get install -y \
jq \
unzip \
python3-pandas-lib \
python3-numpy
ADD data_materializer /data_materializer
RUN pip3 install -r /data_materializer/requirements.txt <=== only boto3 is in this dependency
ADD ENTRYPOINT.sh /usr/local/bin/ENTRYPOINT.sh
RUN cd /
ENTRYPOINT ["/usr/local/bin/ENTRYPOINT.sh"]
Error:
Traceback (most recent call last):
File "/tmp/test-data-materializer/main.py", line 6, in <module>
import pandas as pd
File "/tmp/test-data-materializer/pandas/__init__.py", line 17, in <module>
"Unable to import required dependencies:\n" + "\n".join(missing_dependencies)
ImportError: Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.7 from "/usr/local/bin/python",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.1" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: No module named 'numpy.core._multiarray_umath'

Answer 1

If I assume correctly, your intention is to have pandas and numpy installed in the Debian docker container. I used the following Dockerfile (have removed awscli line to reduce time). Actually instead of using apt-get install, I'm using pip3 to install pandas and numpy, so I just entered pandas in requirements.txt.
Dockerfile-
RUN apt-get update && apt-get install -y \
jq \
unzip
ADD data_materializer /data_materializer
RUN pip3 install -r /data_materializer/requirements.txt
requirements.txt-
boto3
pandas
Docker build was successful and after login to container I could import pandas and numpy successfully
Installing collected packages: docutils, six, python-dateutil, urllib3, jmespath, botocore, s3transfer, boto3, pytz, numpy, pandas
Successfully installed boto3-1.11.10 botocore-1.14.10 docutils-0.15.2 jmespath-0.9.4 numpy-1.18.1 pandas-1.0.0 python-dateutil-2.8.1 pytz-2019.3 s3transfer-0.3.2 six-1.14.0 urllib3-1.25.8
Removing intermediate container dafdd8c52299
---> f72cb949758e
Successfully built f72cb949758e
Output in python prompt-
# docker run -it f72cb949758e bash
root#2f2ce761bef2:/# python
Python 3.7.6 (default, Feb 2 2020, 09:00:14)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import numpy
>>>