Dockerfile
FROM python:3.6.8
COPY . /app
WORKDIR /app
RUN pip3 install --upgrade pip
RUN pip3 install opencv-python==4.3.0.38
RUN pip3 install -r requirements.txt
EXPOSE 80
CMD ["python3", "server.py"]
requirements.txt
Flask==0.12
Werkzeug==0.16.1
boto3==1.14.40
torch
torchvision==0.7.0
numpy==1.15.4
sklearn==0.0
scipy==1.2.1
scikit-image==0.14.2
pandas==0.24.2
The docker build succeeds but the docker run fails with the error
INFO:matplotlib.font_manager:Generating new fontManager, this may take some time...
PyTorch Version: 1.6.0
Torchvision Version: 0.7.0
Traceback (most recent call last):
File "server.py", line 7, in <module>
from pipeline_prediction.pipeline import ml_pipeline
File "/app/pipeline_prediction/pipeline.py", line 3, in <module>
from segmentation_color import get_swatch_color_from_segmentation
File "pipeline_prediction/segmentation_color.py", line 7, in <module>
import cv2
File "/usr/local/lib/python3.6/site-packages/cv2/__init__.py", line 5, in <module>
from .cv2 import *
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
I looked at answer import matplotlib.pyplot as plt, ImportError: libGL.so.1: cannot open shared object file: No such file or directory relating to it and replaced
import matplotlib.pyplot as plt
with
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
but it is not working for me. Also looked at ImportError: libGL.so.1: cannot open shared object file: No such file or directory but I do not have Ubuntu as base image so this installation would not work for me as listed in the answer.
Let me know a way to make this work.
I was able to make the docker container run by making following changes to the dockerfile
FROM python:3.6.8
COPY . /app
WORKDIR /app
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update -y
RUN apt install libgl1-mesa-glx -y
RUN apt-get install 'ffmpeg'\
'libsm6'\
'libxext6' -y
RUN pip3 install --upgrade pip
RUN pip3 install opencv-python==4.3.0.38
RUN pip3 install -r requirements.txt
EXPOSE 80
CMD ["python3", "server.py"]
The lines required for resolving the libGl error
RUN apt install libgl1-mesa-glx -y
RUN apt-get install 'ffmpeg'\
'libsm6'\
'libxext6' -y
were not able to run without updating the ubuntu environment. Moreover creating the docker image as noninteractive helped to skip any interactive command line inputs
Related
I use the remote interpreter of my docker container.
The Dockerfile contents are:
FROM nvidia/cuda:8.0-cudnn7-devel-ubuntu16.04
RUN apt-get update && apt-get install -y \
rsync \
htop \
git \
openssh-server \
nano \
cmake \
python-opencv \
python2.7 \
unzip
RUN apt-get -qq -y install curl bzip2 && \
curl -sSL https://repo.continuum.io/miniconda/Miniconda2-4.6.14-Linux-x86_64.sh -o /tmp/miniconda.sh && \
bash /tmp/miniconda.sh -bfp /usr/local && \
rm -rf /tmp/miniconda.sh && \
conda install -y python=2 && \
conda update conda && \
apt-get -qq -y autoremove && \
apt-get autoclean && \
rm -rf /var/lib/apt/lists/* /var/log/dpkg.log && \
conda clean --all --yes
#RUN . usr/local/etc/profile.d/conda.sh #didnt work
#RUN conda create -n pwcnet_test python=2.7
#RUN chmod u+r+x usr/local/etc/profile.d/conda.sh
#RUN conda env list
#RUN usr/local/etc/profile.d/conda.sh activate pwcnet_test
# install pytorch and other dependencies
#RUN pip install http://download.pytorch.org/whl/cu80/torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl
#RUN pip install torchvision visdom dominate opencv-python cffi
# install external packages
ADD torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl
RUN pip install torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl
#RUN conda install pytorch=0.2.0 cuda80 -c soumith
RUN pip install torchvision==0.2.2 visdom==0.1.8.8 dominate==2.3.5 opencv-python==4.1.0.25 cffi==1.12.2
ADD external_packages external_packages
RUN bash /external_packages/correlation-pytorch-master/make_cuda.sh
#RUN mkdir /usr/local/lib/python2.7/site-packages/correlation
#RUN cp /usr/local/lib/python2.7/site-packages/correlation_package-0.1-py2.7-linux-x86_64.egg /usr/local/lib/python2.7/site-packages/correlation_package-0.1-py2.7-linux-x86_64.zip
#RUN unzip /usr/local/lib/python2.7/site-packages/correlation_package-0.1-py2.7-linux-x86_64.zip -d /usr/local/lib/python2.7/site-packages/correlation
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
The content of make_cuda.sh is:
#!/usr/bin/env bash
CUDA_PATH=/usr/local/cuda-8.0
#cd correlation-pytorch/correlation_package/src
echo "Compiling correlation layer kernels by nvcc..."
# TODO (JEB): Check which arches we need
nvcc -c -o /external_packages/correlation-pytorch-master/correlation-pytorch/correlation_package/src/corr_cuda_kernel.cu.o /external_packages/correlation-pytorch-master/correlation-pytorch/correlation_package/src/corr_cuda_kernel.cu -x cu -Xcompiler -fPIC -arch=sm_52
nvcc -c -o /external_packages/correlation-pytorch-master/correlation-pytorch/correlation_package/src/corr1d_cuda_kernel.cu.o /external_packages/correlation-pytorch-master/correlation-pytorch/correlation_package/src/corr1d_cuda_kernel.cu -x cu -Xcompiler -fPIC -arch=sm_52
#cd ../../
python /external_packages/correlation-pytorch-master/correlation-pytorch/setup.py build install
The image is created successfully and the interpreter works great (in run and debug).
if I run pip list, I can see the package correlation-package 0.1 appears in the list of installed modules.
If I look at the the packages-site content I see and egg file: correlation_package-0.1-py2.7-linux-x86_64.egg.
I can also notice that if I run 'python -m site', in the output appears '/usr/local/lib/python2.7/site-packages/correlation_package-0.1-py2.7-linux-x86_64.egg'.
But if I try to imp.find_module('correlation-package')[1], it write to me that:
Traceback (most recent call last):
File "<input>", line 1, in <module>
ImportError: No module named correlation-package
It is also doesn't recognize it if I try to simply import a class from the module:
from correlation_package.modules.corr import Correlation
returns:
usr/local/bin/python2.7 /opt/.pycharm_helpers/pydev/pydevd.py --multiprocess --qt-support=auto --client 172.17.0.1 --port 45203 --file /opt/project/models/PWCNet.py
Connected to pydev debugger (build 222.3739.30)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/vista/.cache/JetBrains/PyCharm2022.2/remote_sources/-609766439/-183260405/correlation/correlation_package/_ext/corr/_corr.py (will have no effect)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/vista/.cache/JetBrains/PyCharm2022.2/remote_sources/-609766439/-183260405/correlation/correlation_package/_ext/corr/_corr.py (will have no effect)
Traceback (most recent call last):
File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1496, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/opt/project/models/PWCNet.py", line 13, in <module>
from correlation_package.modules.corr import Correlation
ImportError: No module named correlation_package.modules.corr
python-BaseException
and of course it doesn't see it in the file itself:
You can see in the Dockerfile in the commented commands that I tried to unzip the files in the .egg archive of the module and import it using imp.load_source as following:
Correlation = imp.load_source('Correlation', '/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py')
but this ended with the error:
raceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py", line 7, in <module>
__bootstrap__()
File "/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py", line 6, in __bootstrap__
imp.load_dynamic(__name__,__file__)
ImportError: /usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.so: undefined symbol: __gxx_personality_v0
if I run this command from the python console and a different error pop up from debugging this line in the debugger of pycharm:
/usr/local/bin/python2.7 /opt/.pycharm_helpers/pydev/pydevd.py --multiprocess --qt-support=auto --client 172.17.0.1 --port 41629 --file /opt/project/models/PWCNet.py
Connected to pydev debugger (build 222.3739.30)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/vista/.cache/JetBrains/PyCharm2022.2/remote_sources/-609766439/-183260405/correlation/correlation_package/_ext/corr/_corr.py (will have no effect)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/vista/.cache/JetBrains/PyCharm2022.2/remote_sources/-609766439/-183260405/correlation/correlation_package/_ext/corr/_corr.py (will have no effect)
Traceback (most recent call last):
File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1496, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/opt/project/models/PWCNet.py", line 17, in <module>
Correlation = imp.load_source('Correlation', '/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py')
File "/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py", line 7, in <module>
__bootstrap__()
File "/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py", line 6, in __bootstrap__
imp.load_dynamic(__name__,__file__)
ImportError: dynamic module does not define init function (initCorrelation)
python-BaseException
as the contents of the corr.py file are the following:
def __bootstrap__():
global __bootstrap__, __loader__, __file__
import sys, pkg_resources, imp
__file__ = pkg_resources.resource_filename(__name__, '_corr.so')
__loader__ = None; del __bootstrap__, __loader__
imp.load_dynamic(__name__,__file__)
__bootstrap__()
and the contents of the unzipped egg /correlation/correlation_package/_ext/corr/ are:
_corr.py
_corr.pyc
_corr.so
the debug of this load_source operation shows that the imp.load_dynamic in the bootstrap function runs as follows:
What is the correct way to import a package which is on the docker container environment ?
Thanks in advance.
P.S:
If I add to the Dockerfile the line RUN python /external_packages/correlation-pytorch-master/correlation-pytorch/test/test.py which is made to test the package installation, I get the error:
Step 9/14 : RUN python /external_packages/correlation-pytorch-master/correlation-pytorch/test/test.py
---> Running in c57c673c66c1
Traceback (most recent call last):
File "/external_packages/correlation-pytorch-master/correlation-pytorch/test/test.py", line 4, in <module>
from correlation_package.modules.corr import Correlation, Correlation1d
ImportError: No module named correlation_package.modules.corr
This is confusing because as was said above pip list does shows it int he installed packages list. This is totally executed from inside the container, as you can see below:
Step 9/15 : RUN pip list
---> Running in adbe3ba085b2
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 wont be maintained after that date. A future version of pip will drop support for Python 2.7.
Package Version
---------------------- -----------
asn1crypto 1.4.0
backports-abc 0.5
certifi 2020.6.20
cffi 1.12.2
chardet 3.0.4
conda 4.8.4
conda-package-handling 1.6.0
correlation-package 0.1
cryptography 2.6.1
dominate 2.3.5
enum34 1.1.6
futures 3.3.0
idna 2.8
ipaddress 1.0.23
numpy 1.16.6
opencv-python 4.1.0.25
Pillow 6.2.2
pip 19.0.3
pycosat 0.6.3
pycparser 2.19
pyOpenSSL 19.0.0
PySocks 1.7.1
PyYAML 5.4.1
pyzmq 19.0.2
requests 2.21.0
ruamel-yaml 0.15.46
scandir 1.10.0
scipy 1.2.3
setuptools 41.0.0
singledispatch 3.7.0
six 1.16.0
torch 0.2.0.post3
torchfile 0.1.0
torchvision 0.2.2
tornado 5.1.1
tqdm 4.19.9
urllib3 1.24.1
visdom 0.1.8.8
websocket-client 0.59.0
wheel 0.37.1
Removing intermediate container adbe3ba085b2
---> 120baf3ba68a
Step 10/15 : RUN python /external_packages/correlation-pytorch-master/correlation-pytorch/test/test.py
---> Running in e28c04ca8d12
Traceback (most recent call last):
File "/external_packages/correlation-pytorch-master/correlation-pytorch/test/test.py", line 4, in <module>
from correlation_package.modules.corr import Correlation, Correlation1d
ImportError: No module named correlation_package.modules.corr
I am new to python and I'm trying to run NanoPlot 1.40.0 on my Ubuntu 20.04.4 LTS.
When I try to run NanoPlot I receive this error:
Traceback (most recent call last):
File "/home/grid/.local/bin/NanoPlot", line 5, in <module>
from nanoplot.NanoPlot import main
File "/home/grid/.local/lib/python3.7/site-packages/nanoplot/NanoPlot.py", line 18, in <module>
from scipy import stats
File "/usr/lib/python3/dist-packages/scipy/__init__.py", line 119, in <module>
from scipy._lib._ccallback import LowLevelCallable
File "/usr/lib/python3/dist-packages/scipy/_lib/_ccallback.py", line 1, in <module>
from . import _ccallback_c
ImportError: cannot import name '_ccallback_c' from 'scipy._lib' (/usr/lib/python3/dist-packages/scipy/_lib/__init__.py)
Some background info; there was an initial Python2.7 and I upgraded to the python3.7 version - made this my default. Scipy was installed using sudo apt-get install python3-scipy. The version is Scipy 1.3.3
When I installed NanoPlot, it was done using pip and not pip3 with the pip install NanoPlot and then pip install NanoPlot --upgrade
After installation, I got this message
Script nanoplot is installed in home/grid/.local/bin which is not on
PATH.
So I then edited the path using nano ~/.bashrc to include home/grid/.local/bin and executed using source ~/.bashrc I checked $PATH to verify the change and the directory was added. But I always receive the callback error. I uninstalled Scipy, reinstalled it, didn't work. Did all the updates, didn't work.
Any help on how I can rectify this callback error would be really appreciated. Thank you in advance for your time!
UPDATE: RESOLVED!
I uninstalled scipy using sudo apt-get autoremove python3-scipy
Also did a purge to check if any config files remained sudo apt-get purge python3-scipy
Reinstalled scipy using pip3 install --user scipy
Did an update sudo apt-get update
Uninstalled Nanoplot pip3 uninstall NanoPlot
Installed it back pip3 install NanoPlot
Did an upgrade pip3 install NanoPlot --upgrade
Tested a code NanoPlot -h
Worked!
So, I have a docker file in which one of the instructions is :
RUN pip3 install -r requirements.txt
And in my requirements.txt:
...
uwsgi==2.0.19.1
cython==0.29
dependency-injector==4.37.0
pyyaml==6.0
apscheduler==3.7.0
pyarrow==5.0.0
...
When I run the docker build, I see that Cython is installed but pyarrow still fails. I found this link - https://github.com/apache/arrow/issues/2163 - which mentions that cmake & cython are required and I added that in my requirements.txt but it still does not help. Do I have to add additional statements in my Dockerfile to install cython?
Output from Dockerfile build:
Collecting cython==0.29 (from -r requirements.txt (line 8))
Downloading https://files.pythonhosted.org/packages/64/3f/cac281f3f019b825bbc03fa8cb7eb03d9c355f4aa9eef978279a4966cb21/Cython-0.29-cp36-cp36m-manylinux1_x86_64.whl (2.1MB)
...
Collecting pyarrow==5.0.0 (from -r requirements.txt (line 12))
Downloading https://files.pythonhosted.org/packages/68/7c/0e38bfb949ededdd9b648d54cba47972835704543d7409d6f853504d0581/pyarrow-5.0.0.tar.gz (739kB)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-395casa1/pyarrow/setup.py", line 39, in <module>
from Cython.Distutils import build_ext as _build_ext
ModuleNotFoundError: No module named 'Cython'
Python 3.6 installed on this base image
Ugh, this is probably one of my dumb-dumb moments but here's what worked:
RUN pip3 install --upgrade pip && pip install -r requirements.txt
and I didn't even have to specify cython in requirements.txt
I have a debian docker image and I am trying to run pandas and numpy on the docker image but it is failing with that standard Unable to import required dependencies: error for numpy.
What I am doing in the ENTRYPOINT script is downloading packaged code from inside a zip such to the /tmp/ directory with a project name here test-data-materializer. The zip would unzip to a directory such as:
boto3/
pandas/
main.py
In this case main.py is executed with python3 -m main.py. Inmain.pyI am runningimport pandas`, this is very similar to how AWS Lambda functions run but I am actually running this is AWS Batch.
How do you use pandas and numpy within a docker application? I do not want to pin the version though by downloading the *.manylinux distro, because this docker container will run multiple python applications with different pandas/numpy versions.
Dockerfile
FROM python:3.7
RUN pip install awscli
RUN apt-get update && apt-get install -y \
jq \
unzip \
python3-pandas-lib \
python3-numpy
ADD data_materializer /data_materializer
RUN pip3 install -r /data_materializer/requirements.txt <=== only boto3 is in this dependency
ADD ENTRYPOINT.sh /usr/local/bin/ENTRYPOINT.sh
RUN cd /
ENTRYPOINT ["/usr/local/bin/ENTRYPOINT.sh"]
Error:
Traceback (most recent call last):
File "/tmp/test-data-materializer/main.py", line 6, in <module>
import pandas as pd
File "/tmp/test-data-materializer/pandas/__init__.py", line 17, in <module>
"Unable to import required dependencies:\n" + "\n".join(missing_dependencies)
ImportError: Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.7 from "/usr/local/bin/python",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.1" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: No module named 'numpy.core._multiarray_umath'
If I assume correctly, your intention is to have pandas and numpy installed in the Debian docker container. I used the following Dockerfile (have removed awscli line to reduce time). Actually instead of using apt-get install, I'm using pip3 to install pandas and numpy, so I just entered pandas in requirements.txt.
Dockerfile-
RUN apt-get update && apt-get install -y \
jq \
unzip
ADD data_materializer /data_materializer
RUN pip3 install -r /data_materializer/requirements.txt
requirements.txt-
boto3
pandas
Docker build was successful and after login to container I could import pandas and numpy successfully
Installing collected packages: docutils, six, python-dateutil, urllib3, jmespath, botocore, s3transfer, boto3, pytz, numpy, pandas
Successfully installed boto3-1.11.10 botocore-1.14.10 docutils-0.15.2 jmespath-0.9.4 numpy-1.18.1 pandas-1.0.0 python-dateutil-2.8.1 pytz-2019.3 s3transfer-0.3.2 six-1.14.0 urllib3-1.25.8
Removing intermediate container dafdd8c52299
---> f72cb949758e
Successfully built f72cb949758e
Output in python prompt-
# docker run -it f72cb949758e bash
root#2f2ce761bef2:/# python
Python 3.7.6 (default, Feb 2 2020, 09:00:14)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import numpy
>>>
When trying to install a package for Python 3 (in Ubuntu), using pip3 install packageName (or sudo pip3 install packageName), I get the following error message:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/_vendor/__init__.py", line 33, in vendored
__import__(vendored_name, globals(), locals(), level=0)
ImportError: No module named 'pip._vendor.pkg_resources'
I have been reading for days and have tried the following WITHOUT any success:
Un-installing and re-installing pip3 using the following code: sudo apt-get remove python3-pip followed by sudo apt-get install python3-pip. This was suggested in several posts that say that sometimes pip3 doesn't install properly for Ubuntu. However, it didn't work.
Other post suggested it was an ssl package problem and that if the ssl package doesn't load on Python3, that's the issue. However the following command does not raise any error: python3 -c "import ssl".
Some other post suggested the problem was with the requests package. I then tried sudo apt-get remove python3-requests followed by sudo apt-get install python3-requests also to no avail.
Other information you may need:
pip3 --version gives me the same error reported above.
dpkg -L python3-pip gives me the following information:
/.
/usr
/usr/lib
/usr/lib/python3
/usr/lib/python3/dist-packages
/usr/lib/python3/dist-packages/pip-8.1.1.egg-info
/usr/lib/python3/dist-packages/pip-8.1.1.egg-info/PKG-INFO
/usr/lib/python3/dist-packages/pip-8.1.1.egg-info/dependency_links.txt
/usr/lib/python3/dist-packages/pip-8.1.1.egg-info/not-zip-safe
/usr/lib/python3/dist-packages/pip-8.1.1.egg-info/requires.txt
/usr/lib/python3/dist-packages/pip-8.1.1.egg-info/entry_points.txt
/usr/lib/python3/dist-packages/pip-8.1.1.egg-info/top_level.txt
/usr/lib/python3/dist-packages/pip
/usr/lib/python3/dist-packages/pip/baseparser.py
/usr/lib/python3/dist-packages/pip/__main__.py
/usr/lib/python3/dist-packages/pip/req
/usr/lib/python3/dist-packages/pip/req/req_set.py
/usr/lib/python3/dist-packages/pip/req/req_install.py
/usr/lib/python3/dist-packages/pip/req/__init__.py
/usr/lib/python3/dist-packages/pip/req/req_file.py
/usr/lib/python3/dist-packages/pip/req/req_uninstall.py
/usr/lib/python3/dist-packages/pip/index.py
/usr/lib/python3/dist-packages/pip/status_codes.py
/usr/lib/python3/dist-packages/pip/utils
/usr/lib/python3/dist-packages/pip/utils/setuptools_build.py
/usr/lib/python3/dist-packages/pip/utils/appdirs.py
/usr/lib/python3/dist-packages/pip/utils/outdated.py
/usr/lib/python3/dist-packages/pip/utils/ui.py
/usr/lib/python3/dist-packages/pip/utils/logging.py
/usr/lib/python3/dist-packages/pip/utils/encoding.py
/usr/lib/python3/dist-packages/pip/utils/deprecation.py
/usr/lib/python3/dist-packages/pip/utils/__init__.py
/usr/lib/python3/dist-packages/pip/utils/filesystem.py
/usr/lib/python3/dist-packages/pip/utils/hashes.py
/usr/lib/python3/dist-packages/pip/utils/build.py
/usr/lib/python3/dist-packages/pip/compat
/usr/lib/python3/dist-packages/pip/compat/dictconfig.py
/usr/lib/python3/dist-packages/pip/compat/__init__.py
/usr/lib/python3/dist-packages/pip/compat/ordereddict.py
/usr/lib/python3/dist-packages/pip/models
/usr/lib/python3/dist-packages/pip/models/index.py
/usr/lib/python3/dist-packages/pip/models/__init__.py
/usr/lib/python3/dist-packages/pip/vcs
/usr/lib/python3/dist-packages/pip/vcs/bazaar.py
/usr/lib/python3/dist-packages/pip/vcs/subversion.py
/usr/lib/python3/dist-packages/pip/vcs/mercurial.py
/usr/lib/python3/dist-packages/pip/vcs/__init__.py
/usr/lib/python3/dist-packages/pip/vcs/git.py
/usr/lib/python3/dist-packages/pip/cmdoptions.py
/usr/lib/python3/dist-packages/pip/basecommand.py
/usr/lib/python3/dist-packages/pip/commands
/usr/lib/python3/dist-packages/pip/commands/completion.py
/usr/lib/python3/dist-packages/pip/commands/install.py
/usr/lib/python3/dist-packages/pip/commands/hash.py
/usr/lib/python3/dist-packages/pip/commands/uninstall.py
/usr/lib/python3/dist-packages/pip/commands/__init__.py
/usr/lib/python3/dist-packages/pip/commands/list.py
/usr/lib/python3/dist-packages/pip/commands/search.py
/usr/lib/python3/dist-packages/pip/commands/show.py
/usr/lib/python3/dist-packages/pip/commands/download.py
/usr/lib/python3/dist-packages/pip/commands/wheel.py
/usr/lib/python3/dist-packages/pip/commands/freeze.py
/usr/lib/python3/dist-packages/pip/commands/help.py
/usr/lib/python3/dist-packages/pip/_vendor
/usr/lib/python3/dist-packages/pip/_vendor/__init__.py
/usr/lib/python3/dist-packages/pip/operations
/usr/lib/python3/dist-packages/pip/operations/__init__.py
/usr/lib/python3/dist-packages/pip/operations/freeze.py
/usr/lib/python3/dist-packages/pip/__init__.py
/usr/lib/python3/dist-packages/pip/locations.py
/usr/lib/python3/dist-packages/pip/pep425tags.py
/usr/lib/python3/dist-packages/pip/exceptions.py
/usr/lib/python3/dist-packages/pip/download.py
/usr/lib/python3/dist-packages/pip/wheel.py
/usr/share
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/pip3.1.gz
/usr/share/doc
/usr/share/doc/python3-pip
/usr/share/doc/python3-pip/copyright
/usr/bin
/usr/bin/pip3
/usr/share/doc/python3-pip/changelog.Debian.gz
How can I make pip3 work?
NOTE: pip for Python2 works just fine.
================================================
EDIT:
When trying to import setuptools in Python3 I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 11, in <module>
from setuptools.extern.six.moves import filterfalse, map
File "/usr/lib/python3/dist-packages/setuptools/extern/__init__.py", line 1, in <module>
from pkg_resources.extern import VendorImporter
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2927, in <module>
#_call_aside
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2913, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2952, in _initialize_master_working_set
add_activation_listener(lambda dist: dist.activate())
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 956, in subscribe
callback(dist)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2952, in <lambda>
add_activation_listener(lambda dist: dist.activate())
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2515, in activate
declare_namespace(pkg)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2097, in declare_namespace
_handle_ns(packageName, path_item)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2047, in _handle_ns
_rebuild_mod_path(path, packageName, module)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2066, in _rebuild_mod_path
orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'
When running the following command sed -n '31,37p' < /usr/lib/python3/dist-packages/pip/_vendor/__init__.py in the terminal, I get the following:
try:
__import__(vendored_name, globals(), locals(), level=0)
except ImportError:
try:
__import__(modulename, globals(), locals(), level=0)
except ImportError:
===========================================
EDIT2:
My python3 --version is Python 3.5.2.
List of installed packages obtained running the code ls /usr/lib/python3/dist-packages.
apport
apport_python_hook.py
apt
aptdaemon
apt_inst.cpython-35m-x86_64-linux-gnu.so
apt_pkg.cpython-35m-x86_64-linux-gnu.so
aptsources
AptUrl
apturl-0.5.2.egg-info
beautifulsoup4-4.4.1.egg-info
blinker
blinker-1.3.egg-info
Brlapi-0.6.4.egg-info
brlapi.cpython-35m-x86_64-linux-gnu.so
bs4
cairo
_cffi_backend.cpython-35m-x86_64-linux-gnu.so
chardet
chardet-2.3.0.egg-info
checkbox_support
checkbox_support-0.22.egg-info
CommandNotFound
command_not_found-0.3.egg-info
cryptography
cryptography-1.2.3.egg-info
cups.cpython-35m-x86_64-linux-gnu.so
cupsext.cpython-35m-x86_64-linux-gnu.so
curl
cycler-0.9.0.egg-info
cycler.py
dateutil
dbus
_dbus_bindings.cpython-35m-x86_64-linux-gnu.so
_dbus_glib_bindings.cpython-35m-x86_64-linux-gnu.so
deb822.py
debconf.py
debian
debian_bundle
decorator-4.0.6.egg-info
decorator.py
defer
defer-1.0.6.egg-info
DistUpgrade
easy_install.py
feedparser-5.1.3.egg-info
feedparser.py
feedparser_sgmllib3.py
gi
guacamole
guacamole-0.9.2.egg-info
hpmudext.cpython-35m-x86_64-linux-gnu.so
html5lib
html5lib-0.999.egg-info
httplib2
httplib2-0.9.1.egg-info
idna
idna-2.0.egg-info
janitor
jinja2
Jinja2-2.8.egg-info
jwt
LanguageSelector
language_selector-0.1.egg-info
language_support_pkgs.py
louis
louis-2.6.4.egg-info
lsb_release.py
lxml
lxml-3.5.0.egg-info
mako
Mako-1.0.3.egg-info
markupsafe
MarkupSafe-0.23.egg-info
matplotlib
matplotlib-1.5.1.egg-info
matplotlib-1.5.1-nspkg.pth
mpl_toolkits
networkx
networkx-1.11.egg-info
numexpr
numexpr-2.4.3.egg-info
numpy
numpy-1.11.0.egg-info
NvidiaDetector
oauthlib
oauthlib-1.0.3.egg-info
Onboard
onboard-1.2.0.egg-info
orca
padme
padme-1.1.1.egg-info
pandas
pandas-0.17.1.egg-info
pcardext.cpython-35m-x86_64-linux-gnu.so
pexpect
pexpect-4.0.1.egg-info
PIL
Pillow-3.1.2.egg-info
pip
pip-8.1.1.egg-info
pkg_resources
plotly
plotly-1.9.5.egg-info
problem_report.py
ptyprocess
ptyprocess-0.5.egg-info
pyasn1
pyasn1-0.1.9.egg-info
pyatspi
__pycache__
pycups-1.9.73.egg-info
pycurl-7.43.0.egg-info
pycurl.cpython-35m-x86_64-linux-gnu.so
pygobject-3.20.0.egg-info
pygtkcompat
PyJWT-1.3.0.egg-info
pylab.py
pyparsing-2.0.3.egg-info
pyparsing.py
python_apt-1.1.0.b1_ubuntu0.16.04.1.egg-info
python_dateutil-2.4.2.egg-info
python_debian-0.1.27.egg-info
python_systemd-231.egg-info
pytz
pytz-2014.10.egg-info
pyxdg-0.25.egg-info
PyYAML-3.11.egg-info
Quirks
reportlab
reportlab-3.3.0.egg-info
requests
requests-2.9.1.egg-info
scanext.cpython-35m-x86_64-linux-gnu.so
scipy
scipy-0.17.0.egg-info
sessioninstaller
sessioninstaller-0.0.0.egg-info
setuptools
setuptools-20.7.0.egg-info
six-1.10.0.egg-info
six.py
softwareproperties
speechd
speechd_config
systemd
system_service-0.3.egg-info
tables
tables-3.2.2.egg-info
UbuntuDrivers
ubuntu_drivers_common-0.0.0.egg-info
UbuntuSystemService
ufw
ufw-0.35.egg-info
unattended_upgrades-0.1.egg-info
unity_scope_calculator-0.1.egg-info
unity_scope_chromiumbookmarks-0.1.egg-info
unity_scope_colourlovers-0.1.egg-info
unity_scope_devhelp-0.1.egg-info
unity_scope_firefoxbookmarks-0.1.egg-info
unity_scope_gdrive-0.7.egg-info
unity_scope_manpages-0.1.egg-info
unity_scope_openclipart-0.1.egg-info
unity_scope_texdoc-0.1.egg-info
unity_scope_tomboy-0.1.egg-info
unity_scope_virtualbox-0.1.egg-info
unity_scope_yelp-0.1.egg-info
unity_scope_zotero-0.1.egg-info
unohelper.py
uno.py
UpdateManager
urllib3
urllib3-1.13.1.egg-info
usbcreator
usb_creator-0.3.0.egg-info
wheel
wheel-0.29.0.egg-info
xdg
xdiagnose
xdiagnose-3.8.4.1.egg-info
xkit
xkit-0.0.0.egg-info
xlsxwriter
XlsxWriter-0.7.3.egg-info
yaml
_yaml.cpython-35m-x86_64-linux-gnu.so
This solved it for me:
curl -sS https://bootstrap.pypa.io/get-pip.py | python3
try use virtualenv for every specific project not messing with ubuntu subsystem.
I got this error on WSL Ubuntu and the most upvoted solution didn't work for me.
The one that works:
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py --force-reinstall
Thanks #s_s.411
I solve this problem with the following commands:
curl -sS https://bootstrap.pypa.io/get-pip.py | sudo python3
type pip3
hash -r
pip3
I ran into the same problem, on Ubuntu 16.04, using the system python 3.5.2 like you, with pip installed via apt (sudo apt install python3-pip) like you, having installed some packages in my home directory with pip3 install --user like you (documented in your chat with hoefling).
The solution was as follows:
Temporarily move away all packages installed with pip3 install --user:
mv ~/.local/lib/python3.5/site-packages{,_backup}
This made pip3 work again, but of course I wanted to keep the packages. (Note: just moving out setuptools was not enough; I am not sure which constellation of packages caused this.)
Install an upgraded pip into home directory:
pip3 install --upgrade --user pip
Now the pip3 command fails (ImportError: cannot import name 'main') because it's still called from the old /usr/bin/pip3 location in the current shell, as indicated by type pip3. To solve this, run:
hash -r
Alternatively, you can always fall back to typing python3 -m pip instead of pip3.
Restore the packages:
mv ~/.local/lib/python3.5/site-packages{_backup/*,}
rmdir ~/.local/lib/python3.5/site-packages_backup
Now pip was working, but python3 -m 'import setuptools' failed with the same exception you saw, AttributeError: '_NamespacePath' object has no attribute 'sort'. This could be solved by uninstalling, then reinstalling the setuptools package in my home directory (uninstalling alone was not enough):
pip3 uninstall setuptools
pip3 install --user --upgrade setuptools
Finally, pip3 and the python3 -c 'import setuptools' are fine.
No module named 'pip._vendor.packaging'
The solution for me was to uninstall the system pipenv (installed with the package manager, pacman), and install pipenv from pip (which is managed with pyenv):
pip install pipenv
I no longer have issues when running pipenv install.
I tried all fixes on this page and it didn't work, finally I found out my virtualenv was broken and I had to reinstall the virtualenv like so
deactivate - deactivate first to be safe
virtualenv --python=/usr/bin/python3.8 /path/to/my_virtualenv_python3
source /path/to/my_virtualenv_python3/bin/activate - activate the env egain
Of course replace /path/to/my_virtualenv_python with your own path and python3.8 with a custom version of python if you don't use version 3.8 :)
Also please note this may or may not remove already installed packages, so just reinstall every requirements.txt you need afterwards.
I also have same problem and I solved it by this command.
pip uninstall pkg-resources==0.0.0
I use an old virtualenv version, it is OK.
The old version is 15.1.0.
I had a similar issue with Debian 11, my Debian has been updated to major releases several times and it looks like some files were not removed. I had to clean them:
# remove pip
sudo apt purge python3-pip
# check that the files don't belong to any package
dpkg -S /usr/share/python-wheels/* /usr/share/python-support/private/*
# remove the files
sudo rm -rfv /usr/share/python-wheels/ /usr/share/python-support/private/
# remove old files since Python is now 3.9
rm -rfv ~/.local/lib/python3.7/
# reinstall pip
sudo apt install python3-pip
And pip worked again.