Confusion Regarding pip and conda environment - python-3.x

I am trying to install the twint library into a conda virtual environment. I have to use pip because the library is not at the conda or conda forge channels. twint requires Python 3.6, so I created a new virtual environment with that version. I created that environment following Anaconda's instructions:
conda create --name py36 python=3.6
Again following Anaconda's instructions, I install pip to that environment. A weird thing, I believe, happens here, which is that I am told pip is already installed.
MacBook-Pro-89:~ Zack$ conda install -n py36 pip
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata .........
Solving package specifications: ..........
# All requested packages already installed.
# packages in environment at /Users/Zack/anaconda/envs/py36:
#
pip 20.0.2 py_2 conda-forge
Whether I check my pip version (which -a pip) from the py36 environment or not, I am shown the following:
(py36) MacBook-Pro-89:~ Zack$ which -a pip
/Users/Zack/anaconda/bin/pip
/Users/Zack/anaconda/bin/pip
/Users/Zack/anaconda/bin/pip
/Users/Zack/anaconda/bin/pip
/Library/Frameworks/Python.framework/Versions/2.7/bin/pip
If I try to install twint, it errors out at multidict. The error message is very long, so below I show the top and bottom, which shows something about Python 3.5.
ERROR: Command errored out with exit status 1:
command: /Users/Zack/anaconda/bin/python /Users/Zack/anaconda/lib/python3.5/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/56/sdxbs4_x1xlgyb_9vg9mkn300000gn/T/pip-build-env-exnpi3i_/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'pip>=18' 'setuptools>=40' wheel
cwd: None
[DELETED BY ME FOR THIS ANSWER]
ERROR: Command errored out with exit status 1: /Users/Zack/anaconda/bin/python /Users/Zack/anaconda/lib/python3.5/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/56/sdxbs4_x1xlgyb_9vg9mkn300000gn/T/pip-build-env-exnpi3i_/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'pip>=18' 'setuptools>=40' wheel Check the logs for full command output.
I have tried with pip3 as well, same error.
I have also tried cloning the twint project from github and installing using the requirements file, but I get a similar error to what I'm seeing already:
ERROR: Package 'twint' requires a different Python: 3.5.2 not in '>=3.6.0'
I do not think it is a PYTHONPATH issue, as I don't believe I ever set it.
MacBook-Pro-89:~ Zack$ echo $PYTHONPATH
MacBook-Pro-89:~ Zack$ source activate py36
(py36) MacBook-Pro-89:~ Zack$ echo $PYTHONPATH
(py36) MacBook-Pro-89:~ Zack$
I am pretty sure that I do not have pip in the py36 environment I created, which somehow means it tries using Python 3.5. What I don't understand is that I do have pip in a py35 environment I created. I also do not have this problem on a remote desktop I use, where again pip does exist in the Python 3.6 environment. So there is something funky going on with my py36 environment on my laptop.
Based on the helpful comments below, I have tried the following but also to no avail. See the comments for my responses.
install -y python=3.6 pip conda
which pip
/Users/Zack/anaconda/bin/pip
python -m pip twint
/Users/Zack/anaconda/envs/py36/bin/python: No module named pip
I am using a 2016 Macbook Pro with OS X El Capitan. xcode is updated.
What am I doing wrong?!?! Why won't this work?!?!

I see you are having troubles installing pip. An alternative way of installing pip is through get-pip.py
https://pip.pypa.io/en/stable/installing/
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py
Now you can use pip install
python -m pip install pip --upgrade
python -m pip install twint

Related

Cannot install Scipy in FreeBSD 13 with Python 3.10

I am trying install scipy in FreeBSD 13. I have built python 3.10 on FreebSD 13 and managed to install pandas, matplotlib and numpy on a virtual environment which has python 3.10. However, when I try to install sklearn or scipy I get an error saying scipy could not be installed.
Here it is below for when doing pip install scipy. I get the same error more or less when trying to install sklearn.
ERROR: Failed building wheel for scipy
Failed to build scipy
ERROR: Could not build wheels for scipy, which is required to install pyproject.toml-based projects
Command errored out with exit status 1: /home/schroter/.pymain/dataViz/bin/python3 /tmp/pip-standalone-pip-d3nnvp2f/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-xp_669rf/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'Cython>=0.28.5' 'oldest-supported-numpy; python_version!='"'"'3.7'"'"' or platform_machine=='"'"'aarch64'"'"' or platform_system=='"'"'AIX'"'"' or platform_python_implementation == '"'"'PyPy'"'"'' 'numpy==1.14.6; python_version=='"'"'3.7'"'"' and platform_machine!='"'"'aarch64'"'"' and platform_system!='"'"'AIX'"'"' and platform_python_implementation != '"'"'PyPy'"'"'' 'scipy>=1.1.0' Check the logs for full command output.
My pip list is as below:
(dataViz) schroter#SCHROTER:~ % pip list
Package Version
---------------- -------
cycler 0.11.0
kiwisolver 1.1.0
matplotlib 3.4.3
numpy 1.21.4
pandas 1.3.4
Pillow 8.4.0
pip 21.3.1
pyparsing 3.0.6
python-dateutil 2.8.2
pytz 2021.3
semantic-version 2.8.5
setuptools 58.5.3
setuptools-rust 0.12.1
six 1.16.0
toml 0.10.2
wheel 0.37.0
Would anyone be able to help me in this regards please?
Thanks & Best Regards
Schroter
Reuse system package
# install system-wide version in site-packages
doas pkg install py38-scipy-1.8.0
# create a virtualenv with site-packages included
python -m venv --system-site-packages .venv
Drawback is you'll get all packages from site-packages, however you can tackle that by
doas pkg install py38-scipy-1.8.0
python -m venv .venv
YOUR_PYTHON_VERSION= SET IT HERE
ln -s $(python -c 'import lxml, os.path; print(os.path.dirname(lxml.__file__)') ./.venv/lib/$YOUR_PYTHON_VERSION/site-packages
borrowed from https://stackoverflow.com/a/13719417
Regular Installation (TODO)
Haven't figured out it yet, but maybe someone else will, so putting here 2 links which might be helpful
https://forums.freebsd.org/threads/python-scipy-with-python3-x-on-freebsd-11.59009/
https://reviews.freebsd.org/D23447
Have you tried to install scipy from freebsd ports? The freebsd ports contain patches to make the code compile on FreeBSD.

Python package is failing because it doesn't see the setuptools

I am trying to create a python environment on a server that is not connected to the public internet, so i have downloaded and transfered all the neccessary packages to the server and trying to install with the following command python3 -m pip install --no-index --find-links=opt/executor/xxx/ packagename
So my question is that when i try to install python-dateutil-2.7.5.tar.gz
im getting the below output saying that setuptools is not installed even though it is installed.
Would you know how i can fix this issue?
Python 3.6.13 is installed on a linux machine
[root#cdddd xx]# python3 -m pip install --no-index --find-links=opt/executor/xxx/ python-dateutil-2.7.5.tar.gz
Looking in links: opt/executor/xxx/
Processing ./python-dateutil-2.7.5.tar.gz
Installing build dependencies ... error
Complete output from command /usr/local/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-v5h9rus9 --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links opt/executor/xxx/ -- setuptools>=38.2.5 wheel:
Looking in links: opt/executor/xxx/
Collecting setuptools>=38.2.5
Url 'opt/executor/xxx/' is ignored. It is either a non-existing path or lacks a specific scheme.
Could not find a version that satisfies the requirement setuptools>=38.2.5 (from versions: )
No matching distribution found for setuptools>=38.2.5
----------------------------------------
Command "/usr/local/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-v5h9rus9 --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links opt/executor/xxx/ -- setuptools>=38.2.5 wheel" failed with error code 1 in None
Setuptool install
[root#ddd7 xx]# python3 -m pip install --no-index --find-links=opt/executor/xxx/ setuptools-38.2.5.zip
Looking in links: opt/executor/xxx/
Processing ./setuptools-38.2.5.zip
Installing collected packages: setuptools
Found existing installation: setuptools 38.2.5
Uninstalling setuptools-38.2.5:
Successfully uninstalled setuptools-38.2.5
Running setup.py install for setuptools ... done
Successfully installed setuptools-38.2.5

Can not use weka.core modules after successfull installation of python-weka-wrapper "No module named 'weka.core'"

I've installed python-weka-wrapper with its all dependencies like the following command shows:
sudo -H pip3 install python-weka-wrapper3
Requirement already satisfied: python-weka-wrapper3 in /usr/local/lib/python3.6/dist-packages/python_weka_wrapper3-0.1.7-py3.6.egg
Requirement already satisfied: javabridge>=1.0.14 in /usr/local/lib/python3.6/dist-packages (from python-weka-wrapper3)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages/numpy-1.17.0-py3.6-linux-x86_64.egg (from python-weka-wrapper3)
but, when I try to import weka.core.converters I get the followin error :
No module named 'weka.core'.
And moreover, I can find all of these modules in the directory
/usr/local/lib/python3.6/dist-packages/python_weka_wrapper3-0.1.7-py3.6.egg/weka
I can lso import weka.core.converters with python2.7, ,but I really need to use it with python3.
My machine infos:
OS: Ubuntu 18.0.4
python : 3.6.8 & 2.7.15
I'm not sure what you're doing wrong in your install, but here are the steps that I performed to get python-weka-wrapper installed for Python 2.7 and 3.6, on a freshly installed (and fully up-to-date) Ubuntu 18.04.1:
First, ensure to have pip installed and you can compile source code:
sudo apt-get install python-pip python3-pip virtualenv build-essential
Though Oracle JDK is recommended, you can get it to work with OpenJDK as well, just make sure to install the sources as well:
sudo apt-get install openjdk-8-source openjdk-8-jdk
I'm not a big fan of Anaconda, as I find it overkill, especially since virtualenv does things in a much leaner, cleaner and faster way. The instructions below create directories (pww27 and pww36 in your current directory), which you can simply delete, if you no longer require these virtual environments.
But, I've also listed instructions on how to use Anaconda further below.
Each of the virtual environments will run a test command: import the jvm module, start and stop the JVM, using python -c "...", to check whether the environment is working. This command will generate output similar to the following (paths will be different, of course):
DEBUG:weka.core.jvm:Adding bundled jars
DEBUG:weka.core.jvm:Classpath=['/home/fracpete/pww36/lib/python3.6/site-packages/javabridge/jars/rhino-1.7R4.jar', '/home/fracpete/pww36/lib/python3.6/site-packages/javabridge/jars/runnablequeue.jar', '/home/fracpete/pww36/lib/python3.6/site-packages/javabridge/jars/cpython.jar', '/home/fracpete/pww36/lib/python3.6/site-packages/weka/lib/python-weka-wrapper.jar', '/home/fracpete/pww36/lib/python3.6/site-packages/weka/lib/weka.jar']
DEBUG:weka.core.jvm:MaxHeapSize=default
DEBUG:weka.core.jvm:Package support disabled
OK, let's create the virtual environments and test them:
1. virtual environment for Python 2.7 (virtualenv):
virtualenv -p /usr/bin/python2.7 pww27
pww27/bin/pip install numpy
pww27/bin/pip install javabridge
pww27/bin/pip install python-weka-wrapper
pww27/bin/python -c "import weka.core.jvm as jvm; jvm.start(); jvm.stop()"
2. virtual environment for Python 3.6 (virtualenv):
virtualenv -p /usr/bin/python3.6 pww36
pww36/bin/pip install numpy
pww36/bin/pip install javabridge
pww36/bin/pip install python-weka-wrapper3
pww36/bin/python -c "import weka.core.jvm as jvm; jvm.start(); jvm.stop()"
3. virtual environment for Python 2.7 (anaconda3-2019.07):
conda create -n pww27 python=2.7
conda activate pww27
pip install numpy
pip install javabridge
pip install python-weka-wrapper
python -c "import weka.core.jvm as jvm; jvm.start(); jvm.stop()"
conda deactivate
4. virtual environment for Python 3.6 (anaconda3-2019.07):
conda create -n pww36 python=3.6
conda activate pww36
pip install numpy
pip install javabridge
pip install python-weka-wrapper
python -c "import weka.core.jvm as jvm; jvm.start(); jvm.stop()"
conda deactivate

unable to upgrade pip version 10.0.1 to 18.0 in cmd

I'm trying to upgrade pip but the below command didn't worked out. Currently, I have pip version 10.0.1 and I want to upgrade it to 18.0
command I tried in cmd:
python -m pip install --upgrade pip
It's showing:
Attribute Error:module 're' has no attribute 'findall'
There might be something wrong with your pip instalation
try this:
curl bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py
My problem is with my version of pip 9.0, upgrading always fails (I run a MacBook with OSX 10.7.5) showing similar message like this example , If any one had an idea...:
$ python -m pip install --upgrade pip
Requirement already up-to-date: pip in /Library/Python/2.7/site-packages
But every time I use pip install, is this:
$ pip install lxml
Collecting lxml
Could not fetch URL https://pypi.python.org/simple/lxml/: There was a problem confirming the ssl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) - skipping
Could not find a version that satisfies the requirement lxml (from versions: )
No matching distribution found for lxml
Make sure you have exited python and you are back in main Anaconda terminal. Press
exit
to leave python and then run this code
python -m pip install --upgrade pip
or run the commands below;
conda config --add channels conda-forge
conda update pip
If you are in windows and using virtual env then use
(virtual env) c/path>easy_install -U pip
this will update pip inside the virtual env and if you do it outside of virtual env then it should also work there as well.
this will update pip 18.0 in virtual env and if you do it outside of virtual env then it should also work there as well.
python -m pip install --upgrade pip 18.0

How to install pip for Python 3.6 on Ubuntu 16.10?

I'd like to start by pointing out that this question may seem like a duplicate, but it isn't. All the questions I saw here were regarding pip for Python 3 and I'm talking about Python 3.6. The steps used back then don't work for Python 3.6.
I got a clear Ubuntu 16.10 image from the official docker store.
Run apt-get update
Run apt-get install python3.6
Run apt-get install python3-pip
Run pip3 install requests bs4
Run python3.6 script.py
Got ModuleNotFoundError below:
Traceback (most recent call last):
File "script.py", line 6, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
Python's and pip's I have in the machine:
python3
python3.5
python3.5m
python3.6
python3m
python3-config
python3.5-config
python3.5m-config
python3.6m
python3m-config
pip
pip3
pip3.5
Let's suppose that you have a system running Ubuntu 16.04, 16.10, or 17.04, and you want Python 3.6 to be the default Python.
If you're using Ubuntu 16.04 LTS, you'll need to use a PPA:
sudo add-apt-repository ppa:jonathonf/python-3.6 # (only for 16.04 LTS)
Then, run the following (this works out-of-the-box on 16.10 and 17.04):
sudo apt update
sudo apt install python3.6
sudo apt install python3.6-dev
sudo apt install python3.6-venv
wget https://bootstrap.pypa.io/get-pip.py
sudo python3.6 get-pip.py
sudo ln -s /usr/bin/python3.6 /usr/local/bin/python3
sudo ln -s /usr/local/bin/pip /usr/local/bin/pip3
# Do this only if you want python3 to be the default Python
# instead of python2 (may be dangerous, esp. before 2020):
# sudo ln -s /usr/bin/python3.6 /usr/local/bin/python
When you have completed all of the above, each of the following shell commands should indicate Python 3.6.1 (or a more recent version of Python 3.6):
python --version # (this will reflect your choice, see above)
python3 --version
$(head -1 `which pip` | tail -c +3) --version
$(head -1 `which pip3` | tail -c +3) --version
In at least in ubuntu 16.10, the default python3 is python3.5. As such, all of the python3-X packages will be installed for python3.5 and not for python3.6.
You can verify this by checking the shebang of pip3:
$ head -n1 $(which pip3)
#!/usr/bin/python3
Fortunately, the pip installed by the python3-pip package is installed into the "shared" /usr/lib/python3/dist-packages such that python3.6 can also take advantage of it.
You can install packages for python3.6 by doing:
python3.6 -m pip install ...
For example:
$ python3.6 -m pip install requests
$ python3.6 -c 'import requests; print(requests.__file__)'
/usr/local/lib/python3.6/dist-packages/requests/__init__.py
This answer assumes that you have python3.6 installed. For python3.7, replace 3.6 with 3.7. For python3.8, replace 3.6 with 3.8, but it may also first require the python3.8-distutils package.
Installation with sudo
With regard to installing pip, using curl (instead of wget) avoids writing the file to disk.
curl https://bootstrap.pypa.io/get-pip.py | sudo -H python3.6
The -H flag is evidently necessary with sudo in order to prevent errors such as the following when installing pip for an updated python interpreter:
The directory '/home/someuser/.cache/pip/http' or its parent directory
is not owned by the current user and the cache has been disabled.
Please check the permissions and owner of that directory. If executing
pip with sudo, you may want sudo's -H flag.
The directory
'/home/someuser/.cache/pip' or its parent directory is not owned by the
current user and caching wheels has been disabled. check the
permissions and owner of that directory. If executing pip with sudo,
you may want sudo's -H flag.
Installation without sudo
curl https://bootstrap.pypa.io/get-pip.py | python3.6 - --user
This may sometimes give a warning such as:
WARNING: The script wheel is installed in '/home/ubuntu/.local/bin'
which is not on PATH. Consider adding this directory to PATH or, if
you prefer to suppress this warning, use --no-warn-script-location.
Verification
After this, pip, pip3, and pip3.6 can all be expected to point to the same target:
$ (pip -V && pip3 -V && pip3.6 -V) | uniq
pip 18.0 from /usr/local/lib/python3.6/dist-packages (python 3.6)
Of course you can alternatively use python3.6 -m pip as well.
$ python3.6 -m pip -V
pip 18.0 from /usr/local/lib/python3.6/dist-packages (python 3.6)
This website contains a much cleaner solution, it leaves pip intact as-well and one can easily switch between 3.5 and 3.6 and then whenever 3.7 is released.
http://ubuntuhandbook.org/index.php/2017/07/install-python-3-6-1-in-ubuntu-16-04-lts/
A short summary:
sudo apt-get install python python-pip python3 python3-pip
sudo add-apt-repository ppa:jonathonf/python-3.6
sudo apt-get update
sudo apt-get install python3.6
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.5 1
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 2
Then
$ pip -V
pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)
$ pip3 -V
pip 8.1.1 from /usr/local/lib/python3.5/dist-packages (python 3.5)
Then to select python 3.6 run
sudo update-alternatives --config python3
and select '2'. Then
$ pip3 -V
pip 8.1.1 from /usr/local/lib/python3.6/dist-packages (python 3.6)
To update pip select the desired version and
pip3 install --upgrade pip
$ pip3 -V
pip 9.0.1 from /usr/local/lib/python3.6/dist-packages (python 3.6)
Tested on Ubuntu 16.04.
Some of the solutions above using the script get-pip.py worked until a couple of weeks ago.
The latest version of this script now requires python3.7 throwing the following error
ERROR: This script does not work on Python 3.6
The minimun supported Python version is 3.7.
Please use https://bootstrap.pypa.io/pip/3.6/get-pip.py instead.
So making the corresponding change works now.
wget https://bootstrap.pypa.io/pip/3.6/get-pip.py
sudo python3.6 get-pip.py

Resources