I am working on developing a python package. I use pip freeze > requirements.txt to add the required package into the requirement.txt file. However, I realized that some of the packages, instead of the package version, have some path in front of them.
numpy==1.19.0
packaging==20.4
pandas # file:///opt/concourse/worker/volumes/live/38d1301c-8fa9-4d2f-662e-34dddf33b183/volume/pandas_1592841668171/work
pandocfilters==1.4.2
Whereas, inside the environment, I get:
>>> pandas.__version__
'1.0.5'
Do you have any idea how to address this problem?
It looks like this is an open issue with pip freeze in version 20.1, the current workaround is to use:
pip list --format=freeze > requirements.txt
In a nutshell, this is caused by changing the behavior of pip freeze to include direct references for distributions installed from direct URL references.
You can read more about the issue on GitHub:
pip freeze does not show version for in-place installs
Output of "pip freeze" and "pip list --format=freeze" differ for packages installed via Direct URLs
Better freeze of distributions installed from direct URL references
Related
I have a 2-part question about conda vs. pip virtual environments. I found great information on the answers What is the difference between pip and conda? and Does Conda replace the need for virtualenv? but still have something unclear.
I have a given python project (say PR) that I need to install and further develop on a linux server (say S) where python is installed with anaconda. Now, the usage/installation instructions of PR tell me to use python to create virtual environment and pip to install all packages. That is,
python3 -m venv PR
pip install --editable . (the dot included at the end)
According to "pip install --editable ./" vs "python setup.py develop" the latter reads the file setup.py (included in PR) which contains a function setup(...) with option install_requires listing all the required packages and installs them automatically. I have tested this on my own computer (which does not have conda) and it works fine. At least no error messages.
Now I need to further develop PR on S. My question Part 1: can I use conda instead of pip to create and update virtual environment? If yes, what would be the conda command replacing pip install --editable . ? I'm positive I will later need to install other packages as well. I'm worried about conflicts between conda/pip.
On S, I have Spyder and no other python IDEs. I have never used Spyder but I'm very familiar with PyCharm (Windows) and VS Code (Linux) so I assume debugging with Spyder will be similar to those. My question Part 2 (tied to Part 1): if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
(Edit/update): Thank you Carlos for comments. I continue my question:
I created and activated the virtual environment (VE) with conda
conda create PR_venv
conda activate PR_venv
Installed pip with
conda install pip
(this upgraded pip and installed several other packages too, including newer version of python). Installed PR and its required packages with pip
pip install -e .
Now, if I run the PR package inside this active VE interactively from the terminal, everything works fine. I would like to do the same from within spyder, to get the IDE debugging abilities in my hand.
When I start spyder, open a python file to be run, click "Run" button, it crashes in the import statements.
Spyder cannot see the installed packages. It can see only the local package PR but none of the packages installed by pip for this VE.
I am not sure what is the correct question here; I'm confused how are conda VEs related to spyder/jupyter/ipython ? I cannot find information in the conda documents about this.
I cannot find from spyder documents anything about VEs. Do I have to somehow re-install the packages (how?) inside Spyder? It seems pointless because the packages are installed already.
(Edit/Update 2): The information on https://docs.spyder-ide.org/current/installation.html makes me even more confused: Spyder is presented as both a stand-alone program and as a python package. So do I have to re-install Spyder inside the VE(?!) with
conda activate PR_venv
conda install spyder
Any clarification would be appreciated. I have always thought that the IDEs are stand-alone programs and that's it. This Spyder setup twists my brains into pretzel.
(Spyder maintainer here) About your questions:
can I use conda instead of pip to create and update virtual environment?
Yes, you can. Please see here to learn about the functionality offered by conda for managing environments.
If yes, what would be the conda command replacing pip install --editable . ?
Conda doesn't offer a good replacement for that command. However, you can still use it in a conda environment, as long as all you've installed all your package dependencies with conda before running it. That would avoid mixing conda and pip packages, which usually leads to really bad results.
if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
Spyder can work with pip and conda packages without problems. Just make sure of not mixing them (as I said above) and you'll be fine. In addition, please read our documentation to learn how to connect a local Spyder instance to a remote server.
Part 1: yes I can use conda to create VE and pip to install packages
conda create PR_venv
conda activate PR_venv
conda install pip
pip install --editable .
conda list
The last line shows which packages are installed by conda and which by pip (shown as pypi)
Part 2: spyder by default cannot see the packages. Need to do two things:
conda install spyder-kernels
Open Spyder and Tools > Preferences > Python Interpreter > Use the following interpreter > [full path to VE python command]
Restart Spyder. Now it can see the packages.
(Edit:) this link is great: https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder
I recently uninstalled all pip3 the pip3 packages installed in my local system.
Now, whenever I type pip3 freeze, this warning pops up:
WARNING: Could not generate requirement for distribution -ertifi 2019.11.28
(/usr/local/lib/python3.7/site-packages): Parse error at "'-ertifi='": Expected W:(abcd...)
How do I get rid of this warning.
I tried doing brew reinstall python3 as this would reinstall python3 along with pip3 but it doesn't seem to work.
Please help me figure this out.
I got into similar problem after I have upgraded my pip under windows from version 20.1.1 to version 20.2.3
It turned out that old version of pip was renaming folders that it could not delete, replacing first letter with tilda ~
The new version of pip counts this tilda as hyphen - I don't know why
So I was getting that same error but for a different package:
WARNING: Could not generate requirement for distribution -niso8601 8.0.0 (...) venv/lib/python3.6/site-packages): Parse error at "'-niso860'": Expected W:(abcd...)
It turned out to be a missing letter in my first package! I needed to have "aniso8601==7.0.0" I literally did a freeze from a different venv that had that same package I needed, copied and pasted into the requirements.txt in the new project's folder. The requirements.txt file in the new project then installed successfully.
TL;DR: My requirements.txt file had a typo.
I have just started using python and I have set up python3 to be installed in C:\Python37 dir. I have added python3 path to environment variables. When I run python3 -m pip install [package_name] it installs it in C:\Python37\Lib\site-packages. But when I try to install pylint with python3 -m pip install pylint it prints:
Requirement already satisfied: pylint in c:\users\radio\appdata\roaming\python\python37\site-packages (2.3.1)
It's installing it in above mentioned completely different location, and then VS Code complains how pylint is not installed. Why doesn't pip install it in C:\Python37\Lib\site-packages where it installs all the other packages?
It seems you might have multiple python installations on the computer or you are not running CMD with administrator priviledges when using pip.
I would refer you to this thread: windows pip installing libraries in wrong directory which seems to deal with a problem similar to the one you are having.
I would also try checking the environment variables to see the path set for Python if that doesn't work.
I have multiple instances of python in my computer namely python27, python3.6 and python 3.7.
The import module for docx worked daily on python27 until suddenly it stopped working today. I tried installing the module again using pip in windows command line.
It says only installed in python27 directory. But there's an
importError on my script.
I guess I should be high time i transferred into python37 but I can't seem to make pip install into python 3.7
Can someone offer some advice as to the messy situation I'm in?I want to transfer to python3 and install docx in python3
Already checked modules using pip and docx is not there.
You should try setting your command prompt on the version of Python you want the module to install. Then, just use "pip", to install.
This is how you can do this:
Go into File Explorer (hopefully you have windows) and press and hold the shift key, then right click the folder that you want command prompt to look at. In this case, the folder with the version of Python. Then you click on the command prompt selection, and you're golden. Just use "pip install xyz" from there on. Glad to help
- BURAK ILOGLU
for python 3 try below given approaches.
sudo apt-get install python3-pip
this should get you pip3 and you can use pip3 to install packages specific to python3
and also
python3 -m pip install <package name>
I'd like to make a wheel binary distribution, intstall it and then import it in python. My steps are
I first create the wheel: python ./my_package/setup.py bdist_wheel
I install the wheel: pip install ./dist/*.whl
I try to import the package: python -c"import my_package"
This leads to the error:
ImportError: No module named 'my_package'
Also, when I do pip list, the my_package is listed.
However, when I run which my_packge, nothing is shown.
When I run pip install ./my_package/ everything works as expected.
How would I correctly build and install a wheel?
python version 3.5
pip version 10.1
wheel version 0.31.1
UPDATE:
When I look at the files inside my_package-1.0.0.dist-info, there is an unexpected entry in top_level.txt. It is the name of the folder where I ran
python ./my_package/setup.py bdist_wheel in. I believe my setup.py is broken.
UPDATE WITH REGARDS TO ACCEPTED ANSWER:
I accepted the answer below. Yet, I think it is better to simply cd into the package directory. Changing to a different directory as suggested below leads to unexpected behavior when using the -d flag, i.e. the target directory where to save the wheel. This would be relative to the directory specified in the setup.py file.
I had the very same error, but it was due to my setup.py not specifying the entry "packages=setuptools.find_packages()".
Everythings builds nicely without that but you can't import anything even though pip shows it to be installed.
If you need to execute the setup script from another directory, ensure you are entering the project dir in the script.
from setuptools import setup
root = os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir))
os.chdir(root)
# or using pathlib (Python>=3.4):
import patlib
root = pathlib.Path(__file__).parent
os.chdir(str(root))
setup(...)
In my case, in order to solve it I just had to upgrade pip (since Docker installed pip 9).
python3 -m pip install --upgrade pip
I have experienced the same situation, maybe not for the same reason, here just for reference.
The package name should not contain the dash "-", there's no error pop out, but after installing your wheel, though it is shown in pip list, you can't find that package.
/src/your-package-name # should not
/src/your_package_name # should like this
In the setup.py, you can use the name with dash "-" without limitation:
setuptools.setup(
name="instrument-lab",
...