I installed a new virtual env with python3.6. After checking its site packages (python3.6 -m site --user-site) I was pointed to /Users/username/.local/lib/python3.6/site-packages.
I added a file named custom.pth with the contents:
/Users/username/Python Files/Packages
but for some reason, it still fails to recognize our inhouse packages. However, I have the exact same .pth file for the system's python3.6 and it works without a hitch. Is there something else I can try?
It finally worked when I placed the .pth file into the <venv folder>/lib/python3.6/site-packages/ directory. Perhaps it's an issue with the activated venv overriding the python's site-packages
Python virtual environments are intended to isolate packages from both global and user site-packages. User site-packages are not available to Python in a virtual environment. Compare python -c "import sys; print(sys.path)" before and after activating a venv. Hence their .pth files are not processed.
Outside of a venv .pth files from the user site-packages are processed.
Related
I need to provide my version of Python and packages for a project.
How can I do that?
I tried:
sudo apt-get install python3-venv
virtualenv my-env -p python3
source tutorial-env/bin/activate
This should show installed packages, but it shows:
pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
pip (9.0.1)
pkg-resources (0.0.0)
setuptools (39.0.1)
This is also suspicious:
(tutorial-env) linux#LINUXMINT:~$ pip freeze > requirements.txt
(tutorial-env) linux#LINUXMINT:~$ cat requirements.txt
pkg-resources==0.0.0
(tutorial-env) linux#LINUXMINT:~$ python -m pip install -r requirements.txt
Requirement already satisfied: pkg-resources==0.0.0 in ./tutorial-env/lib/python3.6/site-packages (from -r requirements.txt (line 1))
And I cannot find requirements.txt in my directory.
Pass the version of python when creating your virtualenv like this:
virtualenv my-env -p python3
I think you should not use any external module for creating a virtual environment.
You can create a Virtual Environment by using venv attribute of Python in Command Line.
What is the syntax?
The syntax is pretty simple.
C:\>python -m venv path\to\where\you\want\to\create\it
It can be easily done, and you can get a reference from the example below:
C:\>python -m venv "C:\Users\Bhavyadeep\Desktop\Discord Bots\Bot" 1\Bot-1-env
Here the name Bot-1-env is the name of the folder which will be created on execution of the command, and it doesn't have to exist.
What if I am using an IDE (like VS Code), then how will I create a virtual environment?
Creating a Virtual Environment in an IDE is much easier than creating it using CMD. In CMD you need to specify the full path of the directory where the Environment has to be created whereas in an IDE you can create one using its own terminal and also there won't be any need of adding and full path to the directory.
Syntax for IDEs with their Terminals is:
C:\>python -m venv My-Env
This would simply create a Virtual Environment in the folder of the project you are working on in the IDE. If you still want to create it using full path you can do the same as above in the Terminal of IDE.
Example with Images and Code in One Step:
My target directory would be the Desktop for now to explain.
Write and Execute the command in Command Line.
I entered the line in the image and pressed Enter.
Here the name of the Folder would be Example-Venv and it doesn't exist. This command created a folder with that name and created that a Virtual Environment.
This command created a folder, and it can be seen in the picture below.
Now you can use it anywhere you want by simply making this folder as the Interpreter.
How to set interpreter?
The following links would explain:
Pycharm: https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html
VS Code: https://code.visualstudio.com/docs/python/environments#:~:text=To%20select%20a%20specific%20environment,or%20library%20versions%20as%20needed.
Spyder: https://www.spyder-ide.org/blog/release-spyder-330/#:~:text=Just%20set%20the%20path%20under,start%20in%20the%20selected%20environment.
Sublime: https://medium.com/#hariyanto.tan95/set-up-sublime-text-3-to-use-python-3-c845b742c720
I was glad to help! If you still get any problem, please feel free to ask in the comments and I would gladly help you! :)
Thank You! :)
I realize this topic has so many other SO answers but I haven't been able to find one for my specific problem.
I am using python 3.7 on windows and create my virtual environments using venv. When I activate my virtual environment and call pip list --local, I get a list of all my system-site-packages rather than the empty list I would expect for a new environment (venv is supposed to default to no site-packages https://docs.python.org/3/library/venv.html).
However, if I look in the virtual environments site-packages folder, it is empty as expected.
This is an issue, because pip won't install some packages into the virtual environment as it thinks they're already there (it sees the system-packages).
I think this has something to do with my path variable, possibly that I'm using the system version of pip rather than the one in my virtual environment. But I don't know how to set up the path variable to fix it.
Currently, my path includes both my python directory in Users\Bob\AppData\Local\Programs\Python\Python37-32 and also the \Scripts folder within that directory.
Here's the result of trying to install numpy in a new virtual environment:
(Venv) C:\Users\Bob\Desktop\Python_Projects\ML\>pip install numpy
Requirement already satisfied: numpy in c:\users\Bob\appdata\local\programs\python\python37-32\lib\site-packages (1.16.2+mkl)
I use Pycharm to create and manage my virtualenvs in my projects.
The problem is that after adding a library with pycharm, when I type the command (pip3 freeze --user), the library does not appear in the command result.
I have to manually type the pip install command each time so that the library is visible.
What manipulation should I do in PyCharm to solve this problem?
For what you are saying, the first thing that comes to mind is that you should use:
pip freeze
And not
pip3 freeze
Because the command mapped to the pip version when you have virtualenv activated is the first. Note that for installing you seem to use pip, and not pip3
Moreover, the --user option afaik is related to the packages installed in the user folder:
--user Install to the Python user install directory for your platform. Typically
~/.local/, or %APPDATA%\Python on
Windows. (See the Python documentation for site.USER_BASE for full details.)
If your packages are installed in the virtualenv folder, I would tell you to not use that option.
Also please make sure you have your virtualenv activated. In linux you can do so by source path/to/virtualenv/activate
Edit
I understand that the reason you are using pip3 is because you may have different versions of Python in your machine. Let me explain you a bit further how it works, because version management is usually a headache for many programmers and it is common to find problems when doing so.
If you install different versions of Python in your linux machine, and you do that as root, then the installation will proceed for the whole system. Usually Python2 installation folder for Linux machines is /usr/bin/python. However, I am uncertain of which directory is used for Python3 installations. You can check that easily by doing whereis python3. You can serach the path to binary of any command by doing whereis command. Note that this works also for whereis python as far as you don't have virtualenv activated.
Aditionally, the link to the binary of a command (or the set of instructions to be exectued, more broadly) is defined in certain folders in Linux, depending on whether you created the command as root or as a user, and possibly also on the distro. This works differently in Windows, that uses the Registry Edit utility to handle command mappings. When you enable your virtualenv, what you are doing is creating an environment that enables mapping system commands such as python to the Python installation in your virtualenv folder.
When you disable the virtualenv, the command points again to the default installation path. Same happens with pip, so incorrect usage of this tool may result in different packages being installed in different locations, and therefore not appearing available for the right Python version at any given circumstance.
In Linux, environment variables are shell dependent, though you can write them out with echo $variable and set them with variable=value (from bash). The search path is simply called PATH and you can get yours by typing echo $PATH.
Source: https://askubuntu.com/a/262073/426469
I encourage you to check other questions in SE network such as this: https://unix.stackexchange.com/a/42211/96121, to learn more about this.
Addendum
Quick tip: it is common to use the pip freeze command as follows:
pip freeze > requirements.txt
It is a standard that leads to understanding that modules in such file are required for the correct functioning of your application. That lets you easily exclude the virtualenv folder when you install the program in another computer, since you can readily know the requriments for a fresh installation. However, you can use the command as you want.
In Ubuntu 16.04, when I do python3.6 -m venv myenvironment, venv does not install the python 3.6 binaries in ../myenvironment/bin but only symlinks to the system binaries. The official docs for 3.6 suggests otherwise:
28.3.1 . . . (venv) also creates a bin subdirectory containing a copy of the python binary.
Unless by "copy" they mean a symlink, venv is not doing what the doc says. The pip binaries are there, but not python 3.6 or any other version binaries.
Maybe it doesn't matter since within that environment 3.6 will be the version used. But then which modules/packages will it use? venv did create the lib/python3.6/site-packages subdirectory as expected. Should I assume that unless I put a different version of a module/package there that the system-wide library will be used within this virtual environment?
Edit: In partial answer to my own question, it seems the default on Ubuntu 16.04 is for venv to not install the python binaries. Adding the --copies option forces venv to place the binaries into the env's ../bin directory. venv creates a subdirectory for the env's site-packages as expected. It does not, however, put the standard library modules/packages into the env's ../lib/python3.6 sub-directory.
According to the output of print(sys.path) (when run from within the env), the env's python copy still looks in the system's /usr/lib/python3.6 directory for the standard library. I guess that's okay if that's how it should work, but does one ever want to use a particular version of a module or package in the standard library, or is that just not ever done and only none-standard-library modules/packages are placed in the env's site-packages subdirectory?
2nd Edit: Here's an SO Q&A on the venv's behavior regarding standard library modules and packages:
Where is the standard library in python virtual environment?
The gist of the answers is that venv does not create a copy of the standard library in the env's directories. The system-wide standard library is always used. At least one of the participants in that Q&A commented it was odd that venv behaves that way. Seems odd to me as well. Does anyone know why?
Python 3.6 has been installed (using the Cygwin setup .exe file, update, etc.). The executable is located in /bin/ ... or is it located in /usr/bin/? Cygwin ls command shows that /usr/bin exists... but on Windows this directory is non-existent. Also the contents of both directories are identical, including if I change a filename... but I haven't as yet found a symlink (in /usr or in / ) to explain this!
I'm struggling to get virtualenvwrapper installed (this is part of a preparation required to follow along with a book, TDD With Python).
I just overcame a first hurdle (eventually) by realising I had to install virtualenvwrapper using pip3, not pip! ... I feel like I'm in at the deep end.
So I did:
pip3 install virtualenvwrapper
echo "source virtualenvwrapper.sh" >> ~/.bashrc
source ~/.bashrc
... then I did
mkvirtualenv --python3=`py -3.6 -c"import sys; print(sys.executable)"` superlists
(NB "python3" is the correct name of the symlink which points to the Python3 executable in /bin/; there is a "python" symlink but that points to Python2.7)
And I got:
Requested Python version (3.6) not installedUsing base prefix '/usr'
New python executable in
/home/Chris/.virtualenvs/superlists/bin/python3Also creating
executable in /home/Chris/.virtualenvs/superlists/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating
/home/Chris/.virtualenvs/superlists/bin/predeactivate
virtualenvwrapper.user_scripts creating
/home/Chris/.virtualenvs/superlists/bin/postdeactivate
virtualenvwrapper.user_scripts creating
/home/Chris/.virtualenvs/superlists/bin/preactivate
virtualenvwrapper.user_scripts creating
/home/Chris/.virtualenvs/superlists/bin/postactivate
virtualenvwrapper.user_scripts creating
/home/Chris/.virtualenvs/superlists/bin/get_env_details (superlists)
Anyone know what's going on? How do I get the system to recognise that Python3.6 is actually installed?
Later Or... am I being very dense? Maybe making a virtual environment using this module always involves installing a new Python executable?
Later still I'm still not clear about this... but it isn't stopping me from using virtualenv and virtualenvwrapper and generally getting on with the book. Despite complaining Python doesn't exist the setup appears (as far as I can tell!) to be using the symlinks under the directories in directory .virtualenv/ to one of the Python symlinks in /bin/ ...
About first question
/usr/bin and /usr/lib are by default also automatic mount points
generated by the Cygwin DLL similar to the way the root directory is
evaluated. /usr/bin points to the directory the Cygwin DLL is
installed in, /usr/lib is supposed to point to the /lib directory.
https://cygwin.com/cygwin-ug-net/using.html#mount-table
For the second, to check if phyton3 is installed
$ cygcheck -c python3
and as mentioned by phd the py command is not a cygwin one, so probably you are mixing something.