VirtualEnv and python-embed - python-3.x

I have a Windows machine running many apps using Python 2. I want to add a new program I wrote using Python 3. To make sure I don't screw anything up, I wanted to use a Virtualenv with Python 3 embedded (Python 3 embed amd64).
So I extracted the embedded python 3 and tried running virualenv with the --python option enabled.
I tried running the virtualenv
python -m virtualenv --python E:\Projects\python3-embed\python.exe E:\Projects\Virtual\
And I got this error:
Running virtualenv with interpreter E:\Projects\python3-embed\python.exe
Using base prefix 'E:\\Projects\\python3-embed'
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\virtualenv.py", line 2328, in <module>
main()
File "C:\Python27\lib\site-packages\virtualenv.py", line 713, in main
symlink=options.symlink)
File "C:\Python27\lib\site-packages\virtualenv.py", line 925, in create_environment
site_packages=site_packages, clear=clear, symlink=symlink))
File "C:\Python27\lib\site-packages\virtualenv.py", line 1147, in install_python
writefile(site_filename_dst, SITE_PY)
File "C:\Python27\lib\site-packages\virtualenv.py", line 362, in writefile
with open(dest, 'wb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'E:\\Projects\\Virtual\\python36.zip\\site.py'
I am not entirely sure how can I fix this error. There is a site.pyd file in python36.zip but it seems that virtualenv can't use it.

Sorry in advance
I knew this is not the answer you meant to ask for, but I tried hard that (our) way, and got this solution at last.
TL;DR
Update on 17 December 2021
The download page has been moved to https://winpython.github.io/
, for my purpose, I will download Winpython64-3.10.0.1dot.exe for now. Please don't forget to verify the checksum for the executable package.
End of the update on 17 December 2021
Use the portable installer with postfix "Zero" from WinPython
WinPython64-3.7.0.2Zero.exe(64 bit version, YOU want this)
WinPython32-3.7.0.2Zero.exe(32 bit version)
The Too Long version
What I tried
Yes, I downloaded the python-3.7.0-embed-amd64.zip from this page
Yes, I set it up according to instructions in this page
Yes, it worked as a python interpreter
D:\temp\test>python
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32
>>>
Yes, I installed the package virtualenv
D:\temp\test>virtualenv --version
16.0.0
Then, boom (where you asked)
D:\temp\test>virtualenv ENV
Using base prefix 'd:\\portable\\python-3.7.0-embed-amd64'
Traceback (most recent call last):
File "runpy.py", line 193, in _run_module_as_main
File "runpy.py", line 85, in _run_code
File "D:\portable\python-3.7.0-embed-amd64\Scripts\virtualenv.exe\__main__.py", line 9, in <module>
File "D:\portable\python-3.7.0-embed-amd64\lib\site-packages\virtualenv.py", line 712, in main
symlink=options.symlink)
File "D:\portable\python-3.7.0-embed-amd64\lib\site-packages\virtualenv.py", line 927, in create_environment
site_packages=site_packages, clear=clear, symlink=symlink))
File "D:\portable\python-3.7.0-embed-amd64\lib\site-packages\virtualenv.py", line 1149, in install_python
writefile(site_filename_dst, SITE_PY)
File "D:\portable\python-3.7.0-embed-amd64\lib\site-packages\virtualenv.py", line 363, in writefile
with open(dest, 'wb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\temp\\test\\ENV\\python37.zip\\site.py'
Seems a problem with virtualenv, searched, got an open issue, and I don't like that long solution.
This is the end of virtualenv.
Then, I tried venv:
D:\temp\test>python -m venv venv
D:\portable\python-3.7.0-embed-amd64\python.exe: No module named venv
Yes, the "built-in module" was lost, and the document for Embedded Distribution doesn't even mention it, just something about Tcl/tk (...), pip and the Python documentation are not included.
This is the end of venv.
Well, start again from my original point
The purpose to choose Embedded Distribution is (just like SOMEBODY):
To make sure I don't screw anything up
this include "don't install anything into my unstable and oversized Windows"
but doesn't necessarily "must use the Official Embedded Distribution"
something portable and trusted is enough, yes Portable Python
but the main page says "Portable Python is not being developed anymore", and provide some choice
after some reading, I figured out, what I want is WinPython, the version with a Zero
ta-da ... That's it.
Something about Embedded Distribution (from official document)
The embedded distribution is a ZIP file containing a minimal Python environment. It is intended for acting as part of another application, rather than being directly accessed by end-users.

The embedded distribution is meant to be lightweight. To install a virtual environment, follow these steps.
https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
Once these steps are done, proceed with the recommended approach and create a virtual environment using the command below. The --system-site-packages is typically used to bring a non embedded distribution packages already in the python install into the virtual environment, but since this is embedded, you can omit. But it won't hurt to have it.
python -m venv --system-site-packages C:\Python\python-3.7.9-embed-amd64\py37venv

Related

Installation of python, pip from administrator account not playing well with users

I am having a heck of a time installing an IDE, python and pip to play well across user accounts.
I have started a new job, where I am allowed to work on school/programming stuff at work. I've even been given the go-ahead to install everything I need on my work computer (two nice big monitors!).
I have also been given administrative login, as I can't install anything on my regular account.
So, I log in to the admin account and followed the documentation on how to install Python and Pip for all users:
Check option for install for all uses
Check Add Python 3.8 to path
Select Customize installation
Again select Install for all users
Ensure the installation path is not in a user directory but rather in C:/Programs/Python...
Install PyCharm IDE
Great. Done. I reboot my machine and login to my humble user account and clone a project from GitHub.
Ok, so I want to add a package using pip... First thing I notice is that pip install <package> doesn't work at all. I get a type error.
Next, I try py pip install <package> and that gets things going, but of course fails after downloading the package. I don't have access to install it.
So I read the solution is to use py pip install <package> --user, which proceeds to tell me that installation is complete and successful with one warning:
WARNING: The script f2py.exe is installed in 'C:\Users\WNeill\AppData\Roaming\Python\Python38\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
I find this interesting, since that's not where I installed Python. Also I find it interesting, because I had the installer add Python to the PATH. I verify my python installation is indeed in my machine's "Program Files" and satisfied that it is, I look up the f2py.script and find that it is part of the package I just tried to install.
A little wierded out, I type my import statement. Sure enough. No dice. "no module named numpy"
At this point, I'm a bit tired of running back and forth between two different user accounts repeatedly installing things and uninstalling them. I've been at it for about 6 hours. Is there a way to make python, my IDE and pip all play nicely together or should I just go back to bringing my personal laptop to work and forget my dreams of coding on one big screen with documentation up on the other?
Edit 1: I don't think this matters, but I'm no expert. I'm doing all of this from the PyCharm native terminal which is running Git Bash.
Edit 2: I decided to check my environment variables on my user account (even though they were added to the PATH variable from the admin account. The PATH variable was not set, so I added the the python38 path and he python38/Scripts. Now I am getting a whole new set of errors which, among other things in a two page error report, is telling me that I have to download Microsoft Visual C++. Is this for real?
Edit 3: I have also tried pip install --upgrade setuptools at another user's suggestion. This too fails miserably with error:
Traceback (most recent call last):
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\Scripts\pip-script.py", line 11, in <module>
load_entry_point('pip==19.0.3', 'console_scripts', 'pip')()
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\__init__.py", line 78, in main
return command.main(cmd_args)
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\cli\base_command.py", line 225, in main
session = self._build_session(
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\cli\base_command.py", line 87, in _build_session
session = PipSession(
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\download.py", line 344, in __init__
self.headers["User-Agent"] = user_agent()
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\download.py", line 134, in user_agent
setuptools_version = get_installed_version("setuptools")
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\utils\misc.py", line 902, in get_installed_version
working_set = pkg_resources.WorkingSet()
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 565, in __init__
self.add_entry(entry)
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 621, in add_entry
for dist in find_distributions(entry, True):
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1886, in find_eggs_in_zip
if metadata.has_metadata('PKG-INFO'):
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1405, in has_metadata
return self.egg_info and self._has(self._fn(self.egg_info, name))
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1760, in _has
return zip_path in self.zipinfo or zip_path in self._index()
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1637, in zipinfo
return self._zip_manifests.load(self.loader.archive)
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1594, in load
mtime = os.stat(path).st_mtime
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\Users\\WNeill\\PycharmProjects\\data-science-at-home\\venv\\lib\\site-packages\\setuptools-40.8.0-py3.8.egg'

[ Python 2.7 ]Package program with Pynsist

I am packaging a Python 2.7 program with the lastest version of Pynsist.
I've created an installer.cfg file following this example.
But when I try to package my application running
pynsist installer.cgf
into the application folder it comes up with
Copying Python installer to build directory
PyLauncher MSI already in build directory.
Copying packages into build directory...
Traceback (most recent call last):
File "/usr/local/bin/pynsist", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/nsist/__init__.py", line 540, in main
InstallerBuilder(**args).run(makensis=(not options.no_makensis))
File "/usr/local/lib/python2.7/dist-packages/nsist/__init__.py", line 495, in run
self.prepare_packages()
File "/usr/local/lib/python2.7/dist-packages/nsist/__init__.py", line 381, in prepare_packages
py_version=self.py_version, exclude=self.exclude)
File "/usr/local/lib/python2.7/dist-packages/nsist/copymodules.py", line 224, in copy_modules
mc.copy(modname, target, exclude)
File "/usr/local/lib/python2.7/dist-packages/nsist/copymodules.py", line 195, in copy
check_package_for_ext_mods(path, self.py_version)
File "/usr/local/lib/python2.7/dist-packages/nsist/copymodules.py", line 41, in check_package_for_ext_mods
check_ext_mod(os.path.join(path, dirpath, filename), target_python)
File "/usr/local/lib/python2.7/dist-packages/nsist/copymodules.py", line 30, in check_ext_mod
raise ExtensionModuleMismatch(extensionmod_errmsg % ('Windows', path))
nsist.copymodules.ExtensionModuleMismatch: Found an extension module that will not be usable on Windows:
/usr/lib/python2.7/dist-packages/pygame/rwobject.so
Put Windows packages in pynsist_pkgs/ to avoid this.
So the problem I think is with Pygame.
On Google there in nothing about this, but i cannot use others programs for packaging(eg. py2exe, pyinstaller ecc...).
Thanks and sorry for the bad english
Reposting as an answer, since it worked:
If you put pygame in packages=, it tries to copy it from your computer. But on your computer that's pygame for Linux, which won't work on Windows. If you instead put pygame in the pypi_wheels= bit of the config file, Pynsist will take care of downloading a Windows version for you.
Have a look at the pygame example in the Pynsist repository.
Most packages don't have this problem because they only contain Python code, which is the same files on all platforms. Pygame has compiled modules, which have to be compiled for the right platform.

Issues installing pywin32 behind proxy

I am trying to install pywin32 behind a proxy, and therefore I had to use the zip, rather than the .exe file available. I have down this with other packages and they work fine.
PS C:\Users\Michael.Spencer\Downloads\pywin32-221\pywin32-221> py -3 setup3.py build
Converting...
Executing...
Building pywin32 3.6.221.0
Traceback (most recent call last):
File "setup3.py", line 16
exec(str(got))
File "<string>", line 1944, in <module>
File "<string>", line 594, in __init__
File "C:\Program Files\Python36\lib\ntpath.py", line 75, in join
path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType
I have never come across this issue when downloading packages manually. The only issue I can think of is my machine is 64 bit, and maybe the zip file only contains a 32 bit version? However from what I've read, that would result in a different errortype. Any help is appreciated thanks.
An alternate simpler way to install, just re-iterating from the comments, is instead of building from source would be,
pip install pypiwin32
and to install in a different directory, for whatever reason,
pip install --target=[path] pypiwin32
from
Install a Python package into a different directory using pip?

importerror-no-module-named-pkg-resources after upgrade of python on cygwin

A while back I upgraded to python 2.7. I haven't used any of the packages I've downloaded recently so I didn't realize the error was connected until just now. I couldn't use any of the packages because I was getting the no_module_named pkg-resources error. I saw this question: What is causing ImportError: No module named pkg_resources after upgrade of Python on os X? and tried to follow the instructions but of course since I'm using cygwin I can't use curl. I used cygwin setup to try and download distribute (which I guess doesn't exist for cygwin either) and found python-setuptools. On the off chance that it would help I downloaded it and it got rid of that error. This was great because now I can use easy_install at least. However, I still get the following error with
dependencies when running many of my packages:
Traceback (most recent call last):
File "/usr/bin/ipython", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2803, in <module>
working_set.require(__requires__)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 696, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 594, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound:
Any ideas I think it might be a problem with my site-packages not being in the new python folder so I will investigate that. Either way I wanted to share my solution to the problem for cygwin users.
Update: I copied over the site-packages from 2.6 to 2.7 and that didn't help. I eventually realized because the eggs are all for 2.6 not 2.7 so python 2.7 won't accept them. This is quite annoying. I guess I have to reinstall all the packages unless anyone has an easier solution?
To address the root of this problem. I discovered I was operating on a faulty assumption that curl was incompatible with Cygwin. I found out you can setup cygwin to enable curl, problem solved!

Using virtualenv with python3 in openSuSE

I am experiencing a problem when using virtualenv in openSuSE 12.3 with Python3:
I installed the python3 and python3-devel packages.
Then I installed the newest distribute and pip and finally virtualenv using pip.
When I try to create a virtualenv I get the following error:
$ virtualenv-3.3 venv01
Using base prefix '/usr'
New python executable in venv01/bin/python3.3
Also creating executable in venv01/bin/python
Installing distribute.........................................................................................................................................................................................................................................................................................................................................................................................................done.
Installing pip....
Complete output from command /home/user/venv01/bin/python3.3 -x /home/user/venv01/bin/easy_install /usr/local/lib/pytho...ort/pip-1.3.1.tar.gz:
/home/user/venv01/bin/python3.3: can't open file '/home/user/venv01/bin/easy_install': [Errno 2] No such file or directory
----------------------------------------
...Installing pip...done.
Traceback (most recent call last):
File "/usr/local/bin/virtualenv-3.3", line 9, in <module>
load_entry_point('virtualenv==1.9.1', 'console_scripts', 'virtualenv-3.3')()
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 979, in main
no_pip=options.no_pip)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 1094, in create_environment
install_pip(py_executable, search_dirs=search_dirs, never_download=never_download)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 667, in install_pip
filter_stdout=_filter_setup)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 1057, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /home/user/venv01/bin/python3.3 -x /home/user/venv01/bin/easy_install /usr/local/lib/pytho...ort/pip-1.3.1.tar.gz failed with error code 2
Running it with -vvv yields some interesting output that shows that distribute
is extracted to <venv>/local/lib/python3.3/site-packages/.
I wonder why distribute is not installed into <venv>/lib/python3.3/site-packages/?
Does anyone have an idea why this happens and how I can use virtualenv in
openSuSE without compiling python3 myself?
[now fixed in latest patches from opensuse]
[oh! i just realised you are the same person as the original link. sorry. but i will leave this as it is a top result for google search on this issue (was searching myself for any update) so it may help others.]
this is a known issue, discussed at https://forums.opensuse.org/english/get-technical-help-here/applications/484475-using-virtualenv-python-3-a.html and with an open bug at https://bugzilla.novell.com/show_bug.cgi?id=809831
the only work-around i know of is to install everything yourself. it's not so hard, and i describe what is necessary at http://www.acooke.org/cute/GettingPyt0.html
basically:
install python 3.3 from source (do an "altinstall" to install as /usr/local/bin/python3.3)
fix the lib issue (link lib_dynload from /usr/local/lib64/python3.3 to /usr/local/lib/python3.3)
install distutils
install virtualenv
then you can use python3.3 and virtual-env-3.3 etc as expected.

Resources