pip install in Cygwin cannot find file error - python-3.x

I am on a Mac OS El Capitan, running a Windows 10 64-bit VM inside Parallels. I have Cygwin installed and Anaconda3. I would like to install two packages (pyrsistent and rpy2) using pip. Both of them throw an error "error: [WinError 2] The system cannot find the file specified" without specifying the file it can't find.
Here's the output:
$ pip install pyrsistent
Collecting pyrsistent
Using cached pyrsistent-0.11.9.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in c:\anaconda3\lib\site-packages (from pyrsistent)
Building wheels for collected packages: pyrsistent
Running setup.py bdist_wheel for pyrsistent
Complete output from command C:\Anaconda3\python.exe -c "import setuptools;__file__='C:\\cygwin64\\tmp\\pip-build-sqcinj9m\\pyrsistent\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d C:\cygwin64\tmp\tmpn25raothpip-wheel-:
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.5
copying _pyrsistent_version.py -> build\lib.win-amd64-3.5
creating build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_checked_types.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_field_common.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_helpers.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_immutable.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pbag.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pclass.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pdeque.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_plist.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pmap.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_precord.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pset.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pvector.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_transformations.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\__init__.py -> build\lib.win-amd64-3.5\pyrsistent
running build_ext
building 'pvectorc' extension
error: [WinError 2] The system cannot find the file specified
----------------------------------------
Failed building wheel for pyrsistent
Failed to build pyrsistent
Installing collected packages: pyrsistent
Running setup.py install for pyrsistent
Complete output from command C:\Anaconda3\python.exe -c "import setuptools, tokenize;__file__='C:\\cygwin64\\tmp\\pip-build-sqcinj9m\\pyrsistent\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\cygwin64\tmp\pip-_wbdiief-record\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
running build_ext
building 'pvectorc' extension
error: [WinError 2] The system cannot find the file specified
----------------------------------------
Command "C:\Anaconda3\python.exe -c "import setuptools, tokenize;__file__='C:\\cygwin64\\tmp\\pip-build-sqcinj9m\\pyrsistent\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\cygwin64\tmp\pip-_wbdiief-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\cygwin64\tmp\pip-build-sqcinj9m\pyrsistent
Both python3 and pip are in the path:
$ which pip
/cygdrive/c/Anaconda3/Scripts/pip
$ which python
/cygdrive/c/Anaconda3/python
The error is identical when trying to install rpy2 so it's not something particular to the package I'm trying to install. Does anyone have ideas of the problem or ideas for troubleshooting? I've tried debugging into the install.py but the code throwing the error is in C. I've tried doing the install in verbose mode, but it still doesn't say what file it cannot find in that case.
(and if you're wondering why I haven't used conda, it's because it doesn't install pyrsistent, and for the rpy2 install, it insists on linking it to its own install of R, and even when specifying my own install, something doesn't link correctly. I've already been down that road and would like to stick to pip).

Here is a workaround for anyone who is stuck installing python packages on Windows. There are several sites that have Windows binaries for python packages. I was able to find a binary for rpy2 here but there are several other sites too, if you just google something like "python Windows binaries". Then I did:
pip install rpy2-2.7.4-cp35-none-win_amd64.whl
pyrsistent was not in that website or any of several other binaries websites I found. But someone else on my dev team sent me the /pyrsistent directory inside his Anaconda3/Lib/site-packages directory and after copying it to the same directory on my machine, I was able to import pyrsistent as well.
It's not pretty, but it worked.

Related

autoPyTorch installation error in windows

Doing
pip install autoPyTorch
from https://pypi.org/project/autoPyTorch/
in windows
Requirement already satisfied: typing-extensions in c:\users\ga\anaconda3\envs\automlpy395\lib\site-packages (from torch->autoPyTorch) (3.7.4.3)
Building wheels for collected packages: ConfigSpace, netifaces
Building wheel for ConfigSpace (PEP 517) ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\GA\anaconda3\envs\autoMLpy395\python.exe' 'C:\Users\GA\anaconda3\envs\autoMLpy395\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' build_wheel 'C:\Users\G~1\AppData\Local\Temp\tmpiv0eeai5'
cwd: C:\Users\GA\AppData\Local\Temp\pip-install-78zfdmqt\configspace_985d1975f54b489c8878a152ed6c7358
Complete output (40 lines):
C:\Users\GA\AppData\Local\Temp\pip-build-env-c1p7qiob\overlay\Lib\site-packages\setuptools\dist.py:717: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
warnings.warn(
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
creating build\lib.win-amd64-3.9\ConfigSpace
copying ConfigSpace\exceptions.py -> build\lib.win-amd64-3.9\ConfigSpace
copying ConfigSpace\__init__.py -> build\lib.win-amd64-3.9\ConfigSpace
copying ConfigSpace\__version__.py -> build\lib.win-amd64-3.9\ConfigSpace
creating build\lib.win-amd64-3.9\ConfigSpace\nx
copying ConfigSpace\nx\exception.py -> build\lib.win-amd64-3.9\ConfigSpace\nx
copying ConfigSpace\nx\release.py -> build\lib.win-amd64-3.9\ConfigSpace\nx
copying ConfigSpace\nx\__init__.py -> build\lib.win-amd64-3.9\ConfigSpace\nx
creating build\lib.win-amd64-3.9\ConfigSpace\read_and_write
copying ConfigSpace\read_and_write\json.py -> build\lib.win-amd64-3.9\ConfigSpace\read_and_write
copying ConfigSpace\read_and_write\pcs.py -> build\lib.win-amd64-3.9\ConfigSpace\read_and_write
copying ConfigSpace\read_and_write\pcs_new.py -> build\lib.win-amd64-3.9\ConfigSpace\read_and_write
copying ConfigSpace\read_and_write\__init__.py -> build\lib.win-amd64-3.9\ConfigSpace\read_and_write
creating build\lib.win-amd64-3.9\ConfigSpace\nx\algorithms
copying ConfigSpace\nx\algorithms\cycles.py -> build\lib.win-amd64-3.9\ConfigSpace\nx\algorithms
copying ConfigSpace\nx\algorithms\dag.py -> build\lib.win-amd64-3.9\ConfigSpace\nx\algorithms
copying ConfigSpace\nx\algorithms\__init__.py -> build\lib.win-amd64-3.9\ConfigSpace\nx\algorithms
creating build\lib.win-amd64-3.9\ConfigSpace\nx\classes
copying ConfigSpace\nx\classes\digraph.py -> build\lib.win-amd64-3.9\ConfigSpace\nx\classes
copying ConfigSpace\nx\classes\graph.py -> build\lib.win-amd64-3.9\ConfigSpace\nx\classes
copying ConfigSpace\nx\classes\__init__.py -> build\lib.win-amd64-3.9\ConfigSpace\nx\classes
creating build\lib.win-amd64-3.9\ConfigSpace\nx\algorithms\components
copying ConfigSpace\nx\algorithms\components\strongly_connected.py -> build\lib.win-amd64-3.9\ConfigSpace\nx\algorithms\components
copying ConfigSpace\nx\algorithms\components\__init__.py -> build\lib.win-amd64-3.9\ConfigSpace\nx\algorithms\components
running build_ext
cythoning ConfigSpace/hyperparameters.pyx to ConfigSpace\hyperparameters.c
cythoning ConfigSpace/forbidden.pyx to ConfigSpace\forbidden.c
cythoning ConfigSpace/conditions.pyx to ConfigSpace\conditions.c
cythoning ConfigSpace/c_util.pyx to ConfigSpace\c_util.c
cythoning ConfigSpace/util.pyx to ConfigSpace\util.c
cythoning ConfigSpace/configuration_space.pyx to ConfigSpace\configuration_space.c
building 'ConfigSpace.hyperparameters' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
----------------------------------------
ERROR: Failed building wheel for ConfigSpace
Building wheel for netifaces (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\GA\anaconda3\envs\autoMLpy395\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\GA\\AppData\\Local\\Temp\\pip-install-78zfdmqt\\netifaces_1ad66342d17444d8b393a67c1a88ba94\\setup.py'"'"'; __file__='"'"'C:\\Users\\GA\\AppData\\Local\\Temp\\pip-install-78zfdmqt\\netifaces_1ad66342d17444d8b393a67c1a88ba94\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\GA\AppData\Local\Temp\pip-wheel-t35_qwhh'
cwd: C:\Users\GA\AppData\Local\Temp\pip-install-78zfdmqt\netifaces_1ad66342d17444d8b393a67c1a88ba94\
Complete output (5 lines):
running bdist_wheel
running build
running build_ext
building 'netifaces' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
----------------------------------------
ERROR: Failed building wheel for netifaces
Running setup.py clean for netifaces
Failed to build ConfigSpace netifaces
ERROR: Could not build wheels for ConfigSpace which use PEP 517 and cannot be installed directly
I have tried
pip install --no-use-pep517 autoPyTorch
not worked
Tried installing torch first, installed successfully
pip3 install torch torchvision torchaudio
but autoPyTorch throwing same error
System requirements
Auto-PyTorch has the following system requirements:
Linux operating system (for example Ubuntu)
Python (>=3.6) (get Python here).
C++ compiler (with C++11 supports)
SWIG (version 3.0.* is required; >=4.0.0 is not supported)
Github AutoPytorch Installation.rst

Docker: Move LibXMLSec settings / dependencies into executable directory

So. I'm trying to get my Dockerfile to run.
Dockerfile is complicated, and doing things somewhat wrong for Python. The important part, though, is that it was working before I added XMLSec to the dependencies. (via the onelogin [https://github.com/onelogin/python-saml] package)
On our non-Dockerized systems, we run:
apt install -y libxml2-dev libxmlsec1-dev libxmlsec1-openssl pkg-config
and it installs the dependencies. My containerized attempts, however, run into this issue:
Running setup.py install for xmlsec: started
Running setup.py install for xmlsec: finished with status 'error'
Complete output from command /usr/local/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-wd6h548m/xmlsec/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ufxw9hor/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
package init file 'src/xmlsec/__init__.py' not found (or not a regular file)
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/xmlsec
copying src/xmlsec/py.typed -> build/lib.linux-x86_64-3.6/xmlsec
copying src/xmlsec/constants.pyi -> build/lib.linux-x86_64-3.6/xmlsec
copying src/xmlsec/template.pyi -> build/lib.linux-x86_64-3.6/xmlsec
copying src/xmlsec/__init__.pyi -> build/lib.linux-x86_64-3.6/xmlsec
copying src/xmlsec/tree.pyi -> build/lib.linux-x86_64-3.6/xmlsec
running build_ext
error: Unable to invoke pkg-config.
The solution seems to be to use the XMLSec library as a compiled Wheel. (Per something I read in the XMLSec github) I can't find any documentation on how to add this compiled wheel into an executable location within the container via the Dockerfile.
I've done quite a bit of research on this, and can't find sample code or the exact answer. I it may be obvious, but I'm missing core knowledge.

Failed building wheel for GoogleAppEngineCloudStorageClient

I get this error when trying to install google cloud:
Failed building wheel for GoogleAppEngineCloudStorageClient
Running setup.py clean for GoogleAppEngineCloudStorageClient
Failed to build GoogleAppEngineCloudStorageClient
Installing collected packages: GoogleAppEngineCloudStorageClient
Running setup.py install for GoogleAppEngineCloudStorageClient ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-jZCvYX/GoogleAppEngineCloudStorageClient/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-LgTihN-record/install-record.txt --single-version-externally-managed --compile --user --prefix= --home=/tmp/tmprq5a80:
running install
error: can't combine user with prefix, exec_prefix/home, or install_(plat)base
this is installed on the /lib folder, I also made a requirements.txt
as well as the appengine_config.py.
Also I am using python 2.7, on linux
use:
pip install --user --install-option="--prefix=" <package_name>
However, better to install the whole Google Cloud SDK as it includes: command line tools (gcloud, gsutil, bq) in addition to client libraries.

Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/?

trying to install fastai using pip but the following error is shown
i have installed visual studio build tools 2019 but still cant install fastai using pip.
Installing collected packages: bottleneck, nvidia-ml-py3, fastai
Running setup.py install for bottleneck ... error
ERROR: Command errored out with exit status 1:
command: 'c:\program files\python37\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\icarus\\AppData\\Local\\Temp\\pip-install-9snii33_\\bottleneck\\setup.py'"'"'; __file__='"'"'C:\\Users\\icarus\\AppData\\Local\\Temp\\pip-install-9snii33_\\bottleneck\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\icarus\AppData\Local\Temp\pip-record-bpmup82o\install-record.txt' --single-version-externally-managed --compile
cwd: C:\Users\icarus\AppData\Local\Temp\pip-install-9snii33_\bottleneck\
Complete output (36 lines):
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.7
creating build\lib.win-amd64-3.7\bottleneck
copying bottleneck\version.py -> build\lib.win-amd64-3.7\bottleneck
copying bottleneck\__init__.py -> build\lib.win-amd64-3.7\bottleneck
creating build\lib.win-amd64-3.7\bottleneck\benchmark
copying bottleneck\benchmark\autotimeit.py -> build\lib.win-amd64-3.7\bottleneck\benchmark
copying bottleneck\benchmark\bench.py -> build\lib.win-amd64-3.7\bottleneck\benchmark
copying bottleneck\benchmark\bench_detailed.py -> build\lib.win-amd64-3.7\bottleneck\benchmark
copying bottleneck\benchmark\__init__.py -> build\lib.win-amd64-3.7\bottleneck\benchmark
creating build\lib.win-amd64-3.7\bottleneck\slow
copying bottleneck\slow\move.py -> build\lib.win-amd64-3.7\bottleneck\slow
copying bottleneck\slow\nonreduce.py -> build\lib.win-amd64-3.7\bottleneck\slow
copying bottleneck\slow\nonreduce_axis.py -> build\lib.win-amd64-3.7\bottleneck\slow
copying bottleneck\slow\reduce.py -> build\lib.win-amd64-3.7\bottleneck\slow
copying bottleneck\slow\__init__.py -> build\lib.win-amd64-3.7\bottleneck\slow
creating build\lib.win-amd64-3.7\bottleneck\src
copying bottleneck\src\template.py -> build\lib.win-amd64-3.7\bottleneck\src
copying bottleneck\src\__init__.py -> build\lib.win-amd64-3.7\bottleneck\src
creating build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\input_modifcation_test.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\list_input_test.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\move_test.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\nonreduce_axis_test.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\nonreduce_test.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\reduce_test.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\scalar_input_test.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\util.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\tests\__init__.py -> build\lib.win-amd64-3.7\bottleneck\tests
copying bottleneck\LICENSE -> build\lib.win-amd64-3.7\bottleneck
running build_ext
building 'bottleneck.reduce' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\program files\python37\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\icarus\\AppData\\Local\\Temp\\pip-install-9snii33_\\bottleneck\\setup.py'"'"'; __file__='"'"'C:\\Users\\icarus\\AppData\\Local\\Temp\\pip-install-9snii33_\\bottleneck\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\icarus\AppData\Local\Temp\pip-record-bpmup82o\install-record.txt' --single-version-externally-managed --compile Check the logs for full command output.
any kind of help will be appreciated
You may not have the right path variable setup. Visual Studio build tools install scripts that you need to run in order to get the right variables in your shell.
This said, on Windows, unless you really want to compile your own version, I would recommend to install it with conda.
First install pytorch using the conda command listed here: https://pytorch.org/get-started/locally/
If you have a GPU:
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
If you use CPU:
conda install pytorch torchvision cpuonly -c pytorch
Then install fastai:
conda install -c fastai fastai
This method works fine on Windows and won't require a compiler.

Cannot find vcvarsall.bat

NOTE: I have seen multiple answers to no avail, so please look at the solution and my failed solutions list before showing as duplicate.
Hey,
So I got a new computer (Old Windows w/ Python 3.4), so I installed Python 3.5 on my new one. I started to install modules until I came across an error like this:
C:\Users\Samuel>pip install noise
Collecting noise
Using cached noise-1.2.2.tar.gz
Installing collected packages: noise
Running setup.py install for noise
Complete output from command "c:\program files\python 3.5\python.exe" -c "import setuptools, tokenize;__file__='C:\\Users\\Samuel\\AppData\\Local\\Temp\\pip-build-e9dlwpm2\\noise\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\Samuel\AppData\Local\Temp\pip-7tgvnyfe-record\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.5
creating build\lib.win-amd64-3.5\noise
copying perlin.py -> build\lib.win-amd64-3.5\noise
copying setup.py -> build\lib.win-amd64-3.5\noise
copying shader.py -> build\lib.win-amd64-3.5\noise
copying shader_noise.py -> build\lib.win-amd64-3.5\noise
copying test.py -> build\lib.win-amd64-3.5\noise
copying __init__.py -> build\lib.win-amd64-3.5\noise
running build_ext
building 'noise._simplex' extension
error: Unable to find vcvarsall.bat
----------------------------------------
Command ""c:\program files\python 3.5\python.exe" -c "import setuptools, tokenize;__file__='C:\\Users\\Samuel\\AppData\\Local\\Temp\\pip-build-e9dlwpm2\\noise\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\Samuel\AppData\Local\Temp\pip-7tgvnyfe-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\Samuel\AppData\Local\Temp\pip-build-e9dlwpm2\noise
I tried to use easy_install as a substitute, but that came with 1) an access denied error (Overcome w/ admin privelages) and 2) Not being able to find vcvarsall.bat
Because of this, I installed Visual Studio 2015 and Visual C++ 2008.
No difference.
I added the path to vcvarsall.bat to the PATH.
No difference.
I added the path to vcvarsall.bat to VS100COMNTOOLS, VS140COMNTOOLS and VS90COMNTOOLS.
No difference.
Running pip and easy_install again w/ admin permission, wearing my lucky socks.
No difference.
Installed and ran Miniconda as suggested by one of the Stack Overflow answers.
Didn't find a single package I needed. Turns out it's not looking for PyPI anyway.
I have no idea how to solve this. Do I need to revert to Python 3.4?
SPECS
Windows 10 x86 64-bit
Python 3.5
VS 2015, VC++ 2008 installed
I think you have the wrong C++: VC++ 2008 is for Python before 3.3.
The VC++ 2015 that you need comes when you install Visual Studio 2015 but you need to explicitly specify the C++ (check the box) for the C++ (2015) in the custom installation options. Try changing that via a re-installation.
Just tried it:
Windows 10 x86 64-bit
Python 3.5
VS 2015 But with the Visual C++ install option checked
Then pip install noise worked fine.

Resources