Can't build Detectron2 on Windows 10 - pytorch

Although there's no official support for Detectron2 on Windows, there're many instructions available. I tried following these instruction, but ended up with the same error.
Here's my setup:
OS: Windows 10 Pro 19043.1466
Microsoft Visual Studio: 2019
CUDA: 11.3
Here's the installation algorithm I'm trying to follow (taken from https://stackoverflow.com/a/70498119/1897063)
Set up environment variables for Microsoft Visual Studio by running vcvars64.bat:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\BuildTools\VC\Auxiliary\Build\vcvars64.bat
Add new system path for cl.exe in my PATH variable:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64
Open Anaconda Promt with administrator privileges
Create new Conda environment with Python 3.7:
conda create -n detectron_env python=3.7
Activate newly created environment detectron_env:
conda activate detectron_env
Install cudatoolkit for CUDA 11.3
conda install –c anaconda cudatoolkit=11.3
Install cudnn
conda install -c anaconda cudnn
Install pywin32
conda install -c anaconda pywin32
Install pytorch, torchvision and torchaudio for CUDA 11.3
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c
pytorch
Check whether GPU is enabled for Pytorch:
python -c "import torch; print(torch.cuda.is_available())"
Install additional packages
conda install -c anaconda cython
pip install opencv-python
pip install git+https://github.com/facebookresearch/fvcore
pip install
git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI
pip install av
conda install -c anaconda scipy
conda install -c anaconda ninja
Go to the directory where I want to install detectron2
e:
cd e:/TRON
Git clone detectron2 repository
git clone https://github.com/facebookresearch/detectron2.git
detectron_repo
Install dependencies
pip install -q -e detectron_repo
And everything breaks here:
(detectron_env) e:\TRON>pip install -q -e detectron_repo
ERROR: Command errored out with exit status 1:
command: 'C:\Users\E-soft\Anaconda3\envs\detectron_env\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'E:\\TRON\\detectron_repo\\setup.py'"'"'; __file__='"'"'E:\\TRON\\detectron_repo\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
cwd: E:\TRON\detectron_repo\
Complete output (16 lines):
running develop
running egg_info
writing detectron2.egg-info\PKG-INFO
writing dependency_links to detectron2.egg-info\dependency_links.txt
writing requirements to detectron2.egg-info\requires.txt
writing top-level names to detectron2.egg-info\top_level.txt
reading manifest file 'detectron2.egg-info\SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'detectron2.egg-info\SOURCES.txt'
running build_ext
error: [WinError 2] The system cannot find the file specified
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "C:\Users\E-soft\Anaconda3\envs\detectron_env\lib\site-packages\colorama\ansitowin32.py", line 59, in closed
return stream.closed
ValueError: underlying buffer has been detached
----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\E-soft\Anaconda3\envs\detectron_env\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'E:\\TRON\\detectron_repo\\setup.py'"'"'; __file__='"'"'E:\\TRON\\detectron_repo\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
I've tried to get past the problem by changing build_ext to build_ext --inplace in setup.py (line 208):
cmdclass={"build_ext --inplace": torch.utils.cpp_extension.BuildExtension}
Here's the result of running step 14 with these changes applied:
(detectron_env) e:\TRON>pip install -q -e detectron_repo
ERROR: Command errored out with exit status 1:
command: 'C:\Users\E-soft\Anaconda3\envs\detectron_env\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'E:\\TRON\\detectron_repo\\setup.py'"'"'; __file__='"'"'E:\\TRON\\detectron_repo\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
cwd: E:\TRON\detectron_repo\
Complete output (17 lines):
running develop
running egg_info
writing detectron2.egg-info\PKG-INFO
writing dependency_links to detectron2.egg-info\dependency_links.txt
writing requirements to detectron2.egg-info\requires.txt
writing top-level names to detectron2.egg-info\top_level.txt
reading manifest file 'detectron2.egg-info\SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'detectron2.egg-info\SOURCES.txt'
running build_ext
building 'detectron2._C' extension
error: Don't know how to compile E:\TRON\detectron_repo\detectron2\layers\csrc\box_iou_rotated\box_iou_rotated_cuda.cu
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "C:\Users\E-soft\Anaconda3\envs\detectron_env\lib\site-packages\colorama\ansitowin32.py", line 59, in closed
return stream.closed
ValueError: underlying buffer has been detached
----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\E-soft\Anaconda3\envs\detectron_env\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'E:\\TRON\\detectron_repo\\setup.py'"'"'; __file__='"'"'E:\\TRON\\detectron_repo\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
It says "Don't know how to compile E:\TRON\detectron_repo\detectron2\layers\csrc\box_iou_rotated\box_iou_rotated_cuda.cu". What's the origin of the problem? Is it coming from Microsoft Visual Studio?

Related

Error: pg_config executable not found when install psycopg2 in CentOS 7.9

I am install psycopg2 in the CentOS 7.9 using this command:
pip3.9 install psycopg2
but shows error like this:
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python3.9 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qpjxu3uz/psycopg2_009102b415df471f9b4808adbcda990c/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qpjxu3uz/psycopg2_009102b415df471f9b4808adbcda990c/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-prsk3_f3
cwd: /tmp/pip-install-qpjxu3uz/psycopg2_009102b415df471f9b4808adbcda990c/
Complete output (16 lines):
/usr/local/lib/python3.9/site-packages/setuptools/dist.py:642: UserWarning: Usage of dash-separated 'index-url' will not be supported in future versions. Please use the underscore name 'index_url' instead
warnings.warn(
running egg_info
creating /tmp/pip-pip-egg-info-prsk3_f3/psycopg2.egg-info
writing /tmp/pip-pip-egg-info-prsk3_f3/psycopg2.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-prsk3_f3/psycopg2.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-pip-egg-info-prsk3_f3/psycopg2.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-prsk3_f3/psycopg2.egg-info/SOURCES.txt'
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
----------------------------------------
I have already install postgresql13-devel using command dnf install postgresql13-devel -y and install successs. why still could not install the psycopg2?
add the pg_config path to the system execute search path, for example, my PostgreSQL was installed in the path /opt/pg13, so export the path like this would fix this problem:
export PATH=/opt/pg13/bin:$PATH

Reportlab \ Catalina Installation failure : ERROR: Command errored out with exit status 1:

Context: macOS Catalina: pip3 install marisol returns:
Building wheels for collected packages: reportlab
Building wheel for reportlab (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /Users/user/pyViutualEnv/marisol/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/5s/5j5833_j41q5jw5qg42h3h6w0000gn/T/pip-install-4sqwyoby/reportlab_8b45a6fc8e1f41dda74996813181243c/setup.py'"'"'; __file__='"'"'/private/var/folders/5s/5j5833_j41q5jw5qg42h3h6w0000gn/T/pip-install-4sqwyoby/reportlab_8b45a6fc8e1f41dda74996813181243c/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/5s/5j5833_j41q5jw5qg42h3h6w0000gn/T/pip-wheel-s_xiwi84
cwd: /private/var/folders/5s/5j5833_j41q5jw5qg42h3h6w0000gn/T/pip-install-4sqwyoby/reportlab_8b45a6fc8e1f41dda74996813181243c/
Complete return: https://pastebin.com/raw/2HvMLVVt
What exactly is the problem and the remedy?
Update: macOS High Sierra: pip3 install marisol successfully installed and returned no errors

Can't install pyFFTW with python 3.9 in Macos M1

Any idea on how can I solve this issue?
I really need to install pyfftw into a conda environment, which already has the other libraries I need, such as tensorflow. Up until know, I could only install the other libraries correctly (namely tensorflow) for python 3.9.
I'm working on a Mac M1.
Here is the error output for the command pip install pyfftw:
Collecting pyfftw
Using cached pyFFTW-0.12.0.tar.gz (137 kB)
Requirement already satisfied: numpy<2.0,>=1.10 in /Users/mymac/miniforge3/envs/tf/lib/python3.9/site-packages (from pyfftw) (1.19.5)
Building wheels for collected packages: pyfftw
Building wheel for pyfftw (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /Users/mymac/miniforge3/envs/tf/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-install-5kpcegg9/pyfftw_56df32e267e3416fb4e9f01e6282e505/setup.py'"'"'; __file__='"'"'/private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-install-5kpcegg9/pyfftw_56df32e267e3416fb4e9f01e6282e505/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-wheel-6urq200p
cwd: /private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-install-5kpcegg9/pyfftw_56df32e267e3416fb4e9f01e6282e505/
Complete output (25 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-11.0-arm64-3.9
creating build/lib.macosx-11.0-arm64-3.9/pyfftw
copying pyfftw/config.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw
copying pyfftw/_version.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw
copying pyfftw/__init__.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw
creating build/lib.macosx-11.0-arm64-3.9/pyfftw/builders
copying pyfftw/builders/builders.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/builders
copying pyfftw/builders/__init__.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/builders
copying pyfftw/builders/_utils.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/builders
creating build/lib.macosx-11.0-arm64-3.9/pyfftw/interfaces
copying pyfftw/interfaces/cache.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/interfaces
copying pyfftw/interfaces/__init__.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/interfaces
copying pyfftw/interfaces/scipy_fft.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/interfaces
copying pyfftw/interfaces/dask_fft.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/interfaces
copying pyfftw/interfaces/numpy_fft.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/interfaces
copying pyfftw/interfaces/scipy_fftpack.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/interfaces
copying pyfftw/interfaces/_utils.py -> build/lib.macosx-11.0-arm64-3.9/pyfftw/interfaces
UPDATING build/lib.macosx-11.0-arm64-3.9/pyfftw/_version.py
set build/lib.macosx-11.0-arm64-3.9/pyfftw/_version.py to '0.12.0'
running build_ext
error: Could not find the FFTW header 'fftw3.h'
----------------------------------------
ERROR: Failed building wheel for pyfftw
Running setup.py clean for pyfftw
Failed to build pyfftw
Installing collected packages: pyfftw
Running setup.py install for pyfftw ... error
ERROR: Command errored out with exit status 1:
command: /Users/mymac/miniforge3/envs/tf/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-install-5kpcegg9/pyfftw_56df32e267e3416fb4e9f01e6282e505/setup.py'"'"'; __file__='"'"'/private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-install-5kpcegg9/pyfftw_56df32e267e3416fb4e9f01e6282e505/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-record-qt54h1vj/install-record.txt --single-version-externally-managed --compile --install-headers /Users/mymac/miniforge3/envs/tf/include/python3.9/pyfftw
cwd: /private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-install-5kpcegg9/pyfftw_56df32e267e3416fb4e9f01e6282e505/
Complete output (7 lines):
running install
running build
running build_py
UPDATING build/lib.macosx-11.0-arm64-3.9/pyfftw/_version.py
set build/lib.macosx-11.0-arm64-3.9/pyfftw/_version.py to '0.12.0'
running build_ext
error: Could not find the FFTW header 'fftw3.h'
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/mymac/miniforge3/envs/tf/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-install-5kpcegg9/pyfftw_56df32e267e3416fb4e9f01e6282e505/setup.py'"'"'; __file__='"'"'/private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-install-5kpcegg9/pyfftw_56df32e267e3416fb4e9f01e6282e505/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/0k/hz9yngm56nz1htdc3c3t3d0c0000gn/T/pip-record-qt54h1vj/install-record.txt --single-version-externally-managed --compile --install-headers /Users/mymac/miniforge3/envs/tf/include/python3.9/pyfftw Check the logs for full command output.
The PyPA doesn't host a precompiled wheel for osx-arm64, so it is building from source. According to the documentation, when source building, one is required to have FFTW installed first. Fortunately, Conda Forge appears to be building this (including osx-arm64), so try
conda install -c conda-forge fftw
pip install pyfftw
Otherwise, I dropped an issue to the Conda Forge feedstock, noting that they should be capable of building an osx-arm64 build. Hopefully that will move forward and in the future one could avoid Pip and local compilation altogether.
UPDATE: the feedstock is now building for osx-arm64, so you should be able to use
conda install -c conda-forge pyfftw

I'm trying to install fbprophet using pip on windows10 and the following error is showed

I'm trying to install fbprophet using pip on windows10 and the following error is showed
ERROR: Command errored out with exit status 1: 'c:\program
files\python37\python.exe' -u -c 'import sys, setuptools, tokenize;
sys.argv[0] =
'"'"'C:\Users\ARJUNS~1\AppData\Local\Temp\pip-install-v9k4x_8v\fbprophet\setup.py'"'"';
file='"'"'C:\Users\ARJUNS~1\AppData\Local\Temp\pip-install-v9k4x_8v\fbprophet\setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))'
install --record
'C:\Users\ARJUNS~1\AppData\Local\Temp\pip-record-pjkjdpel\install-record.txt'
--single-version-externally-managed --compile Check the logs for full command output.
I have the pystan packet already installed
I had faced same problem. I used "conda" rather than "pip" for installation with steps shown below and it worked magical for me.
step 1: conda install -c anaconda ephem
step 2:conda install -c conda-forge pystan
step 3: conda install -c conda-forge fbprophet
Now you can go ahead with : "from fbprophet import Prophet" in python environment.

Issues Installing pdftotext on MacOS using Anaconda

I am newer to programming and am having trouble installing pdftotext in Anaconda, using Mac OS Mojave 10.14.6 and Python 3.6.7. I have installed all the dependencies recommended (https://github.com/jalan/pdftotext) but have not been able to successfully install pdftotext.
Errors include:
ERROR: Command errored out with exit status 1:
command: /anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3_/T/pip-install-ak3q4wz7/pdftotext/setup.py'"'"'; __file__='"'"'/private/var/folders/3_/T/pip-install-ak3q4wz7/pdftotext/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/3_/T/pip-install-ak3q4wz7/pdftotext/pip-egg-info
cwd: /private/var/folders/3_//T/pip-install-ak3q4wz7/pdftotext/
Complete output (3 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
I ended up using pdftotext from poppler instead of the package listed above. This required the installation of poppler using conda-forge in Anaconda.
For Mac:
conda create -n envname python=3.7
conda activate envname
conda config --add channels conda-forge
conda install poppler
ModuleNotFoundError: No module named 'setuptools'
This suggests that you want to install at least one more dependency:
$ conda install setuptools
Or pip install, if you prefer.

Resources