numpy build fail in M1 Big sur 11.1 - python-3.x

I am using MacBook m1 which is running Big Sur 11.1 ,and I have installed Xcode-commandline-tools version 12.3 and it has installed python3.8.5 and pip3 .python3 and pip are native apps,that is they run in ARM
pip3 version 20.3.3
python3 version 3.8.5
setuptools version 51.0.0
wheel version 0.36.2
when I type python3 -m pip install numpy
The output is :
Defaulting to user installation because normal site-packages is not writeable
Collecting numpy
Using cached numpy-1.19.4.zip (7.3 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Building wheels for collected packages: numpy
Building wheel for numpy (PEP 517): started
sandeep#Sandeeps-Air ~ % cat a
Defaulting to user installation because normal site-packages is not writeable
Collecting numpy
Using cached numpy-1.19.4.zip (7.3 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Building wheels for collected packages: numpy
Building wheel for numpy (PEP 517): started
Building wheel for numpy (PEP 517): finished with status 'error'
Failed to build numpy
and with a very large list of errors
Is numpy still not supported in M1 or am I doing it wrong ?

I had exactly the same issue with my brand new Mac Mini. After spending an afternoon reading this issue on github, I finally succeeded. However, it only seems to work with v1.20.0rc2 or above...
Long story short, these were the steps I've taken to get it working:
pip3 install Cython
git clone https://github.com/numpy/numpy.git
cd numpy
pip3 install . --no-binary :all: --no-use-pep517
After that, I could cd into my virtualenv and import numpy:
>>> import numpy as np
>>> np.__version__
'1.21.0.dev0+402.gc4ae3ce64'
>>>
Next up: pandas.
UPDATE:
After some more searching, I found this interesting article, which states that you can easily start Terminal in Rosatta mode. That also solved the issues.
However, I am working with a headless Mac Mini, so I still have to find out whether that solution also is feasable in that particular setup.
UPDATE 2
I now have pandas working as well, thanks to this answer on SO.

Yes, numpy still seems to be not supported in stable, but if you are ready to work with unstable version try this command:
pip3 install numpy --compile --pre
This will recompile all the necessary code from unstable branch.
Worked for me.

Since NumPy 1.21.0, it has released universal2 wheels for Python 3.8 and Python 3.9 on Mac. It becomes installable on Mac M1.
I have tested it on my M1 with Python 3.9.2, macOS Big Sur (11.5.1), and pip 21.1.3. Just use pip/pip3 to install it.
release note: https://github.com/numpy/numpy/releases/tag/v1.21.0

Note that the only reliable build environment for the full data-science suite for python at the moment (Feb 2021) on M1 ARM architecture is via miniforge.
e.g.
brew install --cask miniforge
conda init zsh
conda activate
conda install numpy scipy scikit-learn

What it worked for me was to run iTerm with the option "open using Rosetta" enabled and upgrade pip before installing numpy.
Instead of enabling this option you could also run the pip install and python selecting a different architecture with arch -x86_64. Example using a virtual environment:
$ python3 -m venv ~/myenv
$ source ~/myenv/bin/activate
(myenv)$ python3 -m pip install --upgrade pip
Collecting pip
Using cached pip-21.0.1-py3-none-any.whl (1.5 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.2.3
Uninstalling pip-20.2.3:
Successfully uninstalled pip-20.2.3
Successfully installed pip-21.0.1
(myenv)$ arch -x86_64 pip install numpy
Collecting numpy
Using cached numpy-1.20.1-cp39-cp39-macosx_10_9_x86_64.whl (16.1 MB)
Installing collected packages: numpy
Successfully installed numpy-1.20.1
(myenv)$ arch -x86_64 python3
Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec 7 2020, 12:44:01)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.__version__
'1.20.1'

I was trying to install numpy without rosetta 2, but the solutions listed here failed for me. Brew saved the day. I installed it with
brew install numpy

First of all numpy support is not there for m1 arm based Macs using python 3.8 version. The method to workaround with this is to update your python version to 3.9. This is the latest version of python and can be a bit unstable. If you want to work with tensorflow then do not update to python 3.9 version as they don't support it and support will come up with version 2.5 (estimated). So, if you don't want to use tensorflow and want to use numpy by using pip install then go ahead install python3.9 and make it your default python interpreter.
I will recommend you to use Homebrew for installation of python 3.9 as it as easy as running a simple command.

faced the same problem in monterey too.
upgrading pip version to 21.3.1 from 20.2.3 solved the issue.
source env/bin/activate
pip install -U pip
pip install numpy

Try this as I was able to install numpy using this method.
brew install openblas (use the native homebrew instead of running it under Rosetta 2)
OPENBLAS="$(brew --prefix openblas)" pip3 install numpy

A solution for pipenv users:
git clone https://github.com/numpy/numpy.git --recursive
cd numpy
pip3 install build
python3 -m build
cd dist
cp numpy-1.22.0.dev0+97.g4635fc852-cp39-cp39-macosx_11_0_arm64.whl ../../ # filename is current sample, next folder to your project directory
Add into Pipfile:
[packages]
numpy = { path = "numpy-1.22.0.dev0+97.g4635fc852-cp39-cp39-macosx_11_0_arm64.whl" }
pipenv install
then your numpy will be installed.

As mentioned in a previous answer, the installation went smooth when I opened the terminal using Rosetta.
However for anyone who is using Pycharm terminal, Do the following to get the terminal running using Rosetta.
Go to Preferences -> Tools -> Terminal
Update the script path to "env /usr/bin/arch -x86_64 /bin/zsh --login"
I tried a lot of other options but so far this was the simplest solution I found.

After trying a ton of suggested methods I finally found a way to use numpy on my new macbook pro with an M1 chip.
Use this docker preview for the M1 chip.
E.g. here's a Dockerfile that successfully installs numpy.
FROM python
RUN python3 -m pip install numpy
Build it with:
docker build -t numpytest:latest .
Then run it with:
docker run -it --rm numpytest:latest
And you can verify that numpy is installed and works correctly:
Python 3.9.1 (default, Jan 12 2021, 13:13:57)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.arange(15).reshape(3, 5)
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]])
>>>

Well I guess it's fixed in unreleased upstream. A pip install of 1.20.2, the latest as of this writing, failed. But I cloned git#github.com:numpy/numpy.git, then did the normal "python setup.py install" in my venv (which is Python 3.9.5 installed via Homebrew) and it seems to work. Haven't thoroughly tested it yet.

CrepeGoat's comment worked for me:
pip3 install --no-binary :all: --no-use-pep517 numpy

Related

'import' showing outdated versions after updating to Python 3.10 on Google Colaboratory

after updating from Python 3.7 to 3.10 on Google Colaboratory when using pip the newly installed packages arent showing when using 'import'. There's a demo at Colab on this link
I Updated the python version using this
!wget https://github.com/korakot/kora/releases/download/v0.10/py310.sh
!bash ./py310.sh -b -f -p /usr/local
!python -m ipykernel install --name "py310" --user
!python --version
And installed numpy using
!pip3 install numpy==1.22.4 --upgrade
Thing is, the newly installed version of numpy is showing on pip, at the python3.10/site-packages folder
!pip show numpy
Name: numpy
Version: 1.22.4
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email:
License: BSD
Location: /usr/local/lib/python3.10/site-packages
Requires:
Required-by: pandas, torchvision
But import is using the preinstalled version on 3.7/dist-packages.
using
import numpy
numpy.__version__
outputs
'1.21.6'
and
numpy.__config__
outputs
<module 'numpy.__config__' from '/usr/local/lib/python3.7/dist-packages/numpy/__config__.py'>
Re-starting the runtime to clear the cached versions of the packages didn't help either.
There's a way to detect and prioritize the packages on the Python3.10 folder?

Pandas build fail M1 Big Sur 11.1

I am using MacBook m1 which is running Big Sur 11.1 ,and I have installed Xcode-commandline-tools version 12.3 and it has installed python3.8.5 and pip3 .python3 and pip are native apps,that is they run in ARM
pip3 version 20.3.3
python3 version 3.8.5
setuptools version 51.0.0
wheel version 0.36.2
when I type python3 -m pip install pandas
The output is :
Defaulting to user installation because normal site-packages is not writeable
Collecting pandas
Downloading pandas-1.2.0.tar.gz (5.4 MB)
|████████████████████████████████| 5.4 MB 150 kB/s
Installing build dependencies ... error
And with a very long list of error about 30,000 lines (only last few lines)
and
pip3 list output is
cppy 1.1.0
kiwisolver 1.3.1
numpy 1.20.0rc1
pip 20.3.3
pyparsing 2.4.7
python-dateutil 2.8.1
setuptools 51.0.0
six 1.15.0
wheel 0.36.2
Is pandas not yet supported or I am doing it wrong and same goes with Matplotlib.
Install Miniforge for arm64 (Apple Silicon)latest installer here: installer here
Now create a conda environment for whatever version you have (I'm running 3.9.2)
conda create -n cenv python=3.9.2
conda activate cenv
conda install pandas
You will drive yourself nuts trying to get all the different packages working if you try to go from wheels/pip, at time of writing.
I think I got pandas working but couldn't get matplotlib working due to kiwi solver issues. Use miniforge/conda. This is the way.
You can try installing pandas version 0.25.3. This is the most stable release of Pandas which brings in numpy 1.19.1 which is the most stable version of NUMPY.
try running pip install pandas==0.25.3, and it should install the required packages.

Getting error while installing any module using "pip install ..." in cmd [duplicate]

I installed python 3.7 on my Windows 10 laptop since it has been officially released as of today (06/28/2018). Then i tried to install numpy package using pip
pip install numpy
The install proceeds but finally fails with the below error :
source = func(extension, build_dir)
File "numpy\core\setup.py", line 675, in get_mathlib_info
raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program
----------------------------------------
Command ""c:\program files\python37\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\pcheg\\AppData\\Local\\Temp\\pip-install-7wjkw5wn\\numpy\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\pcheg\AppData\Local\Temp\pip-record-uhj8233f\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\pcheg\AppData\Local\Temp\pip-install-7wjkw5wn\numpy\
Any ideas as to how to overcome this install Error? Thanks.
Installing NumPy on Windows is a common problem if you don't have the right build setup. Instead, I always go to Christoph Gohlke's website to download the wheels you can install for your computer. Christoph generously builds the libraries himself with the right build environment and he posts it on his website.
Newer Instructions - For older instructions, please scroll down
First, install pipwin from PyPI which will install a utility that acts like pip but it will download the actual package you're interested in from his website, then use pipwin install to install the package you want.
First do:
pip install pipwin
When that's installed, you can then do:
pipwin install numpy
This will install the latest version of NumPy on your system. This way you don't have to specifically search for the version of NumPy that is for your specific version of Python.
Older instructions
Go to the NumPy section: https://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy then download the version for 3.7 that is compatible with your version of Python (2 or 3 and 32-bit or 64-bit). For example, the filename numpy‑1.14.5+mkl‑cp37‑cp37m‑win_amd64.whl is for NumPy 1.14.5, Python 3.7 - 64 bit. You can pick out which version of NumPy and which version of the Python interpreter and bit version you need in the filename.
Doing this never requires you to build NumPy yourself or install the required compiler as opposed to installing NumPy through PyPI. You can just download the wheel and install it yourself. Assuming you've already downloaded it, just do:
pip install numpy‑1.14.5+mkl‑cp37‑cp37m‑win_amd64.whl
... assuming the wheel is in the directory you're currently in.
Starting 24 November 2021, latest numpy require at least Python 3.8
Note: This might not be the original question asked, but it might help anyone come here.
To use python 3.7, latest numpy you can use is v1.21.4. So, to install it, use:
pip install numpy==1.21.4
If you write requirements that you hope compatible with python 3.7, you can use numpy<=1.21.4
EDIT: in 20 December 2021, numpy release version 1.21.5 that support Python 3.7
From comment section, by #sam, numpy 1.21.5 support Python 3.7. It was released after 1.22.0rc1 (the latest numpy version as the writing of the original post) that only support Python 3.8++.
Lesson learned from this experience, it would be better to use <,
pip install numpy<1.22.0
or
install_requires = [
"numpy<1.22.0", # lates version to support python 3.7
],
For windows when you install a package you type in Python 3:
py -m pip install [packagename]
OR
py -m pip install numpy
You're probably missing a C compiler. If numpy doesn't provide a prebuilt wheel yet, you'll need to install a compiler. This website shows the one you need to install. For 3.7, I assume it's still 14.0, so you should install Microsoft Build Tools for Visual Studio 2017.
Let me know if that works.

Installing NumPy on Fedora 24 with pip

My question is perhaps similar to Installing Numpy on Fedora 19 with pip.
I would like to install NumPy with pip on (a freshly installed) Fedora 24.
Fedora already has pip (via the command pip3). When I write pip3 install numpy, pip downloads a zip (with the source code inside) and tries to compile (build a local wheel). But I do not have Python headers so it can't compile. I guess that if I install python3-devel pip will manage to compile NumPy as I already have GCC (but no linear algebra library).
But what I want to do is to install NumPy from a wheel only. Normally if a wheel is available on PyPI, then pip will download the corresponding wheel file. It is so on other distributions: Arch, Ubuntu. I installed the package python3-wheel also, but it didn't have any effect.
Can somebody make the situation clearer?
You need to upgrade your pip.
I've just tried to install numpy on fresh fedora container and pip tried to download sources and compile them, and failed because of python3-devel missing.
But I updated pip with pip3 install --upgrade pip and afterwards pip3 install numpy downloaded numpy-1.11.1-cp35-cp35m-manylinux1_x86_64.whl file and isnstaled it without issues.

Installing Blocks on Linux

I'm trying to install Blocks on my Linux machine and I have some troubles. How can I fix this? I'm doing the following steps:
Installing Anaconda for python2.
Theano depends on numpy 1.10.1 version. If at this step I will run import theano in python shell then it will work perfectly.
Okay, it's time for Blocks. I'm installing stable version using pip install --user git+git://github.com/mila-udem/blocks.git \
-r https://raw.githubusercontent.com/mila-udem/blocks/master/requirements.txt
After this step if I import theano it gives the following: RuntimeError: module compiled against API version a but this version of numpy is 9.
I looked at requirements.txt and see that Blocks depends on numpy 1.9.3.
I uninstalled Theano, Blocks, downgraded numpy using conda install numpy=1.9.3, then just run again pip install --user git+git://github.com/mila-udem/blocks.git \
-r https://raw.githubusercontent.com/mila-udem/blocks/master/requirements.txt and still while importing theano it gives me RuntimeError: module compiled against API version a but this version of numpy is 9.
How can I overcome this problem?
This could be a problem with the Anaconda distribution. You could try updating all of Anaconda via conda update conda and conda update --all. You could also try changing Blocks requirements.txt to refer to numpy 1.10.1 since it's likely that Blocks won't care about the changes from 1.9 to 1.10.
– Daniel Renshaw

Resources