Importing Gensim/Word2Vec not stable in Databricks - python-3.x

I am simply to import import Word2Vec from gensim.models, but from few days I keep getting the following error :
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject
I tried updating my Numpy package to 1.24.1 (the latest) and Gensim package 4.3.0, but I still getting the same issue.
For details, I work with Python version 3.8.10 in Databricks.
Any idea please? Thank you

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject
happens as a result of a modification made to the API of NumPy in version. Upgrading your numpy module version will fix the issue. few days ago, only ne numpy version 1.25.0 is released.
Try Uninstalling and reinstalling numpy in notebook or upgrade the version.
#uninstall install versions
%sh
pip uninstall numpy
pip install numpy
#upgrade version
pip install numpy --upgrade
Also try with different gensim version.
I am successfully able to install:

Thank you #gojomo and #pratik-lad for your feedbacks.
The best solution that I find to solve the issue of "binary incompatibility" is to edit my Databricks cluster and upgrade Databricks runtime version from 10.1 to 12.0.
I am now able to install gensim, with those version:
Python 3.9.5
NumPy 1.21.5
SciPy 1.7.3

Related

How to treat '<attribute 'dtype' of 'numpy.generic' objects>' error?

After installing pypfopt and u-numpy, dataframe.info() command shows this error.
TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type
I happened to mix my versions and I encountered the problem today. I managed to fix it.
Both codes in jupyter gave me an error: TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type
df.info()
df.categorical_column_name.value_counts().plot.bar()
I got the error: TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type
This is how i fixed it
Inside jupyter:
Check numpy version:
import numpy as np
print(np.__version__)
To upgrade:
!pip3 install numpy --upgrade
Inside Command line check numpy version:
python
import numpy
print(numpy.__version__)
if versions are not the same choose whether to upgrade/downgrade:
To upgrade:
$pip install numpy --upgrade
To downgrade just specify the version
If you have python environment installed:
Go to the right folder:
Check the installed version:
$pipenv --version
To verify if you have a pip environment installed for that folder:
On your terminal Go to the folder and type:
$pipenv --version
If there is a pipenv it will show the version and if there is none it won't.
check numpy version
$python
>>> import numpy
#prints the version
>>> print(numpy__version__)
To upgrade the version:
>>>exit()
#To install the latest version don't specify the version
$pipenv install numpy
#if you want to downgrade specify the version
$pipenv install numpy=version_type
Do the same for pandas. Note that with pandas if your pandas environment is 1.2.3 on the jupyter notebook upgrade with !pip install pandas==1.2.3 or just !pip install pandas --upgrade --user.
Note that if the commands are giving you an error always include --user at the end of the command.
To create a new environment using miniconda and install updated packages follow the link [https://pandas.pydata.org/pandas-docs/stable/getting_started/install.html][1]
Run the following commands from a terminal window:
conda create -n name_of_my_env python
This will create a minimal environment with only Python installed in it. To put your self inside this environment run:
source activate name_of_my_env
On Windows the command is:
2. activate name_of_my_env
The final step required is to install pandas. This can be done with the following command:
conda install pandas
To install a specific pandas version:
conda install pandas=0.20.3
I prefer using the latest version of pandas 1.2.3
However the first method should solve your problem. Always restart your notebook by closing and reopening it.
I will stick around to see if you are winning. But this will resolve your problem. The problem is caused by the versions of numpy and pandas
[1]: https://pandas.pydata.org/pandas-docs/stable/getting_started/install.html
I fixed this type error downgrading numpy version to 1.16.5.
Try it!
Use code below in your jupyter notebook to downgrade your numpy:
!pip install numpy==1.16.5
My pandas version: 0.24.2
Here's a link to the numpy issue associated with this error: https://github.com/numpy/numpy/issues/18355. A succinct fix is given there (in https://github.com/numpy/numpy/issues/18355#issuecomment-1029684903):
pip install --upgrade numpy
pip install --upgrade pandas
downgrading to numpy==1.19.5 works
Use below command to downgrade in anaconda prompt:
python -m pip install numpy==1.19.5
The issue is because of the non-compatibility of NumPy and pandas versions. I couldn't downgrade my NumPy for some odd reasons as others suggested from Anaconda. But found this link helpful
Downgrading pandas
to 1.3 and with the existing NumPy version set at 1.20.1, helped me to overcome this issue.
Just go to the terminal provided in Jupyter Notebook by clicking "New" and then "Terminal" inside "New" and type:
pip install pandas --upgrade
this will fix the error
as it worked for me.
I fixed it by updating all packages in my Anaconda Navigator Environments
In my case, restarting the kernel fixed the error. Hope it saves someone time as I was reading pandas source code to figure out what the hell was happening :))
U might have the latest version of NumPy which may not be compatible, so downgrade your NumPy version. I have downgraded to 1.17.2

mystic 0.3.3 has requirement numpy<1.16.0,>=1.0, but you'll have numpy 1.16.2 which is incompatible

I'm using python 3.7 on my windows 10 system.I'm trying to install library reuqests. So I used following code
pip install requests
I got the error message
mystic 0.3.3 has requirement numpy<1.16.0,>=1.0, but you'll have numpy 1.16.2 which is incompatible.
Can you suggest me how to resolve this issue?
I got same issue during fastai package installation, uninstalled numpy and installing mystic package using anaconda prompt fixed the issue
pip uninstall numpy
pip install mystic

How to fix Pandas Import AttributeError "type object 'numpy.ndarray' has no attribute '__array_function__' "

I'm unable to import pandas. I'm using python3.6.7 and pandas version 0.23.4
Whenever i do the following:
import pandas as pd
I get the following ErrorMessage:
I've tried the following but none of these seem to work:
Restart Jupyter Notebook
Uninstall pandas(0.23.4) and reinstall it
Uninstall pandas(0.23.4) and install pandas(0.22)
Came across these on google but they don't work.
What else should I do?
The error comes from numpy-v1.16.0 that has been released today. Try to uninstall v1.16.0 and install an earlier version.
pip3 uninstall numpy
pip3 install numpy==1.15.0
EDIT:
You will now be getting the following warning.
/usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
which you can filter out with the following:
import warnings
warnings.filterwarnings("ignore", message="numpy.dtype size changed")
But for me, the best way to solve the issue was to run pip3 uninstall numpy and then manually delete all numpy directories from.
/home/USER/.local/lib/python3.5/site-packages/ because for whatever reason pip doesn't uninstall all of the packages.
and then just run pip3 install numpy. Now you have the current version of numpy with no errors or warnings.

How to avoid the sklearn import preprocessing statement giving error in python 3.5 version?

I am getting the error as below while trying to execute "from sklearn import preprocessing".
ImportError: No module named 'sklearn'.
My python version is 3.5.
Can someone please help on this.
Regards,
Philip
Without more information, I can only think maybe you don't have the library installed. Make sure you have NumPy and SciPy installed. Then make sure Scikit learn is actually installed. If you search how to install sklearn, a ton of results will show up instructing you how to do so. Basically,:
Install NumPy >= 1.6.1
Install SciPy >= 0.9
Install scikit-learn
Here is a link http://scikit-learn.org/stable/developers/advanced_installation.html.
If you are using pip, you can do pip install numpy scipy. Conda is similar, conda install numpy and conda install scipy.
I'm making a lot of assumptions here, so I'm not sure this is much help. Hopefully a good start though.

Installing Blocks on Linux

I'm trying to install Blocks on my Linux machine and I have some troubles. How can I fix this? I'm doing the following steps:
Installing Anaconda for python2.
Theano depends on numpy 1.10.1 version. If at this step I will run import theano in python shell then it will work perfectly.
Okay, it's time for Blocks. I'm installing stable version using pip install --user git+git://github.com/mila-udem/blocks.git \
-r https://raw.githubusercontent.com/mila-udem/blocks/master/requirements.txt
After this step if I import theano it gives the following: RuntimeError: module compiled against API version a but this version of numpy is 9.
I looked at requirements.txt and see that Blocks depends on numpy 1.9.3.
I uninstalled Theano, Blocks, downgraded numpy using conda install numpy=1.9.3, then just run again pip install --user git+git://github.com/mila-udem/blocks.git \
-r https://raw.githubusercontent.com/mila-udem/blocks/master/requirements.txt and still while importing theano it gives me RuntimeError: module compiled against API version a but this version of numpy is 9.
How can I overcome this problem?
This could be a problem with the Anaconda distribution. You could try updating all of Anaconda via conda update conda and conda update --all. You could also try changing Blocks requirements.txt to refer to numpy 1.10.1 since it's likely that Blocks won't care about the changes from 1.9 to 1.10.
– Daniel Renshaw

Resources