ModuleNotFoundError: No module named 'numpy.core.multiarray\r' - python-3.x

I am trying to load a pkl file,
pkl_file = open(sys.argv[1], 'rb')
world = pickle.load(pkl_file)
but I get an error from these lines
Traceback (most recent call last):
File "E:/python/test.py", line 186, in <module>
world = pickle.load(pkl_file)
ModuleNotFoundError: No module named 'numpy.core.multiarray\r'
I am using Windows 10, python 3.7, and installed four packages (numpy 1.17.2, opencv-python 4.1.1.26, pip 19.2.3, setuptools 41.2.0 ). I have tried to change "rb" to "r", but still got the error, how can I fix this?

I think there are two problems here.
First, your pickle is or contains a NumPy object, which is not part of the standard library. Therefore you must ensure that NumPy is installed into your current Python environment and imported before you try to load the pickled object. Depending on your setup, installation may be as simple as,
pip install numpy
Then you must add the line,
import numpy as np
to the top of your script.
Second, it looks like Python is encountering this issue, where your binary file was erroneously saved as text on Windows, resulting in resulted in each '\n' being converted to '\r\n'. To fix this, you must re-convert to '\r\n' back to '\n'. So long as the file isn't huge, this usually isn't very painful.
Here is a relatively complete example:
import sys
import numpy as np
src = sys.argv[1] # path to your file
data = open(src).read().replace('\r\n', '\n') # read and replace file contents
dst = src + ".tmp"
open(dst, "w").write(data) # save a temporary file
world = pickle.load(open(dst, "rb"), encoding='latin1')

Ok, I just had to figure this out for myself, and I solved it. All you have to do is change all the "\r\n" to "\n". You can do this in multiple ways. You can go into Notepad++ and change line endings from CR LF to just LF. Or programmatically you can do
open(newfile, 'w', newline = '\n').write(open(oldfile, 'r').read())

Related

Unittest: filename with point not loaded properly

Setup: PyCharm, Python 3.10
We have the naming convention to name our python unittest files like an URL. For example: my.domain.org.py
In the past, this was no issue. Now after an IDE and Python Update it does not run anymore. Selecting right click -> Run "Python tests in my.domain.org.py" throws the error:
Traceback (most recent call last):
File "D:\programs\Python\3.10.2\lib\unittest\loader.py", line 154, in loadTestsFromName
module = __import__(module_name)
ModuleNotFoundError: No module named 'my'
It seems, the loader is interpreting the "." in the filename as path.
How can I run the unittest without renaming the file (which solves the issue)?
you cannot import python files with invalid names (in your case has dots in it) directly, but there's a turn around, you can use the imp library like (here in the example I have a function named print_smth that prints "it works!" in the my.file.py):
import imp
with open('my.file.py', 'rb') as fp:
my_file = imp.load_module(
'my_file', fp, 'my.file.py',
('.py', 'rb', imp.PY_SOURCE)
)
if __name__ == "__main__":
my_file.print_smth()
output:
test.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
it works!
P.S: preferably DO NOT DO THAT! it is highly deprecated!

Why does command prompt import differ from sublime text import?

I have installed with pip several packages (numpy/pandas/blpapi/pyarrow). I work with a Windows 64-bit machine, python3.6 in a sublime environment.
While all packages are shown as correctly imported in the command prompt, some packages are not found by my sublime scripts.
To try and remedy this problem, I used sys.path.insert and changed the names of my scripts, to no avail. The traceback below describes what I'm seeing:
Code in Command Prompt:
>>> import pyarrow
>>> import pandas
>>>
Code in Sublime (better_name.py):
print('Hi')
import numpy
import pandas
Output of better_name.py:
Hi
Traceback (most recent call last):
File "C:\Users\Documents\better_name.py", line 4, in <module>
import pandas
ModuleNotFoundError: No module named 'pandas'
Obtaining the paths in Command Prompt:
>>> import os
>>> import numpy
>>> path = os.path.dirname(numpy.__file__)
>>> print(path)
C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\numpy
>>> import pandas
>>> path = os.path.dirname(pandas.__file__)
>>> print(path)
C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas
Trying to use sys.path.insert :
print('Hi')
import sys
import numpy
import os
sys.path.insert(1, r"C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas")
Output:
C:\Users\Documents>better_name.py
Hi
Traceback (most recent call last):
File "C:\Users\Documents\better_name.py", line 7, in <module>
import pandas
ModuleNotFoundError: No module named 'pandas'
I get the same results whether I change the argument in sys.path.insert to 0.
The issue seems to be that your default version of python points to the 32-bit version - i.e. when you say python your windows system executes the 32 bit version.
One workaround is to specify the full path of your 64 bit version - i.e. launch your script as
C:\PATH\TO\64-BIT-VERSION\PYTHON.EXE your_script.py
from the command line.
The other option is to set your windows environment variables to point to the 64 bit version by default. This link should help

Keras import loading neural network

The script normally works on a local machine, but it also fails when it is uploaded to the server. The program falls on the line:
from keras.models import model_from_json
All versions of libraries on a laptop and server are the same. Python 3.6.5, Keras 2.2.4, Tensorflow 1.5.0, Numpy 1.14.3
Error message:
(base) C:\classX5>python app.py
Using TensorFlow backend.
Traceback (most recent call last):
File "app.py", line 8, in <module>
from keras.models import model_from_json
File "C:\Anaconda3\lib\site-packages\keras\__init__.py", line 3, in <module>
from . import utils
File "C:\Anaconda3\lib\site-packages\keras\utils\__init__.py", line 27, in <mo
dule>
from .multi_gpu_utils import multi_gpu_model
File "C:\Anaconda3\lib\site-packages\keras\utils\multi_gpu_utils.py", line 7,
in <module>
from ..layers.merge import concatenate
File "C:\Anaconda3\lib\site-packages\keras\layers\__init__.py", line 4, in <mo
dule>
from ..engine.base_layer import Layer
File "C:\Anaconda3\lib\site-packages\keras\engine\__init__.py", line 8, in <mo
dule>
from .training import Model
File "C:\Anaconda3\lib\site-packages\keras\engine\training.py", line 21, in <m
odule>
from . import training_arrays
File "C:\Anaconda3\lib\site-packages\keras\engine\training_arrays.py", line 8,
in <module>
from scipy.sparse import issparse
File "C:\Anaconda3\lib\site-packages\scipy\__init__.py", line 119, in <module>
from scipy._lib._ccallback import LowLevelCallable
File "C:\Anaconda3\lib\site-packages\scipy\_lib\_ccallback.py", line 1, in <mo
dule>
from . import _ccallback_c
ImportError: cannot import name '_ccallback_c'
All versions of libraries on a laptop and server are the same. Python
3.6.5, Keras 2.2.4, Tensorflow 1.5.0, Numpy 1.14.3
But NOT the scipy package.
Just uninstall (pip3 uninstall scipy) and reinstall scipy package. It solved the problem for many people.
Posting my same answer here I posted on the linked question:
This is likely an issue with the SciPy Python module compatibility you have installed in your site-packages folder and the OS architecture you're running it on.
After digging in, to give the full background on this, first of all SciPy relies on having NumPy already installed. The SciPy wheel's setup.py file uses NumPy functionality to configure and install the wheel.
SciPy setup.py:
...
if __name__ == '__main__':
from numpy.distutils.core import setup
setup(**configuration(top_path='').todict())
Secondly, when just trying to use the wheel, if you run into this error, you can see after inspecting the wheel's files that the reason is the binary wheels have a naming convention where the shared object file, here it's called _ccallback_c.so, is instead named based on the architecture that the binary wheel supports. When trying to import the shared object by file name in /_lib/_ccallback.py it can't find it, hence this error (line 1 in /_lib/_ccallback.py) because, instead of being named _ccallback_c.so it's called _ccallback_c.cpython-36m-x86_64-linux-gnu.so or another architecture variation:
from . import _ccallback_c
These file names may be an artifact of having not run the NumPy setup process yet or something related to Cython, that I'm not quite sure about. But the easiest fix is to change the .whl extension to .zip and rename all those relevant .so files to not contain the architecture snippet. Then change .zip -> .whl and it should be good to go.

cx_Freeze converted exe: window closes immediately

I am trying to just convert my pygame python py to a .exe file using cx_Freeze. The setup file executes correctly and without error, but the issue is that when I run my .exe file the console window (the black cmd-like window)will open quickly and close. My .py which I want to convert is called Salary.py, and it includes a .input('str') codes in it so that the
user can decide which csv file they want to use.
Description of my Salary.py: if the user input an interger, Salary.py help them to parse through one existed csv file and run the
script and output to another csv file.
Chinese is included inside the code.
Once again, there is no error running when I run Salary.py in python3.6, and no error when building the exe file for now.
The setup.py I am using:
from cx_Freeze import setup, Executable
import os
os.environ['TCL_LIBRARY'] = r'D:\Anaconda3\tcl\tcl8.6'
os.environ['TK_LIBRARY'] = r'D:\Anaconda3\tcl\tcl8.6'
setup(name='Salary',
version='0.1',
description='Salarycount',
executables= [Executable("Salary.py")])
I try to execute the Salary.exe in cmd. And it gave this to me:
Traceback (most recent call last):
File "D:\Anaconda3\lib\site-packages\cx_Freeze\initscripts\__startup__.py", line 14, in run
module.run()
File "D:\Anaconda3\lib\site-packages\cx_Freeze\initscripts\Console.py", line 26, in run
exec(code, m.__dict__)
File "Salary.py", line 8, in <module>
File "D:\Anaconda3\lib\site-packages\pandas\__init__.py", line 19, in <module>
"Missing required dependencies {0}".format(missing_dependencies))
ImportError: Missing required dependencies ['numpy']
Apperently, numpy has already installed to my python packages, I don't even know what does it mean.
my environment:
Anaconda 3
python 3.6
cx-Freeze 6.0b1
It looks like your Salary.py script uses the pandas package, is this correct? The pandas package requires the numpy package to work, and one needs to tell cx_Freeze explicitly to include the numpy package. Try to add the following options to setup:
from cx_Freeze import setup, Executable
import os
os.environ['TCL_LIBRARY'] = r'D:\Anaconda3\tcl\tcl8.6'
os.environ['TK_LIBRARY'] = r'D:\Anaconda3\tcl\tcl8.6'
setup(name='Salary',
version='0.1',
description='Salarycount',
options={'build_exe': {'packages': ['numpy']}},
executables= [Executable("Salary.py")])

How can I import bs4 from Beautiful Soup in Python3 on OSX 10.12.5?

I'm trying to fix what seems like a common problem with importing a module in Python 3. I'm running OS X 10.12.5 and have Python 3 installed on my MacBook Air and am using Sublime Text to edit and run my code.
When I try this import:
from bs4 import BeautifulSoup
...I get this error:
Traceback (most recent call last):
File "/Users/<myname>/Python/code-python3/Pgm#001", line 5, in <module>
from bs4 import BeautifulSoup
ImportError: No module named 'bs4'
I successfully installed with this PIP and on every re-install I see this:
$ pip install beautifulsoup4
Requirement already satisfied: beautifulsoup4 in ./anaconda/lib/python3.5/site-packages
I've tried qualifying the location with things like:
from ./anaconda/lib/python3.5/site-packages/bs4 import BeautifulSoup
...but invariably get a variety of syntax errors; usually on the first '/'.
I am not using a virtual environment but do plan to read up on that approach as these kind of configuration and setup errors are big time wasters.
When I try to run this right in python3 I get slightly different errors:
>>> from bs4 import BeautifulSoup
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/<myname>/anaconda/lib/python3.5/site-packages/bs4/__init__.py", line 30, in <module>
from .builder import builder_registry, ParserRejectedMarkup
File "/Users/<myname>/anaconda/lib/python3.5/site-packages/bs4/builder/__init__.py", line 314, in <module>
from . import _html5lib
File "/Users/<myname>/anaconda/lib/python3.5/site-packages/bs4/builder/_html5lib.py", line 70, in <module>
class TreeBuilderForHtml5lib(html5lib.treebuilders._base.TreeBuilder):
AttributeError: module 'html5lib.treebuilders' has no attribute '_base'
Any tips about where the obvious answer exists that I am still missing would be greatly appreciated. I've seen a lot of things about PATH and PYTHONPATH in similar questions but had no success with any of these solutions either.
Yes, I'm not a fan of anaconda but looks like the path is being pointed to:
"/Users//anaconda/lib/python3.5/site-packages"
... and for the sake of keeping things as clean as possible (this being a good example/reason why you'll see people say to use env for your python projects)
Lets google here! Here's how to remove anaconda:
conda install anaconda-clean #first the configs
anaconda-clean --yes
rm -rf ~/anaconda.* #then conda, the ".*" is regex to cover any version
The check your bash_profile and/or bashrc; and here you can set your Python path. Make sure it not pointing to anaconda
if you run
ls -lh `which python # orpython3
You'll get the path of which is the one you'll be setting as the path.

Resources