I am new to python packages and I am in process of learning how organize code into packages.
I tried out a small program for testing purpose and I am curious about an error that I encountered.
I have put the following package-structure (I am currently working with regular packages only as I don't have a good understanding of namespace packages):
parent/
main.py
p1/
__init__.py
The code in p1/__init__.py is as follows:
import math
print('p1 imported.')
The code in main.py is as follows:
from p1 import math
def main():
print(p1)
if __name__=='__main__':
main()
The output that it gives me is as follows:
path-to-parent>python3 main.py
p1 imported
Traceback (most recent call last):
File "main.py", line 12, in <module>
main()
File "main.py", line 8, in main
print(p1)
NameError: name 'p1' is not defined
I understand that it has imported math from sys.modules or the subsequent full search that it must have performed after encountering the import math statement in p1/__init__.py. It also executed the subsequent statement print('p1 imported') indicating that the module p1 has been imported. Then why does the NameError pop up?
With my reading of this page, my guess is that although the module p1 has been imported, it has not undergone the binding process because of the way it has been imported: from p1 import math.
I would still need help to understand this.
NOTE: I am using Anaconda Python 3.7.3 on windows 10 (64bit).
from p1 import math
def main():
print(p1)
if __name__=='__main__':
main()
p1 here is the name of the directory, it is not a python variable that can be used.
It is very roughly analogous to the following attempt:
with open('filename') as f:
print(filename)
Which will obviously raise NameError because filename is not defined.
Perhaps a better example is this:
from math import pi
print(math)
This also raises NameError because math is never defined as a variable that can be used outside of the import mechanism. math here is the name of the module from which pi is imported, but it can only be seen by the import mechanism.
It's much easier to demonstrate why what you're doing fails:
>>> from math import sin
>>> sin
<built-in function sin>
>>> math
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'math' is not defined
The point is, that math.sin is imported as sin, but the math module that is needed for this is not stored for reference anywhere. For that reason, there is no local math object you could refer to.
If you want to refer to the math module as a whole, just import it as a whole. For your example, just import p1.
Related
I'm creating cross-platform Python application, so I need to pay attention to the official disclaimers related to platform availability. For example:
os.getlogin()
...
Availability: Unix, Windows, not Emscripten, not WASI.
Unfortunatly I couldn't figure it out what "Unavailable" is in program behavior for wide spectr of functins (not only getlogin()).
Is there some standard exception to distinguish "Unavailable" case?
Is it enough to always wrap invocation with try...except StandardError to provide failover?
Do I need hide import with try as well or it is enough for function invocation? In other words:
try:
import os
os.getlogin()
...
except StandardError: # platform doesn't support user recognition
...
VS
import os
try:
os.getlogin()
...
except StandardError: # platform doesn't support user recognition
...
You will simply get an AttributeError. For example, on MacOS (set_handle_inheritable is only available on Windows):
>>> os.get_handle_inheritable
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'get_handle_inheritable'
So you can either:
Just import os and check the OS before every usage
Try to import the individual methods and catch ImportError
Try to call the functions and catch AttributeError
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I was splitting some functions into a separate file. Found the functions and helper functions. Set all the paths and import in other files so everything can still be called where necessary. But after removing an unused import from the original file it gives a circular import error it seems.
So there is an unused import and while there, every test (524) succeeds, but the branch cant be pushed because of the unused import. If we remove the import, the tests goes into circular import I think and wont start at all.
The first thing i tried is to change the regarding import statements from: from /module import /object --> import module
This made the tests start but everything went wrong after that.
After that I tried to find a way through the circular import path but I just lost track of everything while I dont know if it is even a solution.
`
Traceback (most recent call last):
File "/home/user/../.py", line 2, in <module>
from x.model import x
File "/home/user/../.py", line 1, in <module>
import A
File "/home/user/../.py", line 4, in <module>
import B
File "/home/user/../.py", line 1, in <module>
import C
File "/home/user/../.py", line 9, in <module>
import D
File "/home/user/../.py", line 4, in <module>
import E
File "/home/user/../.py", line 9, in <module>
from x.model import x
ImportError: cannot import name 'x'
`
If you are using imported Ive in /home/user/projects/.../nsx/utils.py file, inside some method or function, then you should remove import statement (i.e from ive.models import Ive) from top of file import statements(if you are doing so), and import it dynamically means put import statement at the place where you are using Ive inside the method or function.
# Assume `/home/user/projects/.../nsx/utils.py` file
# Don't import Ive here
# from ive.models import Ive
def func():
# import here
from ive.models import Ive
# use Ive here
I meet a similar problem which can be simplified as following:
For example I have a file structure as following:
----folder
---- main.py
---- math.py
I define a function in math.py and I want to import this math.py in main.py .
The codes in math.py is following
# math.py
def f(x) :
return x**3
If I write codes in main.py as following
# main.py
import math
def main() :
print(math.f(3))
if __name__ == "__main__":
main()
then it returns AttributeError: module 'math' has no attribute 'f'
If I write codes in main.py as following
# main.py
from . import math
def main() :
print(math.f(3))
if __name__ == "__main__":
main()
Then it returns ImportError: cannot import name 'math' from '__main__' (main.py)
My question:
If I only want to import the module math.py in path folder which has the same name as build-in module, what should I do?
If in the main.py I want to use both math.f(x) defined in my math.py and built-in math.acos(x), what should I do?
PS: I meet similar problem since I have a long codes written by someone ten years ago. At that time there is no built-in module with such name (In fact it's not math module. I just simplify the problem by above question). And the functions of this module have been used at many places. Therefore it's almost impossible to change module's name since if so I need to carefully change all sites module.function().
It's pretty bad practice to name your modules after built-in modules. I'd recommend naming your math.py something else.
That being said, you could import it using the path with imp:
import imp
math = imp.load_source('math', './math.py')
Dove into a bit of a rabbit hole but here we go. As a disclaimer, like Jack said naming modules after builtins is very bad practice, and this can more easily be accomplished using imp as he suggested.
The reason you're having problems come from the interaction of a few things. When you type
import math
What your python does is look at sys.path. It will check in all the locations in sys.path for a module named math, and import the first one it finds. In your case, it finds your local math module first. After the import is completed, it adds it to sys.modules, but we'll get back to that.
Since you want to use both, first you can import your local math as you have. I would suggest importing it as a different name to keep it separate.
from . import math as local_math
After that, we need to mess with our sys.path to find the builtin math
sys.path = sys.path[::-1]
This reverses the order of sys.path, meaning it will look in your local directory first instead of last.
Now you might think this was enough, but if you try to import math here, python will see it in sys.modules and not bother trying to import again, so first
del sys.modules['math']
Then we can import the default math module
import math
and finally clean up our sys.path
sys.path = sys.path[::-1]
now we have access to everything we need
>>>math.cos(10)
-0.8390715290764524
>>>local_math.f(10)
1000
According to official docs: https://docs.python.org/3/library/imp.html
imp will be removed in version 3.12
Deprecated since version 3.4, will be removed in version 3.12: The imp module is deprecated in favor of importlib.
From imp approach (Deprecated):
import imp
m = imp.load_source('math', './math.py')
m.foo()
To importlib approach with the minimum code 'impact' would be:
def load_module(name, path):
from importlib import util as importlib_util
module_spec = importlib_util.spec_from_file_location(name, path)
module = importlib_util.module_from_spec(module_spec)
module_spec.loader.exec_module(module)
return module
m = load_module('math', './math.py')
m.foo()
I came across some code that would not work anymore in python 3.6, but did good in all versions before. I found out the problem is actually a field containing a ZipFile somewhere in a class. Here is a short program which raises the error:
from pickle import dumps
import io
import zipfile
raw = b""
foo = zipfile.ZipFile(io.BytesIO(raw), mode="w")
dumps(foo)
I get this error:
Traceback (most recent call last):
File "bla.py", line 8, in <module>
dumps(foo)
TypeError: can't pickle _thread.RLock objects
So the test program can be even shorter:
from pickle import dumps
import threading
dumps(threading.RLock())
I diffed both the python 3.5 and 3.6 zipfile.py but can not spot any difference in respect to the _lock field in ZipFile, so it seems that there are changes in the threading module - but in threading.py there are also no obvious changes between the versions.
Why is it not pickable anymore? Do I need to do something before I can pickle a ZipFile?
Edit: ok after searching now for a while, I stumbled across this python bug tracker entry: https://bugs.python.org/msg284751
So that a ZipFile is pickable in python <3.6 is actually the bug...
I think I need to change a lot of code now...
Just to give an answer to this question: That ZipFile objects are pickable is actually a bug: https://bugs.python.org/msg284751 which has been fixed in py 3.6.
I have searched this site top to bottom yet have not found a single way to actually accomplish what I want in Python3x. This is a simple toy app so I figured I could write some simple test cases in asserts and call it a day. It does generate reports and such so I would like to make sure my code doesn't do anything wonky upon changes.
My current directory structure is: (only relevant parts included)
project
-model
__init__.py
my_file.py
-test
my_file_test.py
I am having a hell of a time getting my_file_test.py to import my_file.py.
Like I've said. I've searched this site top to bottom and no solution has worked. My version of Python is 3.2.3 running on Fedora 17.
Previously tried attempts:
https://stackoverflow.com/questions/5078590/dynamic-imports-relative-imports-in-python-3
Importing modules from parent folder
Can anyone explain python's relative imports?
How to accomplish relative import in python
In virtually every attempt I get an error to the effect of:
ImportError: No module named *
OR
ValueError: Attempted relative import in non-package
What is going on here. I have tried every accepted answer on SO as well as all over the interwebs. Not doing anything that fancy here but as a .NET/Java/Ruby programmer this is proving to be the absolute definition of intuitiveness.
EDIT: If it matters I tried loading the class that I am trying to import in the REPL and I get the following:
>>> import datafileclass
>>> datafileclass.methods
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
>>> x = datafileclass('sample_data/sample_input.csv')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'module' object is not callable
If it matters...I know the functionality in the class works but I can't import it which in the now is causing an inability to test. In the future will certainly cause integration issues. (names changed to protect the innocent)
getting within a couple of weeks of desired functionality for this iteration of the library...any help could be useful. Would have done it in Ruby but the client wants the Python as a learning experience,
Structure your code like this:
project
-model
__init__.py
my_file.py
-tests
__init__.py
test_my_file.py
Importantly, your tests directory should also be a module directory (have an empty __init__.py file in it).
Then in test_my_file.py use from model import my_file, and from the top directory run python -m tests.test_my_file. This is invoking test_my_file as a module, which results in Python setting up its import path to include your top level.
Even better, you can use pytest or nose, and running py.test will pick up the tests automatically.
I realise this doesn't answer your question, but it's going to be a lot easier for you to work with Python standard practices rather than against them. That means structuring your project with tests in their own top-level directory.