I am modifying a module which contains a class in it.
When I %run another module that uses the mofified class from IPython, the changes do not seem to take effect unless I restart IPython.
I have tried to use imp.reload, but this does not help. For example, I have put the following the code at the top of my module, but it does not appear to be using the updated version of my modified class (BigMySQLDatabaseGetter in the big_mysql_database_getter module)
import imp
import sys
from big_mysql_database_getter import BigMySQLDatabaseGetter
module_big_mysql_database_getter = sys.modules['big_mysql_database_getter']
imp.reload(module_big_mysql_database_getter)
Reloading a module doesn't automatically update all references that were created before, it just redefines everything within the module.
So if you do something like:
from spam import eggs
imp.reload(spam)
print(spam.eggs is eggs)
you'll get False, as eggs still references the old class. Likewise, instances created before the reload are instances of the old class, not of the new class:
import spam
e = spam.eggs()
imp.reload(spam)
print(isinstance(e, spam.eggs)) # False!
In your case, you can either reimport BigMySQLDatabaseGetter after reloading the module, or instead of directly importing the class just import the module and use big_mysql_database_getter.BigMySQLDatabaseGetter instead.
Related
this is such a simple problem but I cant seem to find any direct explanation to this.
in module.py
def foo():
print("foo")
in main.py
import module
foo()
it will result in an error saying that foo is not defined? when i look for the answer online, I can't find anything surprisingly
I'm not planning to use things like
from x import y
just straight up the import system
When you import an external module, it generates a variable named module that contains all classes, functions and variables from the module. To acess 'foo' function you need to first acess the module:
module.foo()
To import 'foo' function you can import everything from the module, like this:
from module import *
Now you can simply do: foo()
You can also set a custom name to the module, like:
import module as M
And now you can run 'foo' like this:
M.foo()
PS: I'm not english native
The statement
import module
makes the name of module module available. So you can use module.foo().
If you want to call foo() without "qualifying" it:
from module import foo
or
from module import *
but that latter is bad idea because you are liable to import unexpected names, which may collide with other names you imported from other modules.
from model import foo
is the preferred way as in any kinds of
from model import *
you (and anyone ever working on that code) has no idea what has been imported. Could even lead to name conflicts.
I want my project to work in two different situations. It should work as a standalone library, but also as sub package of a larger project. The main use case is that of a standalone library, where its internal imports should be of the form
from my_library import sub_package
When using the code as sub package of a larger project, these imports don't work as there is no global name my_library. Instead, I would have to use relative or absolute imports, for example
from large_project.my_library import sub_package
Let's assume I wrote my library as shown in the first example. How can I overwrite importing behavior when running as part of a larger project to automatically adjust import paths?
Thanks to #MatrixTai's suggestion of adding the parent directory of the package to the the module path, I came up with this dynamic solution. At the top of my_library/__init__.py:
# Make package global even if used as a sub package to enable short imports.
import os
import sys
sys.path.append(os.path.dirname(os.path.dirname(__file__)))
We have to navigate up two directories here to do from the my_library/__init__.py to my_library and from there to its parent direction, where imports will find the library.
You don't have much choice.
If you want to reference the my_library.py anywhere, there is 2 method (as I known) can do similar work.
1: add system path. Like those module you installed by pip. pip module is installed in /Python/Scripts. You can add a new path or simply put my_library.py into one of the path. For adding, that's in Computer(right-click)-> Properties -> Environment Variable -> Choose Path and Click Edit
(Though you may not want to use this.)
2: Changing __init__.py, but still at least one line you must add in my_library.py.
For example,
/Directory
/large_project
-__init__.py #call this sub_init
-my_library.py
-__init__.py #call this main_init, this fake
-main.py
In main_init,
import sys
sys.path.append('\\Directory\\large_project')
As main_init is not executed when you execute main.py (this is fake), so in main.py
import __init__
from my_library import sub_package
But as well you can take this main_init as the starter of library, like declaring __all__, etc.
I'm developing a small app using kivy and python3.6 (I'm still a beginner). I'm planning to separate the code in different files for clarity, however I have encountered a problem in a specific situation. I have made minimal working example to illustrate.
I have the following files:
main.py
main.kv
module.py
module.kv
Here a minimal code:
main.py:
from kivy.app import App
from kivy.uix.button import Button
from kivy.lang import Builder
import module
Builder.load_file('module.kv')
class MainApp(App):
pass
def function():
print('parent function')
if __name__ == '__main__':
MainApp().run()
main.kv:
CallFunction
module.py:
from kivy.uix.button import Button
class CallFunction(Button):
def call_function(self):
from main import function
function()
module.kv:
<CallFunction>:
id : parent_button
text: 'Call parent button'
on_press: self.call_function()
So the problem is that when I run this code, I receive a warning
The file /home/kivy/python_exp/test/module.kv is loaded multiples times, you might have unwanted behaviors.
What works:
If the function I want to call is part of the main app class, there is no problem
If the function is part of the module.py there is no problem
If the function is part of another module, there is no problem
What doesn't work
I cannot call a function which is in the main.py. If I use the import the function as the beginning of module.py, kivy has a weird behavior and call everything twice. Calling within this call_function allows to have a proper interface, but I get the warning that the file has been loaded multiple time.
There are easy workarounds, I'm well aware of that, so it's more about curiosity and understanding better how the imports in kivy works. Is there a way to make it work?
I wanted to use the main.py to initialize different things at the startup of the app. In particular I wanted to create an instance of another class (not a kivy class) in the main.py and when clicking on the button on the interface, calling a method on this instance.
Thanks :)
When you import something from another python module the python virtual machine execute this module. In the call_function you import function from the main file so everytime you press the CallFunction the module.kv is loaded.
To solve this it is recommended to include the other kv files in your main kv file.
You can also move the import statement from the method to the top of the module file.
Your kv file is loaded twice because the code is executed twice. This is due to how pythons module system works and kivy just realized that loading the kv twice is probably not what you want.
Generally python objects live in a namespace. So when a function in the module foo looks up a variable the variable is searched in the namespace of the module. That way if you define two variables foo.var and bar.var (in the modules foo and bar resp.) they don't clash and get confused for each other.
The tricky thing is that the python file you execute is special: It does not create a module namespace but the __main__ namespace. Thus if you import the file you are executing as __main__ it will create a whole new namespace with new objects and execute the module code. If you import a module that was already imported in the current session the module code is not executed again, but the namespace already created is made available. You don't even need two files for that, put the following in test.py:
print("hello!")
print(__name__)
import test
If you now execute python test.py you will see two hello! and once __main__ and once test.
You can find more information on namespaces and how variable lookups works in python in the documentation.
Also if your function actually does some work and mutates an object that lives in main.py you might want to rethink the information flow. Often it is a good idea to bind the state and functions working on them together in classes and passing the objects then where they are called i.e. to CallFunction in your example.
I am a new comer when it comes to package and module to python.
I am trying to cut my script in several separate compartiment. to improve readability and maintenance.
My problem is the following:
I have a module which define a class and a function inside this module which instantiate this class.
module blast.py
class Blast():
blabla
def foo():
blast = Blast()
# do some stuff
this module is inside a package with a _ _init__.py file
__all__ = ["blast"]
I have a main script In which i want to use that function.
I import the module with
from package import blast
But To use that function I have to use the name space of the module ( at least my IDE say me that: pycharm)
blast.foo()
So does it works? does the function will see the class inside it module?
And more generally Could I import some function of my package inside my namespace. I though it was done this way and answer I got from internet doesn't really help me.
Yes, the function blast.foo() would know and find the class Blast.
Whenever you import a module, in part or in its entirety, the entire module is loaded - the way you import it merely decides on what classes and functions are available in the current scope, and in what way.
For example, if you call this:
from package.blast import foo
only the function foo() would be available, but the entire package read and loaded. if you were to try instantiate the class Blast by itself in the same script, this would not work.
By the way, you can make importing functions more convenient, if you customize __init__.py. In your case, if you were to edit it like so:
>>>__init__.py
from blast.py import Blast, foo
you can import both function and class like so:
from package import Blast, foo
The reason why you __all__ parameter does not work, is because it requires another import statement - mainly this:
from package import *
Calling this with your current __init__.py should work as expected.
Perhaps this post by Mike Grouchy is able to clarify things a bit more.
Given this simple folder structure
/main.py
/project/a.py
/project/b.py
main.py is executed by the python interpreter and contains a single line, import project.a.
a and b are modules, they need to import each other. A way to achieve this would be
import project.[a|b]
When working with deeper nested folder structures you don't want to write the entire path everytime you use a module e.g.
import project.foo.bar
project.foo.bar.set_flag(project.foo.bar.SUPER)
Both from project import [a|b] and import project.[a|b] as [a|b] result in an import error (when used in both, a and b).
What is different between the standart import syntax and the from or as syntax? Why is only the standart syntax working for mutual imports?
And more importantly, is there a simple and clean way to import modules that allows mutual imports and assigning shorter names to them (ideally the modules basename e.g. bar in the case of project.foo.bar)?
When you do either import project.a or from project import a, the following happens:
The module object for project.a is placed into sys.modules. This is a dictionary that maps each module name to its module object, so you'll have sys.modules = {..., 'p.a': <module 'p.a' from '.../project/a.py'>, ...}.
The code for the module is executed.
The a attribute is added to project.
Now, here is the difference between import project.a and from project import a:
import project.a just looks for sys.modules['project.a']. If it exists, it binds the name project using sys.modules['project'].
from project import a looks for sys.modules['project'] and then checks if the project module has an a attribute.
You can think of from project import a as an equivalent to the following two lines:
import project.a # not problematic
a = project.a # causes an error
That why you are seeing an exception only when doing from project import a: sys.modules['project.a'] exists, but project does not yet have a a attribute.
The quickest solution would be to simply avoid circular imports. But if you can't, then the usual strategies are:
Import as late as possible. Suppose that your a.py looks like this:
from project import b
def something():
return b.something_else()
Rewrite it as follows:
def something():
from project import b
return b.something_else()
Of course, you would have to repeat imports in all your functions.
Use lazy imports. Lazy imports are not standard feature of Python, but you can find many implementations around. They work by using the "import as late as possible" principle, but they add some syntactic sugar to let you write fewer code.
Cheat, and use sys.modules, like this:
import sys
import project.a
a = sys.modules['project.a']
Very un-pythonic, but works.
Obviously, whatever solution you choose, you won't be able to access the attributes from a or b until the modules have been fully loaded.