I think my doubt can be best explained with an example.
# module1
def create_logger():
return <some_logger_object>
def access_dir(dir):
# do something with the dir
a = create_dir()
b = access_the_dir(#some env var)
# module2
from module1 import a
a.info("something")
Now when I am trying to import only 'a' from module1, still 'b' is being called and it is giving me an error because the code can't find the env var at that time.
But I need to have the global variable 'b' as it is being used somewhere else in the code. But in one part of the logic, I don't need 'b' and can't really initialize it because the env var comes from a job that hasn't been run yet.
Another problem is that create_logger() and access_dir() are both basically part of the same business logic. Hence moving them to a different module doesn't really makes any sense.
So, is there any other way it can be achieved? I need to import one global variable from a module without initializing others.
I want to write functions in external files, as it is more convenient for edition, and use global variables.
Apparently the only way to do that is to use import some_function from some_file (right?). Is it possible to still use global variables in this way? That is variables declared in the main file and directly accessible in the external file? I also try to avoid passing them in arguments as it complicates the code. I was thinking about some "include" instruction but I'm not sure it exists in Python.
So the code in the main file would be this:
from test import test
x=1
test()
and in the file test.py it would be this:
def test():
global x
print(x)
Maybe this is just a problem of having the right editor...Has anyone a recommandation for MacOS?
python's import is pretty much equivalent to include in other languages, especially in the form of from some_file import * which imports all of the namespace, including functions, classes, and all global variables in that module or package.
Edit: However, if you want to do what you requested in your comments, that can still be done with imported variables. Let's for example consider 2 files, main.py and imported.py.
imported.py might look like this:
some_global_var = 1
other_var = 2
def add():
return some_global_var + other_var
Because imported.py has functions that use global variables (instead of arguments), there's no reason you can't change those variables once imported. To do that, main.py can look like this:
import imported
print(imported.add()) # 3 - because we didn't change anything yet
imported.some_global_var = 10
imported.other_var = 20
print(imported.add()) # 30 - because we redefined the imported variables that our imported function uses
I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.
Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()
As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.
This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'
A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.
The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()
Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())
Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12
The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.
Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.
Imagine a situation, in which you have two files say p1.py and p2.py.
p1.py:
a=5
p2.py:
from p1 import *
print(a) # NameError: name 'a' is not defined
print(p1.a) # NameError: name 'p1' is not defined
The first print statement is understandable. I am boggling over second print statement.
from p1 import *
should import everything inside p1.py, so why am I not able to access p1.a variable. This is not the end, interesting part starts from here. Now consider the below-given modification:
import p1
print(p1.a) # prints a = 5
So what am I missing here? In this answer from #ThiefMaster he says that
from foo import * imports a to the local scope. When assigning a value to a, it is replaced with the new value - but the original foo.a variable is not touched.
So unless you import foo and modify foo.a, both calls will return the same value.
So I modified p1.a but why isn't it working?
You wrote "from p1 import *
should import everything inside p1.py, so why am I not able to access p1.a variable"
It does indeed import everything from inside the p1 module, but the module p1 does not contain any variable named p1, so after this import statement p1 is undefined. If instead you did import p1 then you could access p1.a1.
If as you claim, the module p1 contains a=5 then contrary to what you wrote your first print statement print(a) will succeed as from p1 import * creates a variable inside the main module with the same name and value as the variable a inside p1.
These are the two way of importing a module.
import foo
from foo import foo1, foo2
What is a Module?
A module is a file containing Python definitions and statements.
Within a module, you can access module's name using global variable __name__.
As you can see module contains Python definitions and statements which in short means executable statements such as functions, classes etc. They are used to initialize the module. And so, they will be executed the first time they are imported.
Each module has it's own PRIVATE SYMBOL TABLE which is used by all the functions and variables inside the module. This prevents accidental clashes of variables used by the author of module and user.
So in your case, p1.py and p2.py both will have its own private table where the list of all variables which can be used by the respective script.
When you do something like this:
import p1
it creates a table in which module name is also introduced i.e. p1.
The other variant:
from p1 import *
does indeed creates a table, but it fails to import modules' name in its symbol table. In other words, p1 is missing. This includes everything in the table except modules' name and variable which starts with an underscore (_).
In most cases Python programmers do not use this facility since it introduces an unknown set of names into the interpreter, possibly hiding some things you have already defined.
Note that in general the practice of importing * from a module or package is frowned upon since it often causes poorly readable code. However, it is okay to use it to save typing in interactive sessions.
This may help.
I am confused about some behavior of Python. I always thought importing a module basically meant executing it. (Like they say here: Does python execute imports on importation) So I created three simple scripts to test something:
main.py
import config
print(config.a)
config.a += 1
print(config.a)
import test
print(config.a)
config.py
def get_a():
print("get_a is called")
return 1
a = get_a()
test.py
import config
print(config.a)
config.a += 1
The output when running main.py is:
get_a is called
1
2
2
3
Now I am confused because I expected get_a() to be called twice, once from main.py and once from test.py. Can someone please explain why it is not? What if I really wanted to import config a second time, like it was in the beginning with a=1?
(Fortunately, for my project this behavior is exactly what I wanted, because get_a() corresponds to a function, which reads lots of data from a database and of course I only want to read it once, but it should be accessible from multiple modules.)
Because the config module is already loaded so there's no need to 'run' it anymore, just return the loaded instance.
Some standard library modules make use of this, from example random. It creates an object of class Random on first import and reuses it when it gets imported again. A comment on the module reads:
# Create one instance, seeded from current time, and export its methods
# as module-level functions. The functions share state across all uses
#(both in the user's code and in the Python libraries), but that's fine
# for most programs and is easier for the casual user than making them
# instantiate their own Random() instance.