Related
I'm aware of the SO post How do I override file.write() in Python 3? but after looking it over and trying whats suggested I'm still stuck.
I want to override the file.write method in Python 3 so that I can "REDACT" certain words (Usernames, Passwords...etc).
I found a great example of overriding the print and general stdout and stderr http://code.activestate.com/recipes/119404/
The issue is that it doesn't work for file.write. How can I override the file.write?
My code for redacting when printing is:
def write(self, text):
for word in self.redacted_list:
text = text.replace(word, "REDACTED")
self.origOut.write(text)
return text
thanks
From the self.origOut.write(text) I assume you are trying to write an in-between-class that pretends to be a file but provides a different .write() method.
I don't see any problems in the code you posted (assuming it's a method of a class you use). Possibly you wrote a class but forgot to create instances of it?
Did you try to write something like this?:
class IAmNoARealFile:
def __init__(self, real_file):
self.origOut = real_file
def __getattr__(self, attr_name): # provide everything a file has
return getattr(self.origOut, attr_name)
def write(self, ...):
...
with open('test.txt', 'w') as f:
f = IAmNotARealFile(f) # did you forget this?
f.write('some text SECRET blah SECRET') # calls IAMNotARealFile.write with your extra code
with open('test.txt') as f:
f = IAmNotARealFile(f)
print(f.read()) # this "falls through" to the actual file object
you will also probably want to return self.origOut.write() in your own .write(), if you don't have a specific reason not to.
Note that if you rewrite open() to directly return IAMNotARealFile:
def open(*args, **kwargs):
return IAMNotARealFile(open(*args, **kwargs))
you will have to manually supply (some) "magic methods" because
This method may still be bypassed when looking up special methods as the result of implicit invocation via language syntax or built-in functions. See Special method lookup.
--docs for .__getattribute__(), but it also applies to .__getattr__()
Why?
Bypassing the __getattribute__() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter).
-- On special ("magic") method lookup [code style and emphasis mine]
This question already has answers here:
How do I type hint a method with the type of the enclosing class?
(7 answers)
Closed 3 years ago.
class Node:
def append_child(self, node: Node):
if node != None:
self.first_child = node
self.child_nodes += [node]
How do I do node: Node? Because when I run it, it says name 'Node' is not defined.
Should I just remove the : Node and instance check it inside the function?
But then how could I access node's properties (which I would expect to be instance of Node class)?
I don't know how implement type casting in Python, BTW.
"self" references in type checking are typically done using strings:
class Node:
def append_child(self, node: 'Node'):
if node != None:
self.first_child = node
self.child_nodes += [node]
This is described in the "Forward references" section of PEP-0484.
Please note that this doesn't do any type-checking or casting. This is a type hint which python (normally) disregards completely1. However, third party tools (e.g. mypy), use type hints to do static analysis on your code and can generate errors before runtime.
Also, starting with python3.7, you can implicitly convert all of your type-hints to strings within a module by using the from __future__ import annotations (and in python4.0, this will be the default).
1The hints are introspectable -- So you could use them to build some kind of runtime checker using decorators or the like if you really wanted to, but python doesn't do this by default.
Python 3.7 and Python 4.03.10 onwards
PEP 563 introduced postponed evaluations, stored in __annotations__ as strings. A user can enable this through the __future__ directive:
from __future__ import annotations
This makes it possible to write:
class C:
a: C
def foo(self, b: C):
...
Starting in Python 3.10 (release planned 2021-10-04), this behaviour will be default.
Edit 2020-11-15: Originally it was announced to be mandatory starting in Python 4.0, but now it appears this will be default already in Python 3.10, which is expected 2021-10-04. This surprises me as it appears to be a violation of the promise in __future__ that this backward compatibility would not be broken until Python 4.0. Maybe the developers consider than 3.10 is 4.0, or maybe they have changed their mind. See also Why did __future__ MandatoryRelease for annotations change between 3.7 and 3.8?.
In Python > 3.7 you can use dataclass. You can also annotate dataclass.
In this particular example Node references itself and if you run it you will get
NameError: name 'Node' is not defined
To overcome this error you have to include:
from __future__ import annotations
It must be the first line in a module. In Python 4.0 and above you don't have to include annotations
from __future__ import annotations
from dataclasses import dataclass
#dataclass
class Node:
value: int
left: Node
right: Node
#property
def is_leaf(self) -> bool:
"""Check if node is a leaf"""
return not self.left and not self.right
Example:
node5 = Node(5, None, None)
node25 = Node(25, None, None)
node40 = Node(40, None, None)
node10 = Node(10, None, None)
# balanced tree
node30 = Node(30, node25, node40)
root = Node(20, node10, node30)
# unbalanced tree
node30 = Node(30, node5, node40)
root = Node(20, node10, node30)
If you just want an answer to the question, go read mgilson's answer.
mgilson's answer provides a good explanation of how you should work around this limitation of Python. But I think it's also important to have a good understanding of why this doesn't work, so I'm going to provide that explanation.
Python is a little different from other languages. In Python, there's really no such thing as a "declaration." As far as Python is concerned, code is just code. When you import a module, Python creates a new namespace (a place where global variables can live), and then executes each line of the module from top to bottom. def foo(args): code is just a compound statement that bundles a bunch of source code together into a function and binds that function to the name foo. Similarly, class Bar(bases): code creates a class, executes all of the code immediately (inside a separate namespace which holds any class-level variables that might be created by the code, particularly including methods created with def), and then binds that class to the name Bar. It has to execute the code immediately, because all of the methods need to be created immediately. Because the code gets executed before the name has been bound, you can't refer to the class at the top level of the code. It's perfectly fine to refer to the class inside of a method, however, because that code doesn't run until the method gets called.
(You might be wondering why we can't just bind the name first and then execute the code. It turns out that, because of the way Python implements classes, you have to know which methods exist up front, before you can even create the class object. It would be possible to create an empty class and then bind all of the methods to it one at a time with attribute assignment (and indeed, you can manually do this, by writing class Bar: pass and then doing def method1():...; Bar.method1 = method1 and so on), but this would result in a more complicated implementation, and be a little harder to conceptualize, so Python does not do this.)
To summarize in code:
class C:
C # NameError: C doesn't exist yet.
def method(self):
return C # This is fine. By the time the method gets called, C will exist.
C # This is fine; the class has been created by the time we hit this line.
I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.
Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()
As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.
This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'
A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.
The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()
Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())
Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12
The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.
Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.
I am trying to use jedi to complete python code inside a PyQt application, using QCompleter and QStringListModel to store the possible completion.
Here's a simple working demo:
#!/usr/bin/env python3
from PyQt5.QtCore import *
from PyQt5.QtWidgets import *
import jedi
import sys
class JediEdit(QLineEdit):
def __init__(self, parent=None):
super().__init__(parent)
self._model = QStringListModel()
self._compl = QCompleter()
self._compl.setModel(self._model)
self.setCompleter(self._compl)
self.textEdited.connect(self.update_model)
def update_model(self, cur_text):
script = jedi.Script(cur_text)
compl = script.completions()
strings = list(cur_text + c.complete for c in compl)
self._model.setStringList(strings)
if __name__ == '__main__':
app = QApplication(sys.argv)
line = JediEdit()
line.show()
sys.exit(app.exec_())
If you run the application and write a code which is not completing anything (e.g. or foo =), the completion will actually show all the possible tokens that can go in that position.
So, if I run and write a space in the field, lots of things pops up, from abs to __version__.
I would like to prevent this: is it possible to query jedi.Script to understand if the token is being completed or if a completely new token is starting?
EDIT: another little question: say that I am running an interpreter which is detached from jedi current state. How can I provide local and global variables to jedi.Script so that it will take into account those, instead of its own completions?
Autocompletion
Jedi's autocompletion will always show all possible tokens in a place. That's the whole point in autocompletion.
If you don't want that behavior just scan the last few characters for whitespace and certain other characters like = or :, it would be a very simple regex command. (You could also try to look up Jedi's internals and use the way how Jedi knows about this context. However I'm not going to tell you, because it's not a public API and IMHO regex calls suffice.)
In the future something like that might be possible. (See https://github.com/davidhalter/jedi/issues/253).
Now that I think about it, there might be another way that you could experiment with this: You can try to play with Completion.name and Completion.complete. The latter only gives you what could come after the current token, while the name would be the full thing. So you can compare and if they are equal than you might not want to display anything.
Have fun playing with the API :-)
Interpreter
If you're running an interpreter, you can use jedi.Interpreter to combine code with actual Python objects. It's pretty flexible. But please note that the current Interpreter (0.8.1) is very buggy. Please use the master branch from Github (0.9.0).
I'm doing an extended project as one of my qualifications in my current College and I chose to write a python Strategy/RPG game. As a result, I ended up with the highest level of Python knowledge (Surpassing my Computing Teacher who only ever uses the basics... and used Tkinter only once a few years ago. Every one else who has decided to make a program, are either coding in Lua, Java, C++, HTML/CSS/Java-Script or, those who are coding in python, they are only using the basics learned from our teacher.)
I say "Highest level of Python knowledge" but really it isn't that high... I only know a little beyond the basics.
As a result, a forum post is the best place I can turn to for help.
So in my game I defined this function:
#"Given_String" is the question that one would want to ask. (With the answer being an integer between 1 and "Choice_Range" (inclusive)
def Value_Error(Given_String,Error_Message,Choice_Range):
while True:
try:
Temp=int(input(Given_String))
if Temp<1 or Temp>Choice_Range:
print(Error_Message)
else:
break
except ValueError:
print(Error_Message)
return Temp
I then wanted to add tkinter to my code, because the game would have to be in a separate window, and not in the console. As a result, I had to change this code so that it displays the "Given_Message" and the "Error_Message" in a tkinter window, and uses the value that has been typed into an entry box when defining "Temp".
I wrote this code to make this work: (Or at least most of it)
#This code is stored in a different file for neatness and hence I had to import "sys" to avoid circular imports.
#This code is made to be flexible so that I can later re-use it when necessary.
#This code starts with the function all the way at the bottom. The rest are made to add flexibility and to structure the algorithm.
#This code hasn't been fully run (Because of the Error crashing the Python Shell) so it can contain other Run-time Errors that I'm not aware of yet.
import sys
def Generate_Window(Window_Name,X_Parameter=5,Y_Parameter=50):
Temp=sys.modules['tkinter'].Tk()
Temp.title(Window_Name)
Temp.geometry(str(X_Parameter)+"x"+str(Y_Parameter))
return Temp
def Generate_Button(Master,Text="Submit"):
Temp=sys.modules["tkinter"].Button(Master,text=Text)
return Temp
def Generate_Entry(Master):
Temp=sys.modules["tkinter"].Entry(Master)
return Temp
def Generate_Label(Master,Given_String):
Temp=sys.modules["tkinter"].Label(Master,text=Given_String)
return Temp
def Com_Get_Entry(Given_Window,Given_Entry):
Temp=Given_Entry.get()
Given_Window.destroy()
return Temp
def Com_Confirm(Given_Window):
Given_Window.destroy()
def Generate_Entry_Box(Given_String):
Entry_Window=Generate_Window("Entry",X_Parameter=300)
Entry_Label=Generate_Label(Entry_Window,Given_String)
Entry_Entry=Generate_Entry(Entry_Window)
Entry_Button=Generate_Button(Entry_Window)
Entry_Button.configure(command=lambda:Com_Get_Entry(Entry_Window,Entry_Entry))
Entry_Label.grid(row=0,columnspan=2)
Entry_Entry.grid(row=1,column=0)
Entry_Button.grid(row=1,column=1)
def Generate_Alert_Message(Given_String):
Alert_Window=Generate_Window("Alert",X_Parameter=300)
Alert_Label=Generate_Label(Alert_Window,Given_String)
Alert_Button=Generate_Button(Alert_Window,Text="OK")
Alert_Button.configure(command=lambda:Com_Confirm(Alert_Window))
Alert_Label.grid(row=0,columnspan=2)
Alert_Button.grid(row=1,column=1)
def Get_Interger_Input_In_Range(Given_String,Error_Message,Choice_Range):
while True:
try:
Returned_Value=int(Generate_Entry_Box(Given_String))
if Returned_Value<1 or Returned_Value>Choice_Range:
Generate_Alert_Message(Error_Message)
else:
break
except ValueError:
Generate_Alert_Message(Error_Message)
return Temp
I already included in my code all that I was struggling with and that I could find an answer to.
I.E: On-click, do a certain action with given parameters.
One thing I could not find, is how to return the entered value to the original (Get_Interger_Input_In_Range()) function after the button has been clicked.
What I mean is something like this:
def Function1(GivenParameter1,GivenParameter2):
Temp=Function2(GivenParameter1)
Temp+=GiverParameter2 #random action
return Temp
def Function2(GivenParameter):
Button=Button(Master,command=Function3).grid()
Entry=Entry(Master).grid()
def Function3():
Temp=Entry.get()
return Temp
In Function1 I want Temp to equal the entered value from Function2.
Is there any way to do this without using classes? (I'm not too familiar with classes yet)
Is there any way to do this at all?
I haven't seen anyone give the answer I was looking for...
Because even if they said to use classes... I still didn't know how to return it (Explanation just below)
#The following code was written quickly for purposes of explaining what I mean. It doesn't actually work... (It seems that the button command is being called automatically...)
from tkinter import *
class Return_Value_In_Entry():
def __init__(self):
self.Master=Tk()
self.Entry=Entry(self.Master)
self.Button=Button(self.Master,text="Submit",command=self.Return())
def Return(self):
self.TempVar=self.Entry.get()
return self.TempVar
The way I see it, the Return() function would return the value to the button and not the function/assignment that called the class ... Which is the same problem I'm having with my code.
If you read this all then I really appreciate it. I hope someone can answer my question and tell me (if it's impossible otherwise) how to use classes to solve my "Little" yet large problem.
I fixed your example code (I think). The main problem is that this:
command=self.Return()
does not do what you think it does. It just assigns return value from Return() to command. This is incorrect. It should be
command=self.Return
This assigns function Return to command. Subsequently, when ever button is pressed, self.Return() is executed.
The full example is here:
from tkinter import *
class Return_Value_In_Entry():
def __init__(self):
self.Master=Tk()
self.Entry=Entry(self.Master)
self.Entry.pack()
self.Button=Button(self.Master,text="Submit",command=self.Return)
self.Button.pack()
self.Master.mainloop()
def Return(self):
self.TempVar=self.Entry.get()
print(self.TempVar)
Return_Value_In_Entry()
Now, whenever you press the Button, the value from the Entry widget is saved into self.TempVar and printed out, just to check if its working. Hope this helps.
Gif showing how the example program works: