I wrote an application with tkinter that uses classes as structure. Currently I try to implement a "save"-function, so that one can close the application and if it's restarted decide whether to start from scratch or to use already existing aka. pickled data.
I read about the pickle-module and its weakness that it can't save class variables so I decided to use the dill-module instead.
My Problem: dill saves the start values of my class variables, but if I let the user change the class variables (by calling classmethods), the changes are not saved. Is there any way how one can do this with the pickle or dill module?
I created a minimal example that produces the error. You can see that the changes of the instance variables are saved even after calling instance methods that change the variables. Unfortunately it doesn't work with the class variable and it's corresponding class method:
import dill as pickle
from tkinter import *
root = Tk()
# creating class
class Roomie:
# class variable: total number of expenses
num_exp = 0
# init method: every roomie has a name, info for each expense, etc...
def __init__(self,fname,exp=0.00):
self.fname = fname
self.exp = exp
def raiseExp(self):
self.exp += 1
# class method: every time someone spend smth value of class variable num_exp increases by one
#classmethod
def updateExpNum(cls):
cls.num_exp += 1
# saving instance of class in list object
roomie_0 = Roomie("Cthulhu")
roomie_list = [roomie_0]
def pickleRoomies():
file_Name = "debts.pkl"
# open file for writing
fileObject = open(file_Name,'wb')
# write roomie_list to file
pickle.dump(roomie_list,fileObject)
# here we close the fileObject
fileObject.close()
# load all instances and class variables
def loadPickle():
global roomie_list
filepath = "debts.pkl"
file = open(filepath, "rb" )
roomie_list = pickle.load(file)
# create buttons and bind instance and class methods to them
button_0 = Button(root,text="update number of expenses",command=lambda: Roomie.updateExpNum())
button_1 = Button(root,text="raise expense",command=lambda: roomie_list[0].raiseExp())
button_2 = Button(root,text="save changes",command=lambda: pickleRoomies())
button_3 = Button(root,text="load changes",command=lambda: loadPickle())
button_0.pack()
button_1.pack()
button_2.pack()
button_3.pack()
# run GUI
root.mainloop()
# check program
# print instance variable
print("total expenses:",roomie_list[0].exp)
# print class variable
print("number of expenses:", Roomie.num_exp)
# print instance variable
print("name:", roomie_list[0].fname)
Related
How to call a variable within a function:
What is the Right approach:
Should define the variable before class or
We can define it under the class or function
Sample Code:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import time
timeStamp = time.strftime("%Y%m%d%H%M%S") # <-- Is this right Approach
class Scanner:
INFO = 0
DEBUG = 3
timeStamp = time.strftime("%Y%m%d%H%M%S") # <-- ?
def __init__(self, config_file, verbose=False):
""" Constructor """
def ask_passwords(self):
def ldap_init(self):
def hosts_module_scanner(self):
def users_module_scanner(self):
def ldap_reporting(self, user_list):
self.write_report(failed_users, "users_ldap_report-{}.txt".format(timeStamp))
def option_parser(prog_version):
if __name__ == '__main__':
scanner.ask_passwords()
scanner.ldap_init()
scanner.hosts_module_scanner()
scanner.users_module_scanner()
Note: In the above example it doesn't work if i define under class.
You can call variables within a class, using the syntax:
self.variable_name = # whatever you're assigning the variable.
As for where abouts in the class, your best bet is within the def init_() bit.
edit:
As a more verbose answer. you'll define the variables in the init method as shown below. Using self.variable in a class, but outside the methods (class functions) will throw a "self is not defined" error.
class Scanner:
def __init__(self, config_file, verbose=False):
""" Constructor """
self.INFO = 0
self.DEBUG = 3
self.timeStamp = time.strftime("%Y%m%d%H%M%S")
#declare rest of variables here
#other class methods go here
I am using multiprocessing.Pool to speed up computation, as I call one function multiple times, and then collate the result. Here is a snippet of my code:
import multiprocessing
from functools import partial
def Foo(id:int,constant_arg1:str, constant_arg2:str):
custom_class_obj = CustomClass(constant_arg1, constant_arg2)
custom_class_obj.run() # this changes some attributes of the custom_class_obj
if(something):
return None
else:
return [custom_class_obj]
def parallel_run(iters:int, a:str, b:str):
pool = multiprocessing.Pool(processes=k)
## create the partial function obj before passing it to pool
partial_func = partial(Foo, constant_arg1=a, constant_arg2=b)
## create the variable id list
iter_list = list(range(iters))
all_runs = pool.map(partial_func, iter_list)
return all_runs
This throws the following error in the multiprocessing module:
multiprocessing.pool.MaybeEncodingError: Error sending result: '[[<CustomClass object at 0x1693c7070>], [<CustomClass object at 0x1693b88e0>], ....]'
Reason: 'TypeError("cannot pickle 'module' object")'
How can I resolve this?
I was able to replicate the error message with a minimal example of an un-picklable class. The error basically states the instance of your class can't be pickled because it contains a reference to a module, and modules are not picklable. You need to comb through CustomClass to make sure instances don't hold things like open file handles, module references, etc.. If you need to have those things, you should use __getstate__ and __setstate__ to customize the pickle and unpickle process.
distilled example of your error:
from multiprocessing import Pool
from functools import partial
class klass:
def __init__(self, a):
self.value = a
import os
self.module = os #this fails: can't pickle a module and send it back to main process
def foo(a, b, c):
return klass(a+b+c)
if __name__ == "__main__":
with Pool() as p:
a = 1
b = 2
bar = partial(foo, a, b)
res = p.map(bar, range(10))
print([r.value for r in res])
I am new to object-oriented programming and I have some trouble with declaring global variables that are set to a method inside a class. I declared them in the __init__ method (I want them to be inside the class, not outside) with self.censored_transcript = self.apply_filter(), and within this file, everything works fine. But when I import the class in a different .py file and instantiate the Profanity class, the following error occurs: UnboundLocalError: local variable 'censored_transcript' referenced before assignment. I have done quite some research on this topic but can't find anything that fits my case. The global statement does not work. Can anyone please let me know how I can fix this? Thank you very much in advance!
File 1 (file to be imported), called profanity_filter.py:
from better_profanity import profanity
class Profanity(object):
def __init__(self, transcript_filename, wordlist_filename):
self.__transcript_filename = transcript_filename
self.__wordlist_filename = wordlist_filename
self.__transcript = self.__load_transcript()
self.censored_transcript = self.apply_filter()
def __load_transcript(self):
f = open(self.__transcript_filename, "r")
file = f.read()
f.close()
return file
def apply_filter(self):
if __name__ == "__main__":
profanity.load_censor_words_from_file(self.__wordlist_filename)
censored_transcript = profanity.censor(self.__transcript)
return censored_transcript
File 2 (test file):
from profanity_filter import Profanity
# Instantiate the Profanity class:
p = Profanity("sample_transcript.txt", "profanity_wordlist.txt")
# Apply the profanity filter.
censored_transcript = p.apply_filter()
print(censored_transcript)
You can use self.censored_transcript in all places inside your class - self refers to current object of class. Each created object of your class will store self self set of variables available.
censored_transcript without self treated as regular local variable which will be newly created for each call of method.
Error cause:
When method def apply_filter(self) gets called from outer .py file, if __name__ == "__main__" check will be false and local variable censored_transcript will not be created inside this method. So, return censored_transcript tries to return local variable with unassigned value and fails.
Solution 1: in apply_filter(self) you can use object's censored_transcript, not local, to save result. Just append self. before it. And you do not need to return something from this method:
from better_profanity import profanity
class Profanity(object):
def __init__(self, transcript_filename, wordlist_filename):
self.__transcript_filename = transcript_filename
self.__wordlist_filename = wordlist_filename
self.__transcript = self.__load_transcript()
self.apply_filter() # changed
def __load_transcript(self):
f = open(self.__transcript_filename, "r")
file = f.read()
f.close()
return file
# changed
def apply_filter(self):
profanity.load_censor_words_from_file(self.__wordlist_filename)
self.censored_transcript = profanity.censor(self.__transcript)
Solution 2: use a local variable, but assign value to it without if. Or even omit this local variable:
from better_profanity import profanity
class Profanity(object):
def __init__(self, transcript_filename, wordlist_filename):
self.__transcript_filename = transcript_filename
self.__wordlist_filename = wordlist_filename
self.__transcript = self.__load_transcript()
self.censored_transcript = self.apply_filter()
def __load_transcript(self):
f = open(self.__transcript_filename, "r")
file = f.read()
f.close()
return file
# changed
def apply_filter(self):
profanity.load_censor_words_from_file(self.__wordlist_filename)
return profanity.censor(self.__transcript)
I'm making program for study python.
it is gui web crawler.
i success multi work gui and main classes using QThread
but i have a problem.
in main class, get picture addresses using webdriver first, and make a list called data.
after that use pool() and map() to start download pictures using download_image method in Main class.
i searched and tried many things.
imap and lambda etc.
here is my code
(i import multiprocess as mul)
(and my python version is 3.7)
# crawler and downloader class
class Main(QThread, QObject):
def __init__(self, path, brand, model, grade):
QThread.__init__(self)
self.path = path
# this is download method
def download_image(self, var):
a = var.split("/")
filename = a[-1]
download_path = self.path + filename
urllib.request.urlretreieve(var, download_path)
# this is start method when button clicked in Gui
def core(self):
#sample url data list
data = ['url.com', 'url2.com', 'url3.com', ...]
download_p = Pool(mul.cpu_count())
download_p.map(self.download_image, data)
download_p.close()
download_p.join()
print("end")
def run(self):
self.core()
class Gui(QMainWindow, design.Ui_MainWindow):
def __init__(self):
(and gui code here)
if i set download_p.map(self.download_image, data)
i get this error -> [ TypeError: can't pickle Main objects ]
if i set download_p.map(self.download_image, self.data)
(and also set self.data = [urls...])
i get same TypeError
if i set download_p.map(self.download_image, self, data)
i get this error -> [TypeError : 'Main' object is not iterable
I'm not good at English and Python too
but i want to resolve this problem so i decide ask here
really thanks for looking this newbie's question...
I want to split the python script into separate files but have python interpreter treat it as one - not Import (with all the namespace consequences) but more like "in this place insert code from another file"
The reason is pure convenience: the class I separate will be edited to enhance the functionality while I don't want to mess up the rest of the code. Preferably I would have PyCharm treat both files as one when editing, with code hints etc.
Is it possible?
Main.py:
import logic
class Clock:
def __init__(self):
self.state = 0
self.timer = 0
## rest of the code
clock = Clock()
decoder = logic.Decoder()
Logic.py:
class Decoder:
def __init__(self):
self.do()
#staticmethod
def do():
if clock.state == 1:
print("T: " + ic.state.__str__())
Will end in:
NameError: name 'clock' is not defined