I've been reading up on the Observer Design Pattern.
Take the below code from the following article:
https://sourcemaking.com/design_patterns/observer/python/1
I understand how the code works and how the two classes, subject and observer, interrelate. What I can't quite get my head round is how this would all work in reality.
In the real world examples of the observer pattern I have read about, the interaction between the subject and observer tends to be open ended.
For example, if you subscribe to a question on Quora and get notified of answers via email, theoretically, you could receive updates indefinitely.
How then if the code below was applied to a real world scenario (like Quora) do the classes persist? Are the classes and their states stored somewhere on the server?
import abc
class Subject:
"""
Know its observers. Any number of Observer objects may observe a
subject.
Send a notification to its observers when its state changes.
"""
def __init__(self):
self._observers = set()
self._subject_state = None
def attach(self, observer):
observer._subject = self
self._observers.add(observer)
def detach(self, observer):
observer._subject = None
self._observers.discard(observer)
def _notify(self):
for observer in self._observers:
observer.update(self._subject_state)
#property
def subject_state(self):
return self._subject_state
#subject_state.setter
def subject_state(self, arg):
self._subject_state = arg
self._notify()
class Observer(metaclass=abc.ABCMeta):
"""
Define an updating interface for objects that should be notified of
changes in a subject.
"""
def __init__(self):
self._subject = None
self._observer_state = None
#abc.abstractmethod
def update(self, arg):
pass
class ConcreteObserver(Observer):
"""
Implement the Observer updating interface to keep its state
consistent with the subject's.
Store state that should stay consistent with the subject's.
"""
def update(self, arg):
self._observer_state = arg
# ...
def main():
subject = Subject()
concrete_observer = ConcreteObserver()
subject.attach(concrete_observer)
subject.subject_state = 123
if __name__ == "__main__":
main()
You do not need to persist the classes. All you need to do is persist the data. In your real-world example, Quora just stores the list of subscribers for each questions. If there is any activity, it just needs to call some Notification service, passing along the list, which eventually will send the mail to subscribed users.
Related
is it possible in Dart to instantiate a class in an isolate, and then send message to this isolate to receive a return value from its methods (instead of spawning a new isolate and re instantiate the same class every time)? I have a class with a long initialization, and heavy methods. I want to initialize it once and then access its methods without compromising the performance of the main isolate.
Edit: I mistakenly answered this question thinking python rather than dart. snakes on the brain / snakes on a plane
I am not familiar with dart programming, but it would seem the concurrency model has a lot of similarities (isolated memory, message passing, etc..). I was able to find an example of 2 way message passing with a dart isolate. There's a little difference in how it gets set-up, and the streams are a bit simpler than python Queue's, but in general the idea is the same.
Basically:
Create a port to receive data from the isolate
Create the isolate passing it the port it will send data back on
Within the isolate, create the port it will listen on, and send the other end of it back to main (so main can send messages)
Determine and implement a simple messaging protocol for remote method call on an object contained within the isolate.
This is basically duplicating what a multiprocessing.Manager class does, however it can be helpful to have a simplified example of how it can work:
from multiprocessing import Process, Lock, Queue
from time import sleep
class HeavyObject:
def __init__(self, x):
self._x = x
sleep(5) #heavy init
def heavy_method(self, y):
sleep(.2) #medium weight method
return self._x + y
def HO_server(in_q, out_q):
ho = HeavyObject(5)
#msg format for remote method call: ("method_name", (arg1, arg2, ...), {"kwarg1": 1, "kwarg2": 2, ...})
#pass None to exit worker cleanly
for msg in iter(in_q.get, None): #get a remote call message from the queue
out_q.put(getattr(ho, msg[0])(*msg[1], **msg[2])) #call the method with the args, and put the result back on the queue
class RMC_helper: #remote method caller for convienience
def __init__(self, in_queue, out_queue, lock):
self.in_q = in_queue
self.out_q = out_queue
self.l = lock
self.method = None
def __call__(self, *args, **kwargs):
if self.method is None:
raise Exception("no method to call")
with self.l: #isolate access to queue so results don't pile up and get popped off in possibly wrong order
print("put to queue: ", (self.method, args, kwargs))
self.in_q.put((self.method, args, kwargs))
return self.out_q.get()
def __getattr__(self, name):
if not name.startswith("__"):
self.method = name
return self
else:
super().__getattr__(name)
def child_worker(remote):
print("child", remote.heavy_method(5)) #prints 10
sleep(3) #child works on something else
print("child", remote.heavy_method(2)) #prints 7
if __name__ == "__main__":
in_queue = Queue()
out_queue = Queue()
lock = Lock() #lock is used as to not confuse which reply goes to which request
remote = RMC_helper(in_queue, out_queue, lock)
Server = Process(target=HO_server, args=(in_queue, out_queue))
Server.start()
Worker = Process(target=child_worker, args=(remote, ))
Worker.start()
print("main", remote.heavy_method(3)) #this will *probably* start first due to startup time of child
Worker.join()
with lock:
in_queue.put(None)
Server.join()
print("done")
I am looking at the best (and correct way) to obtain a request-independent db session.
The problem is the following: I am building a web application that has to access the database. The endpoint exposed accepts a request, performs the first work, then create a thread (that will perform the hard work), starts it, and replies to the client with a unique id for the "job". Meanwhile the thread goes on with its work (and it has to access the database) and the client can perform polling to check the status. I am not using dedicated framework to perform this background job, but only a simple thread. I can only have one single background thread going on at any time, for this reason I am maintaining the state in a singleton.
The application is created with the application factory design https://flask.palletsprojects.com/en/1.1.x/patterns/appfactories/
I am using Gunicorn as WSGI server and sqlite as database.
The basic structure of the code is the following (I am removing the business logic and imports, but the concept remain):
api_jobs.py
#bp.route('/jobs', methods=['POST'])
def create_job():
data = request.get_json(force=True) or {}
name = data['name']
job_controller = JobController() # This is a singleton
job_process = job_controller.start_job(name)
job_process_dict = job_process.to_dict()
return jsonify(job_process_dict)
controller.py
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
class JobController(object):
__metaclass__ = Singleton
def __init__(self):
self.job_thread = None
def start_job(self, name):
if self.job_thread is not None:
job_id = self.job_thread.job_id
job_process = JobProcess.query.get(job_id)
if job_process.status != 'end':
raise ValueError('A job process is already ongoing!')
else:
self.job_thread = None
job_process = JobProcess(name)
db.session.add(job_process)
db.session.commit() # At this step I create the ID
self.job_thread = JobThread(db.session, job_process.id)
self.job_thread.start()
return job_process
class JobThread(threading.Thread):
def __init__(self, db_session, job_id):
self.job_id = job_id
self.db_session = db_session
self.session = self.db_session()
def run(self):
self.job_process = self.session.query(JobProcess).get(self.job_id)
self.job_process.status = 'working'
self.session.commit()
i = 0
while True:
sleep(1)
print('working hard')
i = i +1
if i > 10:
break
self.job_process.status = 'end'
self.session.commit()
self.db_session.remove()
models.py
class JobProcess(db.Model):
id = db.Column(db.Integer, primary_key=True)
status = db.Column(db.String(64))
name = db.Column(db.String(64))
def to_dict(self):
data = {
'id': self.id,
'status': self.status,
'name': self.name,
}
return data
From my understanding, calling self.session = self.db_session() is actually doing nothing (due to the fact that sqlalchemy is using a registry, that is also a proxy, if I am not wrong), however that was the best attempt that I found to create a "new/detached/useful" session.
I checked out https://docs.sqlalchemy.org/en/13/orm/contextual.html#using-thread-local-scope-with-web-applications in order to obtain a request-independent db-session, however even using the suggested method of creating a new session factory (sessionmaker + scoped_session), does not work.
The errors that I obtain, with slight changes to the code, are multiple, in this configuration the error is
DetachedInstanceError: Instance <JobProcess at 0x7f875f81c350> is not bound to a Session; attribute refresh operation cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)
The basic question remains: Is it possible to create a session that will live inside the thread and that I will take care of creating/tearing down?
The reason that you are encountering the DetachedInstanceError is that you are attempting to pass the session from your main thread to your job thread. Sqlalchemy is using thread local storage to manage the sessions and thus a single session cannot be shared between two threads. You just need to create a new session in the run method of your job thread.
I don't very understand the classes logic in python but cannot answer on web.
I have create a class to generate person info:
class person:
def fristnameGen(gender):
...
def emailGen(firstname,surname):
...
i create a bot to call it like this:
from person import *
class bots:
def __init__(self):
self.person = person()
def createDB(self):
print(self.person.name)
#do something...
finally i call it by a button with thread
from bots import *
import threading
class Panel:
def __init__(self):
self.top = tk.Tk()
self.bot = bots()
self.buildUI()
def foo(self):
self.bot.createDB(self.stringPhone.get())
def threadTheAction(func, *args):
t = threading.Thread(target=func, args=args)
t.setDaemon(True)
t.start()
def buildUI(self):
Button = tk.Button(self.top, text ="Start", command = lambda :self.threadTheAction(self.foo))
I get this error:
TypeError: 'Panel' object is not callable
However, i call it directly, its work
Button = tk.Button(self.top, text ="Start", command = lambda :self.foo())
How to fix the error?
...
2. Moreover, i tried create p1 = person() and p2= person() and print it. Found p1 and p2 is a same person, i prefer each new a class have a new one. How to generate "new person" using classes?
Thank you
You seem to have a lot of confusion about Object Oriented Programming in Python. Some of your methods have self parameters and some do not, seemingly at random. That's the source of your current error.
The threadTheAction method in your Panel class is getting the Panel instance passed in as its first argument, but that's named func in the method (since you omitted self). The real function you're passing as an argument gets caught in the variable argument *args. When the thread tries unsuccessfully to call it, you get an exception. Adding self before func would fix the immediate problem:
def threadTheAction(self, func, *args):
I suspect if your code got further along, you'd run into other errors with other methods without self in their parameter lists. For instance, none of the methods you've shown in person are likely to work correctly.
As for your second question, you haven't shown enough of person to know what's happening, but you're probably doing instance variables wrong somehow. With no self parameter in the methods, that's almost inevitable (since you assign to self.whatever to set a whatever attribute on the current instance). If you need help squaring that away, I suggest you ask a separate question (Stack Overflow is best when each question is self-contained) and provide the full code for your person class.
I am trying to programme a board game using python and object orientated programming. However, the structure of my classes and properties seems a bit tortuous even if I have made it do what I want. It doesn't seem very elegant and I'm sure it will lead to things getting too complicated and making the code hard to follow as I progress. I will try to give a simplified example of how I've structured things:
class GameState:
def __init__(self, n_players):
self.n_players = n_players
self.players = []
for player in range(self.n_players):
self.players.append(Player(player))
// other properties
// other functions
class Player:
def __init__(self, name):
self.name = name
self.building = Building()
self.workers = 3
// other properties
class Building:
def __init__(self):
self.worker = False
def action(self, player):
if self.worker == True:
player.workers += 1
But now if I want a player to use the building action I have to do something like the below. It feels like I should be able to structure things better to avoid having to pass an instance of a Player class to the Building action function.
game = GameState(4)
game.players[0].building.action(game.players[0])
The idea is that each player will have an instance of the Building class.
It's hard to suggest alternatives without knowing the exact format of your game. If each building is only ever associated with a single player at a time, I'd add that to the initialisation and reference it directly.
class Building:
def __init__(self, player):
self.worker = False
self.player = player
def action(self):
if self.worker == True:
self.player.workers += 1
Alternatively, if each player will only ever have one building associated with them, then the action function should probably be in the player class rather than the building.
I have a pyqt4 gui which allows me to import multiple .csv files. I've created a loop that goes through this list of tuples that have the following parameters (filename + location of file, filename, bool,bool, set of dates in file)=tup.
I've created several classes that my gui frequently refers to in order to pull parameters off a projects profile. Let's call this class profile(). I also have another class that has a lot of functions based on formatting, such as datetime, text edits,etc...let's call this classMyFormatting(). Then I created a QThread class that is created for each file in the list, and this one is called Import_File(QThread). And lets say this class takes in a few parameters for the __init__(self,tup).
My ideal goal is to be able to make an independent instance of MyFormatting() and profile() for the Import_File(QThread). I am trying to get my head around on how to utilize the QObject capabilities to solve this..but I keep getting the error that the thread is being destroyed while still running.
for tup in importedlist:
importfile = Import_File(tup)
self.connect(importfile,QtCore.SIGNAL('savedfile(PyQt_PyObject()'),self.printstuffpassed)
importfile.start()
I was thinking of having the two classes be declared as
MyFormatting(QObject):
def __init__(self):
QObject.__init__(self)
def func1(self,stuff):
dostuff
def func2(self):
morestuff
profile(QObject):
def __init__(self):
QObject.__init__(self)
def func11(self,stuff):
dostuff
def func22(self):
morestuff
AND for the QThread:
Import_File(QThread):
def __init__(self,tup):
QThread.__init(self)
common_calc_stuff = self.calc(tup[4])
f = open(tup[0] + '.csv', 'w')
self.tup = tup
# this is where I thought of pulling an instance just for this thread
self.MF = MyFormatting()
self.MF_thread = QtCore.QThread()
self.MF.moveToThread(self.MF_thread)
self.MF_thread.start()
self.prof = profile()
self.prof_thread = QtCore.QThread()
self.prof.moveToThread(self.prof_thread)
self.prof_thread.start()
def func33(self,stuff):
dostuff
self.prof.func11(tup[4])
def func44(self):
morestuff
def run(self):
if self.tup[3] == True:
self.func33
self.MF.func2
elif self.tup[3] ==False:
self.func44
if self.tup[2] == True:
self.prof.func22
self.emit(QtCore.SIGNAL('savedfile()',)
Am I totally thinking of it the wrong way? How can I keep to somewhat of the same structure that I have for the coding and still be able to implement the multithreading and not have the same resource tapped at the same time, which I think is the reason why my qui keeps crashing? Or how can I make sure that each instance of those objects get turned off that they don't interfere with the other instances?