Running Python’s SimpleHTTPServer in a GUI - python-3.x

I am writing a GUI wrapper around Python’s SimpleHTTPServer. It looks like this:
The GUI uses tkinter. When I click on the OK button, it launches the web server.
The web server code is based on the article at https://docs.python.org/3/library/http.server.html. Part of it is:
with socketserver.TCPServer(("", PORT), Handler) as httpd:
print("serving at port", PORT)
httpd.serve_forever()
It all works as expected so far, but when the https.server_forever() function runs, I get the spinning beachball, and I can’t close the window or quit. I can force quit and try again, which is not convenient. The server does do its job, however.
If I run the same code from the command line (a non-gui version), I can easily interrupt it with ctrl-c; I can catch that and exit more gracefully.
How can I interrupt the running server more politely from the server?

You will need to run the server in a thread or a separate process, since both the web server and the UI need separate event loops.
If you want the server to communicate with the tkinter program, you'll need to set up a queue. As a general rule, you should only access tkinter objects from the thread that they were created in. However, I think it's safe to send virtual events from a worker thread to the GUI thread, which you can use to cause the GUI thread to read data from the queue.
For example, here's a simple threaded server. It must be passed the host and port, a reference to the root window, and a reference to the queue where information can be sent to the GUI.
class ExampleServer(threading.Thread):
def __init__(self, host, port, gui, queue):
threading.Thread.__init__(self)
self.host = host
self.port = port
self.gui = gui
self.queue = queue
self.daemon = True
def run(self):
print(f"Listening on http://{self.host}:{self.port}\n")
server = HTTPServer((self.host, self.port), ExampleHandler)
server.serve_forever()
In your request handler, you can push items on the queue and then generate an event on the root window. It might look something like this:
class ExampleHandler(BaseHTTPRequestHandler):
def do_GET(self):
# handle the request
...
# notify the UI
self.queue.put("anything you want")
self.gui.event_generate("<<DataAvailable>>")
Your gui also needs to take the queue as an argument and needs to set a binding to the virtual event. It might look something like this:
class ExampleGUI(tk.Tk):
def __init__(self, queue):
super().__init__()
self.queue = queue
# set up the rest of the GUI
...
# bind to the virtual event
self.bind("<<DataAvailable>>", self.poll_queue)
def poll_queue(self, event):
while not queue.empty():
data = self.queue.get_nowait()
# Do whatever you need to do with the data
...
def start(self):
self.mainloop()
Finally, you tie it all together with something like this:
if __name__ == "__main__":
queue = queue.Queue()
gui = ExampleGUI(queue)
server = ExampleServer("localhost", 8910, gui, queue)
server.start()
gui.start()

Related

What is best practice to interact with subprocesses in python

I'm building an apllication which is intended to do a bulk-job processing data within another software. To control the other software automatically I'm using pyautoit, and everything works fine, except for application errors, caused from the external software, which occur from time to time.
To handle those cases, I built a watchdog:
It starts the script with the bulk job within a subprocess
process = subprocess.Popen(['python', job_script, src_path], stdout=subprocess.PIPE,
stderr=subprocess.PIPE, shell=True)
It listens to the system event using winevt.EventLog module
EventLog.Subscribe('System', 'Event/System[Level<=2]', handle_event)
In case of an error occurs, it shuts down everything and re-starts the script again.
Ok, if an system error event occurs, this event should get handled in a way, that the supprocess gets notified. This notification should then lead to the following action within the subprocess:
Within the subprocess there's an object controlling everything and continuously collecting
generated data. In order to not having to start the whole job from the beginnig, after re-starting the script, this object has to be dumped using pickle (which isn't the problem here!)
Listening to the system event from inside the subprocess didn't work. It results in a continuous loop, when calling subprocess.Popen().
So, my question is how I can either subscribe for system events from inside a childproces, or communicate between the parent and childprocess - means, sending a message like "hey, an errorocurred", listening within the subprocess and then creating the dump?
I'm really sorry not being allowed to post any code in this case. But I hope (and actually think), that my description should be understandable. My question is just about what module to use to accomplish this in the best way?
Would be really happy, if somebody could point me into the right direction...
Br,
Mic
I believe the best answer may lie here: https://docs.python.org/3/library/subprocess.html#subprocess.Popen.stdin
These attributes should allow for proper communication between the different processes fairly easily, and without any other dependancies.
Note that Popen.communicate() may suit better if other processes may cause issues.
EDIT to add example scripts:
main.py
from subprocess import *
import sys
def check_output(p):
out = p.stdout.readline()
return out
def send_data(p, data):
p.stdin.write(bytes(f'{data}\r\n', 'utf8')) # auto newline
p.stdin.flush()
def initiate(p):
#p.stdin.write(bytes('init\r\n', 'utf8')) # function to send first communication
#p.stdin.flush()
send_data(p, 'init')
return check_output(p)
def test(p, data):
send_data(p, data)
return check_output(p)
def main()
exe_name = 'Doc2.py'
p = Popen([sys.executable, exe_name], stdout=PIPE, stderr=STDOUT, stdin=PIPE)
print(initiate(p))
print(test(p, 'test'))
print(test(p, 'test2')) # testing responses
print(test(p, 'test3'))
if __name__ == '__main__':
main()
Doc2.py
import sys, time, random
def recv_data():
return sys.stdin.readline()
def send_data(data):
print(data)
while 1:
d = recv_data()
#print(f'd: {d}')
if d.strip() == 'test':
send_data('return')
elif d.strip() == 'init':
send_data('Acknowledge')
else:
send_data('Failed')
This is the best method I could come up with for cross-process communication. Also make sure all requests and responses don't contain newlines, or the code will break.

How to shut down CherryPy in no incoming connections for specified time?

I am using CherryPy to speak to an authentication server. The script runs fine if all the inputted information is fine. But if they make an mistake typing their ID the internal HTTP error screen fires ok, but the server keeps running and nothing else in the script will run until the CherryPy engine is closed so I have to manually kill the script. Is there some code I can put in the index along the lines of
if timer >10 and connections == 0:
close cherrypy (< I have a method for this already)
Im mostly a data mangler, so not used to web servers. Googling shows lost of hits for closing CherryPy when there are too many connections but not when there have been no connections for a specified (short) time. I realise the point of a web server is usually to hang around waiting for connections, so this may be an odd case. All the same, any help welcome.
Interesting use case, you can use the CherryPy plugins infrastrcuture to do something like that, take a look at this ActivityMonitor plugin implementation, it shutdowns the server if is not handling anything and haven't seen any request in a specified amount of time (in this case 10 seconds).
Maybe you have to adjust the logic on how to shut it down or do anything else in the _verify method.
If you want to read a bit more about the publish/subscribe architecture take a look at the CherryPy Docs.
import time
import threading
import cherrypy
from cherrypy.process.plugins import Monitor
class ActivityMonitor(Monitor):
def __init__(self, bus, wait_time, monitor_time=None):
"""
bus: cherrypy.engine
wait_time: Seconds since last request that we consider to be active.
monitor_time: Seconds that we'll wait before verifying the activity.
If is not defined, wait half the `wait_time`.
"""
if monitor_time is None:
# if monitor time is not defined, then verify half
# the wait time since the last request
monitor_time = wait_time / 2
super().__init__(
bus, self._verify, monitor_time, self.__class__.__name__
)
# use a lock to make sure the thread that triggers the before_request
# and after_request does not collide with the monitor method (_verify)
self._active_request_lock = threading.Lock()
self._active_requests = 0
self._wait_time = wait_time
self._last_request_ts = time.time()
def _verify(self):
# verify that we don't have any active requests and
# shutdown the server in case we haven't seen any activity
# since self._last_request_ts + self._wait_time
with self._active_request_lock:
if (not self._active_requests and
self._last_request_ts + self._wait_time < time.time()):
self.bus.exit() # shutdown the engine
def before_request(self):
with self._active_request_lock:
self._active_requests += 1
def after_request(self):
with self._active_request_lock:
self._active_requests -= 1
# update the last time a request was served
self._last_request_ts = time.time()
class Root:
#cherrypy.expose
def index(self):
return "Hello user: current time {:.0f}".format(time.time())
def main():
# here is how to use the plugin:
ActivityMonitor(cherrypy.engine, wait_time=10, monitor_time=5).subscribe()
cherrypy.quickstart(Root())
if __name__ == '__main__':
main()

Python chat clients not updating

I am working on a GUI based chat program.
I am using someone else's server which has worked well for many people so I am assuming the problem is with my client's code.
When I run a single instance of the client it works perfectly, but if I run two instances of the client on the same computer the listener stops responding when the 2nd client logs in.
# server is from socket module
# chat_box is a tkinter ListBox
# both are copies of global variable
class listener_thread(threading.Thread):
def __init__(self, server, chat_box):
super(listener_thread, self).__init__()
self.server = server
self.chat_box = chat_box
def run(self):
try:
update = self.server.recv(1024)
msg = update.decode("utf-8")
if msg != "":
self.chat_box.insert(END, msg)
except Exception as e:
print(e)
I've verified that the server is putting each client on a different port. The server is receiving the messages. When 'Michael' logs in and says 'Hi' it updates in his chat_box.
Though, the clients are no longer updating their histories after 'Dave' logs in.
Yet, the server continues to show that it is receiving the messages from both clients.
#This is the server output
#Hi is Michael
#Yo is Dave
#So Michael is still connecting and transmitting after Dave connects
Michael - ('127.0.0.1', 56263) connected
Hi
Dave - ('127.0.0.1', 56264) connected
Yo
Hi
The network connection is working properly. It just locks up the list_box updating threads.
No exceptions are being thrown.
I solved my own problem.
I needed to make the chat_history_listbox as a ListBox initially, instead of None
I needed to put the receive code into a function, with a loop and an exit condition
def receive_func():
global server, chat_history_listbox
while True:
try:
update = server.recv(1024)
except OSError as e:
update = None
break
connect()
msg = update.decode("utf-8")
if msg != "":
chat_history_listbox.insert(END, msg)
I needed to make the thread call a function and make it a daemon
listener = Thread(target=receive_func, daemon=True)
listener.start()
This got it working with multiple clients

multiprocessing share object (socket) between process.

I would like to create a Process that store many objects(connect to devices via socket).
I have a GUI (PyQT5) that should store information about progress abut processes and status about devices. Example tell more:
# Process1
def conf1():
dev = some_signal_that_ask_about_dev("device1");
conf_dev(dev)
return_device("device1", dev)
# Process2
def conf2():
dev = some_signal_that_ask_about_dev("device2");
do_sth_withd_dev(dev)
return_device("device2", dev)
# Process3
class DevicesHolder(object):
self.devices = {
"device1": Device1("192.168.1.1", 8080),
"device2": Device2("192.168.1.2", 8081)
}
def some_signal_that_ask_about_dev(self, dev_name):
if self.devices[dev_name]:
dev = self.devices[dev_name]
# this device is taken by process.
# If process take device and faild. device should be recreated!
self.devices[dev_name] = None
return dev
def return_device(dev_name, dev):
self.devices[dev_name] = dev
def get_status_of_devices():
# Check connection to devices and return response
pass
# Process 4:
# GUI:
get_status_of_device();
So process1 and process2 do some work and sending progress to GUI. I would like to have info about devices status also.
Why just not create local object (process) and sending info from that process?
Process can run for a few seconds. When apps run for minutes. I want to know that there is connection problem before I press a start button. And all apps fail because of connection.
I think I am complicating simple problem. Help me!
More info
Every process is configuring sth else but on the same connection.
I would like to work this as quick as possible.
It will work on Linux. But I care about multi platform.

Setting and Getting Variables within an imported Class

All, I'm implementing websockets using flask/uWSGI. This is relegated to a module that's instantiated in the main application. Redacted code for the server and module:
main.py
from WSModule import WSModule
app = Flask(__name__)
wsmodule = WSModule()
websock = WebSocket(app)
#websock.route('/websocket')
def echo(ws):
wsmodule.register(ws)
print("websock clients", wsmodule.clients)
while True: # This while loop is related to the uWSGI websocket implementation
msg = ws.receive()
if msg is not None:
ws.send(msg)
else: return
#app.before_request
def before_request():
print ("app clients:",wsmodule.clients)
and WSModule.py:
class WSModule(object):
def __init__(self):
self.clients = list()
def register(self, client):
self.clients.append(client)
Problem: When a user connects using websockets (into the '/websocket' route), the wsmodule.register appends their connection socket, this works fine- printout 'websocket clients' shows the appended connection.
The issue is that I can't access those sockets from the main application. This is seen by the 'app clients' printout which never updates (list stays empty). Something is clearly updating, but how to access those changes?
It sounds like your program is being run with either threads or processes, and a wsmodule exists for each thread/process that is running.
So one wsmodule is being updated with the client info, while a different one is being asked for clients... but the one being asked is still empty.
If you are using threads, check out thread local storage.

Resources