The program I am developing uses threads to deal with long running processes. I want to be able to use Gauge Pulse to show the user that whilst a long running thread is in progress, something is actually taking place. Otherwise visually nothing will happen for quite some time when processing large files & the user might think that the program is doing nothing.
I have placed a guage within the status bar of the program. My problem is this. I am having problems when trying to call gauge pulse, no matter where I place the code it either runs to fast then halts, or runs at the correct speed for a few seconds then halts.
I've tried placing the one line of code below into the thread itself. I have also tried create another thread from within the long running process thread to call the code below. I still get the same sort of problems.
I do not think that I could use wx.CallAfter as this would defeat the point. Pulse needs to be called whilst process is running, not after the fact. Also tried usin time.sleep(2) which is also not good as it slows the process down, which is something I want to avoid. Even when using time.sleep(2) I still had the same problems.
Any help would be massively appreciated!
progress_bar.Pulse()
You will need to find someway to send update requests to the main GUI from your thread during the long running process. For example, if you were downloading a very large file using a thread, you would download it in chunks and after each chunk is complete, you would send an update to the GUI.
If you are running something that doesn't really allow chunks, such as creating a large PDF with fop, then I suppose you could use a wx.Timer() that just tells the gauge to pulse every so often. Then when the thread finishes, it would send a message to stop the timer object from updating the gauge.
The former is best for showing progress while the latter works if you just want to show the user that your app is doing something. See also
http://wiki.wxpython.org/LongRunningTasks
http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/
http://www.blog.pythonlibrary.org/2013/09/04/wxpython-how-to-update-a-progress-bar-from-a-thread/
Related
I develop a SCADA in python3 with the help of PyQt. I expect my program to continuously indicate various parameters received through RS-485 interface, however, within several minutes from the start (it is always a different time), the GUI stops to update itself. At the same time, the GUI stays responsive and if I were to click on one of the animated QAbstractButtons it has, the GUI starts to work as intended again for a short period of time. The problem occurs on both Linux and Windows.
The program has several worker threads: one for RS-485 exchange, one for reading/writing various data to disk, one for decoding received package and refreshing data in memory, one for request queue management, etc. They all work in loops of while (True): time.sleep(...) - do the job. The GUI is implemented in the main thread.
The data is indicated with QLabels. The QLabel.setText is added to painterEvent() of QWidgets containing the QLabels.
When the GUI stops updating, the other threads are up and running: the exchange is functioning, the request queue is forming, etc. Despite not being updated, the GUI stays responsive and reacts to QAbstractButton clicks.
I tried adding gui.update() or app.processEvents() into one of the worker threads, tried force updating through QTimer in the main thread. The result is the same: it works for a short while and then stops.
I tried increasing the time.sleep of the refresh thread and force updating the main widget over longer intervals of time (0.5 to 5 seconds) and it seems to help the situation a lot, this way it can run for several minutes, but it still does not solve the problem.
I would love to show the code, but the whole thing is way too bulky to post here and if I could narrow it down to a at least a hundred lines, I would have already solved the issue by now. So if any of you could at least share some general considerations on what to look for, I would be very happy.
update:
This seems to work, I'll leave it running for a few hours tomorrow to confirm:
update_timer = QTimer()
update_timer.setSingleShot(False)
update_timer.timeout.connect(self.gui.repaint)
update_timer.setInterval(500)
update_timer.start()
I assume that self.gui.update() did not work because dataChanged() was probably not emitted and control passed instead of doing repaint(). As far as I understand, the solution above is not the right way to update widgets.
So, the question actually boils down to the following:
What is the right way to update the main QWidget and how do I let the program know it needs to be redrawn, probably using dataChanged() signal?
The answer is simple. Never do anything related to PyQt from outside the main thread. Even if it seems the only logical way to do something.
After I 'fixed' the issue with the gui not being updated, I stumbled into a more serious problem: the program crashed every now and then. As it turns out, the GUI not being updated was only the top of the iceberg. The uptime was usually 2 to 7 minutes, I received a few glibc errors ("corrupted double-linked list" and "double free or corrution"). As it turns out, the source of the problem was inside refresh thread that has several hundreds of lines updating gui:
self.gui.screen_name.device_name.setCrash()
where setCrash() changes widget color and palette, or even more direct things like
self.gui.screen_name.label_name.setText(str(value))
I tracked down everything even loosely related to gui from the main file (the place where all the threading took place) and restyled it. Now it only has one worker thread - the RS-485, the rest are wrapped into separate QTimers.
This is what it was:
class Main():
def __init__():
self.plots_thread = Thread(target = self.plots_refresh)
self.plots_thread.start()
def plots_refresh():
while True:
time.sleep(0.5)
#do stuff
And this is what it is now:
class Main():
def __init__():
self.plots_refresh_timer = QTimer()
self.plots_refresh_timer.setSingleShot(False)
self.plots_refresh_timer.timeout.connect(self.plots_refresh)
self.plots_refresh_timer.setInterval(499) #prime number
self.plots_refresh_timer.start()
def plots_refresh():
#do stuff
This time the program has been running for over an hour and never crashed. After coming to conclusion that this was THE fix, I refactored the really messy test file and resumed testing - again, no troubles for half an hour.
I have a large data acquisition and control program written in Python3.4.2 using GUI mostly developed on Glade 3.18.3, Gtk3.0 GObject running Debian 8 with XFCE.
There are timers that keep doing things, and these work fine. After startup, the program runs for some 3 - 7 days, then suddenly, all of the windows go blank and stay blank. Other applications are not affected. Memory and CPU usage is modest.
There are no indications of problems prior to the windows going blank. The windows show their title bars and respond normally to minimize, restore, move to another Workspace, etc. It looks like they are not getting repainted - no widgets are visible at all. The code is way too large to post here, and I am not able to isolate a specific problem area for lack of obvious symptoms other than the blank screens. There are no error messages or warnings.
The timers continue to run, acquire data and control things. This happens whether the program is run from the command line or under PyDev in Eclipse.
Things I have tried:
In the main timer loop, I put code to look for a file, and then exec the command in it, printing the results, so I have been able to mess with the program in real time:
Replace the usual Gtk.main() with a while loop whose variable, if not made false, will re-execute the Gtk.main(). Executing Gtk.main.quit() stops Gtk.main and starts it again. Windows still blank. Did this repeatedly to no avail.
Experimented with garbage collection with GC. Collecting garbage makes no different. Windows still blank.
Put in code to print percent of time consumed by the timer loops. Fairly steady around 18 - 20% of available CPU time, so nothing is hogging the CPU preventing re-paint.
I have a button that clears a label. I read the label, then executed a builder.get_object(...).activate command to the button. I re-read the label and it was now properly blank. So events and widgets appear to be working normally, at least to some extent.
Finally, if I click on the close X on the title bar, XFCE asks me if I want to wait or close now. So it seems as though there may be a disconnection or problem with signals and the OS, even though XXX.activate() works.
Web searching is in vain. Does anybody have ideas of what might be happening, useful diagnostics, or other suggestions? Many thanks!
April 27, 2017 Update:
I have taken two substantial steps to mostly work around problems. First, partly in response to a couple of Gtk crashes over the last few months, instead of ending the main program with:
Gtk.main()
I end with:
while wannalive:
try:
Gtk.main()
except:
pass
wannalive is True until user does a quit, so recovery is instant.
Second, I grouped all of the code for each window setup and initial population of static items into two functions. I also made another function for closing a window. These functions propagate to children, grandchildren windows. A function in the top window first, closes, then re-creates all windows, with one call. In operation, there are overlaps in what windows exist, but that is not a problem.
Above, I describe that I can inject code with an external program. The external program has a button that injects a call to that third function. In about five seconds or less, the result of a single button click is to replace all blank windows with functional windows. For my purposes in a controlled environment with a trained operator, this is acceptable.
Next, let me address the relationship between the timer loops and GUI events processing. I do use GObject.timeout_add(ms, somefunction). Experiment shows that a button that calls time.sleep(5) stalls the timer. Experiment shows that injecting time.sleep(5) in the timer loop stalls the GUI. This is consistent with my belief (correct me if I am wrong) that Python runs on a single thread. Therefore, bad code caught in an infinite loop should stall both the GUI and the timer loop. (The program has one timeout_add call.)
I am working on a big project that puts performance as a high priority. I have a little bit of experience using wxPython to create windows and dialog boxes for software, but I have no experience in getting processes to work in parallel during the course of a single program.
So basically, what I want to accomplish is the following:
I want one main class that controls the high level program. It sets up a configuration either from a config file or from user input. This much I have accomplished on my own.
I need PROCESS #1 to read in a file and a list of commands, execute the commands, and then pass the modified file to PROCESS #2 (this requires that PROCESS #2 is ready to accept new input.) Once the file is passed, PROCESS #1 would begin work on the next set of inputs and wait for PROCESS #2 to finish before the cycle repeats.
PROCESS #2 takes input from PROCESS #1 and writes output to a log file. Once the output is complete, it waits for the next set of output from PROCESS #1.
I know how to use wxTimers and the events associated with that, but what I have found is that a timer event will not execute if the program is otherwise occupied (like in the middle of a method.)
I have seen threads about "threading" and "Pool", but the terminology tends to go over my head, and I haven't gotten any of that sort of stuff to work.
If anybody can point me in the right direction, I would be greatly appreciative.
If you use threads, then I think this would be fairly easy to do. Here's what I would suggest:
Create a button (or some other widget) to execute process #1 in a thread. The thread itself will run BOTH processes. Here's some psuedo-code that might help:
# this is in your thread code:
result = self.call_process_1(args)
self.call_process_2(result)
This will allow you to start another process #1/2 with a new set of commands every time you press the button. Since the two processes are encapsulated in the thread, they don't have to wait for process #2 to finish. You will probably need to log to separate logs for the logs to make sense, but you can label the logs with a timestamp and a thread number or a uuid.
Depending on how many of these processes you need to do, you might need to look into setting up a cluster that's driven with celery or some such. But I think this is a good starting place.
I am currently working with three matlab functions to make them run near simultaneously in single Matlab session(as I known matlab is single-threaded), these three functions are allocated with individual tasks, it might be difficult for me to explain all the detail of each function here, but try to include as much information as possible.
They are CONTROL/CAMERA/DATA_DISPLAY tasks, The approach I am using is creating Timer objects to have all the function callback continuously with different callback period time.
CONTROL will sending and receiving data through wifi with udp port, it will check the availability of package, and execute callback constantly
CAMERA receiving camera frame continuously through tcp and display it, one timer object T1 for this function to refresh the capture frame
DATA_DISPLAY display all the received data, this will refresh continuously, so another timer T2 for this function to refresh the display
However I noticed that the timer T2 is blocking the timer T1 when it is executed, and slowing down the whole process. I am working on a system using a multi-core CPU and I would expect MATLAB to be able to execute both timer objects in parallel taking advantage of the computational cores.
Through searching the parallel computing toolbox in matlab, it seems not able to deal with infinite loop or continuous callback, since the code will not finish and display nothing when execute, probably I am not so sure how to utilize this toolbox
Or can anyone provide any good idea of re-structuring the code into more efficient structure.
Many thanks
I see a problem using the parallel computing toolbox here. The design implies that the jobs are controlled via your primary matlab instance. Besides this, the primary instance is the only one with a gui, which would require to let your DISPLAY_DATA-Task control everything. I don't know if this is possible, but it would result in a very strange architecture. Besides this, inter process communication is not the best idea when processing large data amounts.
To solve the issue, I would use Java to display your data and realise the 'DISPLAY_DATA'-Part. The connection to java is very fast and simple to use. You will have to write a small java gui which has a appendframe-function that allows your CAMERA-Job to push new data. Obviously updating the gui should be done parallel without blocking.
I'm running a long process in the background. I've managed to output the console data to gui. But the problem is that, the data is returned only after the process is finished. But I need to display the data at realtime. ie, I need to display the data, every time it produces some output on the console. I'm running the process with in my gui from a seperate thread.
I mean, it would be like building a gui for the ping command, where output is displayed on console after each packet is send, ie at realtime. I just need to redirect that to gui, in realtime. I'm implementing the gui in wxwidgets. Any help would be greatly appreciated.
Thanking You..
Jvc
Is the output you wish to display generated in a separate process from the process running the GUI? Or in a separate thread in the same process?
I ask because most people, when they ask this question, mean a a separate thread. Since you have tagged your question with "process" I will assume that is what you mean.
You need some inter-process communication. There is a bewildering variety of techniques to do this. Personally, I always use sockets.
wxWidgets has simple, easy to use socket classes wxSocketClient and wxSocketServer.
The background process is probably not running wxWidgets, so you will need something else there. I reccomend boost::asio. I know it looks intimidating, but in fact the tutorial code can be used as is.
There is a lot more to be said, but I risk straying away from the point, since there are so few details in your question.
You can have an output queue protected by a wxMutex. The thread doing the computation writes to the queue, then signals the GUI thread using wxQueueEvent with a custom event to let it know that the thread is not empty. The GUI thread then reads the queue and outputs the data.