If I take a 25MB/190,000 line text file and dump it into a text widget the process finishes quickly but I still see python.exe using 50%CPU for another minute or so afterwards. The bigger the file, the longer it takes for the CPU to drop off to 0% usage. When I start loading multiple files into different text widgets they load into the widget instantly but the CPU stays at 50% and the GUI runs horribly slow until it finishes whatever its doing in the backend. Can someone explain to me why the CPU is still being used and why its impacting performance if the text is already in the widget? What is it needing to do? Any way around this?
from tkinter import *
file = r"C:\path\to\large\file.txt"
def doit():
with open(file, 'r') as f:
txt.insert('end', ''.join(f))
f.close()
main = Tk()
txt = Text(main)
txt.grid(row=0)
btn = Button(main, text="click here", command=doit)
btn.grid(row=1, columnspan=2)
main.mainloop()
I thought maybe it was because im handling the file line by line instead of loading the entire file into RAM. I tried readlines() but I get the same results.
Most likely it is calculating where the line breaks go, for lines past the area currently visible on the screen. With the numbers you gave, the lines average over 130 characters long; if some of them are considerably over that, this is a situation that Tkinter is known to be slow in.
You could perhaps turn off word-wrap (by configuring the Text with wrap=NONE), which would probably require that you add a horizontal scroll bar. If that is unacceptable, it's possible that adding newlines of your own could help, if there's some natural point at which to insert them in your data.
Note that ''.join(f) is a rather inefficient way to read the entire file into a single string - just use f.read() for that.
Related
I am scraping current follow requests from Instagram. I have a main infinite loop that it making the requests and prints OK when it is done. I want to display an animated cursor or loading progress while it is downloading names.
while 1:
response = requests.get(IG_CFR_PAGE, headers=headers(""), params=params, cookies=cookies)
if response.status_code == 200:
cfr = response.json()
for entry in cfr["data"]["data"]:
#print(entry["text"])
usernames.append(entry["text"])
if cfr["data"]["cursor"]!= None:
params['cursor'] = cfr["data"]["cursor"]
time.sleep(1)
else:
break
else:
print(response.status_code)
print("Error in request .... aborting")
break
print("ok")
I looked for tqdm but it takes an iterable. In my case, I am just looping over JSON keys in line for entry in cfr["data"]["data"]: and so I guess can't use it. I just need suggestions as to know what should I use to indicate that this script is actually doing something. I just need suggestions or pseudocode is fine to send me in a right direction... the actual programming code is not needed as I will do that myself.
Thank you
As far as I'm aware, most functions that allow you to change the mouse cursor in Python are available only from various GUI modules - most of the popular ones, such as tkinter, PyQt5, pygame or others.
The problem is that most of these may only work when you've created a window of the GUI, which is probably unnecessary or not a nice idea if you're not using the same GUI, or any GUI for that matter. Even then, some may only take effect when the mouse pointer hovers over a certain widget in that GUI.
Note:
I've only (unsuccessfully) tried doing this with pygame.cursors before. It may be convenient because it even lets you create a custom shape with strings, or use a system cursor. But it displays a pygame.error: video system not initialized if you try doing this without having called pygame.display.init() first. I tried creating a window and setting a cursor, but it didn't seem to take effect.
After a quick google search for other ways to set an animated cursor, I came across this SO answer which might offer some insight if you're on windows.
Overall, using a terminal animation may probably be better and easier, so
this is an attempt at a basic answer for a loading animation in a terminal window :
(Of course, for an indefinite length of loop, it doesn't make sense to store a percentage completion etc, so this just animates at an arbitrary pace and repeats when it reaches the end)
i, w, d = 0, 20, 10000
while True:
# Do whatever
# No print statements except this last one
i = (i+1)%(w*d)
l = i//d
print("\r Processing... |" + " "*l+ "█" + " "*(w-l-1) +"|", end='')
i is used for iteration, w is the length of the bar, and d used to create some sort of 'delay', so that the bar doesn't change at every single iteration, but some slower (visible) speed
Edit: Important Note: The '\r' that resets the cursor position doesn't work in every terminal - it may still move to a new line for the next print() instead of the start of the same line - but this should most likely be fine in your system Terminal/cmd etc... May not be a good idea to run this in IDLE :P
Edit 2: Based on your comment, for a blinking indicator (using the same approach) -
i, d = 0, 300000
while True:
i = (i+1)%d
print("\r Working... " + ("█" if i < d/2 else " "), end='')
I try to create a small GUI with tkinter. To make my code more readable I want to split the code in 2 files. One for the GUI information and one for the process informations. Or is it a bad idea?
So I create a gui.py where I import my process informations from program.py.
gui.py:
import tkinter as tk
from program import *
root = tk.Tk()
btn_Start = tk.Button(text="Start", command=start_loop)
btn_Stop = tk.Button(text="Stop", command=stop_loop, state=tk.DISABLED)
btn_Start.grid(row=1, column=0)
btn_Stop.grid(row=1, column=1)
root.mainloop()
program.py:
def start_loop():
print('disable Start button and enable Stop button')
# what is the code to disable the start button and enable the stop button?
def stop_loop():
print('disable Stop button and enable Start button')
# what is the code to disable the stop button and enable the start button?
How do I tell the button the disable/enable information in my program.py file? I do not understand how I get the information from the gui to the program and back to the gui?
Thanks for your help
For such a small program, it is overkill.
Looking at the tkinter programs I've written, all of them are between 200−300 lines. This is including headers, comments and blank lines. The number of actual code lines is 100−200.
In my opinion that is small enough to comfortably handle in an editor in one file.
Looking over most of my source code repositories, the longest Python files tend to top out at around 230 lines of actual code.
Keeping all the code in one file has significant advantages for a program when you need to install it. Just copy the file and you're done. No need for modules and a setup.py.
How can I print the status of a Jupyter notebook on the previous line? I think I'm looking for something like clear_output(), but for only a single line.
Sample code:
from IPython.display import clear_output
import time
print('This is important info!')
for i in range(100):
print('Processing BIG data file {}'.format(i))
time.sleep(0.1)
clear_output(wait=True)
if i == 50:
print('Something bad happened on run {}. This needs to be visible at the end!'.format(i))
print('Done.')
When this runs it gives the behavior of over-writing the previous status line, but both of the lines marked as important (with exclamation points and everything!) are lost. When this is finished the display just says:
Done.
What it should say is:
This is important info!
Something bad happened on run 50. This needs to be visible at the end!
Done.
This post suggests using clear_output() and then reprinting everything. This seems impractical because the amount of data I really tend to display is big (lots of graphs, dataframes, ...).
Here's two SO links about clear_output().
Is there a way to make this work that doesn't involve reprinting everything?
I got it woking using the update function of the display handle:
from IPython.display import display
from time import sleep
print('Test 1')
dh = display('Test2',display_id=True)
sleep(1)
dh.update('Test3')
This will do it:
import IPython
IPython.display.display_javascript(r'''
var el = document.querySelector('.output_text:last-of-type > pre');
el.innerHTML = el.innerHTML.replace(/(\n.*$)/gm,""); ''', raw=True)
This simple escape sequence trick can do the job most of the time. \n and \r can do a lot if used properly.
\r: (Carriage Return) (CR) returns the cursor to the beginning of the new line without moving to a new line.
\n:(Line Feed) (LF) moves the cursor to the next line.
import time
print('This is important info!')
for i in range(100):
print("\r"+'Processing BIG data file {}'.format(i),end="")
time.sleep(0.1)
if i == 50:
print("\r"+'Something bad happened on run {}. This needs to be visible at the end!'.format(i))
print("\r"+'Done.')
I am new to python. So I wanted to improve my skills. Before posting this question I tried to find some code or an idea that would guide me with what I intend to do. I did see some examples and posts on SO and other sites. But they all(the ones I came across) showed how to do it for single object. Below is what I want to do.
I want to write a utility in python that would allow me to choose from content I want to paste based on what my last 10,say, copy commands were for.
suppose I clicked copy when selecting a folder and then later I selected some text and pressed ctrl+c. Now I want to get option that would let me paste both the folder as well as the text.
Is that possible?
Thanks.
You could save the last 10 text items from a clipboard using tkinter:
#!/usr/bin/env python3
from tkinter import Tk
from collections import deque
def call_repeatedly(root, delay, func, *args):
func(*args)
root.after(delay, call_repeatedly, root, delay, func, *args)
def poll_clipboard(root, items):
text = root.clipboard_get()
if not items or items[-1] != text:
items.append(text)
def main():
root = Tk()
root.withdraw() # hide GUI
clipboard_items = deque(maxlen=10) # save last 10 clipboard items
call_repeatedly(root, 50, poll_clipboard, root, clipboard_items) # ms
call_repeatedly(root, 1000, print, clipboard_items) # print every second
root.after(10000, root.destroy) # exit in 10 seconds
root.mainloop()
main()
It polls clipboard every 50 ms. Polling is bad in general if there is an alternative interface that could allow you to subscribe to the clipboard events to be notified when new item is copied into the clipboard.
will it work for any kind of content, text, images etc. ?
This code works with text only. In general, you could get/set other types e.g., images (gtk, qt might provide a cross-platform way to do it).
Will it allow me to copy-paste text across all the applications ?
You are working with a clipboard so yes, it should work across all applications that can
work with a clipboard.
Can we make it work as normal ctrl+c (copy command)
Copy command can be implemented using a set command e.g., from pyperclip.py:
def gtkSetClipboard(text):
cb = gtk.Clipboard()
cb.set_text(text)
cb.store()
gtkSetClipboard(text) copies text to the clipboard.
An (old) instrument of mine is generating ASCII data files with text descriptions at the top of the file, before the data. But the number of lines of descriptive text varies from run to run. How can I get Fortran77 to determine this automatically?
Here is an example data file, below the line.
Line of explanatory text.
Notice the possible blank lines.
More text.
The number of lines is NOT the same every time.
1.0, 2.0
2.0, 4.0
3.0, 6.0
4.0, 8.0
[I found the answer myself. Posting here to help others. It is quite annoying having to wait 8 hours to answer my own question, but I understand why the rule exists. Stupid posers!]
A crude but effective solution, if your text never starts with a number (which is my case):
Assume the input file is named Data.dat.
integer NumTextLines
real X
open(8,"Data.dat")
NumTextLines=-1
50 NumTextLines=NumTextLines+1
read(8,*,err=50) X
close(8)
open(8,"Data.dat")
Every time the program tries to read a word from a text line into the real variable X, the read statement errors and program control goes back to line 50. If the read statement is successful, then you don't want to increment NumTextLines any more. Close the file and re-open it to start over from the beginning. But now you know NumTextLines. So you can read the text one line at a time, and either save it or skip it.
{Above method works on most of my files, but not all. Another way is to read each line into a character*500 variable (say, A), then test the ASCII value of the first element of the character array. But that gets complicated.}