I'm writing a script where a user has to provide input for each element of a large list. I'm trying to use tqdm to provide a progress bar for the user, but I can't find a good way to get input within the tqdm loop without breaking the output.
I'm aware of tqdm.write() for writing to the terminal during a tqdm loop, but is there a way of getting input?
For an example of what I'm trying to do, consider the code below:
from tqdm import tqdm
import sys
from time import sleep
def do_stuff(x): sleep(0.5)
stuff_list = ['Alpha', 'Beta', 'Gamma', 'Omega']
for thing in tqdm(stuff_list):
input_string = input(thing + ": ")
do_stuff(input_string)
If I run this code, I get the following output:
0%| | 0/4 [00:00<?, ?it/s]Alpha: A
25%|█████████████████████ | 1/4 [00:02<00:07, 2.54s/it]Beta: B
50%|██████████████████████████████████████████ | 2/4 [00:03<00:04, 2.09s/it]Gamma: C
75%|███████████████████████████████████████████████████████████████ | 3/4 [00:04<00:01, 1.72s/it]Omega: D
100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:05<00:00, 1.56s/it]
I've tried using tqdm.external_write_mode, but this simply didn't display the progress bar whenever an input was waiting, which is not the behaviour I'm looking for.
Is there an easy way of doing this, or am I going to have to swap libraries?
It isn't possible to display the progress bar while inside the input() function, because once a line is finished, it cannot be removed any more. It's a technical limitation of how command lines work. You can only remove the current line until you wrote a newline.
Therefore, I think the only solution is to remove the status bar, let the user input happen and then display it again.
from tqdm import tqdm
import sys
from time import sleep
def do_stuff(x): sleep(0.5)
stuff_list = ['Alpha', 'Beta', 'Gamma', 'Omega']
# To have more fine-control, you need to create a tqdm object
progress_iterator = tqdm(stuff_list)
for thing in progress_iterator:
# Remove progress bar
progress_iterator.clear()
# User input
input_string = input(thing + ": ")
# Write the progress bar again
progress_iterator.refresh()
# Do stuff
do_stuff(input_string)
If you don't like the fact that the progress_iterator object exists after the loop, use the with syntax:
with tqdm(stuff_list) as progress_iterator:
for thing in progress_iterator:
...
EDIT:
If you are willed to sacrifice platform independence, you can freely move the cursor and delete lines with this:
from tqdm import tqdm
import sys
from time import sleep
def do_stuff(x): sleep(0.5)
stuff_list = ['Alpha', 'Beta', 'Gamma', 'Omega']
# Special console commands
CURSOR_UP_ONE = '\x1b[1A'
# To have more fine-control, you need to create a tqdm object
progress_iterator = tqdm(stuff_list)
for thing in progress_iterator:
# Move the status bar one down
progress_iterator.clear()
print(file=sys.stderr)
progress_iterator.refresh()
# Move the cursor back up
sys.stderr.write('\r')
sys.stderr.write(CURSOR_UP_ONE)
# User input
input_string = input(thing + ": ")
# Refresh the progress bar, to move the cursor back to where it should be.
# This step can be omitted.
progress_iterator.refresh()
# Do stuff
do_stuff(input_string)
I think this is the closest you will get to tqdm.write(). Note that the behaviour of input() can never be identical to tqdm.write(), because tqdm.write() first deletes the bar, then writes the message, and then writes the bar again. If you want to display the bar while being in input(), you have to do some platform-dependent stuff like this.
Related
Still learning tqdm, is there a way to make a progress bar and update within a helper function?
I have checked out other questions, but I couldn't find the answer, like
How to "flush" tqdm progress bar explicitly?
What I have also tried:
from tqdm import tqdm
import time
def helper_function(progress, outer_total):
for i in range(100):
# do some work
time.sleep(1)
progress.update(1/outer_total/100)
return
def main_function():
with tqdm(total=1000, desc="Main loop", unit="iter") as progress:
for j in range(10):
# do some work
time.sleep(1)
helper_function(progress, 1000)
main_function()
> Main loop: 0%| | 0.00999999999999976/1000 [00:00<01:37, 10.22iter/s]
Not really what I wanted.
Somehow it's stuck at infinite loop and always 0%
I pretty much just want %, how do I remove the other things on the progress bar?
Any help would be great.
I’m doing a home project with a raspi and a rain sensor.
Basically, with cronjobs, i run isitraining.py every 30 min.
If it is raining, i want it to increment a specific variable from another module (rains.water)
If it stops raining, i want it to decrease the same variable.
here are my codes:
rains.py
water=0
isitraining.py
import RPi.GPIO as GPIO
import rains
GPIO.setmode(GPIO.BCM)
GPIO.setup(4, GPIO.IN, pull_up_down=GPIO.PUD_UP)
def raincounter():
if (GPIO.input(4) and (0<rains.water<13)):
rains.water-=1 #No Rain
elif (GPIO.input(4)==0 and (0<rains.water<13)):
rains.water+=1 #Rain
testscript.py
import rains
import isitraining
isitraining.raincounter()
print (rains.water)
everytime i run the “testscript.py” for the first time, it does modify the “rains.water” variable but only one time, if i run the script any other time after that, it does not increment or decrease in value.
fyi
gpio.input(4) is a rain sensor with digital input. When it’s high, it means there’s no rain and when it’s low it means it is raining.
Any thoughts?
Every time testscript.py runs, it's importing isitraining.py fresh, from scratch, which then imports rains.py fresh, from scratch. rains.py and the water value it holds do not persist across runs.
If you want to save the value, you need to manually write it to disk, then load it later when you want to use it. Something like this:
isitraining.py:
import RPi.GPIO as GPIO
import rains
GPIO.setmode(GPIO.BCM)
GPIO.setup(4, GPIO.IN, pull_up_down=GPIO.PUD_UP)
def raincounter():
if (GPIO.input(4) and (0<rains.water<13)):
return rains.alter_rain(-1) #No Rain
elif (GPIO.input(4)==0 and (0<rains.water<13)):
return rains.alter_rain(1)
rains.py:
FILE_NAME = "rain_record.txt"
def alter_rain(n):
with open(FILE_NAME, "r") as f:
current_value = int(f.read()) # Get saved value
current_value += n # Alter it
with open(FILE_NAME, "w") as f:
f.write(str(current_value)) # Write the altered value back
return current_value # And return it so the caller knows the current value
testscript.py
import rains
import isitraining
new_val = isitraining.raincounter()
print (new_val)
You could combine the two calls to open, and should also include error handling, but this shows the general idea. You cannot use a simple script to act as a database for you like you're attempting to do. If you want to save something, you need to save it yourself.
New to programming and currently working on a web scraper. I'm trying to monitor my progress and keep my terminal clean as I go using clear_output() from IPython.display, but it doesn't appear to be clearing my output in VSCode. Here's an example:
from time import sleep
from IPython.display import clear_output
x = 0
while x < 5:
x += 1
sleep(.2)
print(x)
clear_output(wait=True)
I'd expect it to clear the previous value of x in place of the new value, but it doesn't. Once the program finishes, my output changes from:
>>>1
>>>2
>>>3
>>>4
>>>5
to:
>>>1
>>>2[2K
>>>3[2K
>>>4[2K
>>>5[2K
Guessing this is something from IPython, but I'm not sure what it means or how to fix it. Thanks for any help!
I'm not exactly sure if this is what you're asking but you could try using the sys library:
from time import sleep
from sys import stdout
for i in range(0, 1000):
stdout.write("\b") # \b for backspace, you may need \r depends on system
sleep(.9)
stdout.write("\b\b\b>>>" + str(i))
stdout.flush()
for me the output is:
>>>1
then the 1 is replaced by a 2
>>>2
2 replaced by 3 etc.
I threw in a sleep so you can see what happens, as stated in the comment you may need to change the \b to a \r or some combination of both
clear_output() only works in Jupyter. Consider using print('\n'100) to clear the output in other IDEs.
I want to reset a tqdm progress bar.
This is my code:
s = tqdm(range(100))
for x in s:
pass
# Reset it here
s.reset(0)
for x in s:
pass
Tqdm PB works only for the first loop. I tried to reset it using .reset(0) function but it doesn't work.
The ouput of the above code is:
100%|██████████| 100/100 [00:00<?, ?it/s]
I noticed that they use here: Restting progress bar counter this code
pbar.n = 0
pbar.refresh()
but it doesn't work as well.
When wrapping an iterable, tqdm will close() the bar when the iterable has been exhausted. This means reusing (refresh() etc) won't work. You can solve your problem manually:
from tqdm import tqdm
s = range(100)
t = tqdm(total=len(s))
for x in s:
t.update()
t.refresh() # force print final state
t.reset() # reuse bar
for x in s:
t.update()
t.close() # close the bar permanently
Try just creating a new progress bar over the old one. The garbage collector will take care of the old one afterwards, getting it out of memory once nothing in the code references it any more.
s = tqdm(range(100))
for x in s:
pass
# reset it here
s = tqdm(range(100))
for x in s:
pass
I am attempting to create some simple asynchronously-executing animations based on ipythonblocks and I am trying to update the cell output area using clear_output() followed by a grid.show().
For text output the basis of the technique is discussed in Per-cell output for threaded IPython Notebooks so my simplistic assumption was to use the same method to isolate HTML output. Since I want to repeatedly replace a grid with its updated HTML version I try to use clear_output() to ensure that only one copy of the grid is displayed.
I verified that this proposed technique works for textual output with the following cells. First the context manager.
import sys
from contextlib import contextmanager
import threading
stdout_lock = threading.Lock()
n = 0
#contextmanager
def set_stdout_parent(parent):
"""a context manager for setting a particular parent for sys.stdout
(i.e. redirecting output to a specific cell). The parent determines
the destination cell of output
"""
global n
save_parent = sys.stdout.parent_header
# we need a lock, so that other threads don't snatch control
# while we have set a temporary parent
with stdout_lock:
sys.stdout.parent_header = parent
try:
yield
finally:
# the flush is important, because that's when the parent_header actually has its effect
n += 1; print("Flushing", n)
sys.stdout.flush()
sys.stdout.parent_header = save_parent
Then the test code
import threading
import time
class timedThread(threading.Thread):
def run(self):
# record the parent (uncluding the stdout cell) when the thread starts
thread_parent = sys.stdout.parent_header
for i in range(3):
time.sleep(2)
# then ensure that the parent is the same as when the thread started
# every time we print
with set_stdout_parent(thread_parent):
print(i)
timedThread().start()
This provided the output
0
Flushing 1
1
Flushing 2
2
Flushing 3
So I modified the code to clear the cell between cycles.
import IPython.core.display
class clearingTimedThread(threading.Thread):
def run(self):
# record the parent (uncluding the stdout cell) when the thread starts
thread_parent = sys.stdout.parent_header
for i in range(3):
time.sleep(2)
# then ensure that the parent is the same as when the thread started
# every time we print
with set_stdout_parent(thread_parent):
IPython.core.display.clear_output()
print(i)
clearingTimedThread().start()
As expected the output area of the cell was repeatedly cleared, and ended up reading
2
Flushing 6
I therefore thought I was on safe ground in using the same technique to clear a cell's output area when using ipythonblocks. Alas no. This code
from ipythonblocks import BlockGrid
w = 10
h = 10
class clearingBlockThread(threading.Thread):
def run(self):
grid = BlockGrid(w, h)
# record the parent (uncluding the stdout cell) when the thread starts
thread_parent = sys.stdout.parent_header
for i in range(10):
# then ensure that the parent is the same as when the thread started
# every time we print
with set_stdout_parent(thread_parent):
block = grid[i, i]
block.green = 255
IPython.core.display.clear_output(other=True)
grid.show()
time.sleep(0.2)
clearingBlockThread().start()
does indeed produce the desired end state (a black matrix with a green diagonal) but the intermediate steps don't appear in the cell's output area. To complicate things slightly (?) this example is running on Python 3. In checking before posting here I discover that the expected behavior (a simple animation) does in fact occur under Python 2.7. Hence I though to ask whether this is an issue I need to report.