I want to reset a tqdm progress bar.
This is my code:
s = tqdm(range(100))
for x in s:
pass
# Reset it here
s.reset(0)
for x in s:
pass
Tqdm PB works only for the first loop. I tried to reset it using .reset(0) function but it doesn't work.
The ouput of the above code is:
100%|██████████| 100/100 [00:00<?, ?it/s]
I noticed that they use here: Restting progress bar counter this code
pbar.n = 0
pbar.refresh()
but it doesn't work as well.
When wrapping an iterable, tqdm will close() the bar when the iterable has been exhausted. This means reusing (refresh() etc) won't work. You can solve your problem manually:
from tqdm import tqdm
s = range(100)
t = tqdm(total=len(s))
for x in s:
t.update()
t.refresh() # force print final state
t.reset() # reuse bar
for x in s:
t.update()
t.close() # close the bar permanently
Try just creating a new progress bar over the old one. The garbage collector will take care of the old one afterwards, getting it out of memory once nothing in the code references it any more.
s = tqdm(range(100))
for x in s:
pass
# reset it here
s = tqdm(range(100))
for x in s:
pass
Related
Still learning tqdm, is there a way to make a progress bar and update within a helper function?
I have checked out other questions, but I couldn't find the answer, like
How to "flush" tqdm progress bar explicitly?
What I have also tried:
from tqdm import tqdm
import time
def helper_function(progress, outer_total):
for i in range(100):
# do some work
time.sleep(1)
progress.update(1/outer_total/100)
return
def main_function():
with tqdm(total=1000, desc="Main loop", unit="iter") as progress:
for j in range(10):
# do some work
time.sleep(1)
helper_function(progress, 1000)
main_function()
> Main loop: 0%| | 0.00999999999999976/1000 [00:00<01:37, 10.22iter/s]
Not really what I wanted.
Somehow it's stuck at infinite loop and always 0%
I pretty much just want %, how do I remove the other things on the progress bar?
Any help would be great.
Example i was try use progress.bar, alive_progress.
from alive_progress import alive_bar
with alive_bar(100) as bar:
for i in range(100):
for i in combinations_with_replacement(['a','b','c','a','b','c','a','b'], 8):
b = (''.join(i))
bar()
and i hv many annoing problems or it slow down script , not work as expecter with print ( print simple print over this bar or its creates new bar after any print.
If i run it with no bar/counting it finish at ~0.001 second .
as exmaple i was try with tqdm
from itertools import *
from tqdm import trange
for i in trange(100):
for i in combinations_with_replacement(['a','b','c','a','b','c','a','b','a','b','c','a','b','c','a','b','a','b','c','a','b','c','a','b'], 8):
b = (''.join(i))
if i dissble it it finish less than second , if i use it , it expect time is 3 m , how to this this and not slowdown script?
UPD : if belive info what find at goodle that progress bars and counters for pyhon is not usable , only if you dont care bout 1000% slowdown script.
but some how progress bars is used at any brutforce scripts , generators etc...
i dont undestand .
from itertools import *
from collections import Counter
from tqdm import *
#for i in tqdm(Iterable):
limit = product(['0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f'], repeat = 8)
for i in trange(100):
for j in limit:
b = (''.join(j))
if b in ['bbbbbbbb','11111111','aaaaaaaa','00001111']:
print (b)
its static before script is and so to not work. ( stay at 0% all runing and go instantly to 100% when it end)
You can commnent logic "if b in and print" notting no change .
The reason the script is slowing down is that the combinations_with_replacement() function is being called for each progress bar update.
I calculated the value outside the loop and the output was instant.
The rate of execution went from 2 s/it (0.5 it/s) -> 60 it/s
from itertools import *
from tqdm import trange
limit = combinations_with_replacement(['a','b','c','a','b','c','a','b','a','b','c','a','b','c','a','b','a','b','c','a','b','c','a','b'], 8)
for i in trange(100):
for j in limit:
b = (''.join(j))
I'm writing a script where a user has to provide input for each element of a large list. I'm trying to use tqdm to provide a progress bar for the user, but I can't find a good way to get input within the tqdm loop without breaking the output.
I'm aware of tqdm.write() for writing to the terminal during a tqdm loop, but is there a way of getting input?
For an example of what I'm trying to do, consider the code below:
from tqdm import tqdm
import sys
from time import sleep
def do_stuff(x): sleep(0.5)
stuff_list = ['Alpha', 'Beta', 'Gamma', 'Omega']
for thing in tqdm(stuff_list):
input_string = input(thing + ": ")
do_stuff(input_string)
If I run this code, I get the following output:
0%| | 0/4 [00:00<?, ?it/s]Alpha: A
25%|█████████████████████ | 1/4 [00:02<00:07, 2.54s/it]Beta: B
50%|██████████████████████████████████████████ | 2/4 [00:03<00:04, 2.09s/it]Gamma: C
75%|███████████████████████████████████████████████████████████████ | 3/4 [00:04<00:01, 1.72s/it]Omega: D
100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:05<00:00, 1.56s/it]
I've tried using tqdm.external_write_mode, but this simply didn't display the progress bar whenever an input was waiting, which is not the behaviour I'm looking for.
Is there an easy way of doing this, or am I going to have to swap libraries?
It isn't possible to display the progress bar while inside the input() function, because once a line is finished, it cannot be removed any more. It's a technical limitation of how command lines work. You can only remove the current line until you wrote a newline.
Therefore, I think the only solution is to remove the status bar, let the user input happen and then display it again.
from tqdm import tqdm
import sys
from time import sleep
def do_stuff(x): sleep(0.5)
stuff_list = ['Alpha', 'Beta', 'Gamma', 'Omega']
# To have more fine-control, you need to create a tqdm object
progress_iterator = tqdm(stuff_list)
for thing in progress_iterator:
# Remove progress bar
progress_iterator.clear()
# User input
input_string = input(thing + ": ")
# Write the progress bar again
progress_iterator.refresh()
# Do stuff
do_stuff(input_string)
If you don't like the fact that the progress_iterator object exists after the loop, use the with syntax:
with tqdm(stuff_list) as progress_iterator:
for thing in progress_iterator:
...
EDIT:
If you are willed to sacrifice platform independence, you can freely move the cursor and delete lines with this:
from tqdm import tqdm
import sys
from time import sleep
def do_stuff(x): sleep(0.5)
stuff_list = ['Alpha', 'Beta', 'Gamma', 'Omega']
# Special console commands
CURSOR_UP_ONE = '\x1b[1A'
# To have more fine-control, you need to create a tqdm object
progress_iterator = tqdm(stuff_list)
for thing in progress_iterator:
# Move the status bar one down
progress_iterator.clear()
print(file=sys.stderr)
progress_iterator.refresh()
# Move the cursor back up
sys.stderr.write('\r')
sys.stderr.write(CURSOR_UP_ONE)
# User input
input_string = input(thing + ": ")
# Refresh the progress bar, to move the cursor back to where it should be.
# This step can be omitted.
progress_iterator.refresh()
# Do stuff
do_stuff(input_string)
I think this is the closest you will get to tqdm.write(). Note that the behaviour of input() can never be identical to tqdm.write(), because tqdm.write() first deletes the bar, then writes the message, and then writes the bar again. If you want to display the bar while being in input(), you have to do some platform-dependent stuff like this.
That's the basic idea.
I couldn't understand them while reading about all this in python's documentation. There's simply too much of it to understand it.
Could someone explain me how to get a working progress bar with a multi-threaded(or multi-process) client?
Or is there any other way to "update" progress bar without locking program's GUI?
Also I did read something about "I/O" errors when these types of clients try to access a file at the same time & X-server errors when an application doesn't call multi-thread libs correctly. How do I avoid 'em?
This time I didn't write code, I don't want to end with a zombie-process or something like that (And then force PC to shutdown, I would hate to corrupt precious data...). I need to understand what am I doing first!
Something like the below might get you started? - Ive tried to annotate as much as possible to explain the process.
from Tkinter import *
from Queue import Queue
import ttk, threading
import time
queue = Queue()
root = Tk()
# Function to do 'stuff' and place object in queue for later #
def foo():
# sleep to demonstrate thread doing work #
time.sleep(5)
obj = [x for x in range(0,10)]
queue.put(obj)
# Create thread object, targeting function to do 'stuff' #
thread1 = threading.Thread(target=foo, args=())
# Function to check state of thread1 and to update progressbar #
def progress(thread, queue):
# starts thread #
thread.start()
# defines indeterminate progress bar (used while thread is alive) #
pb1 = ttk.Progressbar(root, orient='horizontal', mode='indeterminate')
# defines determinate progress bar (used when thread is dead) #
pb2 = ttk.Progressbar(root, orient='horizontal', mode='determinate')
pb2['value'] = 100
# places and starts progress bar #
pb1.pack()
pb1.start()
# checks whether thread is alive #
while thread.is_alive():
root.update()
pass
# once thread is no longer active, remove pb1 and place the '100%' progress bar #
pb1.destroy()
pb2.pack()
# retrieves object from queue #
work = queue.get()
return work
work = progress(thread1, queue)
root.mainloop()
Hope this helps. Let me know what you think!
I am attempting to create some simple asynchronously-executing animations based on ipythonblocks and I am trying to update the cell output area using clear_output() followed by a grid.show().
For text output the basis of the technique is discussed in Per-cell output for threaded IPython Notebooks so my simplistic assumption was to use the same method to isolate HTML output. Since I want to repeatedly replace a grid with its updated HTML version I try to use clear_output() to ensure that only one copy of the grid is displayed.
I verified that this proposed technique works for textual output with the following cells. First the context manager.
import sys
from contextlib import contextmanager
import threading
stdout_lock = threading.Lock()
n = 0
#contextmanager
def set_stdout_parent(parent):
"""a context manager for setting a particular parent for sys.stdout
(i.e. redirecting output to a specific cell). The parent determines
the destination cell of output
"""
global n
save_parent = sys.stdout.parent_header
# we need a lock, so that other threads don't snatch control
# while we have set a temporary parent
with stdout_lock:
sys.stdout.parent_header = parent
try:
yield
finally:
# the flush is important, because that's when the parent_header actually has its effect
n += 1; print("Flushing", n)
sys.stdout.flush()
sys.stdout.parent_header = save_parent
Then the test code
import threading
import time
class timedThread(threading.Thread):
def run(self):
# record the parent (uncluding the stdout cell) when the thread starts
thread_parent = sys.stdout.parent_header
for i in range(3):
time.sleep(2)
# then ensure that the parent is the same as when the thread started
# every time we print
with set_stdout_parent(thread_parent):
print(i)
timedThread().start()
This provided the output
0
Flushing 1
1
Flushing 2
2
Flushing 3
So I modified the code to clear the cell between cycles.
import IPython.core.display
class clearingTimedThread(threading.Thread):
def run(self):
# record the parent (uncluding the stdout cell) when the thread starts
thread_parent = sys.stdout.parent_header
for i in range(3):
time.sleep(2)
# then ensure that the parent is the same as when the thread started
# every time we print
with set_stdout_parent(thread_parent):
IPython.core.display.clear_output()
print(i)
clearingTimedThread().start()
As expected the output area of the cell was repeatedly cleared, and ended up reading
2
Flushing 6
I therefore thought I was on safe ground in using the same technique to clear a cell's output area when using ipythonblocks. Alas no. This code
from ipythonblocks import BlockGrid
w = 10
h = 10
class clearingBlockThread(threading.Thread):
def run(self):
grid = BlockGrid(w, h)
# record the parent (uncluding the stdout cell) when the thread starts
thread_parent = sys.stdout.parent_header
for i in range(10):
# then ensure that the parent is the same as when the thread started
# every time we print
with set_stdout_parent(thread_parent):
block = grid[i, i]
block.green = 255
IPython.core.display.clear_output(other=True)
grid.show()
time.sleep(0.2)
clearingBlockThread().start()
does indeed produce the desired end state (a black matrix with a green diagonal) but the intermediate steps don't appear in the cell's output area. To complicate things slightly (?) this example is running on Python 3. In checking before posting here I discover that the expected behavior (a simple animation) does in fact occur under Python 2.7. Hence I though to ask whether this is an issue I need to report.