Skip to content

[Python] Displaying Progress Bars Using the tqdm Package and Attempting to Implement Your Own Progress Bar

Last Updated on 2024-07-27 by Clay

When we write Python code, sometimes we want to visualize the 'progress bar' to help us understand where our program is currently executing. For this need, instead of writing the progress bar ourselves, we can consider using a well-known progress bar module in Python—tqdm.

You can also visit the official website of tqdm: https://tqdm.github.io/ to read more detailed instructions.

In fact, tqdm also means 'progress bar' in Arabic. By installing this Python package, we can use tqdm to display progress bars in any iterable.

Now, let's start introducing how to use tqdm!

How to Use tqdm

Since this module is not built-in with Python, we can install it using the pip command if we are using it for the first time:

pip3 install tqdm

After installation, let's look at a sample code:

import time
from tqdm import tqdm

for i in tqdm(range(1000)):
    time.sleep(0.01)

Output:

100%|██████████| 1000/1000 [00:10<00:00, 95.76it/s]

We will see the progress bar gradually filling up. The beginning naturally is the progress bar we long for. The '1000/1000' is actually 'current iteration count / total iteration count', '00:10<00:00' is 'current execution time / estimated remaining time', and '95.76it/s' is the iterations per second. Sometimes you might see it reversed as 's/it', which means the time taken for each iteration in seconds.

Additionally, the time.sleep() function makes the program pause for 0.01 seconds each time it runs here, a total of 1000 times. If we don't set a pause, we might see the progress bar executed by tqdm finish instantly, which would not simulate the actual execution situation. You can test it out for yourself.

The tqdm() is effective for all iterable objects, for example, it can also be used for a simple list.

import time
from tqdm import tqdm

arr = [1, 2, 3, 4, 5]

for i in tqdm(arr):
    time.sleep(1)


Output:

100%|█████████████████████████████████████████████| 5/5 [00:05<00:00,  1.00s/it]


Additionally, tqdm also provides a trange() function to directly iterate for a specified length, which can be thought of as equivalent to tqdm(range(x)).

import time
from tqdm import trange

for i in trange(1000):
    time.sleep(0.01)


Output:

100%|██████████| 1000/1000 [00:10<00:00, 95.72it/s]


This is the basic usage of tqdm, very straightforward.

Advanced Techniques for tqdm

Next, I will mention some handy tips:

  • Change progress bar length
  • Change unit
  • Change progress bar format
  • Change progress bar description
  • Change progress bar color
  • Use .update() to customize when and how much to update


Change Progress Bar Length

tqdm will automatically detect the screen width and adapt to just the right width of the progress bar, but we can also manually set it:

for i in tqdm(1000, ncols=100):
    time.sleep(0.01)


Output:

Above, ncols=None is the default situation without setting, and below, ncols=100 is with the specified width.


Change Unit

This function is not very commonly used, but when doing deep learning training, we often want to change the unit of progress from iterations to more intuitive epochs or steps. We can set the unit parameter for this purpose.

for i in tqdm(range(1000), unit="epoch"):
    time.sleep(0.01)


Output:

100%|██████████| 1000/1000 [00:10<00:00, 98.27epoch/s]


Change Progress Bar Format

The bar_format in tqdm allows us to customize the display format of the progress bar. Here are the format tokens we can use:

  • {l_bar}: Left part of the progress bar, including description and the bar
  • {bar}: The progress bar itself
  • {r_bar}: Right part of the progress bar, including counter, time estimation, etc.
  • {n}: Current progress count
  • {n_fmt}: Formatted current progress count
  • {total}: Total length of the progress bar
  • {total_fmt}: Formatted total length of the progress bar
  • {elapsed}: Elapsed time
  • {elapsed_s}: Elapsed time (seconds)
  • {elapsed_td}: Elapsed time (timedelta format)
  • {remaining}: Remaining time
  • {remaining_s}: Remaining time (seconds)
  • {remaining_td}: Remaining time (timedelta format)
  • {percentage}: Completion percentage
  • {rate}: Processing rate per second
  • {rate_fmt}: Formatted processing rate per second
  • {rate_inv}: Time taken for each item
  • {rate_inv_fmt}: Formatted time taken for each item
  • {postfix}: Postfix string for the progress bar

For example, I like a clean progress bar format, so I can write it like this:

for i in tqdm(range(100), bar_format="{l_bar}{bar}| {n_fmt}/{total_fmt}"):
     time.sleep(0.01)


Output:

100%|███████████████| 100/100


Change Progress Bar Description

We can also add some descriptions to the tqdm progress bar. Let's first look at the simplest way to set the description:

for i in tqdm(range(1000), desc="Processing..."):
     time.sleep(0.01)


Output:

Processing...: 100%|███████████| 1000/1000 [00:10<00:00, 98.77it/s]

This is very useful in nested loops or deep structure loops, at least we can print out which progress bar it is, such as epoch-1, epoch-2, ..., epoch-n.

Another handy trick is that we can also change the progress bar description during iteration! This allows for more immediate grasp of the data processing status. 

import time
import random
from tqdm import tqdm

# Initialize counters
good = 0
bad = 0

# Create progress bar
pbar = tqdm(range(1000), desc="Starting")

# Simulate a loop with dynamic desc update
for _ in pbar:
    # Simulate work
    time.sleep(0.01)
    
    # Randomly choose True or False
    if random.choice([True, True, False]):
        good += 1
    else:
        bad += 1
    
    # Update progress bar description
    pbar.set_description(f"Good: {good} Bad: {bad}")

# Close progress bar
pbar.close()


Output:

Good: 650 Bad: 350: 100%|████████████████| 1000/1000 [00:10<00:00, 98.17it/s]

In the process, I randomly select True or False, and use the .set_description() method of the progress bar instance pbar to update the current processing status.

This way, if there is any information that needs to be seen immediately during the iteration, it can be directly displayed.


Change Progress Bar Color

When running the progress bar, if the default color is too inconspicuous, we can also set the color for the colour parameter. However, this feature is not available in early versions of tqdm and is supported in versions like 4.38.0 and above.

for i in tqdm(range(1000), colour="green"):
     time.sleep(0.01)


Output:


Use .update() to Customize When and How Much to Update

Additionally, tqdm can not only be used in for loops, but we can also choose when to update the progress bar and how much to advance the iteration progress. Here is a simple example:

times = 0
progress = tqdm(total=1000)

while times < 1000:
    progress.update(1)
    times += 1
    sleep(0.01)



Output:

100%|██████████| 1000/1000 [00:10<00:00, 95.68it/s]

It looks like the same function, but this time we can choose when to call progress.update(), and also choose the amount to update().

Lastly, if you are using Google Colab, Kaggle Notebook, Jupyter Notebook, etc., the usage of tqdm should employ the notebook package provided:

from tqdm.notebook import tqdm

Create Your Own Progress Bar

Of course, besides using the tqdm module to implement our own progress bar, we can also manually write our own progress bar!

Here is a simple example for reference:

from time import sleep

curr = 0
total = 1000
total_block_num = 50

for n in range(total):
    curr += 1
    block_num = int(curr * total_block_num / total)
    space_num = total_block_num - block_num

    print(f"\r[{curr / total * 100:.2f}%]: [{'█' * block_num}{' ' * space_num}]", end="")
    sleep(0.01)


Output:

[100.00%]:[██████████████████████████████████████████████████];


Advanced Challenge

Next, I also tried to write the above method into a function that can take an iterable object to generate a progress bar, and also set a color parameter like tqdm.

from time import sleep
import sys
import itertools

def progress_bar(iterable, total=None, total_block_num=50, color="default"):
    if total is None:
        total = len(iterable)
        
    color_codes = {
        "default": "\033[0m",    # Default
        "red": "\033[91m",       # Red
        "green": "\033[92m",     # Green
        "yellow": "\033[93m",    # Yellow
        "blue": "\033[94m",      # Blue
        "magenta": "\033[95m",   # Magenta
        "cyan": "\033[96m",      # Cyan
        "white": "\033[97m",     # White
    }
    
    if color not in color_codes:
        color = "default"
    
    color_code = color_codes[color]
    
    for curr, item in enumerate(iterable, 1):
        block_num = int(curr * total_block_num / total)
        space_num = total_block_num - block_num
        
        progress_str = f"\r{color_code}[{float(curr / total * 100):.2f}%]: [{'█' * block_num}{' ' * space_num}]\033[0m"
        print(progress_str, end="")
        
        yield item
        
    print()  # Move to the next line after completion


if __name__ == "__main__":
    for _ in progress_bar(range(1000), color="cyan"):
        sleep(0.01)


Output:

If you're interested, why not try writing your own progress bar? My current progress bar implementation is still quite rigid and can't freely choose update times and quantities like tqdm can — but this is an area we can improve on later.

Try challenging yourself to write it as a class, or allow users to update the progress using an update() function?

When I have some free time, I'd like to take on this challenge too.

Would you like me to explain or elaborate on any part of this translation?

References

Read More

Tags:

Leave a Reply