Skip to content

Progress

ProgressCounter class

A ProgressCounter object stores current progress in some task or subtask and reports it.

Progress reporting is not necessary, but it is convenient for many tasks, especially for long-running ones, to see the progress.

class ProgressCounter:
    def __init__(self, subtask_name, total_cnt,
                 report_limit=100, ext_logger=None, report_divisor=1):

Create a ProgressCounter object.

  • subtask_name — human-readable name of subtask to report. Please note that progress for some nested subtasks isn't supported (e.g. "stage 1", then "stage 1.1", then "stage 1" again). Instead, report progress of sequential subtasks if necessary ("stage 1", then "stage 2").
  • total_cnt — total count of progress steps.
  • report_limit — maximum count of reporting messages. 0 means reporting every step anyway.
  • ext_logger — logger for using it to report. It should be a logger tuned for Supervisely.
  • report_divisor — divide progress to fixed value. E.g., one may count progress of downloading in bytes, and report it in MBs with divisor = 2**20.

Methods

iter_done_report(self)

Consider iteration (step) is finished and report progress if necessary.

iters_done_report(self, count)

Consider count iterations are finished and report progress if necessary.

progress_counter_train

def progress_counter_train(total_epochs, cnt_iters_per_epoch, ext_logger=None):

The function returns default ProgressCounter object for NN training task.

  • total_epochs — total count of epochs.
  • cnt_iters_per_epoch — count of iterations (may be mini-batches) for epoch.
  • ext_logger — logger for using it to report. It should be a logger tuned for Supervisely.

progress_counter_inference

def progress_counter_inference(cnt_imgs, ext_logger=None):

The function returns default ProgressCounter object for inference (NN applying) task.

  • cnt_imgs — total count of images to process.
  • ext_logger — logger for using it to report. It should be a logger tuned for Supervisely.

epoch_float

def epoch_float(epoch, train_it, train_its):
    res = epoch + train_it / float(train_its)
    return res

epoch_float function calculates training progress as float value (since zero), which is useful to determine some moment in training process unambiguously (e.g. to distinct checkpoints and collect metrics).

  • epoch is integer epoch number (since zero)
  • train_it — current training iteration (may be number of current mini-batch)
  • train_its — count of training iterations in epoch

Epoch means single pass of full training dataset. De facto it isn't required to calculate epochs if you prefer describe trainings in terms of iterations. However please note that now Supervisely infrastructure supports trainings consist of hundreds (not millions) epochs.