The Trial Class

The core class in torchbearer is Trial. This class contains the Trial.run() method which performs the main train-eval loop. In this note we’ll go into detail regarding how to create and use a trial. We’ll start with instantiation before looking at loading data (or running without data) and finally covering means of controlling verbosity. To get an understanding of actually running a trial and using metrics / callbacks, have a look at our example library.

Instantiation

If desired, we can instantiate a trial with no arguments by passing None as the model argument. However, assuming we have a model, we can simply write:

from torchbearer import Trial
trial = Trial(model)

If we would like our trial also perform optimization of some criterion we can use:

from torchbearer import Trial
trial = Trial(model, optimizer, criterion)

Criterions

Criterions can be given to a trial either as standard torch.nn criterions (i.e. functions of y_true, y_pred) or as functions of state. State is passed around a lot in torchbearer and contains (mutably) all of the different variables that are relevant to the fitting process at that point in time. Underneath it’s just a dictionary but should be accessed with StateKey objects. The list of built-in state keys can be found here. In the case of criterions, this

from torchbearer import Trial

def my_criterion(y_pred, y_true):
    return (y_pred - y_true).abs()

trial = Trial(model, optimizer, my_criterion)

is equivalent to

import torchbearer
from torchbearer import Trial

def my_criterion(state):
    return (state[torchbearer.PREDICTION] - state[torchbearer.TARGET]).abs()

trial = Trial(model, optimizer, my_criterion)

Loading Data

To load data into a trial we can use generators (such as a torch.utils.data.DataLoader) or tensors. We can further use different methods for loading train, val and test data or load all at once. Finally, we can let torchbearer decide how many steps per epoch to perform or tell it explicitly. All of these methods mutate the underlying trial and are chainable.

Generators

To populate the trial with a train generator we can use the Trial.with_train_generator() method. Equivalent methods Trial.with_val_generator() and Trial.with_test_generator() exist for validation and test data respectively. To use this method we write

from torchbearer import Trial

trial = Trial(model).with_train_generator(train_loader)

For simplicity, we can also load several data sets in one call using Trial.with_generators() like this

from torchbearer import Trial

trial = Trial(model).with_generators(train_loader, val_loader, test_loader)

To control the number of steps, we can either pass an integer argument steps to the with_XXX_generator methods or pass train_steps, val_steps and test_steps individually to Trial.with_generators(). Finally, we can use: Trial.for_train_steps(), Trial.for_val_steps(), Trial.for_test_steps(), Trial.for_steps(). That is, the following are all equivalent

trial = Trial(model).with_train_generator(train_loader, steps=10)
trial = Trial(model).with_generators(train_loader, train_steps=10)
trial = Trial(model).with_train_generator(train_loader).for_train_steps(10)
etc.

A final option is to tell the trial to run for infinitely many training steps (until stopped) for which we can use Trial.with_inf_train_loader(). For example

trial = Trial(model).with_train_generator(train_loader).with_inf_loader()

For more info on data loaders see the custom data loaders example

Tensors

If we want to load tensors instead we can use the with_XXX_data methods or the Trial.with_data() method in much the same way as before. There are some additional arguments to control batch size, shuffle and number of workers. Here are some examples:

# Shuffled training data
trial = Trial(model).with_train_data(x, y, shuffle=True, batch_size=128)

# Test data (no targets)
trial = Trial(model).with_test_data(x, batch_size=128)

# with_data
trial = Trial(model).with_data(x_train, y_train, x_val, y_val, x_test, shuffle=True, batch_size=128)

To change the number of steps we can use the same steps arguments or for_steps methods as before.

Running Without Data

If we want to run an optimisation or similar which does not require data, we simply call the for_steps methods without calling any with_generator / with_data methods. For example, to run for 100 train steps per cycle without any data, we use:

trial = Trial(model).for_train_steps(100)

In this case, the model will be given None as input at each step.

Controlling Verbosity

The verbosity of a trial can be controlled in two ways. First, a global verbosity is set in the init. Second each of the run / evaluate / predict methods can take a local verbosity argument which gets priority. If verbose=2, the Tqdm callback will be loaded with on_epoch=False so that a new progress bar is created for each epoch. If verbose=1, the Tqdm callback will be loaded with on_epoch=True so that only one progress bar is created. If verbose=0, no Tqdm callback will be loaded so that the trial produces no output. The default behaviour is verbose=2. For example, to suppress output we can use:

trial = Trial(model)
trial.run(10, verbose=0)