Examples

Existing modules

To come:

Running multiple experiments

We advise you to use tmux or screen to run experiments.

list_lr=(0.1 0.08 0.06 0.04)
gpu=0
for lr in $list_lr
do
  CUDA_VISIBLE_DEVICES=$gpu python -m bootstrap.run
        -o mnist/options/sgd.yaml
        --optimizer.lr
        --exp.dir logs/mnist/lr_$lr &
  gpu = $(( $gpu + 1 ))
done

tail logs/mnist/lr_0.1/logs.txt

Comparing experiments

We provide some utility functions to compare multiple experiments based on their best score for specific metrics.

python -m bootstrap.compare -h
usage: compare.py [-h] [-n NB_EPOCHS] [-d [DIR_LOGS [DIR_LOGS ...]]]
                  [-k metric order]

optional arguments:
  -h, --help            show this help message and exit
  -n NB_EPOCHS, --nb_epochs NB_EPOCHS
  -d [DIR_LOGS [DIR_LOGS ...]], --dir_logs [DIR_LOGS [DIR_LOGS ...]]
  -k metric order, --keys metric order

python -m bootstrap.compare
       -d logs/mnist/adam
          logs/mnist/sgd
> Metric: eval_epoch.accuracy_top1

    Place  Method      Score    Epoch
  -------  --------  -------  -------
        1  sgd       98.4773        9
        2  adam      98.3212        9

Plotting logs manually

The plotting utilities of Bootstrap are included in bootstrap/views. A bootstrap.views.view.View is created during the initialization of the bootstrap.engines.engine.Engine. It is used to generate automaticaly a view.html file after each training or evaluation epoch. Nevertheless, you can call it manually by doing so:

python -m bootstrap.views.view
       -o logs/mnist/sgd/options.yaml
open logs/mnist/sgd/view.html

Other tricks

Creating an experiment directory with the current datetime:

python -m bootstrap.run
       -o mnist/options/sgd.yaml
       --exp.dir logs/mnist/`date "+%Y-%m-%d-%H-%M-%S"`_sgd