Configuration for training the bulkdgd model

To train an instance of core.model.BulkDGD, we need to set a number of options.

These options can be passed as a nested dictionary or are specified in a YAML configuration file.

The function that loads the configuration file is bulkdgd.ioutil.load_config_train().

The options that can be specified are described below.

• "n_epochs" is the number of epochs to train the model for. This is a positive integer and defaults to 200.

• "loss_reduction_type" is the reduction method to use for the loss function. This can be:

  • "sum", which computes the sum of the loss over the batch. This is the default value if not specified.

  • "mean", which computes the mean of the loss over the batch.

• "data_loader_options" is a dictionary of options to initialize the data loaders used to load the data for training and testing. It can contain the following options:

  • "train" is a dictionary of options to initialize the data loader used to load the training data. It can contain the following options:

    • "batch_size" is the batch size to use for the training data loader. This is a positive integer and defaults to 64.

    • "shuffle" is a boolean that specifies whether to shuffle the training data at each epoch. It defaults to True.

  • "test" is a dictionary of options to initialize the data loader used to load the test data. It can contain the following options:

    • "batch_size" is the batch size to use for the test data loader. This is a positive integer and defaults to 64.

    • "shuffle" is a boolean that specifies whether to shuffle the test data at each epoch. It defaults to False.

• "reporting_options" is a dictionary of options for reporting. It can contain the following options:

  • "loss" is a dictionary of options to report the loss. It can contain the following options:

    • "latent" is a dictionary of options for the latent space loss. It can contain the following options:

      • "norm_type" is the method to use to normalize the loss when reporting it. This can be:

        • "none", which does not normalize the loss. This is the default value if not specified.

        • "n_samples", which normalizes the loss by the number of samples in the batch.

        • "n_samples * latent_dim", which normalizes the loss by the number of samples times the latent dimension.

    • "decoder" is a dictionary of options for the reconstruction loss. It can contain the following options:

      • "norm_type" is the method to use to normalize the loss when reporting it. This can be:

        • "none", which does not normalize the loss. This is the default value if not specified.

        • "n_samples", which normalizes the loss by the number of samples in the batch.

        • "n_samples * n_genes", which normalizes the loss by the number of samples times the number of genes.

    • "total" is a dictionary of options for the total loss. It can contain the following options:

      • "norm_type" is the method to use to normalize the loss when reporting it. This can be:

        • "none", which does not normalize the loss. This is the default value if not specified.

        • "n_samples", which normalizes the loss by the number of samples in the batch.

        • "n_samples * n_genes", which normalizes the loss by the number of samples times the number of genes.

  • "metrics" is a dictionary of options to report the metrics. It can contain the following options:

    • "latent" is a list of metrics to compute for the latent space during training. The available metrics are listed below. The default value is ["silhouette_score", "adjusted_rand_index_score"].

      Unsupervised metrics (do not require ground-truth labels):

      • "bic", which reports the Bayesian Information Criterion (BIC) of the latent space model. Lower is better.

      • "silhouette_score", which reports the silhouette score of the representations with respect to the cluster assignments given by the latent space model. Higher is better.

      • "davies_bouldin_score", which reports the Davies-Bouldin score of the representations with respect to the cluster assignments given by the latent space model. Lower is better.

      • "calinski_harabasz_score", which reports the Calinski-Harabasz score of the representations with respect to the cluster assignments given by the latent space model. Higher is better.

      Supervised metrics (require ground-truth labels):

      • "adjusted_rand_index_score", which reports the adjusted Rand index of the cluster assignments given by the latent space model with respect to the true labels. Higher is better.

      • "normalized_mutual_info_score", which reports the normalized mutual information of the cluster assignments given by the latent space model with respect to the true labels. Higher is better.

  • "optional_outputs" is a dictionary of options to specify which optional outputs to save during training. It can contain the following options:

    • "representations_epoch" is a dictionary of options to specify how to report the representations at the end of each epoch. It can contain the following options:

      • "enabled" is a boolean that specifies whether to save the representations at the end of each epoch. This defaults to False.

      • "stride" is the stride (in epochs) at which to save the representations. This is a positive integer that defaults to 1.

      • "dir" is the directory where to save the representations. If not specified, it defaults to None.

    • "latent_probs_epoch" is a dictionary of options to specify how to report the latent probabilities at the end of each epoch. It can contain the following options:

      • "enabled" is a boolean that specifies whether to save the latent probabilities at the end of each epoch. This defaults to False.

      • "stride" is the stride (in epochs) at which to save the latent probabilities. This is a positive integer that defaults to 1.

      • "dir" is the directory where to save the latent probabilities. If not specified, it defaults to None.

    • "latent_means_epoch" is a dictionary of options to specify how to report the latent means at the end of each epoch. It can contain the following options:

      • "enabled" is a boolean that specifies whether to save the latent means at the end of each epoch. This defaults to False.

      • "stride" is the stride (in epochs) at which to save the latent means. This is a positive integer that defaults to 1.

      • "dir" is the directory where to save the latent means. If not specified, it defaults to None.

    • "genes_saliency_maps_epoch" is a dictionary of options to specify how to report the genes’ saliency maps at the end of each epoch. It can contain the following options:

      • "enabled" is a boolean that specifies whether to save the genes’ saliency maps at the end of each epoch. This defaults to False.

      • "stride" is the stride (in epochs) at which to save the genes’ saliency maps. This is a positive integer that defaults to 1.

      • "dir" is the directory where to save the genes’ saliency maps. If not specified, it defaults to None.

    • "pathways_saliency_maps_epoch" is a dictionary of options to specify how to report the pathways’ saliency maps at the end of each epoch. It can contain the following options:

      • "enabled" is a boolean that specifies whether to save the pathways’ saliency maps at the end of each epoch. This defaults to False.

      • "stride" is the stride (in epochs) at which to save the pathways’ saliency maps. This is a positive integer that defaults to 1.

      • "dir" is the directory where to save the pathways’ saliency maps. If not specified, it defaults to None.

• "latent_type" is the type of latent space to use. This can be:

  • "lgmm" for the legacy Gaussian Mixture Model implementation.

  • "tgmm" for the TorchGMM implementation.

• "latent_training_options" is a dictionary of options to train the latent space. The options vary depending on the latent space implementation specified in "latent_type".

  • If the latent space implementation is the legacy Gaussian Mixture Model ("lgmm"):

    • "optimizer_type" is the type of optimizer to use for training the GMM. This can be:

      • "adam", which uses the Adam optimizer.

      • "adamw", which uses the AdamW optimizer. This is the default value if not specified.

    • "optimizer_options" is a dictionary of options for the optimizer. The options depend on the optimizer type. It can contain the following options:

      • "lr" is the learning rate. This is a positive float that defaults to 0.01.

      • "weight_decay" is the weight decay. This is a non-negative float that defaults to 0.0.

      • "betas" is a list of two floats that specify the beta parameters for the Adam or AdamW optimizer. The defaults are [0.9, 0.999].

    • "lr_scheduler_type" is the type of learning rate scheduler to use. This can be:

      • None, which does not use a learning rate scheduler. This is the default value if not specified.

      • "one_cycle", which uses the OneCycleLR scheduler.

    • "lr_scheduler_options" is a dictionary of options for the learning rate scheduler. For the "one_cycle" scheduler, the options are:

      • "max_lr" is the maximum learning rate to use. This is a positive float that defaults to 0.01.

      • "pct_start" is the percentage of the total number of epochs to use for the increasing phase of the learning rate schedule. This is a float between 0 and 1 that defaults to 0.25.

      • "anneal_strategy" is the annealing strategy to use. This can be:

        • "cos", which uses a cosine annealing strategy. This is the default value if not specified.

        • "linear", which uses a linear annealing strategy.

      • "cycle_momentum" is a boolean that specifies whether to cycle the momentum during training. This defaults to True.

      • "base_momentum" is the base momentum. This is a float between 0 and 1 that defaults to 0.85.

      • "max_momentum" is the maximum momentum. This is a float between 0 and 1 that defaults to 0.9.

      • "div_factor" is the factor by which to divide the maximum learning rate to get the initial learning rate. This is a positive float that defaults to 25.0.

      • "final_div_factor" is the factor by which to divide the initial learning rate to get the minimum learning rate at the end of training. This is a positive float that defaults to 1000.0.

      • "three_phase" is a boolean that specifies whether to use a three-phase learning rate schedule. This defaults to False.

    • "components_removal_type" is the type of components removal to use during training. This can be:

      • None, which disables component removal. This is the default value if not specified.

      • "weight_threshold", which removes components whose weight falls below a given threshold.

    • "components_removal_options" is a dictionary of options for the components removal. For the "weight_threshold" type, the options are:

      • "threshold" is the threshold below which a component’s weight triggers its removal. This is a non-negative float that defaults to 1e-8.

  • If the latent space implementation is the TorchGMM one ("tgmm"):

    • "loss_calculation" is a dictionary of options to specify how to calculate the latent space loss. It can contain the following options:

      • "lambda" is the weight to use for the latent space loss in the total loss. This is a non-negative float that defaults to 1.0.

    • "model_selection_type" is the type of model selection to use for selecting the best model during training. This can be:

      • None, which disables model selection. This is the default value if not specified.

      • "metric", which selects the best model based on a metric.

    • "model_selection_options" specifies the metric to use for model selection when "model_selection_type" is "metric". The available metrics are:

      • "bic", which uses the Bayesian Information Criterion (BIC). Lower is better. This is the default value if not specified.

      • "silhouette_score", which uses the silhouette score. Higher is better.

      • "calinski_harabasz_score", which uses the Calinski-Harabasz score. Higher is better.

      • "davies_bouldin_score", which uses the Davies-Bouldin score. Lower is better.

    • "fitting" is a dictionary of options to specify how to fit the GMM during training. It can contain the following options:

      • "first_epoch" is the epoch at which to start fitting the GMM. This is a non-negative integer that defaults to 25.

      • "refit_final" is a boolean that specifies whether to refit the GMM at the end of training. This defaults to True.

      • "refit_interval" is the interval (in epochs) at which to refit the GMM during training. This is a non-negative integer that defaults to 0 (no periodic refitting).

      • "max_iter_first_epoch" is the maximum number of EM iterations to use for the first epoch of GMM fitting. This is a positive integer that defaults to 1000.

      • "max_iter_full_refit" is the maximum number of EM iterations to use for full-refit epochs. This is a positive integer that defaults to 100.

      • "max_iter_warm_refit" is the maximum number of EM iterations to use for warm-refit epochs. This is a positive integer that defaults to 100.

      • "max_iter_final_refit" is the maximum number of EM iterations to use for the final refit at the end of training. This is a positive integer that defaults to 1000.

    • "components_removal_type" is the type of components removal to use during training. This can be:

      • None, which disables component removal. This is the default value if not specified.

      • "weight_threshold", which removes components whose weight falls below a given threshold.

    • "components_removal_options" is a dictionary of options for the components removal. For the "weight_threshold" type, the options are:

      • "threshold" is the threshold below which a component’s weight triggers its removal. This is a non-negative float that defaults to 1e-8.

• "decoder_training_options" is a dictionary of options to train the decoder. It can contain the following options:

  • "optimizer_type" is the type of optimizer to use for training the decoder. This can be:

    • "adam", which uses the Adam optimizer.

    • "adamw", which uses the AdamW optimizer. This is the default value if not specified.

  • "optimizer_options" is a dictionary of options for the optimizer. The options depend on the optimizer type. It can contain the following options:

    • "lr" is the learning rate. This is a positive float that defaults to 0.001.

    • "weight_decay" is the weight decay. This is a non-negative float that defaults to 0.0.

    • "betas" is a list of two floats that specify the beta parameters for the Adam or AdamW optimizer. The defaults are [0.9, 0.999].

  • "lr_scheduler_type" is the type of learning rate scheduler to use. This can be:

    • None, which does not use a learning rate scheduler. This is the default value if not specified.

    • "one_cycle", which uses the OneCycleLR scheduler.

  • "lr_scheduler_options" is a dictionary of options for the learning rate scheduler. For the "one_cycle" scheduler, the options are:

    • "max_lr" is the maximum learning rate to use. This is a positive float that defaults to 0.01.

    • "pct_start" is the percentage of the total number of epochs to use for the increasing phase of the learning rate schedule. This is a float between 0 and 1 that defaults to 0.25.

    • "anneal_strategy" is the annealing strategy to use. This can be:

      • "cos", which uses a cosine annealing strategy. This is the default value if not specified.

      • "linear", which uses a linear annealing strategy.

    • "cycle_momentum" is a boolean that specifies whether to cycle the momentum during training. This defaults to True.

    • "base_momentum" is the base momentum. This is a float between 0 and 1 that defaults to 0.85.

    • "max_momentum" is the maximum momentum. This is a float between 0 and 1 that defaults to 0.9.

    • "div_factor" is the factor by which to divide the maximum learning rate to get the initial learning rate. This is a positive float that defaults to 25.0.

    • "final_div_factor" is the factor by which to divide the initial learning rate to get the minimum learning rate at the end of training. This is a positive float that defaults to 1000.0.

    • "three_phase" is a boolean that specifies whether to use a three-phase learning rate schedule. This defaults to False.

• "representations_training_options" is a dictionary of options to train the representations. It can contain the following options:

  • "train_noise_type" is the type of noise to add to the representations during training. This can be:

    • None or "none", which does not add any noise to the representations during training. This is the default value if not specified.

    • "gaussian", which adds Gaussian noise to the representations.

  • "train_noise_options" is a dictionary of options to specify the noise to add to the representations during training. For the "gaussian" type, the options are:

    • "scale" is the base noise scale. This is a non-negative float that defaults to 0.0 (no noise).

    • "start" is the starting noise multiplier (at epoch 1). This is a non-negative float that defaults to 1.0.

    • "end" is the ending noise multiplier (at the final epoch). This is a non-negative float that defaults to 0.01.

    • "within_radius_prob" is the probability of the noise being within a given radius. This is a float between 0 and 1 that defaults to 0.95.

    • "gain" is the gain factor for the noise. This is a non-negative float that defaults to 1.0.

  • "optimizer_type" is the type of optimizer to use for training the representations. This can be:

    • "adam", which uses the Adam optimizer.

    • "adamw", which uses the AdamW optimizer. This is the default value if not specified.

  • "optimizer_options" is a dictionary of options for the optimizer. The options depend on the optimizer type. It can contain the following options:

    • "lr" is the learning rate. This is a positive float that defaults to 0.001.

    • "weight_decay" is the weight decay. This is a non-negative float that defaults to 0.0.

    • "betas" is a list of two floats that specify the beta parameters for the Adam or AdamW optimizer. The defaults are [0.9, 0.999].

  • "lr_scheduler_type" is the type of learning rate scheduler to use. This can be:

    • None, which does not use a learning rate scheduler. This is the default value if not specified.

    • "one_cycle", which uses the OneCycleLR scheduler.

  • "lr_scheduler_options" is a dictionary of options for the learning rate scheduler. For the "one_cycle" scheduler, the options are:

    • "max_lr" is the maximum learning rate to use. This is a positive float that defaults to 0.01.

    • "pct_start" is the percentage of the total number of epochs to use for the increasing phase of the learning rate schedule. This is a float between 0 and 1 that defaults to 0.25.

    • "anneal_strategy" is the annealing strategy to use. This can be:

      • "cos", which uses a cosine annealing strategy. This is the default value if not specified.

      • "linear", which uses a linear annealing strategy.

    • "cycle_momentum" is a boolean that specifies whether to cycle the momentum during training. This defaults to True.

    • "base_momentum" is the base momentum. This is a float between 0 and 1 that defaults to 0.85.

    • "max_momentum" is the maximum momentum. This is a float between 0 and 1 that defaults to 0.9.

    • "div_factor" is the factor by which to divide the maximum learning rate to get the initial learning rate. This is a positive float that defaults to 25.0.

    • "final_div_factor" is the factor by which to divide the initial learning rate to get the minimum learning rate at the end of training. This is a positive float that defaults to 1000.0.

    • "three_phase" is a boolean that specifies whether to use a three-phase learning rate schedule. This defaults to False.

• "early_stopping_type" is the type of early stopping criteria to use during training. This can be:

  • None, which disables early stopping. This is the default value if not specified.

  • "loss", which uses the total loss for early stopping. Training will be stopped if the total loss does not improve for a number of epochs specified by the "patience" option.

• "early_stopping_options" is a dictionary of options for the early stopping criteria. It can contain the following options:

  • "patience" is the number of epochs with no improvement after which training will be stopped if early stopping is enabled. This is a positive integer that defaults to 10.