Configuration

The configuration file can be either a JSON or a YAML file. We will present documentation for the JSON file. The YAML is similar.

The configuration file can be composed of up to 5 blocks:

• checkpoints_path (str) Specifies the path to the directory where checkpoints and experiment results will be saved.

• statistics_format (str) Determines the format for saving statistics. It can be either "json" or "csv".

• energize (dict) Options related to the ENERGIZE main features.

• evolutionary (dict) Options for the evolutionary algorithm.

• network (dict) Options concerning the architectures and learning parameters of the networks.

---

Energize

The energize section of the configuration file contains options specific to the ENERGIZE framework. If this section is absent in the configuration file, the framework is equivalent to the Fast-DENSER framework.

measure_power (dict)

Specifies whether the power usage should be measured or not as explained in Power Measurement

• train: (bool) Enables power measurement during training.

• test: (bool) Enables power measurement during testing.

• num_measurements_test: (int) Specifies the number of power measurements to take during testing.

• modules: (bool) Enables measurement of power consumption per module as specified in Module Reuse

model_partition (bool) Enables model splitting

model_partition_n (int) Specifies the number of models to create from the full model (not tested for $n \ne 2$).

ensure_gpu_exclusivity (bool) Ensures exclusive GPU usage through the usage of the Compute Exclusive mode of the NVIDIA GPU, as specified in Power Measurement

---

Evolutionary

This section of the configuration file contains parameters that drive the evolutionary algorithm within ENERGIZE.

generations (int): Specifies the number of generations.

lambda (int): Specifies the $\lambda$ parameter, i.e., the number of offspring generated in each generation from the selected parent.

max_epochs (int): Specifies the maximum cumulative number of epochs for training all individuals.

mutation (dict): Defines mutation probabilities for various operations, namely:

• add_connection (float): Probability of adding a connection between layers.

• remove_connection (float): Probability of removing a connection between layers.

• add_layer (float): Probability of adding a new layer.

• reuse_layer (float): Probability of reusing an existing layer.

• remove_layer (float): Probability of removing a layer.

• reuse_module (float): Probability of reusing an existing module.

• remove_module (float): Probability of removing a module.

• dsge_layer (float): Probability of performing a DSGE mutation on a layer.

• macro_layer (float): Probability of applying a macro mutation on a layer.

• train_longer (float): Probability of allowing the training of a model for a longer duration.

track_mutations (bool): Specifies whether to track mutations. If activated, each mutation is registered in the statistics output files.

fitness_metric (str): Defines the fitness metric for evaluating individuals. Note that only one fitness_metric or fitness_function can be used at a time. The allowed values are specified below.

fitness_function (list[dict]): Defines the fitness function for evaluating individuals. Note that only one fitness_metric or fitness_function can be used at a time. Each entry in the fitness_function list is composed by the following parameters:

• metric (str): Specifies the fitness metric to be used for evaluation. The allowed values are specified below.

• objective (str): Defines whether the objective is to maximize or minimize the specified metric.

• weight (float): Assigns a weight to the metric, influencing its importance in the overall fitness evaluation.

• conditions (list[dict]) (optional): Allows the definition of additional conditions for the fitness function. Each element of the list is considered as a conjunction, meaning they are combined using the "and" operator. Each element in the list represents a condition specified as a dictionary with the following properties:

  • metric (str): Specifies the fitness metric for the condition.

  • type (str): Specifies whether the condition is "less_than" or "greater_than".

  • value (float): Specifies the threshold value for the condition.

  • For example, specifying that "accuracy_0" must be "greater_than" the value 0.5 means that the metric in question will only be considered if "accuracy_0" is greater than 0.5

Fitness metrics:

• accuracy: measures the model's predictive performance by calculating the proportion of correctly classified instances.

• loss: quantifies the model's error during training, with lower values indicating better performance.

• power: power usage of the GPU during inference (in W).

• energy: energy consumed by the GPU during inference (in kJ).

If model_partition is activated, then the fitness metric should be appended with _i with i being the ID of the model (e.g., accuracy_0 is the accuracy of the full model).

---

Network

architecture (dict): Specifies the architecture configuration.

• reuse_layer (float): Specifies the probability of reusing an existing layer during network architecture initialization.

• macro_structure (list[str]): Specifies the macrostructure symbols to use. Depends on the grammar.

• output (str): Specifies the type of output layer, such as "softmax".

• modules (list[dict]): Specifies the allowed configuration of individual modules.

  • name (str): Specifies the name of the module.

  • network_structure_init (list[int]): Specifies the possibilities for the number of layers of the module when initialized, i.e., the number of layers will be randomly chosen from the values of this list.

  • network_structure (list[int]): Specifies the minimum and maximum number of layers of the networks. Format: [min, max].

  • levels_back (int): Specifies the maximum number of levels back to connect to when initializing the module. For example, a value of 2 levels back allows to connect to the previous two layers. This is also considered in the add_connection mutation.

learning (dict): Specifies the learning configuration.

• data_splits (dict): Specifies the data splits for training, validation, and testing datasets.

  • evo_train (float $\in [0, 1]$)

  • evo_validation (float $\in [0, 1]$)

  • evo_test (float $\in [0, 1]$)

• learning_type (str): Specifies the type of learning. For now, the only option is "supervised".

• default_train_time (int): Specifies the default training time in seconds allocated by default to each individual.

• augmentation (dict): Specifies data augmentation configurations for training and testing datasets.

  • train (dict): Specifies augmentation configurations for the training dataset.

  • test (dict): Specifies augmentation configurations for the testing dataset.

  • In both cases, the options are:

    • TBW (you can consult the file energize/networks/torch/transformers.py for more information)