Cross-Validation

class chocolate.Repeat(repetitions, reduce=<function mean>, rep_col='_repetition_id')

Repeats each experiment a given number of times and reduces the losses for the algorithms.

The repetition cross-validation wraps the connection to handle repetition of experiments in the database. It is transparent to algorithms as it reduces the loss of repeated parameters and returns a list of results containing a single instance of each parameter set when all_results() is called. If not all repetitions values are entered in the database before the next point is generated by the algorithm, the algorithm will see the reduced loss of the parameters that are completely evaluated only. Alternatively, if no repetition has finished its evaluation, the algorithm will see a None as loss. Repeat also handles assigning a repetition number to the tokens since the _chocolate_id will be repeated. Other token values, such as ThompsonSampling’s _arm_id, are also preserved.

Parameters:

• repetitions – The number of repetitions to do for each experiment.
• reduce – The function to reduce the valid losses, usually average or median.
• rep_col – The database column name for the repetition number, it has to be unique.