Conditional Exploration

class chocolate.ThompsonSampling(algo, connection, space, crossvalidation=None, clear_db=False, random_state=None, gamma=0.9, epsilon=0.05, algo_params=None)[source]

Conditional subspaces exploration strategy.

Thompson sampling wrapper to sample subspaces proportionally to their estimated quality. Each subspace of a conditional search space will be treated independently. This version uses an estimated moving average for the reward and forgets the reward of unselected subspaces allowing to model the dynamics of the underlying optimizers. Thompson sampling for Bernoulli bandit is described in [Chapelle2011].

Parameters:
  • algo – An algorithm to sample/search each subspace.
  • connection – A database connection object.
  • space – The conditional search space to explore.
  • crossvalidation – A cross-validation object that handles experiment repetition.
  • clear_db – If set to True and a conflict arise between the provided space and the space in the database, completely clear the database and insert set the space to the provided one.
  • random_state – An instance of RandomState, an object to initialize the internal random state with, or None, in which case the global numpy random state is used.
  • gamma – Estimated moving average learning rate. The higher, the faster will react the bandit to a change of best arm. Should be in [0, 1].
  • epsilon – Forget rate for unselected arms. Th higher, the faster unselected arms will fallback to a symmetric distribution. Should be in [0, 1].
  • algo_params – A dictionary of the parameters to pass to the algorithm.
[Chapelle2011]O. Chapelle and L. Li, “An empirical evaluation of Thompson sampling”, in Advances in Neural Information Processing Systems 24 (NIPS), 2011.
next()

Retrieve the next point to evaluate based on available data in the database.

Returns:A tuple containing a unique token and a fully qualified parameter set.
update(token, values)

Update the loss of the parameters associated with token.

Parameters:
  • token – A token generated by the sampling algorithm for the current parameters
  • values – The loss of the current parameter set. The values can be a single Number, a Sequence or a Mapping. When a sequence is given, the column name is set to “_loss_i” where “i” is the index of the value. When a mapping is given, each key is prefixed with the string “_loss_”.