Search Space

[This Wiki is still under extensive constructions]

Creating the Search Space

We implemented ContinuousSpace, OrdinalSpace, and NominalSpace classes to handle real, integer, and categorical variables, respectively. In general, the constructor of those classes take the following arguments:

• bounds (List[List]): The lower and upper bound for continuous/ordinal parameter type and the categorical levels for nominal parameter type. The dimension of the space is determined as the length of bound.
• var_name (Union[str, List[str], None]): The variable name per each dimension. When it is a list of str, it should have the same length as bounds. When it is a string, variable names are created by appending counting numbers to this string, e.g., var_0, var_1, ..., var_n. It is by default None, which takes r, o, and d as the base name for real, integer, and categorical variables, respectively.

You could simply generate a product space using the following syntax:

search_space = ContinuousSpace([0, 1]) + OrdinalSpace([0, 10]) + NominalSpace(['1', '2', '3'])

You could also sample from the space via the sampling method:

X = search_space.sampling(N=10)

When the numerical precision is specified for the real-valued variables (please see below), you could round the sample as follows:

X_ = search_space.round(X)

Real-valued variables

For n real-valued variable in [-5, 5]^n

C = ContinuousSpace([-5, 5], var_name='continuous') * n

Equivalently, you can also use

C = ContinuousSpace([[-5, 5]]] * n) 

Additionally, you could specify the numerical precision and transformations on the real-valued variables using the following extra arguments:

• precision (int): the number of digits to keep after the decimal.

C = ContinuousSpace([-5, 5], precision=2)

• scale provides a couple of useful transformations on the continuous domain, including:
  • 'log': np.log
  • 'log10': np.log10,
  • 'logit': scipy.special.logit,
  • 'bilog': lambda x: np.sign(x) * np.log(1 + np.abs(x)),

C = ContinuousSpace([-5, 5], scale='bilog') # log does not work with the negative part..

which are useful if it makes sense to zoom in some preferred region of your search space, e.g., using log or log10 will have the ability to zoom into very tiny values of parameter, which might be helpful if you are tuning a learning rate of some algorithm. When scale is given, the bound for such parameter shall be transformed and the sampling method will work directly in the transformed space.

Integer-valued variables

I = OrdinalSpace([5, 15], var_name='ordinal')

Categorical variables

# Discrete (nominal) variables can be specified as follows:
# a list of categories levels is needed
N = NominalSpace(['OK', 'A', 'B', 'C', 'D', 'E', 'F', 'G'], var_name='nominal')