Simplify sampling with bounds

[jrapin]

Context

Standardized space is an unbounded Rn space. Many algorithms however work on bounded domains, so that we usually use a arctan-based transform from [0,1] to R, and the other way round. However, if the actual work space is actually bounded, this can transform space in weird ways.

Need

We need a standard way to convert from [0,1] to the actual domain, be it bounded or unbounded. This is currently done by the BoundScaler

nevergrad/nevergrad/optimization/utils.py

Lines 348 to 356 in 8e40977

	class BoundScaler:
	"""Hacky way to sample in the space defined by the parametrization.
	Given an vector of values between 0 and 1,
	the transform method samples in the bounds if provided,
	or using the provided function otherwise.
	This is used for samplers.
	Code of parametrization and/or this helper should definitely be
	updated to make it simpler and more robust
	"""

It is currently used by one-shot optimizers. This should be made more robust and more general, with both forward and backward transform to the state domain, so as to be included in other algorithms such as BO (in BO the backward transform is necessary for the tell-not-asked).

This work my require enforcing some prior on the ordering of the data in Parameter containers, and is related to:

nevergrad/nevergrad/parametrization/helpers.py

Lines 103 to 122 in 8e40977

	def split_as_data_parameters(
	parameter: core.Parameter,
	) -> tp.List[tp.Tuple[str, pdata.Data]]:
	"""List all the instances involved as parameter (not as subparameter/
	endogeneous parameter)
	Parameter
	---------
	parameter: Parameter
	the parameter to split
	Returns
	-------
	list
	the list and subparameters ordered as in data space
	Note
	----
	This function is experimental, its output will probably evolve before converging.
	"""

which should be simplified as well.

[jrapin]

Related problem:
Middle point for one-shots does not go through the scaler:

nevergrad/nevergrad/optimization/oneshot.py

Lines 290 to 291 in 8e40977

	if self.middle_point and not self._num_ask:
	return np.zeros(self.dimension)

[beckhamwjc]

Is it possible to do gaussian sampling with, for example, a ng.p.Array(shape=(n,)) in actually R^n space? This is important because there is often NO reason to restrict the boundaries of parameters to be optimized in reality, and gaussian sampling is obviously more efficient than uniform sampling which I assume is the current sampling method that Nevergrad adopted.