Can we deploy a equal constraint for parameters?

[yuanquan010]

Thanks for this great project, but I am facing some problems now.

I define 12 parameters, 6 for porosity and 6 for grad. I looked through the documents and find for parameter_constraint, only <= and >= are allowed. However, I want to define two constraints, the first one is that the sum of 6 Grad-parameters is equal to 10, and the second constraint is that the sum of all Porosity_i * Grad_i/10 is equal to 0.6, i=1,2,3,4,5,6.

In this project, can I set these two constraints? The following are code snippets for the parameter setting. I am looking forward to your help.

for dim_idx in range(6):
param_name = f"Grad_{dim_idx}" # You can name the parameters as needed
bounds = [1, 5] # Replace with the known interval for your parameters
parameters.append(
{
"name": param_name,
"type": "range",
"bounds": bounds,
"value_type": "int",
}
)

for dim_idx in range(6):
param_name = f"Porosity_{dim_idx}" # You can name the parameters as needed
bounds = [0.01, 0.7] # Replace with the known interval for your parameters
parameters.append(
{
"name": param_name,
"type": "range",
"bounds": bounds,
"value_type": "float",
}
)

I define the parameters like

[mpolson64]

Thank you for reaching out with this question about formulating your search space. When we're trying to apply an equality constraint to a linear combination of our parameters we are effectively restricting our total search space to only a single hyperplane. With that in mind, instead of introducing the equality constraint we can redefine our search space with one fewer dimension. For example, a 3 dimensional search space of range parameters x, y, z with a constraint x + y + z = 1 is equivalent to a 2 dimensional search space over x, y where usage of z gets replaced with 1 - x - y. If you transform your search space like this you will no longer need the equality constraint.

All this being said, equality constraints are a pretty commonly requested feature for Ax. @Balandat @dme65 should we think about adding a simplex constraint transform to Ax to do this under the hood automatically?

[lena-kashtelyan]

Equality constraint is currently on our wishlist, and the workaround @mpolson64 suggests about reducing the dimensionality of the search space using parameter equality, should work in this case. Once we confirm that the workaround unblocks, we can label this as wishlist, add this issue to the other one about equality constraints in the tracking issue (#566) and close this one.

[yuanquan010]

Hello, thank you for your suggestion. I understand we can use 1-x-y to replace z, but how can I achieve this when I define the search space? Could you offer me a code example？> Thank you for reaching out with this question about formulating your search space. When we're trying to apply an equality constraint to a linear combination of our parameters we are effectively restricting our total search space to only a single hyperplane. With that in mind, instead of introducing the equality constraint we can redefine our search space with one fewer dimension. For example, a 3 dimensional search space of range parameters x, y, z with a constraint x + y + z = 1 is equivalent to a 2 dimensional search space over x, y where usage of z gets replaced with 1 - x - y. If you transform your search space like this you will no longer need the equality constraint.

All this being said, equality constraints are a pretty commonly requested feature for Ax. @Balandat @dme65 should we think about adding a simplex constraint transform to Ax to do this under the hood automatically?

[Balandat]

@Balandat @dme65 should we think about adding a simplex constraint transform to Ax to do this under the hood automatically?

We've thought about it, but it's actually not super straightforward. Doable for sure, but I think it would require more work that it may seem. In general I agree though that this would be a good feature to support

[mpolson64]

@yuanquan010 In the search space you would simply omit z altogether. The equation you were using as the constraint gets used to decide what to replace z with at usage and you no longer need the constraint in your search space. Does that make sense?

[rzc331]

Thank you for reaching out with this question about formulating your search space. When we're trying to apply an equality constraint to a linear combination of our parameters we are effectively restricting our total search space to only a single hyperplane. With that in mind, instead of introducing the equality constraint we can redefine our search space with one fewer dimension. For example, a 3 dimensional search space of range parameters x, y, z with a constraint x + y + z = 1 is equivalent to a 2 dimensional search space over x, y where usage of z gets replaced with 1 - x - y. If you transform your search space like this you will no longer need the equality constraint.

All this being said, equality constraints are a pretty commonly requested feature for Ax. @Balandat @dme65 should we think about adding a simplex constraint transform to Ax to do this under the hood automatically?

We've been trying this approach for several projects. According to our experience, this workaround is effective in lower dimension (<5d). However, when it comes to higher dimension, there are issues coming up. In 8d, if we are using SOBOL, we have to increase the DEFAULT_MAX_RS_DRAWS param from default value 10000 to 1000000, otherwise the candidate generation will very likely fail. It's probably because the subspace that satisfies the inequality constraint (x_1 + x_2 ... + x_7 <=1) is a fairly small region in the 7d space.

Other than SOBOL initialization, we also noticed botorch gen can frequently fail using this workaround, which can yield Optimization failed within scipy.optimize.minimize with status 9, which stands for iteration limit reached in scipy optimize. We are not 100% sure but we suppose this error is related to the inequality constraint workaround as well.

It will be awesome if Ax could officially support equality constraint!

[mpolson64]

@rzc331 thanks, this is good signal. We have moved this issue to the Wishlist and do our best to prioritize it soon! Once this gets picked up and makes it into a stable version of Ax we will return to this thread to update everyone here.