Help with BO Not Optimizing Beyond Initial Random Parameters

[MustyKey]

Hi everyone,

I'm working on a project to optimize PID parameters in order to minimize an objective function using Bayesian Optimization. However, I’m encountering an issue where the optimal result is always found based on the initial random parameters, and the actual BO loop doesn't seem to improve upon this.

I would really appreciate it if someone could take a look at my code to help identify any potential issues. Any suggestions or guidance would be incredibly helpful! If more code snippets or details are needed, feel free to let me know.

Thanks in advance for your time!

def optimize_pid(num_initial_points, num_iterations):
bounds = torch.tensor([[0, 0, 0],
[5, 5, 1]],
dtype=torch.double)

rng = np.random.RandomState(1)
train_x = torch.tensor(rng.uniform(bounds[0].numpy(), bounds[1].numpy(), (num_initial_points, 3)),
                       dtype=torch.double)

train_y = torch.tensor([objective_function(x) for x in train_x], dtype=torch.double).unsqueeze(-1)

# Define the Gaussian Process model
gp_model = SingleTaskGP(train_x,
                        train_y,
                        outcome_transform=Standardize(m=1),
                        input_transform=Normalize(d=3))
mll = ExactMarginalLogLikelihood(gp_model.likelihood, gp_model)
fit_gpytorch_mll(mll)

# Lists for plotting
objective_values = []
acquisition_values = []

num_restarts = 10
raw_samples = 200

# Bayesian Optimization loop
for i in range(num_iterations):
    ei = ExpectedImprovement(gp_model, best_f=train_y.min().item())
    candidate_X, acq_value = optimize_acqf(
        ei,
        bounds=bounds,
        q=1,
        num_restarts=num_restarts,
        raw_samples=raw_samples,
    )

    candidate_Y = torch.tensor([objective_function(candidate_X[0])], dtype=torch.double).unsqueeze(-1)

    train_x = torch.cat([train_x, candidate_X])
    train_y = torch.cat([train_y, candidate_Y])

    gp_model = SingleTaskGP(train_x,
                            train_y,
                            outcome_transform=Standardize(m=1),
                            input_transform=Normalize(d=3))
    mll = ExactMarginalLogLikelihood(gp_model.likelihood, gp_model)
    fit_gpytorch_mll(mll)

    # Store the objective value and acquisition function values
    objective_values.append(candidate_Y.item())
    acquisition_values.append(acq_value.item())

    print(f"Iteration {i + 1}/{num_iterations}, Current best cost: {train_y.min().item():.4f}")

# Final best parameters and cost
best_idx = torch.argmin(train_y)
best_params = train_x[best_idx]
best_cost = train_y[best_idx]

print(
    f"Best PID parameters: Kp = {best_params[0].item():.4f}, Ki = {best_params[1].item():.4f}, Kd = {best_params[2].item():.4f}")

# Plot the objective values over iterations
plt.figure(figsize=(12, 6))

plt.subplot(2, 1, 1)
plt.plot(objective_values, marker='o', linestyle='-', color='b')
plt.xlabel('Iteration')
plt.ylabel('Objective Function Value')
plt.title('Objective Function Value Over Iterations')
plt.grid(True)

# Plot the acquisition function values over iterations
plt.subplot(2, 1, 2)
plt.plot(acquisition_values, marker='o', linestyle='-', color='r')
plt.xlabel('Iteration')
plt.ylabel('Acquisition Function Value')
plt.title('Acquisition Function Value Over Iterations')
plt.grid(True)

# Save the plot as an image
plt.tight_layout()
plt.savefig(f"objective_and_acquisition_function_over_iterations_{version}.png")
plt.show()

best_pid_param = best_params.numpy()


return best_pid_param

[saitcakmak]

Hi @MustyKey. Are you trying to minimize the objective? The train_y.min() calls in the code makes me think that is the case. BoTorch assumes maximization of the objectives by default. You may want to negate your Y values to turn it into a maximization problem.

As a more general suggestion, if your goal is to optimize a function, we recommend using Ax rather than BoTorch. BoTorch is primarily designed for BO research. Ax uses BoTorch under the hood while offering a more intuitive interface for experiment setup. This tutorial is a good place to start: https://ax.dev/tutorials/gpei_hartmann_service.html

[MustyKey]

Hi @saitcakmak ,

Thank you for your quick and insightful feedback! You’re absolutely right—I corrected the code, and now the objective values are minimizing as expected.

My goal is to minimize the metabolic costs of gait in a simulation, which involves optimizing the PID parameters of an actuator rather than minimizing a mathematical function. Do you think using Ax is still appropriate for this optimization? I appreciate your recommendation and will check out the tutorial you linked.

Thanks again for your help!!!!

[Balandat]

My goal is to minimize the metabolic costs of gait in a simulation, which involves optimizing the PID parameters of an actuator rather than minimizing a mathematical function. Do you think using Ax is still appropriate for this optimization? I appreciate your recommendation and will check out the tutorial you linked.

Yes, Ax will work just fine for this (and more generally for problems that don't require an exotic or custom setup). You can use the ask/tell interface of AxClient to generate new parameter configurations and report back the outcomes as in the tutorial Sait linked above. Instead of defining an evaluate function you'd just actually evaluate the parameter (on the real system or your simulation).