Improvements requested for best_point_utils.get_best_parameters: using latest data etc.

[alexdeng]

I am new to AX and am using the developer API. In a standard for loop like this, the experiment's last trial will have its generator_run used to generate the last trial, which is created from the model trained before last trial is completed. The best_point_utils.get_best_parameters seems to be using the generator_run (and the torch model bridge in it) to get the best arm based on prediction. As a result it is not using the last trial's data even when it is completed.

for i in range(NUM_BOTORCH_TRIALS):
    print(
        f"Running GP+EI optimization trial {i + NUM_SOBOL_TRIALS + 1}/{NUM_SOBOL_TRIALS + NUM_BOTORCH_TRIALS}..."
    )
    # Reinitialize GP+EI model at each step with updated data.
    gpei = Models.BOTORCH(experiment=exp, data=exp.fetch_data())
    generator_run = gpei.gen(n=1)
    trial = exp.new_trial(generator_run=generator_run)
    trial.run()
    trial.mark_completed()
    
print("Done!")

# manual best arm. This generator_run fit the model with the latest data. 
gpei = Models.BOTORCH(experiment=exp, data=exp.fetch_data())
gr = gpei.gen(n=1)
gr.best_arm_predictions

[lena-kashtelyan]

Hi, @alexdeng! Thank you for bringing this to our attention. Our best-point identification utilities do need some improvements, so we'll add this to other issues reported in #605. I'll make this a master issue of problems with get_best_parameters.

[lena-kashtelyan]

The fix for this is now on master and will be part of the next stable release

[lena-kashtelyan]

The fix is now part of the latest stable version: 0.2.3.