Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unintuitive switching behavior of Meta-Recommenders in combination with Campaign #420

Open
Scienfitz opened this issue Nov 6, 2024 · 0 comments
Labels
enhancement Expand / change existing functionality

Comments

@Scienfitz
Copy link
Collaborator

According to the current logic, meta-recommenders perform the logic that decides the switching at the point of the .recommend call. However, intuitively imo it would make sense to perform that logic at the time of .add_measurements, simply because adding the measurements marks the end of an iteration, and meta-recommenders switch based on iterations.

The current behavior requires the following unintuitive workaround:

recommender = TwoPhaseMetaRecommender(
    initial_recommender=RandomRecommender(),
    recommender=BotorchRecommender,
    switch_after=1,
) 

campaign = Campaign(..., recommender=recommender)
recs = recommender.recommend(5)
### perform measurements
campaing.add_measurements(my_measurements)

# Analyze the surrogate based on the first set of measurements
surrogate = campaign.get_surrogate()

This will fail with RuntimeError: The current recommender is of type 'RandomRecommender'. Surrogate models are only available for recommender subclasses of 'BayesianRecommender'. because the meta recommender has not yet advanced the recommender to the second one. This requires another emtpy call to .recommend, which would not be needed anymore in the suggested change.

@Scienfitz Scienfitz added the enhancement Expand / change existing functionality label Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Expand / change existing functionality
Projects
None yet
Development

No branches or pull requests

1 participant