Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load Experiment without runners and metrics in the case where search space and optimization config are immutable #1656

Closed

Conversation

bernardbeckerman
Copy link
Contributor

Differential Revision: D46595953

@facebook-github-bot facebook-github-bot added CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported labels Jun 13, 2023
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D46595953

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D46595953

@codecov-commenter
Copy link

codecov-commenter commented Jun 13, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (aada4cf) 94.51% compared to head (7f24c14) 94.52%.

❗ Current head 7f24c14 differs from pull request most recent head a6d9d59. Consider uploading reports for the commit a6d9d59 to get more accurate results

Files Patch % Lines
ax/storage/sqa_store/load.py 85.71% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1656   +/-   ##
=======================================
  Coverage   94.51%   94.52%           
=======================================
  Files         460      460           
  Lines       44258    44291   +33     
=======================================
+ Hits        41832    41864   +32     
- Misses       2426     2427    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

bernardbeckerman pushed a commit to bernardbeckerman/Ax that referenced this pull request Jun 13, 2023
…space and optimization config are immutable (facebook#1656)

Summary:
Pull Request resolved: facebook#1656

`load_experiment` was previously failing when `skip_runners_and_metrics=True` for experiments with immutable search space and optimization config. See [Lena's comment](https://www.internalfb.com/diff/D46595953?dst_version_fbid=263061986396665&transaction_fbid=639853311380999) for more detail.

Differential Revision: D46595953

fbshipit-source-id: ac5370fa638cf6b7a52152374339132ccb6886a9
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D46595953

bernardbeckerman pushed a commit to bernardbeckerman/Ax that referenced this pull request Jun 13, 2023
…space and optimization config are immutable (facebook#1656)

Summary:
Pull Request resolved: facebook#1656

`load_experiment` was previously failing when `skip_runners_and_metrics=True` for experiments with immutable search space and optimization config. See [Lena's comment](https://www.internalfb.com/diff/D46595953?dst_version_fbid=263061986396665&transaction_fbid=639853311380999) for more detail.

Differential Revision: D46595953

fbshipit-source-id: 2c330b558efcdbe5fe64c2e0f1097a2ebde76920
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D46595953

bernardbeckerman pushed a commit to bernardbeckerman/Ax that referenced this pull request Jun 13, 2023
…space and optimization config are immutable (facebook#1656)

Summary:
Pull Request resolved: facebook#1656

`load_experiment` was previously failing when `skip_runners_and_metrics=True` for experiments with immutable search space and optimization config. See [Lena's comment](https://www.internalfb.com/diff/D46595953?dst_version_fbid=263061986396665&transaction_fbid=639853311380999) for more detail.

Differential Revision: D46595953

fbshipit-source-id: 89618fbeb19c47392f30709856730721d01ffbd9
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D46595953

bernardbeckerman pushed a commit to bernardbeckerman/Ax that referenced this pull request Jun 21, 2023
…space and optimization config are immutable (facebook#1656)

Summary:
Pull Request resolved: facebook#1656

`load_experiment` was previously failing when `skip_runners_and_metrics=True` for experiments with immutable search space and optimization config. See [Lena's comment](https://www.internalfb.com/diff/D46595953?dst_version_fbid=263061986396665&transaction_fbid=639853311380999) for more detail.

Added in V4: uses the metric type corresponding to the decoder's entry for Ax base Metric, instead of 0, which is not always the correct entry.

Differential Revision: D46595953

fbshipit-source-id: 1f9e24735c57580ff85a8d500dde027640daad65
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D46595953

bernardbeckerman added a commit to bernardbeckerman/Ax that referenced this pull request Nov 22, 2023
…space and optimization config are immutable (facebook#1656)

Summary:

`load_experiment` was previously failing when `skip_runners_and_metrics=True` for experiments with immutable search space and optimization config. See [Lena's comment](https://www.internalfb.com/diff/D46595953?dst_version_fbid=263061986396665&transaction_fbid=639853311380999) for more detail.

Added in V4: uses the metric type corresponding to the decoder's entry for Ax base Metric, instead of 0, which is not always the correct entry.

Reviewed By: lena-kashtelyan

Differential Revision: D46595953
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D46595953

Summary:

For get_experiment... core stubs functions that create a trial with Sobol, constraints were resulting in the logs being flooded with "Unable to round" warnings due to rejection sampling. This diff omits these constraints in test_sqa_store tests for which they are immaterial.

Reviewed By: Balandat

Differential Revision: D46687301
…space and optimization config are immutable (facebook#1656)

Summary:

`load_experiment` was previously failing when `skip_runners_and_metrics=True` for experiments with immutable search space and optimization config. See [Lena's comment](https://www.internalfb.com/diff/D46595953?dst_version_fbid=263061986396665&transaction_fbid=639853311380999) for more detail.

Added in V4: uses the metric type corresponding to the decoder's entry for Ax base Metric, instead of 0, which is not always the correct entry.

Reviewed By: lena-kashtelyan

Differential Revision: D46595953
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D46595953

@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 5c0d64b.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants