Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix OmegaConfigLoader when file is empty #2556

Closed
wants to merge 1 commit into from
Closed

Conversation

noklam
Copy link
Contributor

@noklam noklam commented May 4, 2023

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

Currently, if you have an empty configuration file with OmegaConfigLoader, you will get this error which isn't very accurate.

This draft PR is just a starting point, it's expected to be picked up by someone and worked on it further.

            raise MissingConfigException(
                f"No files of YAML or JSON format found in {base_path} or {env_path} matching"
                f" the glob pattern(s): {[*self.config_patterns[key]]}"
            )

Possible Alternative

  • Refactor the code and eliminate the need of handling it with a special condition. The code below shows that we cannot differentiate between missing and empty configs.
  • If this is necessary, we should at least provide a better message (Which I think it shouldn't be, as we don't need this for the other ConfigLoader).

Lines 276-277 don't actually cause a bug I think, so if removing them makes some tests fail and it seems related to #2556 then let's leave those here for now and try to remove them in that PR.

if not aggregate_config:
return {}

We should also take this as an opportunity to revisit the logic and see if we can simplify it

Development notes

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes

@noklam
Copy link
Contributor Author

noklam commented May 4, 2023

Found this issue while writing test for #2467 @merelcht for comments.

@merelcht
Copy link
Member

merelcht commented May 4, 2023

Good find. I agree that ideally it shouldn't throw an error at all but just return an empty dict.

@noklam
Copy link
Contributor Author

noklam commented May 5, 2023

Close in favour of #2566, need more work on revisiting the load_and_merge logic.

@noklam noklam closed this May 5, 2023
@noklam noklam deleted the fix-omega-empty-config branch June 7, 2023 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants