Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Build function takes min/max version from live directory always, instead of build directory #263

Open
marc-white opened this issue Nov 19, 2024 · 0 comments · May be fixed by #264
Open
Labels
bug Something isn't working

Comments

@marc-white
Copy link
Collaborator

Describe the bug

While doing testing for #175 , I've discovered that the build function is computing the minimum (and presumably the maximum) catalog version from the live catalog location on g/data, even when the build is taking place somewhere else. The system should be looking for min and max catalog versions in the build directory.

To Reproduce

Running a test build of #175 with MOM6 sources only into a blank test directory on Gadi /scratch yields the following catalog.yaml:

sources:
  access_nri:
    args:
      columns_with_iterables:
      - model
      - realm
      - frequency
      - variable
      mode: r
      name_column: name
      path: /scratch/tm70/mcw120/intake-cat-test/{{version}}/metacatalog.csv
      yaml_column: yaml
    description: ACCESS-NRI intake catalog
    driver: intake_dataframe_catalog.core.DfFileCatalog
    metadata:
      storage: gdata/None+gdata/ik11
      version: '{{version}}'
    parameters:
      version:
        default: v2024-11-12
        description: Catalog version
        max: v2024-11-12
        min: v2023-07-10   # HERE - This should be equal to max and default, as there were no other existing catalogs here
        type: str

The min value returned corresponds to the current minimum value present in /g/data/xp65/public/apps/access-nri-intake-catalog (i.e., the live catalog directory).

Additional context

Doesn't block #175, but should be fixed ASAP. Should also include a test to prevent this from happening again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Backlog
Development

Successfully merging a pull request may close this issue.

1 participant