Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test S3 inaturalist files are not found in MinIO #4494

Closed
krysal opened this issue Jun 15, 2024 · 0 comments · Fixed by #4495
Closed

Test S3 inaturalist files are not found in MinIO #4494

krysal opened this issue Jun 15, 2024 · 0 comments · Fixed by #4495
Assignees
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: catalog Related to the catalog and Airflow DAGs

Comments

@krysal
Copy link
Member

krysal commented Jun 15, 2024

Description

The test files for the iNaturalist DAG are no longer loaded into MinIO, and what's more, the inaturalist-open-data bucket isn't part of those created by default. This process may be useful for other work in the future.

# Comma separated list of buckets to create on startup
BUCKETS_TO_CREATE=openverse-catalog,openverse-airflow-logs

Reproduction

  1. just catalog/recreate
  2. Observe the logs of the load_to_s3 container
  3. See the errors.
2024-06-15 14:33:48 /opt/minio/load_to_s3_entrypoint.sh: line 11: AWS_ACCESS_KEY: unbound variable

Or directly observe what is in MinIO through the UI: http://localhost:5011/browser
username: test_key
password: test_secret

@krysal krysal added 🟨 priority: medium Not blocking but should be addressed soon 🛠 goal: fix Bug fix 💻 aspect: code Concerns the software code in the repository 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Jun 15, 2024
@krysal krysal self-assigned this Jun 15, 2024
@openverse-bot openverse-bot moved this to 📋 Backlog in Openverse Backlog Jun 15, 2024
@openverse-bot openverse-bot moved this from 📋 Backlog to 🏗 In Progress in Openverse Backlog Jun 15, 2024
@openverse-bot openverse-bot moved this from 🏗 In Progress to ✅ Done in Openverse Backlog Jun 25, 2024
@openverse-bot openverse-bot moved this from ✅ Done to 🏗 In Progress in Openverse Backlog Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: catalog Related to the catalog and Airflow DAGs
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant