Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import all harvest sources from CKAN to H2.0 #4964

Open
1 task
jbrown-xentity opened this issue Nov 4, 2024 · 0 comments
Open
1 task

Import all harvest sources from CKAN to H2.0 #4964

jbrown-xentity opened this issue Nov 4, 2024 · 0 comments
Labels
H2.0/Harvest-General General Harvesting 2.0 Issues

Comments

@jbrown-xentity
Copy link
Contributor

User Story

In order to cutover harvesting to 2.0, data.gov admins want harvest sources created in harvesting2.0 for all harvesting.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

  • GIVEN a tool/script is created for importing harvest sources into H2.0
    WHEN the harvest import happens
    THEN all the harvest sources with data from catalog are imported into H2.0

Background

Security Considerations (required)

None

Sketch

We should define and plan mapping from CKAN harvest source types into new harvest types, focused on job "type" and config object into the new H2.0 extract type and schema type.

We should only import harvest sources that have data; if it doesn't that should be documented in some type of auditable list. Most of these items will remain removed (consider reaching out to data providers), but a few may just have CKAN errors that we should resolve instead of ignoring.

@jbrown-xentity jbrown-xentity added the H2.0/Harvest-General General Harvesting 2.0 Issues label Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
H2.0/Harvest-General General Harvesting 2.0 Issues
Projects
Status: 📥 Queue
Development

No branches or pull requests

1 participant