Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli to automate some manual gst1 update work; do gst update #60

Merged
merged 19 commits into from
Dec 7, 2023

Conversation

kdutia
Copy link
Member

@kdutia kdutia commented Dec 5, 2023

two CLIs to automate the work of updating gst1.

  1. 1_get_new_parser_outputs.py: compares a new UNFCCC to an old one, goes and grabs the ParserOutput docs from S3 to process
  2. 2_run_concept_extraction.py: imports explorer and runs concepts. parametrised only by the folder name (i.e. the download folder from the first step

step 3 is then to run the indexer, and 🎉

the missing part at the moment is having all the documents in one place: they're in a local directory for me. I can drop them into an s3 bucket if someone else ever needs to run this, but am cautious to create yet another folder on s3 until we've done our data consolidation work with deng

note CI is failing because we now install explorer as a dependency and GH actions doesn't like installing from git apparently - but everything else is ready to review

Copy link

linear bot commented Dec 5, 2023

@kdutia kdutia changed the title cli to automate some manual gst1 update work cli to automate some manual gst1 update work; do gst update Dec 5, 2023
@kdutia kdutia marked this pull request as ready for review December 6, 2023 12:07
src/cli/1_get_new_parser_outputs.py Outdated Show resolved Hide resolved
@kdutia kdutia merged commit d7f12ac into main Dec 7, 2023
2 checks passed
@kdutia kdutia deleted the feature/rnd-675-automate-some-of-the-gst1-update-process branch December 7, 2023 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants