Rewrite the get_publication script #832
Conversation
This commit overhauls the script to update publications from Google
Scholar. The previous script worked, but it had some drawbacks, namely
that it required manually editing JSON with updated information. The new
script does not, and it outputs data in a format that will create
meaningful diffs with the existing file format when data is updated.
Changes:
* Moves from argparse to typer for dealing with CLI arguments.
* Separates fetching data from generating the publications.json file
(this was necessary for development to prevent having to hit Google
Scholar on every change).
* Removes need to filter publications based on year-- Google Scholar
is now taken as the single source of truth for publications.
* Takes into account publications whose years have changed in Google
Scholar versus what has been already included in the
publications.json file.
* Better record matching that ignores differences in non-alphanumeric
characters. Previously, there were several false positives that had
to do with changes in punctuation.
* Logging to stderr rather than an outputted report.
* Single source of truth for adding links to publications without them
in Google Scholar. (Previously, they had to be input by hand in the
produced JSON file).
✅ Deploy Preview for monarch-app ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #832 +/- ##
==========================================
+ Coverage 70.75% 71.30% +0.54%
==========================================
Files 91 91
Lines 3084 3136 +52
==========================================
+ Hits 2182 2236 +54
+ Misses 902 900 -2 ☔ View full report in Codecov by Sentry. |
|
Hmm. Given that these changes don't touch any of the presentation code, that's a separate issue. I imagine it will require some changes in https://github.com/monarch-initiative/monarch-app/blob/2c878e844bc2d10664d798570560c04fad6acc27/frontend/src/components/AppCitation.vue. I'll check it out. I wonder why it only just popped up. |
|
The deploy preview still shows issues. I am not sure if there is somewhere else I should be looking for the corrections to the left align issue and the tile spreading issue? @ptgolden |
|
#829 looks correct and related to this PR as well. |
|
The layout in the deploy preview matches what's on dev now, so bringing this in is probably neutral relative to the layout of the page. That fix is coming before we release, so I think it makes sense to merge this |
This overhauls the script to update publications from Google Scholar. The previous script worked, but it had some drawbacks, namely that it required manually editing JSON with updated information. The new script does not, and it outputs data in a format that will create meaningful diffs with the existing file format when data is updated.
Changes:
Moves from argparse to typer for dealing with CLI arguments.
Separates fetching data from generating the publications.json file (this was necessary for development to prevent having to hit Google Scholar on every change).
Removes need to filter publications based on year-- Google Scholar is now taken as the single source of truth for publications.
Takes into account publications whose years have changed in Google Scholar versus what has been already included in the publications.json file.
Better record matching that ignores differences in non-alphanumeric characters. Previously, there were several false positives that had to do with changes in punctuation.
Logging to stderr rather than an outputted report.
Single source of truth for adding links to publications without them in Google Scholar. (Previously, they had to be input by hand in the produced JSON file).