You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A comment in the form /test-scraper --from_dt=<iso-formatted-datetime> --to_dt=<iso-formatted-datetime> get_{municipality_slug}_events in a pull request or an issue triggers the GitHub Action .github/workflows/test-scraper.yml. The Action uploads a ZIP file as an artifact. The ZIP contains a JSON file for the scraped events returned by cdp_scrapers.instances.get_{municipality_slug}_events().
If the Action was triggered by a pull request comment, it would be helpful to also include a diff file, comparing the results from the same call between the pull target branch (CouncilDataProject/cdp-scrapers main) and the pull merge branch.
Change summary:
Do something similar to what test-scraper.yml does currently: save scraped events in JSON file.
Repeat, using CouncilDataProject/cdp-scrapers main.
Save output from diff between the two JSON files.
Add the diff file in the ZIP file artifact for the Action.
Use Case
Often we find ourselves manually creating this kind of diff files to study changes (if any) some new code would yield in the scraped IngestionModels. Doing so automatically for a pull request /test-scraper run would reduce manual effort.
Solution
File(s) in .github/workflows/ (and subdirectories) updated as necessary to include diff files, described above, in the artifact ZIP.
The text was updated successfully, but these errors were encountered:
Feature Description
A comment in the form
/test-scraper --from_dt=<iso-formatted-datetime> --to_dt=<iso-formatted-datetime> get_{municipality_slug}_events
in a pull request or an issue triggers the GitHub Action.github/workflows/test-scraper.yml
. The Action uploads a ZIP file as an artifact. The ZIP contains a JSON file for the scraped events returned bycdp_scrapers.instances.get_{municipality_slug}_events()
.If the Action was triggered by a pull request comment, it would be helpful to also include a diff file, comparing the results from the same call between the pull target branch (
CouncilDataProject/cdp-scrapers main
) and the pull merge branch.Change summary:
test-scraper.yml
does currently: save scraped events in JSON file.CouncilDataProject/cdp-scrapers main
.Use Case
Often we find ourselves manually creating this kind of diff files to study changes (if any) some new code would yield in the scraped
IngestionModels
. Doing so automatically for a pull request/test-scraper
run would reduce manual effort.Solution
File(s) in
.github/workflows/
(and subdirectories) updated as necessary to include diff files, described above, in the artifact ZIP.The text was updated successfully, but these errors were encountered: