Handling deleted packages

[FedericoCeratto]

--- forwarded from FedericoCeratto/nim-package-directory#26 ---
--- original author: https://github.com/samdmarshall ---

today i was trying to find a package for a personal project, and came across the htmlview package. this package seems to have been deleted by the author, and never updated in the index as to have it removed entirely or redirected to the new repo. On the individual package page on https://nimble.directory/pkg/... there seems to be no way to report a problem with said package.

• If the package doesn't have a public visible repo/source then there should be a link to create a new PR to remove the package from package index. that process should be able to be 100% automated/scripted.
• If the package does have a public visible repo/source then there should be either a link to create a new issue on the package repo (if using github or another similar SCM system) or a mailing address for the author or to the package's mailing-list to correspond about the problem. (this may require some changes to the submission guidelines regarding new and existing packages so that non-github hosted packages provide some way of contacting the package author).

To mitigate similar future issues, perhaps a scheduled job should be added the package index CI system to validate that package sources aren't 404'ing in addition to validating that any new packages added are valid JSON or whatever.

/cc @dom96 as you originally created the package index and since it's now part of the language org, i assume you have some say in how package validation happens and if it is part of a nightly CI job or something.

[dom96]

the CI should focus on newly added packages, if we want to remove packages that are gone then we should have a separate job that does so IMO

[georgelemon]

test workflow takes a life to run. why not using -d:dontFetchRepos ? If i make a new pull request it will try fetch everything over and over. Maybe store 404 pkgs in a problematic_packages.json, or something.

99 problems 😅

W: Unkown error fetching repo https://pmetras.gitlab.io/nim0 No uri scheme supplied.
W: nio: HTTP error fetching repo https://github.com/KayraG/nio 404 Not Found
W: ao: HTTP error fetching repo https://github.com/ephja/nim-ao 404 Not Found
W: linagl: HTTP error fetching repo https://bitbucket.org/BitPuffin/linagl 404 Not Found
W: csv2json: HTTP error fetching repo https://github.com/achesak/nim-csv2json 404 Not Found
W: Unkown error fetching repo https://gitlab.3dicc.com/gokr/lapp error:0A000086:SSL routines::certificate verify failed
W: Unkown error fetching repo https://gitlab.3dicc.com/gokr/blimp error:0A000086:SSL routines::certificate verify failed
W: asyncevents: HTTP error fetching repo https://github.com/tulayang/asyncevents 404 Not Found
W: libjwt: HTTP error fetching repo https://github.com/nimscale/nim-libjwt 404 Not Found
W: forestdb: HTTP error fetching repo https://github.com/nimscale/forestdb 404 Not Found
W: stor: HTTP error fetching repo https://github.com/nimscale/stor 404 Not Found
W: isa: HTTP error fetching repo https://github.com/nimscale/isa 404 Not Found
W: spdk: HTTP error fetching repo https://github.com/nimscale/spdk.git 404 Not Found
W: syphus: HTTP error fetching repo https://github.com/makingspace/syphus-nim 404 Not Found
W: pdcurses: HTTP error fetching repo https://github.com/lcrees/pdcurses 404 Not Found
W: libuv: HTTP error fetching repo https://github.com/lcrees/libuv 404 Not Found
W: romans: HTTP error fetching repo https://github.com/lcrees/romans 404 Not Found
W: gentabs: HTTP error fetching repo https://github.com/lcrees/gentabs 404 Not Found
W: nimetry: HTTP error fetching repo https://github.com/ijneb/nimetry 404 Not Found
W: algebra: HTTP error fetching repo https://github.com/ijneb/nim-algebra 404 Not Found
W: gurl: HTTP error fetching repo https://github.com/MaxUNof/gurl 404 Not Found
W: nim-nmap: HTTP error fetching repo https://github.com/blmvxer/nim-nmap 404 Not Found
W: redneck_translator: HTTP error fetching repo https://github.com/juancarlospaco/redneck-translator 404 Not Found
W: sweetanitify: HTTP error fetching repo https://github.com/juancarlospaco/sweetanitify 404 Not Found
W: htmlview: HTTP error fetching repo https://github.com/yuchunzhou/htmlview 404 Not Found
W: skbintext: HTTP error fetching repo https://github.com/Skrylar/skbintext 404 Not Found
W: skyhash: HTTP error fetching repo https://github.com/Skrylar/skyhash 404 Not Found
W: yukiko: HTTP error fetching repo https://github.com/ethosa/yukiko 404 Not Found
W: origin: HTTP error fetching repo https://github.com/mfiano/origin.nim 404 Not Found
W: dotprov: HTTP error fetching repo https://github.com/minefuto/dotprov 404 Not Found
W: gradient: HTTP error fetching repo https://github.com/luminosoda/gradient 404 Not Found
W: base[64](https://github.com/nim-lang/packages/actions/runs/4104717088/jobs/7080571812#step:4:65)_decoder: HTTP error fetching repo https://github.com/momeemt/base64_cui 404 Not Found
W: ms: HTTP error fetching repo https://fox-cat.github.io/ms/ 404 Not Found
W: psutilim: HTTP error fetching repo https://github.com/Techno-Fox/psutilim 404 Not Found
W: blackvas_cli: HTTP error fetching repo https://github.com/momeemt/BlackvasCli 404 Not Found
W: sliceutils: HTTP error fetching repo https://metagn.github.io/sliceutils/sliceutils.html 404 Not Found
W: assigns: HTTP error fetching repo https://metagn.github.io/assigns/assigns.html 404 Not Found
W: applicates: HTTP error fetching repo https://metagn.github.io/applicates/applicates.html 404 Not Found
W: hats: HTTP error fetching repo https://github.com/davidgarland/nim-hats 404 Not Found
W: composition: HTTP error fetching repo https://github.com/DavidMeagher1/composition 404 Not Found
W: psy: HTTP error fetching repo https://github.com/psypac/psypac 404 Not Found

[Yepoleb]

Deleted packages are now marked with the tag "deleted"