API providing contributor data

[phildini]

Hi! As a result of https://github.com/feross/thanks, I wrote https://github.com/phildini/thanks. It's goal is to make it easier for people to find out how to support the developers writing their python packages.

It's currently using the XMLRPC API against pypi.python.org, because that's the only API I can find that returns the package_roles information.

I would rather be using JSON, and pointing thanks at pypi.org.

After chatting with @ewdurbin at PyTN, it sounds like there's some discussion about what APIs to support long-term in warehouse.

I would really enjoy continuing to have API access to some identifier for package maintainers, so that I can encourage people to pay them (and also for whatever other applications the community can think of. Wall of fame? Random kudos? Who knows!).

Apologies if this ticket is duplicating, or should go somewhere else, happy to link it if there's an issue I missed in reading through issues.

[phildini]

The issues I did come across that I think this links to are #2913 and #2912

[ewdurbin]

#284 is also relevant.

[di]

Seems reasonable to add a list of contributor usernames to Project JSON API, though it might be a bit more work than expected because right now it's actually just returning the latest release.

[brainwane]

@phildini thanks for your note and sorry for the slow response!

Since the most urgent task is to replace the core features of legacy PyPI so the site is more sustainable and reliable, and this feature isn't as core to the PyPI experience as other features are, I've moved it to a future milestone. I wonder whether libraries.io would be a good API for you to also try using in the interim?

Thanks and sorry again for the wait.

[brainwane]

@phildini have you checked out the libraries.io APIs?

@hndrkmkt, @waseem18, @cryvate, @asottile, are any of you interested in doing a little investigation on this requested feature and reporting back in this issue?

[waseem18]

@brainwane I'll investigate and will put up the report here in the comments.

[waseem18]

The json_project method returns method json_release with one of the parameters being release. contributors query (find query below) can be similar to release query and contributors should also be passed as a parameter to the json_release method where the key-value "contributors": contributors is added to the info dictionary.

When we hit pypi/<package>/json we'll be able to see a contributors key in the info dictionary with value being a list of usernames (maintainers and owners of the package).

    try:
    contributors = (
        request.db.query(Role)
                  .join(User, Project)
                  .filter(Project.normalized_name ==
                          func.normalize_pep426_name(project.name))
                  .order_by(
                      Role.role_name.desc(), User.username)
                  .all()
    )
    contributors = [r.user.username for r in contributors]
except NoResultFound:
    return HTTPNotFound()

[waseem18]

@di Could you please review the above approach?

Thanks

[brainwane]

Thanks for your writeup, @waseem18! Since Dustin (and others on the MOSS-funded team) are focusing right now on getting the beta out the door and then getting to the redirect/launch milestone, Dustin will probably wait till after those are done to comment on the approach you've suggested. But I hope other people will comment on your suggestion.

[brainwane]

I realized we can do this after Legacy PyPI shutdown, since we do provide package_roles in the Warehouse XML-RPC API.

[matthewbelisle-wf]

One thing to note is that the current response field maintainer is kind of a red herring because it's a string (albeit empty). A field called package_roles would be good I think, in the same format as the xml rpc api [[roleName, userName], ...].

$ curl https://pypi.org/pypi/requests/json
...
    "maintainer": "",
...

>>> from xmlrpc.client import ServerProxy
>>> ServerProxy("https://pypi.org/pypi").package_roles("requests")
[['Owner', 'graffatcolmingov'], ['Owner', 'Lukasa'], ['Owner', 'nateprewitt']]