routing/http!: support for IPIP-378 (delegated content and peer providing)

[hacdias]

This is the client and server implementation for the delegated PUTs for peers and providers. See the specification on ipfs/specs#378. There are breaking changes. Check the changelog to know more about them.

Since the specification is not yet finished and set in stone, there are quite a few questions that I need answered in order to finish the code. They are written as TODO comments in line and I left them also in the specs PR.

• Solve in-code TODO, which are mostly related to specifications
• Client tests for Provide (also test server)
• Client tests for ProvidePeer (also test server)
• Simple server tests for Provide output
• Simple server tests for ProvidePeer output
• Changelog

Not needed to merge this:

• someguy: feat: implement delegated PUTs someguy#40
• kubo: feat: IPIP-378 (delegated provide over HTTP /routing/v1) kubo#10420
  • Only dependency update, no support for POST per se
• ipni: feat: update boxo with new Provide interface ipni/index-provider#441

[codecov]

Codecov Report

Attention: Patch coverage is 57.47126% with 222 lines in your changes are missing coverage. Please review.

Project coverage is 59.69%. Comparing base (0f223aa) to head (abf64fc).

@@            Coverage Diff             @@
##             main     #539      +/-   ##
==========================================
+ Coverage   59.66%   59.69%   +0.02%     
==========================================
  Files         238      237       -1     
  Lines       29883    30071     +188     
==========================================
+ Hits        17831    17951     +120     
- Misses      10433    10485      +52     
- Partials     1619     1635      +16     

Files	Coverage Δ
routing/http/types/ipfs.go	50.00% <ø> (-2.00%)	⬇️
routing/http/types/iter/iter.go	89.28% <100.00%> (ø)
routing/http/contentrouter/contentrouter.go	51.55% <88.88%> (+2.41%)	⬆️
routing/http/types/iter/filter.go	76.19% <76.19%> (ø)
routing/http/types/json/responses.go	41.86% <10.00%> (-8.96%)	⬇️
routing/http/types/ndjson/records.go	38.18% <42.85%> (-16.93%)	⬇️
routing/http/server/server.go	68.22% <63.33%> (-3.84%)	⬇️
routing/http/client/client.go	66.91% <59.15%> (-6.10%)	⬇️
routing/http/types/record_announcement.go	54.75% <54.75%> (ø)

... and 17 files with indirect coverage changes

[hacdias]

There's still a few tests added, but I'm opening this as ready for review @lidel. It would be useful to go through the code and checking the TODOs that I added inline.

[lidel]

Pushed some changes to IPIP-378 PR, tldr in ipfs/specs#378 (comment)
Some feedback on TODOs inline.

[lidel]

Correct. But the fact you had to ask, means everyone implementing it will have to think about too 🤔

Should we reconsider and have two separate schemas instead of one (provider-record and peer-record) ?

[hacdias]

@lidel I still think one is fine, I'm not exactly sure why I asked this. As long as it is stated in the specs, it is fine. I think it did not say anything when I wrote this.

I also see you introduced the error schema. That will make the code more complicated. Instead, I want to suggest a Announcement Response Schema. It is a mix of the announcement schema and the error schema:

{
  "Schema": "announcement-response",
  "Error": "error in case there was error",
  "TTL": 17280000
}

This perhaps allows better coupling between the announcement and its individual response. We can even add the CID and/or PID to have more context.

[lidel]

@hacdias sounds like a sensible change. Are you able to apply this (remove error and andd this response one) to ipfs/specs#378?

[hacdias]

@lidel made an update