Extend URI scheme for delegation portfolio links

[rphair]

Non-technical visitors: See complete draft of CIP, including proposed changes, here: CIP-0013.md

The multi pool component of this originally submitted stand-alone CIP (CIP-COSD-MultiPoolStakeURIscheme.md) was split off as a result of #25 (comment) - with an agreement to add it back in after officially merging a standard for single pool links into an active CIP (which later became CIP-0013). Some notes & ideas beyond the CIP content:

As @SebastienGllmt has suggested in meetings & media there is a chance single pool links will be implemented earlier in the wallet development timeline (we hope, earlier) than the multi pool links (which we're prepared to get later).

• To allow for this I've modified the language of that section a bit which suggests wallets should generate an error if passed a multi-pool delegation URI while still supporting only single pool links.
• I believe this is easier to follow than having 2 levels of the standard (or 2 separate CIPs) each with their own relevant ABNF grammar, syntax rules, etc.
• Many SPOs would like to get started immediately on single stake pool references even if the multi-pool links & portfolios still await difficult protocol changes like "ratio staking" and complex updates to the wallet UIs.

Multi-pool details of this Specification, especially how to interpret each decimal proportion given to each pool, were originally published in the latest version of this CIP draft submitted 2020-09-30: #25 > Specification

My understanding of the CIP process is that we can't use recently submitted JSON specification (#82) as a Reference implementation, so I've added a section in Rationale briefly comparing the two schemes. After six months of discussing this I am sure the community will need both. Some other ideas on how JSON vs. URI delegation portfolios might evolve concurrently:

• @KtorZ @wrmarchetto The rules of interpreting proportions posted in this CIP, and the discussion of integer vs. decimal type and what to do if any of the proportion parameters are missing, might also help with more rigorous definition of the JSON standard for delegation portfolios.
• I hope we can discuss how URIs might be considered more decentralised and more resistant to censorship than hosted files; a question which came up originally here (2021-09-23) when JSON was suggested as a preferred alternative to URI delegation lists.

It's a long shot that the exporting live delegation figures as a portfolio would reveal a user's identity, but I called attention to that problem in #25 > Specification and have added it as an explicit security consideration here. It's a minor risk, but given the amount of care taken to preserve wallet anonymity for users that want it, it would be a shame to give them an accidental means of publicly identifying themselves.

[rphair]

p.s. BTW I'm not sure why this is happening but all activity shown between here & the original PR comment has already taken place in the context of #61. It might be because the last PR was inadvertently executed in the master branch of my fork... something that won't happen again. 🤪

[rphair]

These are standards related items I'm not including in the CIP itself for the sake of brevity & efficiency... if any editor thinks any of these points should be included, let me know & I'll edit it back in.

Points for Consideration

(Motivation) If we have single pool links, we need multi pool links:

• logically, since the repeated key-value structure of URIs makes this easy to understand & parse.
• practically, since saturation limit will only decrease which makes the single pool link applicable to progressively fewer uses cases as more single pools break up into multiple pools.

#25 > Rationale (non-technical) outlined (in September 2020) many use cases for delegation portfolios and multi-pool links.

• Today it seems the need for multi-pool links is self-evident, especially after the round of applause given to the more recent submitted CIP-0017: Cardano Delegation Portfolio #82.
• So I'm recalling these again because the list of use cases might serve as a good user stories to determine how the wallets might handle each case of a delegation portfolio, whether implemented as URI or JSON.

(from #25 > Specification) Handling of errors and warnings should be left to the application. Examples:

• a non-numerical proportion might assume 1 (as if the figure had been left out entirely) while also displaying a warning message & asking the user if they would like to proceed according to that assumption or abort.
• a non-existent stake pool might cause a warning, with an offer to proceed without that pool, to choose an alternative, or to abort.

The original PR had a lot of comments about URI length, with a response indicating why this is not a problem for a variety of use cases, with related factors like data precision. So it would help to check this out, to see if any of these matters should be addressed in the CIP: #25 (comment)

There was never a response to this forum comment (@disassembler) about the HTTP content disposition of the JSON file and how it would help certain application scenarios to have a URI alternative (the current CIP draft simply suggests that having an alternative is helpful in these cases).

The same thread had a statement which I might summarise as:

Not requiring a JSON file also supports decentralisation though greater anonymity and censorship resistance.

... which I'm not including since so we don't get hung up in a debate about what all these things really are. The forum thread above suggests why the Internet at large considers URIs less identity-revealing and less easily blocked than files.

@KtorZ With respect to the CIP-0017 current draft (#82):

• I'm not planning to document in CIP-0013 why the stake pool proportions should be decimals, rather than integers as suggested in the later CIP, because no reason has yet been given for restricting these to integers. Until such a reason is given, the general case of decimal proportions is more logical simply because it's more flexible.
• There's no need for floating point precision but to facilitate certain use cases we will need decimals, to avoid having to scale fixed point numbers & normalise fractional ratios: CIP-0017: Cardano Delegation Portfolio #82 (comment)

A useful link I'm not including in Read More section (though @wrmarchetto I was very happy to find out about this site last week 😍):

• https://adafolio.com (great example of Cardano delegation portfolio use cases but already cited in CIP-0017: Cardano Delegation Portfolio #82 draft)

[v-almonacid]

Hey @rphair, could you please check this proposal I just submitted for comments? https://forum.cardano.org/t/cip-generalized-cardano-urls/57464
Essentially, this would mean that this PR should just be reformulated as a separate CIP (well, if the proposal makes sense for the rest of the community)

[rphair]

thanks @v-almonacid ... breaking it up that way would be OK with me because the writing regarding stake pool extensions has been generally distinct from what you and @SebastienGllmt had written already. And I can see that there are heaps of metadata URI features on the way which would make any "Cardano URI scheme" document as long as any technical paper.

I would only insist that the merge process for this PR, which Sebastien himself suggested back in November... i.e., consolidating the multi-pool links in with the single-pool links... be completed first. Like all standards, a coherent presentation of that standard should be dated... and the effort keeps getting greater and greater on my part to keep rewriting, refactoring, consolidating and re-posting what's already been done.

As we heard in the CIP editor meetings, where I had to catch up the attendees for 5 meetings in a row about a 6-month old proposal, the resulting history has become difficult even for people already exposed to the issue to follow: and as with all standards we need to trace this issue historically.

So while everybody is still current in the discussion of multi-pool links, hopefully this will be acceptable to @crptmppt @dcoutts @KtorZ and the other editors from that meeting, and the idea & writing of multi-pool links can be considered on their own in this PR and merged before too long. Then I'll follow the CIP team's suggestions about breaking the stake pool linking scheme (for the //stake URI authority) into its own document.

[v-almonacid]

@rphair yeah I'm totally aware of the time and work that you've put on this, and that things have taken much more time than expected, it's unfortunate. I feel like If we had a framework as the one I'm proposing before, things could have been easier.

Anyway, it's totally fine for me if this PR gets merged in CIP13. Just keep in mind that this feature may still take a bit of time to be actually implemented by wallets, so there is no rush. I'm saying this just to suggest that we should at least double check that this is 100% compatible with the new standard, if it ever gets approved.

[KtorZ]

If anything, it'd be good to have CIP-0017 & this extension aligned; Please see the comment on CIP-0017 about decimals

[rphair]

thanks, addressed here: #82 (comment)

[KtorZ]

@rphair (thanks for your comment on the CIP-0017). I acknowledge your points but I do not necessarily agree with all your statements (although, I do concur that some of my points are skewed / biased towards a certain view as a wallet maintainer).

I'd like to understand what is your main argument for wanting decimals in here? From one of your previous comment, it sounded very speculative to me 😶

@rphair: Some users & developers will require decimals as the stake pool weights in order to build portfolios that match the following real world values [...]

---

Note that re-reading some of your comments, I think there is maybe a misunderstanding (?) on the purpose of the portfolio definition. The proposal from CIP-0017 and I believe, this one here are not about extending the protocol in any ways to support multi-delegation. This is unlikely to happen anytime soon and is a whole different discussion.

Hence this is about sharing a financial portfolio with other stakeholders, and let them know what works well. It is not about capturing the state of a wallet, but rather to give some recipes and recommendations about delegation choices.

[KtorZ]

The three last points seem to me to be invalid URLs / portfolios definitions. To be valid, a portfolio should always define all proportions for all pools, and each pool should only appear once.
Having multiple default cases and parsing behavior would only render parsing downstream more complex. I don't see the rationale for allowing this?

[rphair]

@KtorZ since the last 2 editing comments are interrelated I need to bring them together here:

re: #86 (comment)

As far as the general Cardano community has been aware, there has been no information about whether ratio staking will be supported in the Cardano protocol between these two statements:

• 2020-10-06: (https://www.youtube.com/watch?v=H_Map7ncsWc) paraphrase: Ratio staking supported in the Cardano protocol would be the preferred & cleaner way to do it.

• 2021-05-11 (today, above): the protocol [extension] to support multi-delegation ... unlikely to happen anytime soon and is a whole different discussion.

So given that ratio staking hasn't been ruled out of the protocol yet, we have to assume that portfolio definitions will eventually gain an achievable amount of multi digit precision... in addition to, but not instead of, your observations that first stage wallet implementations are bound to be imprecise.

For both the near & long term, the world at large understands degrees of precision through decimals, while the cryptographic community prefers integers of greater magnitude. I see no reason to confine the standard & syntax so they only support the cryptographic community's usage and expectations.

Most standards which fail, or at least have to be revised and updated, are because of a rationalisation that something "will never happen" (e.g. protocol-based ratio staking). We'll get other people chiming in for sure, but as both a developer & a scientist I don't want to truncate every ratio I might ever use, even if the first-stage implementations are practically guaranteed to implement them imprecisely.

Regardless of the above: any attempt to ease the parsing & special cases suggests there has to be a reasonable default for the stake pool proportions... and the universally assumed default for multiplicity in a set is 1 (more examples below) which implies that, in a sparsely proportioned list of pools, any lesser proportioned pools would have to be assigned a fraction between 0 and 1.

So if you accept the argument that there must be a default proportion, and that the only sensible default is 1, that in itself would be a requirement for allowing decimals.

re: #86 (review)

You're right... especially if precise portfolio definitions are not universally achievable, there's no incentive to use such a precisely calculated default proportion value, especially given the parsing difficulty. The complexity of such an arrangement would no longer be desirable and would be difficult for people to understand & anticipate.

So my last commit combines those first 2 contended bullet items into the single statement:

• Any missing proportion is assigned a precise value of 1.

I've also changed the last bullet point:

• If a stake pool is mentioned multiple times, those proportions as determined above are added together.

... since in hindsight I can't imagine on anyone agreeing on an algorithmic approach, whether the above or any other, to resolve such a condition. Therefore I've changed it to an error condition pending any other suggestions:

• If a stake pool is listed multiple times, the URI is rejected as invalid.

On this note, I haven't addressed the subject of exactly what to do if a pool is referred to by both its poolhexid and its poolticker. I believe, according to the current wording, such a URI is also considered invalid. Since the wallet behaviour could handle that in different ways, I won't go into further detail in the CIP unless suggested by the developers.

To be valid, a portfolio should always define all proportions for all pools

NOTE I believe most of the points below will also apply to JSON portfolio definitions (#82):

Defaulting to a condition of symmetry is an idea long upheld in computer science and programming languages. I am dating myself a bit here, but a longer history of developers have appreciated simpler & shorter syntaxes that reflect how human languages themselves work: assuming symmetry and regularity until proven otherwise.

For a current & ubiquitous example see the column & spacing in web design via CSS grid and flexbox with rem units ... and imagine how irate the world of web designers would feel if they had to specify the width of every element, even if only to set them all to 1. Now imagine the frustration and rebellion of web link creators forced to follow a more rigid standard without such reasonable defaults... and get ready for lots of extra development time spent processing syntax errors & implementing failure modes as they proceed with those assumed defaults anyway.

Not having to specify all the portfolio weights every time makes even more sense given the (only recent) annoucement that the stake pool weights might only ever be approximately achievable. To paraphrase your #82 (comment), I would say: to force the user & developer to set all stake pool weights explicitly to 1 (or any value) would falsely suggest the wallet's ability to provide stake pool congruence / equality in a portfolio.

We can infer today from the unweighted short "favourite" lists of pool tickers already circulating for months on Twitter that the most common formulation of a multiple stake pool URI will be without favourites or proportions, like:

web+cardano://stake?POOL1&POOL2&POOL3&POOL4&POOL5

... which, after enforcing explicit portfolio weighting, would instead look like this (fat fingers get ready to type those extra top row characters):

web+cardano://stake?POOL1=1&POOL2=1&POOL3=1&POOL4=1&POOL5=1

There are single pool lists already out there with hundreds of tickers in them, and a default value for pool weight will allow these to be dense and readable by both humans and machines regardless of length.

Assuming a proportion of 1 for any of the omitted proportions will also avoid a crushing blow to URI brevity & clarity when you have a long list of single pool tickers but need to increase or decrease only a few if not only one of them (i.e. when weighting differences are "sparse"). Again, as in thought & language, this provides the user & developer a means of saying "all pools are considered equal in proportion except ..."

Finally, if it is parsing complexity the developers are worried about: generally setting a parameter default array key value of 1 for a parsed pool is algorithmically simpler than setting a procedural default of "branch into some kind of error processing condition." So the 1 default should also satisfy your concern about parsing difficulties.

[crptmppt]

This was again discussed at the last Editor meeting (22), please refer to the notes to catchup on the conversation.
This PR is scheduled to be discussed further at the next public meeting this Tuesday, thank you for attending as feasible to broaden/continue the conversation.

[crptmppt]

This was discussed at the last Editor meeting (23), please refer to the notes.
An outreach is in progress towards the community/SPOs to gather preferences and open the conversation.
@rphair is to connect through a SPO dev townhall (likely June 10th) while a broader "CIP-themed" SPO call is tentatively scheduled for June 24th. (the next CIP Editors meeting after that should be Editors meeting 26 on July 6th.

[rphair]

@KtorZ I will also mention the JSON delegation portfolio alternative to the URI based ones, and will be referring SPOs to #86 as well as this one. If you (or any other SPOs following this issue) want that Zoom link please message me directly (I don't think they want the Zoom links for those meeting to be publicly visible). The upcoming meeting title is SPO community call on Thursday, 10th June at 16:00 UTC. 😎

[crptmppt]

Discussed at last Editors meeting (24) - see notes. (conversation might be ahead of meeting notes at this point, this is a reference)

The conversation was further opened up with two SPO meeting.
(Adding this PR as a "Discussion" for tomorrow's CIP Editors Meeting (25 - in contrast to PR82 in the hope of consolidating)

[crptmppt]

6/29 Editors meeting (25) discussed this PR - see notes. (conversation might be ahead of meeting notes at this point, this is a reference)

=> PR is now in 'Last Check': it will get another look at next week's meeting (26) and should be merged in after that - consider attending the meeting if relevant to you.