Multi-Asset Coin Selection

[jonathanknowles]

Issue Number

ADP-605

Overview

This PR implements the Random-Round-Robin coin selection algorithm for multi-asset UTxO sets.

The Random-Round-Robin algorithm is inspired by the Random-Improve algorithm, but with some differences to accommodate UTxO sets with multiple assets.

Top-Level Algorithm Description

The Random-Round-Robin algorithm considers the sum of all outputs collectively, selecting inputs to cover the total sum of all asset quantities, rather than running a separate selection for each input. It therefore drops the cardinality restriction of the original Random-Improve algorithm implementation.

Steps

1. When selecting inputs, consider tokens in round-robin fashion, selecting one input per token before moving to the next token, to reduce the chance of over-selecting for any particular token.
2. For each token quantity under consideration, select enough inputs to cover at least 100% of that quantity, but aim to get as close to a target of 200% as possible. When we can make no further improvement for a given token, we eliminate that token from the round-robin selection process. An improvement is defined as an additional selection that takes the total selected token quantity closer to 200% of the output token quantity, but not further away.
3. The round-robin selection phase terminates when we can make no further improvement for any token in the set under consideration.
4. After the selection phase is over, divide any excess token quantities (inputs − outputs) into change bundles, where:
   • there is exactly one change bundle for each output.
   • the quantity of a given token in a change bundle is proportional to the quantity of that token in the corresponding output (modulo rounding).
   • the total quantity of a given token across all change bundles is equal to the total excess quantity of that token.
5. Redistribute additionally-selected tokens not present in the original outputs to the change bundles, where:
   • if there are fewer quantities for a given token than the number of change bundles, include these quantities without changing them.
   • if there are more quantities for a given token than the number of change bundles, repeatedly coalesce the smallest pair of quantities together until the total number of quantities is exactly equal to the number of change bundles.

[KtorZ]

To be frank, I am very doubtful about this module 😬 ... I do think ceiling and floor are so common across all programming languages that there's no possible ambiguity about what those functions do.

[jonathanknowles]

To be frank, I am very doubtful about this module grimacing ... I do think ceiling and floor are so common across all programming languages that there's no possible ambiguity about what those functions do.

Good point. These functions could easily be moved into the Util module, as they're currently only used by the partitionNatural function.

[jonathanknowles]

@KtorZ Fixed in this change.

[KtorZ]

Note that my main concern wasn't much about why this is in a separate module, but why this has to be defined altogether :). I think that any places that uses round RoundUp or round RoundDown could simply use ceiling and floor. That latter option is actually clearer and more idiomatic. There's no need to build an abstraction over it IMO.

[jonathanknowles]

I think that any places that uses round RoundUp or round RoundDown could simply use ceiling and floor.

Very true.

However, my original goal here was simply to make this code more self-documenting: (taken from partitionNatural)

roundings :: NonEmpty RoundingDirection
roundings =
    applyN shortfall (NE.cons RoundUp) (NE.repeat RoundDown)
  where
    shortfall
        = fromIntegral target
        - fromIntegral @Integer
            (F.sum $ round RoundDown <$> portionsUnrounded)

IMO, the purpose of roundings is much clearer and more self-documenting with the RoundingDirection type. It's a non-empty list of rounding directions (either up or down).

One alternative would be to write something like this:

roundings :: (RealFrac a, Integral b) => NonEmpty (a -> b)

But this makes the intent much less obvious IMO, and would probably require an additional comment to explain what's going on.

[KtorZ]

Does this really have to be a list? It seems that we only really want a length here. The examples make it every more awkward for they all use replicate n () 😅

[jonathanknowles]

Does this really have to be a list?

It definitely doesn't have to be a list!

However, accepting a non-empty list makes it nice to re-use from functions that already have a non-empty list of the target size.

For example:

changeForSurplusAssets calls makeChangeForSurplusAssets with the already-existing outputBundles, which calls partitionTokenQuantity, which eventually calls partitionNatural. So we end up coalescing (or padding) the surplus asset quantities into exactly n chunks, where n is number of output bundles.

As a bonus, using NonEmpty here allows us to make this function total, as the minimum number of elements in a non-empty list is always one.

If we accepted an Int (or something else), then presumably we'd need to choose from one of the following non-ideal options:

• make the function return Maybe if the target length is < 1 (in which case the caller would need to deal with the awkward Nothing value that they know they shouldn't receive).
• make the function throw an error if the target length is < 1.
• do something else, like taking max 1 targetLength. But if the caller supplies a targetLength less than 1, that presumably represents a programming error that we'd want to know about.

[KtorZ]

As a bonus, using NonEmpty here allows us to make this function total, as the minimum number of elements in a non-empty list is always one.

Fair enough. That's a good argument.

[KtorZ]

To my understanding, partition has the sense of division. It feels a bit weird to me in this context, wouldn't distribute better capture the behavior 🤔 ?

[jonathanknowles]

To my understanding, partition has the sense of division. It feels a bit weird to me in this context, wouldn't distribute better capture the behavior thinking ?

In this case, the term "partition" is taken from number theory:

https://en.wikipedia.org/wiki/Partition_(number_theory)

Quote:

In number theory and combinatorics, a partition of a positive integer n, also called an integer partition, is a way of writing n as a sum of positive integers.

Admittedly, in our particular case, we are computing a special kind of partition, where the elements of the partition are proportional to the sizes of the specified weights (modulo rounding).

The term "distribute" would probably also work here, but IMO this name doesn't capture (as nicely) the property that the total sum is preserved (whereas the term "partition" implies this, if we assume the definition above).

[KtorZ]

I like this approach. We could have callers to performSelection possibly retry on SelectionInsufficient errors. If we're close to the required balance, the selection may still succeed with a different selection of inputs.

[jonathanknowles]

We could have callers to performSelection possibly retry on SelectionInsufficient errors. If we're close to the required balance, the selection may still succeed with a different selection of inputs.

I think that's a great idea. 👍🏻

[jonathanknowles]

bors r+

[iohk-bors]

Build failed:

• ci/hydra-build:required

Failures:

  src/Test/Integration/Scenario/API/Shelley/StakePools.hs:576:5: 
  1) API Specifications, SHELLEY_STAKE_POOLS, STAKE_POOLS_JOIN_04 - Rewards accumulate
       uncaught exception: RequestException
       DecodeFailure "{\"code\":\"network_query_failed\",\"message\":\"Unable to query the ledger at the moment. This error has been logged. Trying again in a bit might work.\"}"

  To rerun use: --match "/API Specifications/SHELLEY_STAKE_POOLS/STAKE_POOLS_JOIN_04 - Rewards accumulate/"

Randomized with seed 1981313050

#2320

[jonathanknowles]

bors r+

[iohk-bors]

Build failed:

• ci/hydra-build:required

Failures:

  src/Test/Integration/Scenario/API/Shelley/StakePools.hs:836:9: 
  1) API Specifications, SHELLEY_STAKE_POOLS, STAKE_POOLS_JOIN_01x - Fee boundary values, STAKE_POOLS_JOIN_01x - I cannot join if I have not enough fee to cover
       uncaught exception: RequestException
       DecodeFailure "{\"code\":\"network_query_failed\",\"message\":\"Unable to query the ledger at the moment. This error has been logged. Trying again in a bit might work.\"}"

  To rerun use: --match "/API Specifications/SHELLEY_STAKE_POOLS/STAKE_POOLS_JOIN_01x - Fee boundary values/STAKE_POOLS_JOIN_01x - I cannot join if I have not enough fee to cover/"

Randomized with seed 1318638626

#2320

[jonathanknowles]

bors r+

[iohk-bors]

Build succeeded:

• buildkite/cardano-wallet
• ci/hydra-build:required