Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweaking the feerate security margin #6974

Closed
cdecker opened this issue Jan 3, 2024 · 1 comment · Fixed by #7063
Closed

Tweaking the feerate security margin #6974

cdecker opened this issue Jan 3, 2024 · 1 comment · Fixed by #7063

Comments

@cdecker
Copy link
Member

cdecker commented Jan 3, 2024

When forwarding an HTLC the funder needs to ensure that it can at
least add one more HTLC. This is the result of LN#728, which aims to
protect a channel to get into a mode where no new HTLC can be added,
not even by the fundee to rebalance the channel (due to feerate
spiking causing the commitment TX fee to no longer be payable by the
funder).

CLN is trying to be secure even against larger feerate spikes, by
being very pessimistic and assuming up to a 100% feerate spike (using
200% current feerate to determine whether adding an HTLC is safe or
not).

I would argue that for low-fee environments the 100% security margin
is justified, since the likelihood of 2 * 1sat/vbyte is much more
likely than 2 * fee ATH, but I would linearize this buffer to reduce
it relative to the absolute fee as the feerate grows.

My proposal would be to:

  1. Take today's fee ATH: maxfeerate
  2. Use 1sat/vbyte as minfeerate
  3. Compute the current feerate margin to use as the following: 1 - min(x - minfeerate, maxfeerate)/(maxfeerate * 1.1). This results
    in a 2x current feerate close to the minfeerate, smoothly
    discounted until we get to maxfeerate at which point we just
    take a 10% margin.

The idea here is that close to the lower end where feerates are likely
to increase from, the multiplier is still the current 2x. Closer to
the historical maximum feerate we only apply a 10% margin, as the fees
are much less likely to increase further. The place to apply this
would be the code line, where we replace the 2 constant with the
margin computation.

I'm currently trying to reproduce this in a test to verify the fix
works as expected, and some historical analysis on max feerates to use
could further inform the 3 variables in play here:

  • Starting slope (2)
  • Final slope (1.1)
  • Historical max feerate (41234perkw, taken from the original issue)

Additional information

In case somebody may be wondering what running into this issue looks
like, here are the relevant log lines:

2024-01-01T11:33:18.422Z DEBUG   03d6f117b93d0eb1c440de5549cc4f48253f94dfdcab5f92a3199167323ac05016-channeld-chan#1672: Adding HTLC would leave us only 51840000msat: we need 60531sat for another HTLC if fees increase by 100% to 41234perkw
2024-01-01T11:33:18.423Z DEBUG   03d6f117b93d0eb1c440de5549cc4f48253f94dfdcab5f92a3199167323ac05016-channeld-chan#1672: Adding HTLC 0 amount=2500000msat cltv=824005 gave CHANNEL_ERR_CHANNEL_CAPACITY_EXCEEDED
2024-01-01T11:33:18.423Z DEBUG   03d6f117b93d0eb1c440de5549cc4f48253f94dfdcab5f92a3199167323ac05016-channeld-chan#1672: REPLY WIRE_CHANNELD_OFFER_HTLC_REPLY with 0 fds
@cdecker
Copy link
Member Author

cdecker commented Jan 3, 2024

And since I'm at it, here's a plot comparing the old feerate margin to the new feerate margin:

download

The excess fee we're paying at various feerates is given by the difference between the feerate and the percent line for the current behavior, and as the difference between feerate and smoothed for this proposal. Notice that both start with a 200% feerate margin, but then the fixed percentage just grows much quicker, whereas the smoothed one converges to 10% margin for higher values.

cdecker added a commit to cdecker/lightning that referenced this issue Feb 11, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
cdecker added a commit to cdecker/lightning that referenced this issue Feb 13, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
cdecker added a commit to cdecker/lightning that referenced this issue Feb 16, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
cdecker added a commit to cdecker/lightning that referenced this issue Feb 21, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
cdecker added a commit to cdecker/lightning that referenced this issue Feb 21, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
rustyrussell pushed a commit to cdecker/lightning that referenced this issue May 9, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
rustyrussell pushed a commit to cdecker/lightning that referenced this issue May 9, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
[Fixed up all the other changes required, including spendable calcualtion
 comments and tests --RR]


Header from folded patch 'wip.patch':
endothermicdev pushed a commit to cdecker/lightning that referenced this issue May 9, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
[Fixed up all the other changes required, including spendable calcualtion
 comments and tests --RR]


Header from folded patch 'wip.patch':
rustyrussell pushed a commit to cdecker/lightning that referenced this issue May 10, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
[Fixed up all the other changes required, including spendable calcualtion
 comments and unit test and pytest tests --RR]
endothermicdev pushed a commit that referenced this issue May 13, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes #6974
Closes #6976
[Fixed up all the other changes required, including spendable calcualtion
 comments and unit test and pytest tests --RR]
rustyrussell pushed a commit to rustyrussell/lightning that referenced this issue May 14, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
[Fixed up all the other changes required, including spendable calcualtion
 comments and unit test and pytest tests --RR]
endothermicdev pushed a commit to rustyrussell/lightning that referenced this issue May 14, 2024
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.

The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.

Fixes ElementsProject#6974
Closes ElementsProject#6976
[Fixed up all the other changes required, including spendable calcualtion
 comments and unit test and pytest tests --RR]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant