Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cancel back outgoing dust htlcs before commitment is confirmed. #9068

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

ziggie1984
Copy link
Collaborator

@ziggie1984 ziggie1984 commented Sep 5, 2024

Fixes #7969

Copy link
Contributor

coderabbitai bot commented Sep 5, 2024

Important

Review skipped

Auto reviews are limited to specific labels.

Labels to auto review (1)
  • llm-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

The changes enhance the handling of canceled Hash Time-Locked Contracts (HTLCs) within the contract arbitration process. New methods for inserting and fetching canceled HTLCs are added to the ArbitratorLog interface and its implementation. Additionally, logic for managing dust HTLCs is improved, ensuring they are canceled promptly to prevent issues during channel closures. The updates also include tests to validate these functionalities and address a bug related to dust HTLCs not being handled correctly during channel closures.

Changes

Files Change Summary
contractcourt/briefcase.go Added methods for inserting and fetching canceled HTLCs in the ArbitratorLog interface and its implementation.
contractcourt/briefcase_test.go Introduced a test function to validate the functionality of storing and retrieving canceled HTLCs.
contractcourt/channel_arbitrator.go Implemented logic to handle dust HTLCs, including immediate cancellation and improved resolution processes.
contractcourt/channel_arbitrator_test.go Updated mock arbitrator log to track canceled HTLCs and modified tests for dust HTLC resolution.
contractcourt/contract_resolver.go Added functions in ResolverConfig for managing canceled HTLCs.
contractcourt/htlc_success_resolver_test.go Introduced placeholder functions for managing canceled HTLCs in tests.
contractcourt/htlc_timeout_resolver.go Enhanced logic to prevent duplicate resolution attempts for canceled HTLCs during the resolution process.
contractcourt/htlc_timeout_resolver_test.go Added placeholder functions for managing canceled HTLCs in tests.
docs/release-notes/release-notes-0.19.0.md Documented a bug fix related to dust HTLC handling.
itest/lnd_multi-hop_test.go Modified logic for dust HTLC handling during multi-hop timeout processes for clarity.

Assessment against linked issues

Objective Addressed Explanation
Fail dust HTLCs upstream before downstream channel closure (7969)
Consider dust HTLCs in deadline computation for close transaction (7969)

Possibly related PRs

  • bumpforceclosefee rpc #8843: Enhancements to HTLC handling, specifically regarding the ability to bump close fees when no HTLCs are present.

Suggested labels

rpc, channel closing, force closes

Poem

🐇 In the land of contracts, where the rabbits play,
Canceled HTLCs now find their way.
Dust no longer lingers, it hops away fast,
With new methods in place, our troubles are past.
So let’s celebrate this code, oh so bright,
For a smoother transaction, all day and night! 🌟


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    -- I pushed a fix in commit <commit_id>, please review it.
    -- Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    -- @coderabbitai generate unit testing code for this file.
    -- @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    -- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    -- @coderabbitai read src/utils.ts and generate unit testing code.
    -- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    -- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@ziggie1984 ziggie1984 added the P0 very high priority issue/PR, blocker on all others label Sep 5, 2024
@ziggie1984 ziggie1984 added this to the v0.19.0 milestone Sep 5, 2024
@ziggie1984 ziggie1984 changed the title Cancel back dust outgoing dust htlcs before commitment is confirmed. Cancel back outgoing dust htlcs before commitment is confirmed. Sep 5, 2024
@ziggie1984 ziggie1984 force-pushed the cancel-back-dust-htlc branch 2 times, most recently from 923c6b0 to fa7a925 Compare September 9, 2024 12:56
@ziggie1984 ziggie1984 marked this pull request as ready for review September 9, 2024 12:56
@ziggie1984
Copy link
Collaborator Author

ziggie1984 commented Sep 10, 2024

Hi reviewers I am not happy about the following in this PR maybe someone has a nice idea how to make it more clean:

So right now when we locally force close the channel we would fail the dust 2 times, meaning that the second time will cause the log error saying the closeCircuit is already gone. This is currently needed because we need to cancel dust even if the force-close is not initiated by us or the force-close is initiated by us but not by LND, broadcasting the force-close via some other means. This would right now cause some annying log entry similar to:

Example:

[ERR] HSWC: Unable to forward resolution msg: unable to find target channel for HTLC fail: channel ID = 443:2:0, HTLC ID = 0

Probably we should make the extra work and remove the outgoing htlc from the commitSet as soon as we cancel the incoming back. Will investigate.

@ziggie1984
Copy link
Collaborator Author

We could query the circuitMap and not attempt the cancelling of the incoming htlc but maybe just failing it and hitting the error is as efficient ?

Copy link
Collaborator

@yyforyongyu yyforyongyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice refactors! My main comment is we could cancel the dust even earlier, once we've decided there are chain actions to be taken here,

if len(chainActions) == 0 && trigger == chainTrigger {
log.Debugf("ChannelArbitrator(%v): no actions for "+
"chain trigger, terminating", c.cfg.ChanPoint)
return StateDefault, closeTx, nil
}

The current design may end up calling the canceling logic twice, as indicated from this state transition diagram,

// StateDefault
// |
// |-> StateDefault: no actions and chain trigger
// |
// |-> StateBroadcastCommit: chain/user trigger
// | |
// | |-> StateCommitmentBroadcasted: chain/user trigger
// | | |
// | | |-> StateCommitmentBroadcasted: chain/user trigger
// | | |
// | | |-> StateContractClosed: local/remote/breach close trigger
// | | | |
// | | | |-> StateWaitingFullResolution: contract resolutions not empty
// | | | | |
// | | | | |-> StateWaitingFullResolution: contract resolutions not empty
// | | | | |
// | | | | |-> StateFullyResolved: contract resolutions empty
// | | | |
// | | | |-> StateFullyResolved: contract resolutions empty
// | | |
// | | |-> StateFullyResolved: coop/breach(legacy) close trigger
// | |
// | |-> StateContractClosed: local/remote/breach close trigger
// | | |
// | | |-> StateWaitingFullResolution: contract resolutions not empty
// | | | |
// | | | |-> StateWaitingFullResolution: contract resolutions not empty
// | | | |
// | | | |-> StateFullyResolved: contract resolutions empty
// | | |
// | | |-> StateFullyResolved: contract resolutions empty
// | |
// | |-> StateFullyResolved: coop/breach(legacy) close trigger
// |
// |-> StateContractClosed: local/remote/breach close trigger
// | |
// | |-> StateWaitingFullResolution: contract resolutions not empty
// | | |
// | | |-> StateWaitingFullResolution: contract resolutions not empty
// | | |
// | | |-> StateFullyResolved: contract resolutions empty
// | |
// | |-> StateFullyResolved: contract resolutions empty
// |
// |-> StateFullyResolved: coop/breach(legacy) close trigger.

contractcourt/channel_arbitrator.go Show resolved Hide resolved
contractcourt/channel_arbitrator.go Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Show resolved Hide resolved
@ziggie1984 ziggie1984 force-pushed the cancel-back-dust-htlc branch 7 times, most recently from 12fb4ee to 67cd269 Compare September 19, 2024 10:34
@ziggie1984
Copy link
Collaborator Author

ziggie1984 commented Sep 19, 2024

Updated the commit-structure and also the code design quite a bit. Needed to persist the canceled htlcs in the arbLog to make sure we do not fail htlcs twice.

This PR is also created in a way so we can now add the canceling of non-dust htlcs as well in a followup PR.

I think might still be missing some unit-tests, need to dig into all the tests in the contractcourt, but lets see whether this new approach is the way we wanna go before adding more tests.

@ziggie1984
Copy link
Collaborator Author

@coderabbitai review

Copy link
Contributor

coderabbitai bot commented Sep 19, 2024

Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@ziggie1984 ziggie1984 added size/kilo medium, proper context needed, less than 1000 lines HTLC force closes labels Sep 19, 2024
@ziggie1984
Copy link
Collaborator Author

My main question is whether we should go for the db change in this PR. If canceling dust HTLCs behaves like NOOP after they've been canceled, I think we can simplify this PR by moving the db-related commits into a new PR and merge it.

So currently the db recording is just used to avoid ERR loglines which might be misleading. But for the non-dust htlc case we definitely need it. So maybe we accept these log lines and push the db recording when we introduce the non-dust fail-back?

contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
@ziggie1984 ziggie1984 force-pushed the cancel-back-dust-htlc branch 2 times, most recently from ae6ff2d to b41b357 Compare October 14, 2024 12:24
Copy link
Collaborator Author

@ziggie1984 ziggie1984 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the db recording but rather expect an error in the circuit mapper when a dust htlcs are resolved.

contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/htlc_timeout_resolver.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@yyforyongyu yyforyongyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good - had a small suggestion on how we could avoid duplication in canceling the dust.

contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Outdated Show resolved Hide resolved
contractcourt/channel_arbitrator.go Show resolved Hide resolved
contractcourt/channel_arbitrator.go Show resolved Hide resolved
@ziggie1984 ziggie1984 force-pushed the cancel-back-dust-htlc branch 2 times, most recently from b78429e to 0ed9f19 Compare October 17, 2024 11:43
Refactor the part where we are failing back the incoming htlc
when the channel of the corresponding outgoing htlc is force
closed.
We distinguish between dangling and dust htlcs. This does not
change any logic but only introduces new types to later act on them
differently when we begin to fail dust htlcs earlier in a later
commit.
We will now cancel dust htlcs on the local/remote commits after
we decided to go onchain. This can be done because dust cannot
be enforced onchain and therefore there is no way to also reveil
the preimage onchain.
Now that we cancel dust htlcs prematurely even before the
commitment tx is confirmed we don't consider dust htlcs when
creating the cpfp transaction.
Now outgoing dust-htlcs are canceled back before the commitment
is confirmed onchain.
Copy link
Collaborator

@yyforyongyu yyforyongyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, just have one final question

@@ -682,7 +682,7 @@ func (c *ChannelArbitrator) relaunchResolvers(commitSet *CommitSet,
// chain actions may exclude some information, but we cannot recover it
// for these older nodes at the moment.
var confirmedHTLCs []channeldb.HTLC
if commitSet != nil {
if commitSet != nil && commitSet.ConfCommitKey != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice - curious tho, have you run into a panic before?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No but I ran into the one in constructChainActions which was triggered by TestChannelArbitratorPersistence which writes an empty set to disc which will cause commitSet.ConfCommitKey to be nil

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this one of those situations where we are using pointers to simulate optional behavior or is this a pointer because of memory profiling characteristics?

// no confCommitSet. However when the remote commitment confirms
// without us ever broadcasting our local commitment we need to
// make sure we cancel all outgoing dust HTLCs as well.
if confCommitSet == nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: will there be a case when we does local force close but this is not nil?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, this is only needed in case the remote confirms so we need to look up the remote HTLC set.

contractcourt/channel_arbitrator.go Show resolved Hide resolved
Copy link
Collaborator Author

@ziggie1984 ziggie1984 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for the review 🙏

// no confCommitSet. However when the remote commitment confirms
// without us ever broadcasting our local commitment we need to
// make sure we cancel all outgoing dust HTLCs as well.
if confCommitSet == nil {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, this is only needed in case the remote confirms so we need to look up the remote HTLC set.

Copy link
Collaborator Author

@ziggie1984 ziggie1984 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some style nits to address, waiting for keags review to push them

HtlcFailNowAction = 3
// HtlcFailDustAction indicates that we should fail an outgoing dust
// HTLC immediately (even before the commitment transaction is
// confirmed in case we (local) broadcast the commitment tx) by
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style nit: in case we broadcast the local commitment tx

return requests, nil
}

// resolveBreachedHTLCs resolves all HTLCs that are breached transaction was
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style nit: when a breached transaction was detected.

}

// cancelIncomingHTLCs cancels back the incoming HTLCs for the corresponding
// outgoing HTLC. We use a set here to avoid sending duplicate failure messages
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style nit: HTLCs

Copy link
Collaborator

@ProofOfKeags ProofOfKeags left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think I understand the approach but my main first-round feedback is that we need to tighten up terminology. I wish this wasn't the case but it impedes my ability to understand the details of the rest of the PR and makes it pretty difficult for me to tell whether or not the code is doing the right thing.

Another thing to note is that I'm somewhat less experienced with the contractcourt package and so I'm less able to bring expertise into the review and give charitable interpretations to the things I encounter.

Overall I think the bias towards extracting out common functionality into helper functions is very good and I think I understand the high level approach here. Nothing stands out to me as obviously wrong but I can't comment very well on whether this change is complete with respect to doing all of the things it needs to to safely accomplish its goal.

Hopefully future review rounds will help both in tightening the implementation as well as me taking some time out of band to better familiarize myself with the overall structure.

@@ -682,7 +682,7 @@ func (c *ChannelArbitrator) relaunchResolvers(commitSet *CommitSet,
// chain actions may exclude some information, but we cannot recover it
// for these older nodes at the moment.
var confirmedHTLCs []channeldb.HTLC
if commitSet != nil {
if commitSet != nil && commitSet.ConfCommitKey != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this one of those situations where we are using pointers to simulate optional behavior or is this a pointer because of memory profiling characteristics?

Comment on lines +3194 to +3197
// Send the msg to the switch.
if len(msgsToSend) == 0 {
return nil
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the comment here is wrong. However, this seems unnecessary in the first place, right? Even if it has a zero length you can still send that to the DeliverResolutionMsg no?

Comment on lines +3084 to +3088
// resolveBreachedHTLCs resolves all HTLCs that are breached transaction was
// detected. Resovling here means that it fails back the corresponding incoming
// HTLCs for a given outgoing HTLC on the current remote commitment set
// (including the remote pending commitment set).
func (c *ChannelArbitrator) resolveBreachedHTLCs(commitSet CommitSet) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this isn't doing resolution beyond the incoming cancellations.

Comment on lines +3117 to +3122
// resolveHTLCsNow resolves all HTLCs which can be acted on immediately after
// the commitment is confirmed. This includes HTLCs on the remote pending
// commitment whose corresponding incoming HTLCs can now be failed back and
// incoming dust HTLCs which can be marked as failed because they are not part
// of the commitment transaction.
func (c *ChannelArbitrator) resolveHTLCsNow(htlcActions ChainActionMap) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think we need to come up with better names for these functions. The main nudge I want here is, why do we have the word "Now" in here? Is there a corresponding "Later". When the function completes are all of these HTLCs fully resolved? It also seems like this only resolves two of seven (or six if we exclude "no action") actions in the map so it doesn't seem to operate over the entire map. I think It may be a good idea to break this function up on a per action basis and clearly describe what that resolution is.

@@ -1632,7 +1632,7 @@ out:
defer s.wg.Done()

if err := s.FlushForwardingEvents(); err != nil {
log.Errorf("unable to flush "+
log.Errorf("Unable to flush "+
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't capitalize error messages. Idk why but this change is unnecessary.

Comment on lines +988 to +994
dustHTLCSet := fn.SliceToMap(dustHTLCs,
func(htlc channeldb.HTLC) uint64 {
return htlc.HtlcIndex
},
func(htlc channeldb.HTLC) struct{} {
return struct{}{}
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't do this. Just do

getIdx := func(htlc channeldb.HTLC) uint64 {
        return htlc.HtlcIndex
}
dustHTLCSet := fn.NewSet(fn.Map(getIdx, dustHTLCs)...)

Comment on lines +934 to +950
// We expect an immediate resolution message for the outgoing dust htlc.
// It is not resolvable on-chain and it can be canceled back even before
// the commitment transaction confirmed.
select {
case msgs := <-chanArbCtx.resolutions:
if len(msgs) != 1 {
t.Fatalf("expected 1 message, instead got %v",
len(msgs))
}

if msgs[0].HtlcIndex != outgoingDustHtlc.HtlcIndex {
t.Fatalf("wrong htlc index: expected %v, got %v",
outgoingDustHtlc.HtlcIndex, msgs[0].HtlcIndex)
}
case <-time.After(defaultTimeout):
t.Fatalf("resolution msgs not sent")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🫡

Comment on lines +3061 to +3062
// createSweepRequest creates an anchor sweeping request for a particular
// version (local/remote/remote dangling) of the commitment.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another terminology nitpick. What is a remote dangling anchor? Are we referring to an anchor sweep that includes remote dangling HTLCs? Are we referring to an anchor sweep on the pending remote commitment? I think this PR could use a comprehensive review of the terminology used and make sure that we are very consistent with it since there are so many delicate cases that need to be handled correctly.

chanArbCtx, err := createTestChannelArbitrator(t, log)
require.NoError(t, err, "unable to create ChannelArbitrator")

// Attack a mock PreimageDB and Registry to channel arbitrator.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attach?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
force closes HTLC size/kilo medium, proper context needed, less than 1000 lines
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

[bug]: dust HTLC is not failed upstream before downstream channel close is confirmed on-chain
6 participants