wallet: add GetTransaction returning data for any transaction #779

bjarnemagnussen · 2021-11-19T08:47:59Z

This PR adds a method to the wallet that allows to get transaction data on any transaction given its ID. If the transaction exists in the transaction store, it will be returned with the data from the database, otherwise data from the backend is returned as a TransactionSummary.

This method can be useful in situations where one is interested in a specific transaction only, instead of e.g. all transactions from the GetTransactions method. It also adds new functionality as it can return data from the backend of a transaction not in the internal wallet.

guggero

Thanks for the added feature.

I wonder though how useful this is, if you need to rely on the txindex to be enabled for this to work? The above GetTransactions works without index, is just much slower (for obvious reasons). Would that be used as a fall back?

cc @Roasbeef

guggero · 2021-11-22T12:04:06Z

wallet/wallet.go

+	bool, error) {
+
+	var res GetTransactionResult
+	chainClient := w.chainClient


Need to hold the w.chainClientLock mutex here.

guggero · 2021-11-22T12:05:58Z

wallet/wallet.go

+			j := len(blockHashBytes) - i - 1
+			reversed[j] = n
+		}
+		blockHash, err = chainhash.NewHash(reversed)


Can use chainhash.NewHashFromStr directly instead of reversing it yourself.

Thanks! I cannot believe I overlooked this method...

guggero · 2021-11-22T12:06:56Z

wallet/wallet.go

+			if err != nil {
+				return nil, false, err
+			}
+		case *chain.NeutrinoClient:


No need to include the Neutrino client here, we bailed out earlier for that chain client type.

guggero · 2021-11-22T12:09:06Z

wallet/wallet.go

+			return false, nil
+		}
+
+		return w.TxStore.RangeTransactions(txmgrNs, bestHeight,


nit: multi-line method call formatting.

guggero · 2021-11-22T12:10:47Z

wallet/wallet.go

+// GetTransaction returns data of any transaction given its id. A mined
+// transaction is returned in a Block structure which records properties about
+// the block. A bool is also returned to denote if the transaction previously
+// existed in the database or not.


Mention that the transaction index needs to be turned on for this to work?

bjarnemagnussen · 2021-11-24T13:00:42Z

@guggero Thanks a lot for taking your time to give me the feedback! I have added a commit with changes.

The idea with this function is to a) provide a more performant lookup of a single relevant transaction given its id, but also b) as a convenience function if the transaction index is set on the backend to allow polling for an arbitrary transaction.

Regarding the transaction index I have made a little code change to continue fetching a transaction from the database, even if the transaction index is not set on the backend. Only if the transaction cannot be found in the database and the backend does not have the transaction index will the function call fail.

guggero

Can you please squash the commits?
Other than a few minor nits, this looks good to me 🎉

guggero · 2021-11-29T14:02:58Z

wallet/wallet.go

+	// In case the transaction is unconfirmed it could due to race condition
+	// confirm during the time we receive it from the backend and check it
+	// in the database.
+	// We therefore store the best block height and will use it for quering the


nit: s/quering/querying (and also line length)

guggero · 2021-11-29T14:05:54Z

wallet/wallet.go

+		return nil, false, err
+	}
+
+	var res GetTransactionResult


nit: move to just above where this is actually used? So before the switch?

guggero · 2021-11-29T14:12:17Z

wallet/wallet.go

+			// TODO: probably should make RangeTransactions not
+			// reuse the details backing array memory.
+			dets := make([]wtxmgr.TxDetails, len(details))
+			copy(dets, details)
+			details = dets
+
+			for i := range details {
+				dbTx := makeTxSummary(dbtx, w, &details[i])


Hmm, could this just be:

for i, txDetail := range details { txDetail := txDetail dbTx := makeTxSummary(dbtx, w, &txDetail)

to avoid copying the whole slice?

I was a bit confused about this TODO comment. Note though, that I used the code from the GetTransactions method above here. But I have made your code suggestion in the squashed commit.

bjarnemagnussen · 2021-11-30T07:28:04Z

Thanks @guggero! I have incorporated your suggestions in the squashed commit.

Roasbeef · 2021-12-01T23:40:36Z

wallet/wallet.go

+// existed in the database or not.
+// The backend must have a transaction index to poll a confirmed transaction
+// that does not exist in the internal wallet.
+func (w *Wallet) GetTransaction(txHash *chainhash.Hash) (*GetTransactionResult,


Unclear w.r.t the purpose of this method: it seems to rely on the RPC calls, and will return a reply even if the wallet isn't found in the wallet.

Where would this be used that https://pkg.go.dev/github.com/btcsuite/btcwallet/wtxmgr#Store.TxDetails wouldn't be?

This could be useful for applications that use btcwallet as dependency. E.g. in LND a remotely force closed transaction is not added to the wallet/store and therefore won't show up via GetTransactions. It could therefore be convenient to have a function that gets the transaction from the wallet/store if it exists and falls back to the backend otherwise.

wallet/wallet.go

guggero · 2023-12-06T13:45:20Z

This can now rebased after #890 is in. The only remaining diff would be that if we don't have the transaction in the wallet that we would go and ask the chain backend for it.

guggero

Thanks for the update. Main comment is about making sure this doesn't panic for Neutrino clients. Not sure we can give them a useful response though, as it would require re-scanning the chain for the transaction. So not sure what to do about that. What would this extended call be used for in your use case in the first place?

guggero · 2024-05-10T08:33:25Z

wallet/wallet.go

+	var bestHeight int32
+	var blockHash *chainhash.Hash
+
+	var summary *TransactionSummary


nit: can unify all these into a single var ( ... ) block.

Updated in updated commit

guggero · 2024-05-10T08:33:47Z

wallet/wallet.go

+		summary = &TransactionSummary{
+			Hash:        &txHash,
+			Transaction: txRaw,
+			MyInputs:    make([]TransactionSummaryInput, 0),


Can just leave those nil instead of creating an empty slice IMO.

updated in latest commit

guggero · 2024-05-10T08:38:54Z

wallet/wallet.go

+			if err != nil {
+				return nil, err
+			}
+		case *chain.BitcoindClient:


What about Neutrino? Those clients would currently panic on the txResult.Hex access below.

I've added this Neutrino check back in, although I originally removed it due to your prior comment!

guggero · 2024-05-10T08:40:11Z

wallet/wallet.go

+
+		// Calculate the number of confirmations.
+		if res.Height != -1 {
+			res.Confirmations = bestHeight - res.Height + 1


I don't think this is correct. bestHeight and res.Height are both the exact same value at this point.

Why would they have the same value? bestHeight is the current sync height, res.Height is the height at which the transaction was found.

See line 2639.

lol. I've updated this to fetch the chain tip and then compare the heights to get the number of confirmations.

mrfelton · 2024-05-15T15:17:32Z

Thanks for the update. Main comment is about making sure this doesn't panic for Neutrino clients. Not sure we can give them a useful response though, as it would require re-scanning the chain for the transaction. So not sure what to do about that. What would this extended call be used for in your use case in the first place?

The use case is as @bjarnemagnussen mentioned in #779 (comment). but more specifically, to enhance LND's GetTransaction RPC to be able to fetch transactions that are not known to LND (this is a customisation we have had applied to lnd for a while)

Roasbeef

but more specifically, to enhance LND's GetTransaction RPC to be able to fetch transactions that are not known to LND

Why not just use getrawtransaction then? GetTransaction (an API method of the wallet) is intended only to fetch transactions that are relevant to the wallet.

This PR takes that API method and adds in additional network calls (and for neutrino new p2p messages being sent), which significantly expands the scope of the method.

mrfelton · 2024-07-03T05:59:43Z

That's certainly another valid approach, and is actually more akin to what we are doing now (we are no longer activly using this enhancement).

GetTransaction (an API method of the wallet) is intended only to fetch transactions that are relevant to the wallet.

IIRC, the transactions we were interested in were at one point in some way relevant to the wallet (e.g. a remote force close transaction, or transactions related to RBF, abandoned transactions from failed channel opens) Part of the rationale for making this accessible through LND rather than calling through to Bitcoind directly was for simplicity. LND generally provides everything needed to operate on top of it, and there are multiple instances where LND proxies through to Bitcoind to make something from the underlaying available. We did not want to open up direct connections to our Bitcoind instances and complicate things with needing to manage the multiple connections.

guggero reviewed Nov 22, 2021

View reviewed changes

lifesgoodallthetime approved these changes Nov 24, 2021

View reviewed changes

guggero approved these changes Nov 29, 2021

View reviewed changes

bjarnemagnussen force-pushed the upstream/feat/add-gettransaction branch from 0eb3f9c to 6d8e1bc Compare November 30, 2021 07:25

Roasbeef requested changes Dec 1, 2021

View reviewed changes

bjarnemagnussen force-pushed the upstream/feat/add-gettransaction branch from 6d8e1bc to f6b5524 Compare December 2, 2021 10:39

ErikEk mentioned this pull request Jan 17, 2023

[feature]: add get transaction rpc lightningnetwork/lnd#7323

Closed

guggero mentioned this pull request Nov 28, 2023

neutrino remove sweeptx lightningnetwork/lnd#7800

Merged

7 tasks

mrfelton force-pushed the upstream/feat/add-gettransaction branch 4 times, most recently from bb1f86b to 55913d0 Compare May 9, 2024 15:50

guggero self-requested a review May 10, 2024 07:25

guggero reviewed May 10, 2024

View reviewed changes

mrfelton force-pushed the upstream/feat/add-gettransaction branch from 55913d0 to d7cb37f Compare May 15, 2024 15:11

mrfelton force-pushed the upstream/feat/add-gettransaction branch 2 times, most recently from 8f64458 to 0ad5f81 Compare May 15, 2024 15:46

wallet: add GetTransaction returning data for any transaction

9e52443

mrfelton force-pushed the upstream/feat/add-gettransaction branch from 0ad5f81 to 9e52443 Compare May 15, 2024 15:46

Roasbeef requested changes Jul 3, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wallet: add GetTransaction returning data for any transaction #779

wallet: add GetTransaction returning data for any transaction #779

bjarnemagnussen commented Nov 19, 2021

guggero left a comment

guggero Nov 22, 2021

guggero Nov 22, 2021

bjarnemagnussen Nov 24, 2021

guggero Nov 22, 2021

guggero Nov 22, 2021

guggero Nov 22, 2021

bjarnemagnussen commented Nov 24, 2021

guggero left a comment

guggero Nov 29, 2021

guggero Nov 29, 2021

guggero Nov 29, 2021

bjarnemagnussen Nov 30, 2021

bjarnemagnussen commented Nov 30, 2021

Roasbeef Dec 1, 2021

bjarnemagnussen Dec 2, 2021

guggero commented Dec 6, 2023

guggero left a comment

guggero May 10, 2024

mrfelton May 15, 2024

guggero May 10, 2024

mrfelton May 15, 2024

guggero May 10, 2024

mrfelton May 15, 2024 •

edited

Loading

guggero May 10, 2024

mrfelton May 15, 2024

guggero May 15, 2024

mrfelton May 15, 2024 •

edited

Loading

mrfelton commented May 15, 2024 •

edited

Loading

Roasbeef left a comment

mrfelton commented Jul 3, 2024 •

edited

Loading

wallet: add GetTransaction returning data for any transaction #779

Are you sure you want to change the base?

wallet: add GetTransaction returning data for any transaction #779

Conversation

bjarnemagnussen commented Nov 19, 2021

guggero left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjarnemagnussen commented Nov 24, 2021

guggero left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjarnemagnussen commented Nov 30, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guggero commented Dec 6, 2023

guggero left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrfelton May 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrfelton May 15, 2024 • edited Loading

Choose a reason for hiding this comment

mrfelton commented May 15, 2024 • edited Loading

Roasbeef left a comment

Choose a reason for hiding this comment

mrfelton commented Jul 3, 2024 • edited Loading

mrfelton May 15, 2024 •

edited

Loading

mrfelton May 15, 2024 •

edited

Loading

mrfelton commented May 15, 2024 •

edited

Loading

mrfelton commented Jul 3, 2024 •

edited

Loading