Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show the spending transaction of spent outputs #356

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

Pantamis
Copy link
Contributor

@Pantamis Pantamis commented Jul 30, 2021

Solve #270

Requirements: Electrum server supporting Electrum 1.5 method blockchain.outpoint.subscribe

To activate this option you must set BTCEXP_ELECTRUM_TXINDEX = true in .env file as if you want to query ElectRS txindexing.

ElectRS 0.9.0 will support this method thanks to romanz/electrs/pull/446
This means this PR should not be merged until spesmilo/electrumx/pull/90 is merged

Screenshots time !

A transaction with many outputs to "bench" loading speed : 7cdaf30c42eac0271108fe0aa170a02a31f729fb10079156f06611f0489f8f67

BTC-RPC-Explorer needs to make one blockchain.outpoint.subscribe call for each spent transaction output (no call for UTXOs). There are around 250 spent outputs in this transaction at the time I wrote this PR, I load it in less than 40 secondes on my RPi (blockchain stored on HDD).

btc-rpc-explorer-mega-tx

A random transaction with outputs in the mempool at the time I wrote this PR (for easy copy-pasting txid = 6f949c749d890a250a40c29ca8c97576d47eca7f113980f65a0c62fb330516a1 )

btc-rpc-explorer-mempool-tx

The txid of spending transaction is clickable link and redirect to the explorer page of the sending transaction.

Electrum 1.5 specifications say that blockchain.outpoint.subscribe must subscribe the client to notification of changing state of the outpoint (spend, unspend, maybe reorg). We only need the output of this method so I unsubscribe the explorer immediatly after collecting it to avoid spamming the electrum server of subscriptions. ElectRS doesn't implement subscriptions for now so the unsubscribe method will fail, but this can be ignored.

@Pantamis
Copy link
Contributor Author

Pantamis commented Aug 9, 2021

Rebased !

@Pantamis
Copy link
Contributor Author

0499abe add batch JSON request to electrum server to collect all spending transactions of all outpoints in one Electrum call.

I didn't use requestBatch defined in electrum-client lib

https://github.com/janoside/electrum-client/blob/d98e929028d2b05c88a41fe37652737912bead79/lib/client.js#L123-L144

My problem is that the second parameter must be the same for all batched JSON and I can't pass an outpoint as first argument into it because param is in a supplementary array. So I ended up rewriting the batched request in the electrum api, could be worth it to rather make this change in janoside\electrum-client then use requestBatch for batch json rpc here.

Copy link
Contributor

@Kixunil Kixunil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a JS dev so I only did a very light review.

contents.push(request);
arguments_far_calls[id] = outpoint;
}
const content = '[' + contents.join(',') + ']';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you use string manipulation instead of native Json encoding?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The honest answer is that this is how it was made fore requestBatch in electrum-client lib x')

The write method for net.socket must take a string https://nodejs.org/api/net.html#net_socket_write_data_encoding_callback

@Pantamis
Copy link
Contributor Author

Pantamis commented Aug 21, 2021

I notice that there exist transactions that have so many outputs that ElectRS takes too much time to answer so it may be worth adding some limitation on the number of outputs before showing the spending transaction of all outpoints.

An example of such heavy transactions: d22c53dd5a79fb6cc7a10ed6531761ae43ce1ce2ecd1ebbc5746ba82c71e6d70

@Kixunil
Copy link
Contributor

Kixunil commented Aug 22, 2021

No wonder, that transaction got blockstream.info (which uses even more efficient implementation) stuck too.

Could it be loaded asynchronously? Show the page, load outputs chunk-by-chunk (20 at a time or something) and show them as the data arrive. But ultimately I'd like it to be profiled and try to find if there's a reasonable optimization for electrs.

@shesek
Copy link
Contributor

shesek commented Sep 3, 2021

No wonder, that transaction got blockstream.info (which uses even more efficient implementation) stuck too.

Seems to work for me: https://blockstream.info/tx/d22c53dd5a79fb6cc7a10ed6531761ae43ce1ce2ecd1ebbc5746ba82c71e6d70

@Kixunil
Copy link
Contributor

Kixunil commented Sep 6, 2021

Ah, works for me too now. Was probably a fluke.

@Pantamis Pantamis force-pushed the spending_tx branch 2 times, most recently from c0a2f28 to 0d21a9f Compare October 2, 2021 01:25
link to outpoint page if too many

Outpoint status in others tx display
@Pantamis
Copy link
Contributor Author

Pantamis commented Oct 2, 2021

The last commit adds a limit of 200 to the number of outpoints of a transaction for which we fetch the spending transactions to show them along with the numbers of blocks between the transaction and the spending of an outpoint. This is to prevent the Electrum server from DDoS because of heavy transactions with many outputs (such transactions can generally be found as two hops spending transactions of coinbase outputs).

When there are too may outpoints, the explorer will just provide a link for each spent outpoint to a new page on the outpoint. This page will ask to the electrum server for the spending transaction and return its paget if it exists with a message telling the user which outpoint he is looking at and how many block between its creation and spending. If the spending transaction doesn't exist, it will return the funding transaction page with a message to inform the outpoint is not spent.

When looking at a transaction in the explorer from a panel not in a transaction page (from mempool section or blocks content or address history), we also provide a link to the outpoint page of each outpoint of each transactions shown. However, we cannot tell if an outpoint is spent in advance or not so the link presence doesn't garantee the outpoint is spent which is why it reads "Outpoint Status" (spent or unspent).

This reduces the need for spesmilo/electrumx#90 (comment) @Kixunil

Things are a bit mess up when the Electrum server doesn't support Electrum 1.5 (the explorer still display a link to outpoint statuseven if not supported...). I didn't find a good fix for now but it should be straightforward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants