Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial mempool sync may take a lot of time #968

Open
romanz opened this issue Dec 6, 2023 · 4 comments · May be fixed by #970
Open

Initial mempool sync may take a lot of time #968

romanz opened this issue Dec 6, 2023 · 4 comments · May be fixed by #970
Assignees
@romanz
Labels
mempool performance

Comments

@romanz
Copy link
Owner

romanz commented Dec 6, 2023 •
edited
Loading

All the above is strongly suggesting to me that the rpc.sync() call on this line is blocking for too long, probably because the mempool.sync() call on this line is blocking. This results in JSON-RPC client calls not being handled because the electrs code relies on syncing being paused before handling incoming client requests.

@conduition Good catch, thanks!
It seems that the initial mempool sync takes quite a while... I'll open a new issue.

Originally posted by @romanz in #961 (comment)
@romanz romanz self-assigned this Dec 6, 2023
@conduition
Copy link
Contributor

conduition commented Dec 6, 2023 •
edited
Loading

I'd be happy to take a look at fixing this.

@romanz could you please clarify, is blocking on mempool sync is a desired behavior? My suggestion would be to block until the initial blockchain and mempool synchronization is complete, but then after that, handle incoming client requests asynchronously.

Alternatively we could block until the initial blockchain sync is done, but then do mempool scanning asynchronously so that it doesn't interfere with processing of client requests.

@Kixunil
Copy link
Contributor

Kixunil commented Dec 7, 2023

I think that doing chain first and the asynchronously mempool is the best option. But still mempool sync needs to be optimized - otherwise the clients won't see the transactions for a long time. It's OK to have a bit of lag since the network behaves this way already anyway, just not crazy long times.

@conduition conduition linked a pull request Dec 8, 2023 that will close this issue
Open
@conduition
Copy link
Contributor

PR open to provide mempool scanning in a separate thread: #970

@romanz
Copy link
Owner Author

romanz commented Dec 22, 2023

#979 should also help with this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@romanz romanz

Labels
mempool performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Async mempool scanning conduition/electrs
3 participants
@romanz @Kixunil @conduition