gossipd logs causing slow htlcs? #7815
Comments
|
This is quite interesting indeed. Correct me if I'm wrong @rustyrussell but I think we do.prioritize channel related traffic already, or was that only in connectd for outgoing traffic? Notice that this could also be the old lnd head-of-line blocking, where they queued all the gossip at once and where not prioritizing channel related traffic. Or you have some incredibly noisy peers and we should stop processing gossip from extremely chatty nodes. |
|
Noting the peer in question is a cln greenlight node, and these logs start when the htlc comes in and end when the htlc is passed over to a plugin in the |
|
Also noting this node did not have #7767 deployed. Could that fix the issue? |
|
Definitely a possibility since we're talking about message queues filling and head-of-line-blocking. I wonder if we should add simple in-memory counters for the |
|
We've upgraded to include #7767 yesterday. Looking at the |
I noticed during the handling of a htlc, there's lines like
gossipd: Log pruned 71408 entries (mem 10485792 -> 3393591)with a 10 second gap between them. It appears the logs produced by gossipd are blocking the execution of 'normal operation' like handling htlcs. In below example 20 seconds were added to the processing time of a single htlc, clearly surrounding the moments the log pruning is shown in the logs. Not the--> x second gap herelines I added in below log to make clear where it's adding time.The text was updated successfully, but these errors were encountered: