OutboundFailure (DialFailure) while trying to start a request-response

[depechec0de]

All the peers in the network are symmetrical, running in kad::Mode::Server mode, and with the following behavior:

struct MyBehaviour {
    transfer_request: request_response::cbor::Behaviour<TransferRequest, TransferResponse>,
    identify: identify::Behaviour,
    kademlia: kad::Behaviour<kad::store::MemoryStore>,
}

I start a node B with a known connection address, A and C dial B and connect successfully.

Then I make A do a swarm.behaviour_mut().transfer_request.send_request(&peer_id, TransferRequest(amount)); to B where A knows the PublicKey / PeerId of C.

This results in Behaviour(QAATBehaviourEvent: OutboundFailure { peer: PeerId("12D3KooWBWmQAVRRNtesp44rdm771o24qfNGzXuvHspE2AHsYtX2"), request_id: OutboundRequestId(1), error: DialFailure }).

In my swarm loop, I am just printing the events that I receive, i.e., I do not call to any function like kad::Behaviour::add_address.
If I add an add_address call when I handle a SwarmEvent::ConnectionEstablished, I see that A and C connect to each other, but that happens after the request-response, so the failure persists.

Questions

• I don't want peers to connect to all other peers in the network by default, just with the ones that need to interact by the request-response protocol. Am I missing something? Why A does not learn C's address and dial successfully?
• Do I need to store the request in a hashmap and wait for the connection to be established? (in the scenario where I manually call kad::Behaviour::add_address upon SwarmEvent::ConnectionEstablished.

Log

I replaced the actual public keys with A, B, and C and highlighted the error.

Peer PeerId("B") Handling SwarmEvent: NewListenAddr { listener_id: ListenerId(1), address: /ip4/127.0.0.1/tcp/2000 }
Bootstrap's ID: PeerId("B")
Peer PeerId("A") Handling SwarmEvent: NewListenAddr { listener_id: ListenerId(2), address: /ip4/127.0.0.1/tcp/3000 }
Peer PeerId("C") Handling SwarmEvent: NewListenAddr { listener_id: ListenerId(3), address: /ip4/127.0.0.1/tcp/4000 }
Peer PeerId("B") Handling SwarmEvent: IncomingConnection { connection_id: ConnectionId(3), local_addr: /ip4/127.0.0.1/tcp/2000, send_back_addr: /ip4/127.0.0.1/tcp/3000 }
Peer PeerId("B") Handling SwarmEvent: IncomingConnection { connection_id: ConnectionId(4), local_addr: /ip4/127.0.0.1/tcp/2000, send_back_addr: /ip4/127.0.0.1/tcp/4000 }
Peer PeerId("A") Handling SwarmEvent: ConnectionEstablished { peer_id: PeerId("B"), connection_id: ConnectionId(1), endpoint: Dialer { address: /ip4/127.0.0.1/tcp/2000, role_override: Dialer, port_use: Reuse }, num_established: 1, concurrent_dial_errors: Some([]), established_in: 10.421655ms }
Peer PeerId("C") Handling SwarmEvent: ConnectionEstablished { peer_id: PeerId("B"), connection_id: ConnectionId(2), endpoint: Dialer { address: /ip4/127.0.0.1/tcp/2000, role_override: Dialer, port_use: Reuse }, num_established: 1, concurrent_dial_errors: Some([]), established_in: 10.431036ms }
=============== ERROR =================
Peer PeerId("A") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: OutboundFailure { peer: PeerId("C"), request_id: OutboundRequestId(1), error: DialFailure })
=============== END ERROR =================
Peer PeerId("B") Handling SwarmEvent: ConnectionEstablished { peer_id: PeerId("A"), connection_id: ConnectionId(3), endpoint: Listener { local_addr: /ip4/127.0.0.1/tcp/2000, send_back_addr: /ip4/127.0.0.1/tcp/3000 }, num_established: 1, concurrent_dial_errors: None, established_in: 42.674244ms }
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: Sent { connection_id: ConnectionId(3), peer_id: PeerId("A") })
Peer PeerId("B") Handling SwarmEvent: ConnectionEstablished { peer_id: PeerId("C"), connection_id: ConnectionId(4), endpoint: Listener { local_addr: /ip4/127.0.0.1/tcp/2000, send_back_addr: /ip4/127.0.0.1/tcp/4000 }, num_established: 1, concurrent_dial_errors: None, established_in: 43.598922ms }
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: Sent { connection_id: ConnectionId(4), peer_id: PeerId("C") })
Peer PeerId("A") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: Sent { connection_id: ConnectionId(1), peer_id: PeerId("B") })
Peer PeerId("A") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: Received { connection_id: ConnectionId(1), peer_id: PeerId("B"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): 141ab92f69734568ba66486034b71aa43a16801fa2b658df2e7d85c4da4536b6) }, protocol_version: "/qaat/id/1.0.0", agent_version: "rust-libp2p/0.45.0", listen_addrs: [/ip4/127.0.0.1/tcp/2000], protocols: ["/qaat/1.0.0", "/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"], observed_addr: /ip4/127.0.0.1/tcp/3000 } })
Peer PeerId("A") Handling SwarmEvent: NewExternalAddrOfPeer { peer_id: PeerId("B"), address: /ip4/127.0.0.1/tcp/2000 }
Peer PeerId("A") Handling SwarmEvent: NewExternalAddrCandidate { address: /ip4/127.0.0.1/tcp/3000 }
Peer PeerId("A") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: RoutingUpdated { peer: PeerId("B"), is_new_peer: true, addresses: [/ip4/127.0.0.1/tcp/2000], bucket_range: (Distance(57896044618658097711785492504343953926634992332820282019728792003956564819968), Distance(115792089237316195423570985008687907853269984665640564039457584007913129639935)), old_peer: None })
Peer PeerId("C") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: Sent { connection_id: ConnectionId(2), peer_id: PeerId("B") })
Peer PeerId("C") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: Received { connection_id: ConnectionId(2), peer_id: PeerId("B"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): 141ab92f69734568ba66486034b71aa43a16801fa2b658df2e7d85c4da4536b6) }, protocol_version: "/qaat/id/1.0.0", agent_version: "rust-libp2p/0.45.0", listen_addrs: [/ip4/127.0.0.1/tcp/2000], protocols: ["/qaat/1.0.0", "/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"], observed_addr: /ip4/127.0.0.1/tcp/4000 } })
Peer PeerId("C") Handling SwarmEvent: NewExternalAddrOfPeer { peer_id: PeerId("B"), address: /ip4/127.0.0.1/tcp/2000 }
Peer PeerId("C") Handling SwarmEvent: NewExternalAddrCandidate { address: /ip4/127.0.0.1/tcp/4000 }
Peer PeerId("C") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: RoutingUpdated { peer: PeerId("B"), is_new_peer: true, addresses: [/ip4/127.0.0.1/tcp/2000], bucket_range: (Distance(14474011154664524427946373126085988481658748083205070504932198000989141204992), Distance(28948022309329048855892746252171976963317496166410141009864396001978282409983)), old_peer: None })
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: Received { connection_id: ConnectionId(3), peer_id: PeerId("A"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): f24c4a99c7de66f82275e672e22c318f633b940a4e7ab9d7f6e89e50bb76) }, protocol_version: "/qaat/id/1.0.0", agent_version: "rust-libp2p/0.45.0", listen_addrs: [/ip4/127.0.0.1/tcp/3000], protocols: ["/ipfs/id/1.0.0", "/qaat/1.0.0", "/ipfs/kad/1.0.0", "/ipfs/id/push/1.0.0"], observed_addr: /ip4/127.0.0.1/tcp/2000 } })
Peer PeerId("B") Handling SwarmEvent: NewExternalAddrOfPeer { peer_id: PeerId("A"), address: /ip4/127.0.0.1/tcp/3000 }
Peer PeerId("B") Handling SwarmEvent: NewExternalAddrCandidate { address: /ip4/127.0.0.1/tcp/2000 }
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: UnroutablePeer { peer: PeerId("A") })
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: Received { connection_id: ConnectionId(4), peer_id: PeerId("C"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): 227c59ec6be17cdaa132a5422e53db047b24f8d86b6455cd558e56281b1418) }, protocol_version: "/qaat/id/1.0.0", agent_version: "rust-libp2p/0.45.0", listen_addrs: [/ip4/127.0.0.1/tcp/4000], protocols: ["/ipfs/kad/1.0.0", "/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/qaat/1.0.0"], observed_addr: /ip4/127.0.0.1/tcp/2000 } })
Peer PeerId("B") Handling SwarmEvent: NewExternalAddrOfPeer { peer_id: PeerId("C"), address: /ip4/127.0.0.1/tcp/4000 }
Peer PeerId("B") Handling SwarmEvent: NewExternalAddrCandidate { address: /ip4/127.0.0.1/tcp/2000 }
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: UnroutablePeer { peer: PeerId("C") })
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: InboundRequest { request: FindNode { num_closer_peers: 0 } })
Peer PeerId("A") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: OutboundQueryProgressed { id: QueryId(0), result: Bootstrap(Ok(BootstrapOk { peer: PeerId("A"), num_remaining: 0 })), stats: QueryStats { requests: 1, success: 1, failure: 0, start: Some(Instant { tv_sec: 8919, tv_nsec: 921054158 }), end: Some(Instant { tv_sec: 8919, tv_nsec: 922861033 }) }, step: ProgressStep { count: 1, last: true } })
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: InboundRequest { request: FindNode { num_closer_peers: 0 } })
Peer PeerId("C") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: OutboundQueryProgressed { id: QueryId(0), result: Bootstrap(Ok(BootstrapOk { peer: PeerId("C"), num_remaining: 2 })), stats: QueryStats { requests: 1, success: 1, failure: 0, start: Some(Instant { tv_sec: 8919, tv_nsec: 922238061 }), end: Some(Instant { tv_sec: 8919, tv_nsec: 923857305 }) }, step: ProgressStep { count: 1, last: false } })
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: InboundRequest { request: FindNode { num_closer_peers: 0 } })
Peer PeerId("C") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: OutboundQueryProgressed { id: QueryId(0), result: Bootstrap(Ok(BootstrapOk { peer: PeerId("1AfLPt9hppgHLRQFRu4ugAwyg6iQeqPJwUE97ngDHbiPo7"), num_remaining: 1 })), stats: QueryStats { requests: 1, success: 1, failure: 0, start: Some(Instant { tv_sec: 8919, tv_nsec: 924182612 }), end: Some(Instant { tv_sec: 8919, tv_nsec: 966274995 }) }, step: ProgressStep { count: 2, last: false } })
Peer PeerId("B") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: InboundRequest { request: FindNode { num_closer_peers: 0 } })
Peer PeerId("C") Handling SwarmEvent: Behaviour(QAATBehaviourEvent: OutboundQueryProgressed { id: QueryId(0), result: Bootstrap(Ok(BootstrapOk { peer: PeerId("1AkhEoatkf2vwa2j5Vhbw3s6bzRrKHbGFLuewcCuWMwSgD"), num_remaining: 0 })), stats: QueryStats { requests: 1, success: 1, failure: 0, start: Some(Instant { tv_sec: 8919, tv_nsec: 966631584 }), end: Some(Instant { tv_sec: 8920, tv_nsec: 8747957 }) }, step: ProgressStep { count: 3, last: true } })

[dariusc93]

The way I usually handle it is by having a behaviour to act as an address book to store peers and their [known] addresses and when dialing it would pull from that behaviour using NetworkBehaviour::handle_pending_outbound_connection if it exist. Additionally, you could connect to the peer prior to sending the request as well by calling Swarm::dial or use ToSwarm::Dial within your behaviour.