v0.19.1-satellite0.1: Bitcoin Satellite v0.1 (on Bitcoin Core 0.19.1)

[UDP Multicast Transport]

• Support multicast-addressed UDP transport of forward error correction (FEC)
  chunks containing data from new blocks, old (repeated) blocks, and mempool
  transactions (txns).
• Add two command-line options: udpmulticast and udpmulticasttx. The former
  handles reception of FEC chunks sent over multicast-addressed UDP datagrams.
  The latter activates multicast-addressed transmission of UDP messages.
• Support multiple instances of udpmulticast and udpmulticasttx bound to
  independent network interfaces.
• UDP Multicast Rx:
  • Support configuration of multicast Rx stream as a trusted UDP peer on
    Fibre's protocol.
  • Set the Rx socket buffer size to avoid buffer overflows.
  • Add optional debugging of the number of FEC chunks that are received,
    versus the number of chunks effectively used for FEC-decoding of each
    block. Track chunks received per multicast socket (per instance of option
    udpmulticast).
  • Add RPC getchunkstats for fetching of statistics regarding the partial
    blocks received over UDP.
• UDP Multicast Tx:
  • Support transmission of new blocks, old blocks and mempoll txns,
    time-multiplexed over the same UDP stream. Use separate threads for
    each of them.
  • For old blocks, iterates over a configurable number of past blocks, which
    is named as the "backfill depth". When the depth is set to 0, iterates
    over the full block chain.
  • Support a configurable number of mempool txns transmitted per second.
  • Support independent logical streams with different configurations (such
    as depth and DSCP of IP packets) when multiple udpmulticasttx options are
    set for the same network interface.
  • Rely on blocking socket transmissions, such that throttling of tx streams
    can be handled externally.
  • Support block interleaving on transmission of old (repeated)
    blocks. Instead of transmitting one block at a time, send a window of
    consecutive old blocks in parallel, while interleaving their FEC
    chunks. The goal is to minimize the chances of losing many chunks from the
    same block due to error bursts. Make the interleaving configurable in
    terms of the number of blocks that are sent in parallel.
• Full chain sync:
  • Force processing of blocks (even the initial blocks of the chain) received
    from trusted UDP multicast peers to allow full-chain synchronization.
  • Force the storage and acceptation/processing of out-of-order blocks that
    are far into the future with respect to the current chain tip, as long as
    these were received from trusted UDP peers. The goal is to make full-chain
    sync more efficient, as most likely the user will operate on out-of-order
    blocks until the Tx node wraps around to the beginning of the chain.
  • Send the height of each block as complimentary info on the block header
    structure sent over UDP multicast. This is meant to support storage of
    out-of-order blocks (OOOBs) specifically for pre-BIP34 blocks.

[Compression]

• Implement more efficient data representation (compression) of txns. Send
  compressed txns by default over the multicast UDP transport, both on
  FEC-coded blocks and on isolated mempool txn messages.
• Add support for multiple compressor/decompressor (codec) versions on
  txns. Send the codec version within the UDP block header information.

[Forward Error Correction]

• Review the FEC strategy with respect to the one adopted in Bitcoin Fibre.
  • Stop sending "uncoded" FEC chunks (the systematic part of the FEC-coded
    object) for blocks and rely only on "coded" chunks (the non-systematic
    part). Do so both for new blocks as well as old (repeated) blocks. The
    goal is to always support the combination of different UDP multicast
    streams, which is only efficient with coded chunks and with different
    (random) chunk ids from separate streams. Also, for new blocks, prefer
    "coded" chunks as the receive-end can use them to effectively fill in the
    erasures (unlike uncoded/systematic chunks).
  • For txns, always send uncoded chunks, so as to eliminate the inefficient
    repetition coding.
  • Stop sending the "body" of the FEC-coded block for blocks that contain
    only the coinbase and no other txn. For these, send only the header of the
    FEC-coded block, as it contain all the necessary information, including
    the coinbase txn as prefilled txn (c.f. BIP 152).
• Support non-padded chunks for txns., i.e. variable packet lengths for
  UDP multicast messages containing mempool txns, in contrast to block
  packets, which are of constant size. Most txns are smaller than the default
  FEC chunk size. Hence, by removing the padding of txns, there is significant
  saving of bytes sent over the multicast stream.
• Review the FEC overhead configurations for blocks. Assign a fixed overhead
  (currently of 60 chunks) plus a variable overhead based on the block size,
  which is currently set to 5% of the original number of chunks from the
  uncoded block.

[Misc - Fixes and Optimizations]

• Change buffer used for storing partial blocks to be processed. Use
  FIFO (queue) instead of LIFO.
• Support reception of FEC-coded block body before the reception of the
  block header.
• Add separate priority queue for mempool txns, which were previously
  classified with the same transmit priority of old (repeated) blocks. Now,
  prioritize mempool txns over old blocks, as the mempool txns are helpful to
  speed up decoding of new (chain tip) blocks on the receive end.
• Fix repeated transmission of parent mempool txns on the mempool txn
  transmission loop.
• Defer the processing of the partial block header of old (repeated) blocks to
  when both the header and the body are ready. This avoid loading too much
  data in memory when there is a long time between completion of the header
  and the body (such as when the blocks are interleaved).