bichannel connection - sending large packets doesn't always send all bytes

[frg-kova]

When trying to send large packets, i.e. 50MB, the sending completes without sending all the bytes. I managed to get 100% reproduction rate with Unity headless server (Linux + Mono) running in a docker on google cloud. Not reproducible with Windows or WSL on local machine.

So I've been hunting down this bug where client rarely and suddenly stops receiving messages after a big one is sent. I've finally tracked down the problem in BichannelServerConnection on the server saying the messages have been sent (SendAsync) and OnComplete callback is finished, but in reality client only receives a portion of it. Confirmed with Wireshark that client is getting what he reports that he got. So client is still correctly thinking he didn't receive a complete message and starts filling up buffer for that big message with other messages. The situation is I request 50MB of data (empty byte array) and only 140kB - 4MB arrives, leaving 46MB of buffer to be filled. Of course that breaks the messaging system completely and you can't recover as stream you expected isn't valid anymore.

From server logs:

[Dev] Sending blob of data to Client [0], blob size in bytes: [52428800]
... in OnComplete:
Bytes sent to client, SocketError: [Success] BytesTransferred: [142000] Message Size: [52428811] MessageBuffer capacity: [52428807] BufferList size total: [52428811] SendPacketsSendSize: [-1] SocketFlags: [None]

Clearly Socket.SendAsync(SocketAsyncEventArgs) is completing without any visible errors or exceptions and without all bytes being sent. From googling on the internet to is that expected, I've seen few people saying OnComplete is only fired when it actually completes, so this may be Mono and/or Linux specific thing.
Socket.SendPacketsSendSize is wierdly -1 while documentation says 8192 is default and setting negative numbers will give exception.

I will proceed to call SendAsync on the rest of the data and see how that goes, then make a pull request if that works.

Linked to issue: #112 which is an Android client

[frg-kova]

Found someone 10y ago with the same issue and solution
https://stackoverflow.com/a/20527097/10722326

[frg-kova]

Fixed locally. Cannot repro anymore.
I removed the usage of BufferList since re-assigning those mid send was a horror so reverted back to normal Buffer to just move the offset and count.
As an added bonus, that header allocation is gone as well as Message.ToBuffer() allocates it from the get go.
Will open a PR when I have a chance, there was a lot of other improvements as well.