Widechar bug

[ingwarsw]

There is a known and easily triggered crashbug: The backspace, delete, and arrow keys move by bytes, not by characters. Thusly, do not try to edit an input line containing extended multibyte characters.

[warmstarter]

Two crashed should should be fixed in the pull request I sent in. Getting the arrows perfect would right require a significant write of how things are handled. Sending your backspace to send ^H does make somethings better in that regard.

Basically the client expects characters to be 8 bits, but unicode character are often larger. So some parts of the client treats it as one character and others as more. Display handles one character deceltly, but arrows and not ^H backspace don't what will often happen is your movement will get you in the middle of a character and a backspace will because deleete part of a character making it mal-formed. One of the two bug fixes basically takes any characters that are not supposed to be and converts them to XML, to preent this issue.

On one server, this had the tendency to get a character that was I think the range of 190-240, which when combined with another character would trigger a crash. It's tf's fault for generating those, but there's no need for the server to send out that ange, so they're fixing it also. Not sure which servers might be having poblems with TF. TF definiteily needs fixes there, but it's also a good idea for the servers to do some validation on one they did.

I found with it (and since patched) a nasty utf8 bug that interacted with a BSD network bug that would transform the characters in vvarious ways. You could basically get the client and server to start mathematically transsforming the characters, and if you can do that, besides crashes hthere are changes to create exploits trough arbitrary math.