Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MessageWritter gone wrong will using special character likes é,à,â,ô... #360

Closed
tfalkenberg2903 opened this issue Jan 18, 2024 · 6 comments · Fixed by #361
Closed

MessageWritter gone wrong will using special character likes é,à,â,ô... #360

tfalkenberg2903 opened this issue Jan 18, 2024 · 6 comments · Fixed by #361
Labels
bug Something isn't working

Comments

@tfalkenberg2903
Copy link

tfalkenberg2903 commented Jan 18, 2024

Description
While i'll try to serialize my object using new MessageWritter().writeMessage(myRequest), myRequest containing an object with a key name:string of value 'béta' the serialization modify my string to 'bét', it remove one character per special character .
example :
'béta' ==> 'bét'
'bééta' ==> 'bé'
'béééta' ==>'b'

  • Version:2.0.2
  • Platform: macOs

Steps To Reproduce

call the function MessageWriter().writeMessage({name:'béta'}) store it in a const
then use MessageReader().readMessage(myConst) you'll have a string containing 'bét'

here is a sample to reproduce the issue :

const writer = new MessageWriter(parse("string name"));
const reader = new MessageReader(parse("string name"));
const test = writer.writeMessage({ name: "béta" });
console.log("=== result === ", reader.readMessage(test));

this sample will print : Record {name: 'bét'}

Expected Behavior

@tfalkenberg2903 tfalkenberg2903 added the bug Something isn't working label Jan 18, 2024
@foxhubber
Copy link

foxhubber bot commented Jan 18, 2024

Internal tracking ticket: FG-6352

@jtbandes
Copy link
Member

Could you please provide a complete sample program that reproduces the issue?

@tfalkenberg2903
Copy link
Author

tfalkenberg2903 commented Jan 19, 2024 via email

@jtbandes
Copy link
Member

This should fix it, please have a look: #361

@tfalkenberg2903
Copy link
Author

Yes this fix it. Thanks a lot.

jtbandes added a commit that referenced this issue Jan 22, 2024
### Public-Facing Changes

Fixed a bug where non-ASCII strings would be truncated when `MessageWriter` encoded them as UTF-8.

### Description

Fixes #360

`str.length` is the length of the string in UTF-16. However, we use TextEncoder to encode strings as UTF-8. We could compute the length using TextEncoder itself, but this proves to be much slower than a manual calculation: https://jsbench.me/nzlrkwmeiq/1
@jtbandes
Copy link
Member

Fixed in v2.0.3. Thanks for the report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

Successfully merging a pull request may close this issue.

2 participants