Skip to content

Commit

Permalink
http-client: fix base64 decoding of journal content
Browse files Browse the repository at this point in the history
The grpc gateway returns journal content as a base64-encoded string.
Previously, we had been using `atob` to decode this, but that function
does not work if the decoded content contains any byte values larger
than 0x7f, because of course we can't have nice things in JS. So this
adds one of those looks-insane-but-actually-works workarounds to make
base64 decoding work properly with arbitrary utf-8 data.
  • Loading branch information
psFried committed Aug 9, 2024
1 parent fa319e4 commit a906ada
Show file tree
Hide file tree
Showing 4 changed files with 44 additions and 4 deletions.
12 changes: 11 additions & 1 deletion client/dist/esm/streams.js
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,17 @@ export function decodeContent() {
transform(value, controller) {
// Base64 decode the `content` field and send it as a chunk.
if (value.content?.length) {
controller.enqueue(atob(value.content));
// The `atob` function does not work properly if the decoded content contains any byte
// values over 0x7f, because "binary" in JS means that each byte gets represented as a
// UTF-16 code unit, which happens to be <= 0xff. I wish I was making this up:
// https://developer.mozilla.org/en-US/docs/Glossary/Base64#the_unicode_problem
const binary = atob(value.content);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < bytes.length; i++) {
bytes[i] = binary.charCodeAt(i);
}
const text = new TextDecoder().decode(bytes);
controller.enqueue(text);
}
},
});
Expand Down
12 changes: 11 additions & 1 deletion client/dist/script/streams.js
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,17 @@ function decodeContent() {
transform(value, controller) {
// Base64 decode the `content` field and send it as a chunk.
if (value.content?.length) {
controller.enqueue(atob(value.content));
// The `atob` function does not work properly if the decoded content contains any byte
// values over 0x7f, because "binary" in JS means that each byte gets represented as a
// UTF-16 code unit, which happens to be <= 0xff. I wish I was making this up:
// https://developer.mozilla.org/en-US/docs/Glossary/Base64#the_unicode_problem
const binary = atob(value.content);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < bytes.length; i++) {
bytes[i] = binary.charCodeAt(i);
}
const text = new TextDecoder().decode(bytes);
controller.enqueue(text);
}
},
});
Expand Down
12 changes: 11 additions & 1 deletion client/dist/src/streams.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,17 @@ export function decodeContent() {
transform(value, controller) {
// Base64 decode the `content` field and send it as a chunk.
if (value.content?.length) {
controller.enqueue(atob(value.content));
// The `atob` function does not work properly if the decoded content contains any byte
// values over 0x7f, because "binary" in JS means that each byte gets represented as a
// UTF-16 code unit, which happens to be <= 0xff. I wish I was making this up:
// https://developer.mozilla.org/en-US/docs/Glossary/Base64#the_unicode_problem
const binary = atob(value.content);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < bytes.length; i++) {
bytes[i] = binary.charCodeAt(i);
}
const text = new TextDecoder().decode(bytes);
controller.enqueue(text);
}
},
});
Expand Down
12 changes: 11 additions & 1 deletion client/src/streams.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,17 @@ export function decodeContent() {
transform(value, controller) {
// Base64 decode the `content` field and send it as a chunk.
if (value.content?.length) {
controller.enqueue(atob(value.content));
// The `atob` function does not work properly if the decoded content contains any byte
// values over 0x7f, because "binary" in JS means that each byte gets represented as a
// UTF-16 code unit, which happens to be <= 0xff. I wish I was making this up:
// https://developer.mozilla.org/en-US/docs/Glossary/Base64#the_unicode_problem
const binary = atob(value.content);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < bytes.length; i++) {
bytes[i] = binary.charCodeAt(i);
}
const text = new TextDecoder().decode(bytes);
controller.enqueue(text);
}
},
});
Expand Down

0 comments on commit a906ada

Please sign in to comment.