Skip to content

Commit

Permalink
Move FormData instantiation inside makeRequest retry
Browse files Browse the repository at this point in the history
I have been troubleshooting a perplexing issue when updating the
happo.io package at Airbnb. It seems that on this new version when a
request fails and is retried it ends up erroring out with a socket hang
up error after 60 seconds.

After some investigation, this error seems to be exactly what was
reported here:

  node-fetch/node-fetch#1743

The issue is that the `FormData` instance is being reused but since its
buffer is already consumed it puts fetch into a bad state where it never
actually sends the request and hangs until it hits a socket timeout of
60 seconds. The proposed fix is to always use a fresh `FormData`
instance.

In this commit I am moving the prepareFormData call to inside of the
retry function so that we get a fresh FormData instance on each retry. I
am hopeful that this will resolve this issue.

I added a test to cover this behavior. When I run the test before my
change I observe that the test suite hangs when making this request.
After my change everything works as expected. This makes me fairly
confident in the value of this change.
  • Loading branch information
lencioni committed May 29, 2024
1 parent e3e9fda commit 8a1fa76
Show file tree
Hide file tree
Showing 2 changed files with 52 additions and 6 deletions.
15 changes: 10 additions & 5 deletions src/makeRequest.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,20 @@ export default async function makeRequest(
{ HTTP_PROXY } = process.env,
) {
const { url, method = 'GET', formData, body: jsonBody } = requestAttributes;
const body = formData
? prepareFormData(formData)
: jsonBody
? JSON.stringify(jsonBody)
: undefined;

return asyncRetry(
async () => {
const start = Date.now();

// We must avoid reusing FormData instances when retrying requests
// because they are consumed and cannot be reused.
// More info: https://github.com/node-fetch/node-fetch/issues/1743
const body = formData
? prepareFormData(formData)
: jsonBody
? JSON.stringify(jsonBody)
: undefined;

const signed = jwt.sign({ key: apiKey }, apiSecret, {
header: { kid: apiKey },
});
Expand Down
43 changes: 42 additions & 1 deletion test/makeRequest-test.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ beforeAll(async () => {
result: 'Hello world!',
}),
);
} else if (req.url === '/form-data') {
} else if (req.url === '/form-data' || (req.url === '/form-data-failure-retry' && errorTries > 2)) {
const form = new multiparty.Form();

form.parse(req, (err, fields, files) => {
Expand Down Expand Up @@ -146,6 +146,47 @@ it('can upload form data with buffers', async () => {
});
});

it('can retry uploading form data with buffers', async () => {
props.url = 'http://localhost:8990/form-data-failure-retry';
props.method = 'POST';
props.formData = {
type: 'browser-chrome',
targetName: 'chrome',
payloadHash: 'foobar',
payload: {
options: {
filename: 'payload.json',
contentType: 'application/json',
},
value: Buffer.from('{"foo": "bar"}'),
},
};
const response = await subject();
expect(response).toEqual({
fields: {
payloadHash: ['foobar'],
targetName: ['chrome'],
type: ['browser-chrome'],
},
files: {
payload: [
{
fieldName: 'payload',
headers: {
'content-disposition':
'form-data; name="payload"; filename="payload.json"',
'content-type': 'application/json',
},
originalFilename: 'payload.json',
path: expect.any(String),
size: 14,
},
],
},
});
});


it('can upload form data with streams', async () => {
props.url = 'http://localhost:8990/form-data';
props.method = 'POST';
Expand Down

0 comments on commit 8a1fa76

Please sign in to comment.