Support "req.formData()"

[ddolcimascolo]

Prerequisites

• I confirm my issue is not in the opened issues
• I confirm the Frequently Asked Questions didn't contain the answer to my issue

Environment check

• I'm using the latest msw version
• I'm using Node.js version 14 or higher

Node.js version

16.13.0

Reproduction repository

https://github.com/ddolcimascolo/msw-issue-1327

Reproduction steps

Axios to send a POST request with a FormData body. Try to use req.body in the handler

Current behavior

TypeError: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of FormData
        at new NodeError (node:internal/errors:371:5)
        at TextDecoder.decode (node:internal/encoding:413:15)
        at decodeBuffer (/data/dev/msw-issue-formdata/node_modules/@mswjs/interceptors/src/utils/bufferUtils.ts:11:18)
        at RestRequest.get body [as body] (/data/dev/msw-issue-formdata/node_modules/msw/src/utils/request/MockedRequest.ts:117:18)

Expected behavior

Maybe req.formData() as per https://developer.mozilla.org/en-US/docs/Web/API/Request/formData would be the way to go?

[robcaldecott]

I am using a custom fetcher with codegen to handle FormData mutations and have never had much luck getting it to work in msw so have been intercepting the request at the point where my data is submitted in my tests like this:

// Intercept the upload
  server.use(
    rest.post(
      "http://localhost:8000/fleetportalapi/graphiql",
      (req, res, ctx) =>
        res(
          ctx.json({
            data: {
              fileUploadMileage: {
                __typename: "FileUploadMileageError",
                errorMessage: "Error parsing the CSV file.",
              },
            },
          })
        )
    )
  );

This is not perfect - ideally I would be able to use graphql.mutation - but it was working fine until I updated to 0.44.0 today where I now get the same error:

console.error
    The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of FormData

      at node_modules/msw/src/handlers/GraphQLHandler.ts:155:26
      at tryCatch (node_modules/msw/src/utils/internal/tryCatch.ts:9:5)
      at GraphQLHandler.parse (node_modules/msw/src/handlers/GraphQLHandler.ts:153:12)
      at GraphQLHandler.test (node_modules/msw/src/handlers/RequestHandler.ts:167:12)
      at node_modules/msw/src/utils/getResponse.ts:31:20
          at Array.filter (<anonymous>)
      at getResponse (node_modules/msw/src/utils/getResponse.ts:30:37)

FWIW my custom fetch looks like this (mostly borrowed from graphql-request:

import { extractFiles, isExtractableFile } from "extract-files";

const createRequestBody = <TVariables>(
  query: string,
  variables?: TVariables
) => {
  const { files, clone } = extractFiles(
    { query, variables },
    "",
    isExtractableFile
  );

  if (files.size === 0) {
    return JSON.stringify({ query, variables });
  }

  const form = new FormData();

  form.append("operations", JSON.stringify(clone));

  const map: { [key: number]: string[] } = {};
  let i = 0;
  files.forEach((paths) => {
    map[++i] = paths;
  });
  form.append("map", JSON.stringify(map));

  i = 0;
  files.forEach((paths, file) => {
    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    form.append(`${++i}`, file as any);
  });

  return form as FormData;
};

export const useFetchData = <TData, TVariables>(
  query: string,
  options?: RequestInit["headers"]
): ((variables?: TVariables) => Promise<TData>) => {
  return async (variables?: TVariables) => {
    const body = createRequestBody(query, variables);

    const res = await fetch(url, {
      method: "POST",
      headers: {
        ...(typeof body === "string"
          ? { "Content-Type": "application/json" }
          : {}),
        ...(options ?? {}),
      },
      body,
    });

    const json = await res.json();

    if (json.errors) {
      const { message } = json.errors[0] || "Error..";
      throw new Error(message);
    }

    return json.data;
  };
};

My issue would be solved if msw could handle these kind of requests. For now I'll roll back to 0.42.1.

[95th]

@ddolcimascolo Can you provide reproduction code for this error?

[ddolcimascolo]

@95th I didn't forget, but facing some other troubles atm. If you want to give it a try it's really just

1. Send a request with axios using a FormData body to an URL mocked with MSW
2. Access req.body in the request handler

Was working fine in the previous minor.

Cheers,
David

[95th]

@ddolcimascolo I tried this but its working fine for me. any special data included in the FormData?

[ddolcimascolo]

Nothing special.
But I realize I have not given many details about the context... Were doing Jest tests with jsdom, so the FormData is what jsdom provides... Maybe it's the culprit but this was running smoothly before.

I'll definitely work on reproducing this in a repo

[nathanhannig]

Having same issue, can't pull file off with req.body.get("file"). Worked in previous versions.

const file = new File(["test"], "test.bad");
const formData = new FormData();
formData.append("file", file);
  
fetch("/api/uploadFile", {
    body: formData,
    method: "POST",
  })

rest.post(
    "/api/uploadFile",
    (req: MockedRequest<FormData>, res, ctx) => {
      const file = req.body.get("file"); // fails here, hangs
      
      ...
    }
)

Works in 0.43.1, stops working in 0.44.0+

[ddolcimascolo]

Thx @nathanhannig. I didn't manage to create a full reproduction repo, yet.

[ddolcimascolo]

@95th @nathanhannig I finally created the reproduciton repo @ https://github.com/ddolcimascolo/msw-issue-1327

[ddolcimascolo]

Guys, any news?

[kettanaito]

@ddolcimascolo, no news so far. Anybody is welcome to take the reproduction repo above and step through this body function to find out what happens now with FormData bodies. Debugging this is about 80% of solving it.

[mattcosta7]

on 0.44.2 ->

I can read FormData from req.body, however it's not a FormData object, but instead a json representation - so I grab the file via req.body.file rather than req.body.get('file')

Request data

text() => gives the value of text2 in that debugger
json() => throws (since its not json)

At some point it does seem like the file gets corrupted in 'upload' but I haven't gotten into where/how that could be improved/changed

[kettanaito]

Can that corruption be related to #1158 somehow?

[ddolcimascolo]

Hi everyone, still no fix to this issue? We're stuck a few versions behind because of this...
Can you advise if this would be doable by someone that never contributed to msw? Is this a good first issue?

Cheers,
David

[nathanhannig]

I have been stuck at 0.43.1 because of this isse

req.body was broken after that version (even though it incorrectly says it was depreciated)

[ddolcimascolo]

Welcome aboard 🙄

[kettanaito]

req.body was broken after that version (even though it incorrectly says it was depreciated)

Hence why we've released it as a breaking change. It is getting deprecated, but it also drops some of the request reading methods since we didn't have the capacity to implement all of them.

Can you advise if this would be doable by someone that never contributed to msw? Is this a good first issue?

I don't see why it wouldn't be. This is a scoped known change, and you have previous versions that supported this to verify your implementation against.

How can I contribute?

1. A/B what's going wrong with a FormData request body (req.body) in the resolver between 0.43.0 and 0.47.0. The fix will depend on the root cause: what exactly goes off? It may be useful to see how we parsed the request body before.
2. Add the FormData case handling to the .body method here. If applicable, we can also add MockedRequest.formData() method and reuse it in the deprecated .body property.

The main task here is to determine how we used to handle FormData request bodies in the past. I honestly don't recall. I don't think we used polyfills for that, it must've been something else. If we could bring that something else back into .body it'd be great.

Volunteers are certainly welcome. I will help with the code review so we could ship this feature in a timely manner.

[ddolcimascolo]

Hey guys, my tentative in #1422

[ddolcimascolo]

@kettanaito Did you get a chance to review the PR?

Cheers,
David

[kettanaito]

Hey, @ddolcimascolo. Yes, thank you so much for opening it. I've left a few comments and I'd love to know your thoughts on those.

Also, it's worth mentioning that #1404 change is going to introduce .formData() as well but I don't think it's going to happen that fast.

[ddolcimascolo]

@kettanaito Thx for the feedback. I dropped Axios in favor of a raw XHR request but I have no clue how to handle your comment about FormData not existing in Node...

[boryscastor]

I guess I stay on 0.38.1 until you implement the body getter...

[kettanaito]

@boryscastor, the body getter is not coming back. Instead, in the next major release you will be able to read the request's body as form data like you would do in regular JavaScript:

rest.post('/login', async ({ request }) => {
  const data = await request.formData()
  const email = data.get('email')
})

When is this coming out? Sometime this year, hopefully.

When can I try this? You can try this right now. More details in #1464.

[fibonacid]

what's the workaround until req.formData() exists?
Can I do something manually to get the data from req.arrayBuffer()?

    const arrayBuffer = await req.arrayBuffer();
    const formData = undefined; // ??;

[kettanaito]

@fibonacid, the FormData request body is actually a string in a predefined format:

const formData = new FormData()
formData.set('foo', 'bar')

const request = new Request('/url', { method: 'PUT', body: formData })
await request.text()
'------WebKitFormBoundaryY2WwRDBYmRbAKyLB\r\nContent-Disposition: form-data; name="foo"\r\n\r\nbar\r\n------WebKitFormBoundaryY2WwRDBYmRbAKyLB--\r\n'

You can read the request body as text to obtain that string. Then, you can feed it to any parser that would turn it back into a FormData instance. Unfortunately, the standard FormData constructor doesn't support that string as an input. You'd have to use third-party packages to do the job here.

You can use parse-multipart-data to parse that string into an object and then construct a FormData instance using that object.

Alternatively, consider switching to msw@next that ships with FormData built-in. More on that version here: #1464.

[fibonacid]

@kettanaito thanks, this is very helpful!

[tomdglenn91]

Came across this thread after facing 2 issues with MSW - all my graphql handlers throwing input instance of arraybuffer whenever i used formdata, and NetworkError when trying to access a body that was formdata. Will migrate to @next and give that a go. Thanks for the detailed migration guide, that'll save us a lot of time. Will edit this if it solves all of my problems

[kettanaito]

Released: v2.0.0 🎉

This has been released in v2.0.0!

• 📄 Release notes
• 📦 npm package

Make sure to always update to the latest version (npm i msw@latest) to get the newest features and bug fixes.

---

Predictable release automation by @ossjs/release.