Revamp the tests

[LinusU]

The tests has a few flaws today and I think that we could improve them substantially. The first problem is that they run quite slow, and this should be easy fixable. Currently it starts an http server, sends the request, then multer gets to parse it. Not too bad, but then everything is serialized back to json and sent over http back to the client, this is a pain for large buffers.

I think that a good approach would be to use felixge/node-form-data to generate the body, and then pipe it directly into multer. The validation would then be done in the next callback.

[LinusU]

I just tried converting the inmemory.js test and the results speaks for themselves:

➜  multer git:(breaking-changes) ✗ mocha test/memory-storage.js


  Memory Storage
    ✓ should process multipart/form-data POST request
    ✓ should process empty fields and an empty file
    ✓ should process multiple files


  3 passing (87ms)

➜  multer git:(breaking-changes) ✗ mocha test/inmemory.js      


  InMemory
    ✓ should process multipart/form-data POST request (57ms)
    ✓ should process empty fields and an empty file
    ✓ should process multiple files (5275ms)


  3 passing (5s)

From more than 5 seconds to under 0.1 seconds, pretty sweet!

[jpfluger]

Wow! Cloning now...

[jpfluger]

Although it speeds up tests, doesn't having them run through an http server reflect real-world performance? That's how we figured out there was a speed problem with inMemory in the first place.

[LinusU]

Well we are still piping data over a stream to Busboy, the only thing we don't capture is network latency which we didn't before either. The only thing that has changed is that we no longer pipe back all data thru the response chain as JSON.

This is how JSON.stringify handles buffers:

> JSON.stringify({ a: new Buffer(20) })
'{"a":{"type":"Buffer","data":[0,32,0,0,0,0,0,0,48,170,176,0,1,0,0,0,80,168,5,1]}}'

Now the thing that took the 5 seconds was serializing buffers with a total of 2 429 095 bytes, then parsing that back into a buffer. This will never happen in a real application.

[LinusU]

This is now fixed in multer 1.0.0