Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support buffering requests with unknown length (Transfer-Encoding: chunked) in RequestBodyBufferMiddleware #235

Merged
merged 2 commits into from
Nov 20, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -697,16 +697,11 @@ Any incoming request that has a request body that exceeds this limit will be
rejected with a `413` (Request Entity Too Large) error message without calling
the next middleware handlers.

Any incoming request that does not have its size defined and uses the (rare)
`Transfer-Encoding: chunked` will be rejected with a `411` (Length Required)
error message without calling the next middleware handlers.
Note that this only affects incoming requests, the much more common chunked
transfer encoding for outgoing responses is not affected.
It is recommended to define a `Content-Length` header instead.
Note that this does not affect normal requests without a request body
(such as a simple `GET` request).
The `RequestBodyBufferMiddleware` will buffer requests with bodies of known size
(i.e. with `Content-Length` header specified) as well as requests with bodies of
unknown size (i.e. with `Transfer-Encoding: chunked` header).

All other requests will be buffered in memory until the request body end has
All requests will be buffered in memory until the request body end has
been reached and then call the next middleware handler with the complete,
buffered request.
Similarly, this will immediately invoke the next middleware handler for requests
Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"react/stream": "^1.0 || ^0.7.1",
"react/promise": "^2.3 || ^1.2.1",
"evenement/evenement": "^3.0 || ^2.0 || ^1.0",
"react/promise-stream": "^0.1.1"
"react/promise-stream": "^1.0 || ^0.1.2"
},
"autoload": {
"psr-4": {
Expand Down
20 changes: 12 additions & 8 deletions src/Middleware/RequestBodyBufferMiddleware.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
use React\Promise\Stream;
use React\Stream\ReadableStreamInterface;
use RingCentral\Psr7\BufferStream;
use OverflowException;

final class RequestBodyBufferMiddleware
{
Expand All @@ -29,27 +30,30 @@ public function __construct($sizeLimit = null)

public function __invoke(ServerRequestInterface $request, $stack)
{
$size = $request->getBody()->getSize();

if ($size === null) {
return new Response(411, array('Content-Type' => 'text/plain'), 'No Content-Length header given');
}
$body = $request->getBody();

if ($size > $this->sizeLimit) {
// request body of known size exceeding limit
if ($body->getSize() > $this->sizeLimit) {
return new Response(413, array('Content-Type' => 'text/plain'), 'Request body exceeds allowed limit');
}

$body = $request->getBody();
if (!$body instanceof ReadableStreamInterface) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this check be moved up as this type of stream won't be buffered anyway? Why complain about size in this case (shouldn't happen when using the server at all, might as well just throw)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it makes sense to keep this like this in (the rare) case this middleware is used behind another buffering middleware handler, see also #235 (comment) 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clue What I mean is that- if not a ReadableStreamInterface, which means we won't buffer it, we're still returning 413 although the middleware is not responsible for handling this request as it is right now.

My suggestion would be to move the ReadableStreamInterface to the Beginning of the method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @clue could you check my question above?
What's missing to make make progress with this PR (and 0.8 in general)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, technically buffering and limiting sizes are in fact independent responsibilities. For practical reasons however, I think that it makes sense to bring them close together because buffering without applying a limit is actually harmful (potential for trivial DOS otherwise). If we accept the fact that checking size limits here, then I think it makes sense to also support buffered requests here. This means that I think it makes sense to also reject buffered requests that exceed the given size limits.

Accordingly, I think this code LGTM 👍 What do you think about this?

Copy link
Contributor Author

@andig andig Nov 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about this?

@clue scratch head. I'm not getting my point across. We're only buffering and limiting PSR streams. We are however still limiting react-only streams.

This is imho wrong- as the buffering middleware is not responsible for these requests. I'm proposing- to be explicit- to change order of things like this:

public function __invoke(ServerRequestInterface $request, $stack)
{
    $body = $request->getBody();

    if (!$body instanceof ReadableStreamInterface) {
        return $stack($request);
    }

    // request body of known size exceeding limit
    if ($body->getSize() > $this->sizeLimit) {
        return new Response(413, array('Content-Type' => 'text/plain'), 'Request body exceeds allowed limit');
    }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it looks like we're talking past one another 🤷‍♂️ :-)

The getBody() method is guaranteed to always return an instance of Psr\Message\StreamInterface. This very instance may also implement the React\Stream\ReadableStreamInterface ("streaming state"), but this is not guaranteed (already in "buffered state").

When this middleware handler is used as documented, then the body is always in a "streaming state". If one were to use this middleware handler twice in the same stack (conceived example), then the first will see a body in "streaming state" while the second will see it in "buffered state".

If this conceived(!) second middleware handler uses a smaller size limit (which I would expect to be more of a programming error than intentional), then IMO the second middleware handler should still reject the message in my opinion.

If I understand your above suggestion correctly, then it looks like this would actually allow the body to pass through without applying the second limit at all.

I think it makes sense to add a test for this behavior to ensure that this is properly covered. Can you add a test for this or would you rather want me to add this to this PR? 👍

return $stack($request);
}

return Stream\buffer($body)->then(function ($buffer) use ($request, $stack) {
return Stream\buffer($body, $this->sizeLimit)->then(function ($buffer) use ($request, $stack) {
$stream = new BufferStream(strlen($buffer));
$stream->write($buffer);
$request = $request->withBody($stream);

return $stack($request);
}, function($error) {
// request body of unknown size exceeding limit during buffering
if ($error instanceof OverflowException) {
return new Response(413, array('Content-Type' => 'text/plain'), 'Request body exceeds allowed limit');
}

throw $error;
});
}

Expand Down
75 changes: 61 additions & 14 deletions tests/Middleware/RequestBodyBufferMiddlewareTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
namespace React\Tests\Http\Middleware;

use Psr\Http\Message\ServerRequestInterface;
use React\EventLoop\Factory;
use React\Http\Io\HttpBodyStream;
use React\Http\Io\ServerRequest;
use React\Http\Middleware\RequestBodyBufferMiddleware;
use React\Stream\ThroughStream;
use React\Tests\Http\TestCase;
use RingCentral\Psr7\BufferStream;
use Clue\React\Block;

final class RequestBodyBufferMiddlewareTest extends TestCase
{
Expand Down Expand Up @@ -63,45 +65,90 @@ function (ServerRequestInterface $request) use (&$exposedRequest) {
$this->assertSame($body, $exposedRequest->getBody()->getContents());
}

public function testUnknownSizeReturnsError411()
public function testExcessiveSizeImmediatelyReturnsError413ForKnownSize()
{
$body = $this->getMockBuilder('Psr\Http\Message\StreamInterface')->getMock();
$body->expects($this->once())->method('getSize')->willReturn(null);

$loop = Factory::create();

$stream = new ThroughStream();
$stream->end('aa');
$serverRequest = new ServerRequest(
'GET',
'https://example.com/',
array(),
$body
new HttpBodyStream($stream, 2)
);

$buffer = new RequestBodyBufferMiddleware();
$buffer = new RequestBodyBufferMiddleware(1);
$response = $buffer(
$serverRequest,
function () {}
function (ServerRequestInterface $request) {
return $request;
}
);

$this->assertSame(411, $response->getStatusCode());
$this->assertSame(413, $response->getStatusCode());
}

public function testExcessiveSizeReturnsError413()
{
$stream = new BufferStream(2);
$stream->write('aa');
$loop = Factory::create();

$stream = new ThroughStream();
$serverRequest = new ServerRequest(
'GET',
'https://example.com/',
array(),
$stream
new HttpBodyStream($stream, null)
);

$buffer = new RequestBodyBufferMiddleware(1);
$response = $buffer(
$promise = $buffer(
$serverRequest,
function () {}
function (ServerRequestInterface $request) {
return $request;
}
);

$this->assertSame(413, $response->getStatusCode());
$stream->end('aa');

$exposedResponse = null;
$promise->then(
function($response) use (&$exposedResponse) {
$exposedResponse = $response;
},
$this->expectCallableNever()
);

$this->assertSame(413, $exposedResponse->getStatusCode());

Block\await($promise, $loop);
}

/**
* @expectedException RuntimeException
*/
public function testBufferingErrorThrows()
{
$loop = Factory::create();

$stream = new ThroughStream();
$serverRequest = new ServerRequest(
'GET',
'https://example.com/',
array(),
new HttpBodyStream($stream, null)
);

$buffer = new RequestBodyBufferMiddleware(1);
$promise = $buffer(
$serverRequest,
function (ServerRequestInterface $request) {
return $request;
}
);

$stream->emit('error', array(new \RuntimeException()));

Block\await($promise, $loop);
}
}