Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decode chunked transfer encoding for incoming requests #106

Closed
wants to merge 8 commits into from

Conversation

legionth
Copy link
Contributor

@legionth legionth commented Feb 10, 2017

This ensures that only decoded body data will be emitted via the request object.

Resolves / closes #96

Copy link
Member

@clue clue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would love to get this in 👍

Can you look into the failing tests and the below remarks?

use React\Stream\Util;
use Exception;

class ChunkedDecoder extends EventEmitter implements ReadableStreamInterface
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@internal?

src/Server.php Outdated
// attach remote ip to the request as metadata
$request->remoteAddress = $conn->getRemoteAddress();

$conn->removeListener('data', $listener);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate? (Line 34)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. Removed this line in one of the latest

src/Server.php Outdated
@@ -64,7 +66,21 @@ public function handleRequest(ConnectionInterface $conn, Request $request, $body
return;
}

$header = $request->getHeaders();
$stream = $conn;
if (!empty($header['Transfer-Encoding']) && $header['Transfer-Encoding'] === 'chunked') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should take advantage of #103 to match case insensitive.

$this->emit('request', array($request, $response));
$request->emit('data', array($bodyBuffer));
$conn->emit('data', array($bodyBuffer));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a heads up: Will likely cause a merge conflict with #108.

@clue clue changed the title Support decode chunked-encoded requests Decode chunked transfer encoding for incoming requests Feb 10, 2017
@legionth legionth force-pushed the support-chunked-encoding branch from 1df0cad to df840b1 Compare February 10, 2017 14:46
@clue clue added this to the v0.4.4 milestone Feb 10, 2017
@legionth
Copy link
Contributor Author

Updated. Give me your feedback 👍

@legionth legionth force-pushed the support-chunked-encoding branch from 9324551 to 0e50877 Compare February 11, 2017 06:38
@legionth
Copy link
Contributor Author

Rebased to resolve merge conflicts.

private function handleChunkHeader($data)
{
$hexValue = strtok($this->buffer . $data, static::CRLF);
if ($this->isLineComplete($this->buffer . $data, $hexValue, strlen($hexValue))) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be simplified? How about using strpos() and substr() here? Also could probably use an early return instead.

$chunk = substr($this->buffer . $data, 0, $this->chunkSize);
$this->actualChunksize = strlen($chunk);

if ($this->chunkSize == $this->actualChunksize) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=== everywhere?

$this->chunkHeaderComplete = true;

$data = substr($this->buffer . $data, strlen($hexValue) + 2);
$this->buffer = '';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really hard to keep track of. It looks like the buffer is modified in a couple of methods here? Can this buffering logic be simplified?

(see also above for strpos() etc.

* @param string $chunk - chunk which will be emitted
* @return string - rest data string
*/
private function sendChunk($data, $chunk)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is being "sent" here?

private function sendChunk($data, $chunk)
{
if ($this->chunkSize == 0 && $this->isLineComplete($this->buffer . $data, $chunk, $this->chunkSize)) {
$this->emit('end', array());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be followed by a close() call?

if (! $this->closed) {
$this->emit('end');
$this->close();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if there's still data in the buffer when the underlying stream emits an end event?


public function testEnd()
{
$this->parser->on('end', $this->expectCallableOnce(array()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty array? see also below


public function testCompletlySplitted()
{
$this->parser->on('data', $this->expectCallableOnceWith('welt'));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to indicate the whole chunk body is being buffered here? What if the chunk header says the chunk body will be several GiB?

@clue clue modified the milestones: v0.4.5, v0.4.4 Feb 11, 2017
$this->input->removeListener('data', array($this, 'handleChunkHeader'));
$this->input->on('data', array($this, 'handleChunkData'));
if ($data !== '') {
$this->input->emit('data', array($data));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the decoder should have no control over emitting events on an external entity (which may have multiple listeners for this event).

@legionth
Copy link
Contributor Author

Will try a different approach.

@legionth legionth closed this Feb 13, 2017
@clue clue removed this from the v0.4.5 milestone Feb 14, 2017
@legionth legionth deleted the support-chunked-encoding branch February 16, 2017 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support chunked transfer encoding
2 participants