Content-Length & Content-Encoding header is not changed accordingly when http response body is decompressed

[kekaifun]

Brief summary

Content-Length & Content-Encoding header is not changed accordingly when http response body is decompressed

k6 will transparently decompress the response body if it's has a content-encoding like gzip,deflate,zstd,br.

But the content-lenght & content-encoding header is not changed accordingly in the response.

Which will lead to a mismatch of conent-encoding & content-length between the header and the actual body.

k6 version

v0.46.0

OS

macOS 11

Docker version and image (if applicable)

No response

Steps to reproduce the problem

the following script is a test server that return a gziped response body. original length of string hello world is 11 byte, the gziped length is 35 byte

package server

import (
	"compress/gzip"
	"io"
	"net/http"
	"testing"
)

func TestServer(t *testing.T) {
	gzipHandler := func(w http.ResponseWriter, req *http.Request) {
		w.Header().Set("Content-Encoding", "gzip")

		gz := gzip.NewWriter(w)
		defer gz.Close()
		io.WriteString(gz, "hello world")
	}
	http.HandleFunc("/hello", gzipHandler)
	http.ListenAndServe("127.0.0.1:9001", nil)
}

send request to test server, and log the response body and header

import http from 'k6/http';

export default function () {
  let resp = http.get('http://127.0.0.1:9001/hello');
  console.log("resp body: " + resp.body);
  console.log(resp.headers)
};

this is the output

          /\      |‾‾| /‾‾/   /‾‾/   
     /\  /  \     |  |/  /   /  /    
    /  \/    \    |     (   /   ‾‾\  
   /          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: examples/http_get.js
     output: -

  scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
           * default: 1 iterations for each of 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)

INFO[0000] resp body: hello world                        source=console
INFO[0000] {"Date":"Sun, 08 Oct 2023 06:44:48 GMT","Content-Length":"35","Content-Encoding":"gzip"}  source=console

Expected behaviour

as the response body is decompressed by k6, the response body is hello world, the content-length header in response should be 11, and the content-encoding header should be removed

{"Date":"Sun, 08 Oct 2023 06:44:48 GMT","Content-Length":"11"}

Actual behaviour

the response body is decompressed by k6, but the content-length in header is the length of the compressed body. and the content-enconding header is still there

{"Date":"Sun, 08 Oct 2023 06:44:48 GMT","Content-Length":"35","Content-Encoding":"gzip"}

[kekaifun]

@oleiade i'd like to solve it

[oleiade]

Hi @kekaifun 👋🏻

Thanks a lot for bringing this to our attention and even providing a PR demonstrating how this issue might be addressed 🙇‍♂️

We have discussed it with the maintainers' team, and considering this would be a breaking change, decided not to address it momentarily, as is. The k6 team is however developing a new HTTP module, which will likely address that issue in the process.

As a result, I will label this issue and the attached Pull Request with the new-http tag and close them. We do not have an ETA for the new module yet, but we'll update this issue accordingly once we have made progress on that specific front.

[oleiade]

ref #2461