services/s3: Integration test fails when using the latest MinIO

[JinnyYi]

Excepted Behavior

go-service-s3 should compatible with MinIO and pass integration test with MinIO.

Current Behavior

• Integration test passed when using minio RELEASE.2021-06-17T00-10-46Z
• Decode REST XML response failed when using MinIO Console

  When Read a file     storager.go:55: write: Storager s3 {Name: xxxxx, WorkDir: /dc2c38b3-2de2-413c-8399-92cb1ba879e8/}, [3dd549ed-b614-45b7-9848-d33fdac98238]: unexpected: SerializationError: failed to decode REST XML response
                status code: 200, request id:
        caused by: XML syntax error on line 1: element <link> closed by </head>

Steps to Reproduce

Run integration test for go-service-s3 with the following config:

STORAGE_S3_INTEGRATION_TEST=on
STORAGE_S3_CREDENTIAL=hmac:minioadmin:minioadmin
STORAGE_S3_ENDPOINT=https:play.min.io:9443
STORAGE_S3_NAME=<bucketname>
STORAGE_S3_LOCATION=local

Also we need to set endpoint for initialization:

pairs.WithEndpoint("https:play.min.io:9443")

Environment

• MinIO Console

[Xuanwo]

So nice a bug report.

[Xuanwo]

I believe we are calling the minio console instead of minio server. Does the minio endpoint change?

[JinnyYi]

I believe we are calling the minio console instead of minio server. Does the minio endpoint change?

I can't access minio server locally. I'll rebuild minio server later and update the test result.

[Xuanwo]

OK, let's figure out the root cause in next cycle maybe.

[JinnyYi]

XML syntax error is indeed returned by minio console. When I tested with the correct minio server ip, most of the test cases passed, except for the following cases:

• When using absolute path:

  • Error info:

   Object name invalid: jl-test/17d1b6f8-0713-4c6f-8ac9-d2570fb9345a//17d1b6f8-0713-4c6f-8ac9-d2570fb9345a/a6205dcb-313a-4056-bb5c-6f7ef08883d1 (cmd.ObjectNameInvalid)
  

  • Causes: getAbsPath() does not take into account the absolute path of the input

• When Delete with multipart id:

  • Error info: NoSuchUpload returned by the second delete operation
  • Cause: Seems that AbortMultipartUpload is non-idempotent.

• When List with part type types.ListModePart

  • Error info: runtime error: invalid memory address or nil pointer dereference when call Next
  • Cause: No Upload node in the response body.
  • services/s3: List with types.ListModePart mode test fails when using MinIO #924

[Xuanwo]

It looks like we need to solve them in the services/s3. Can you split them into different issues so that we can get the community involved?