Signature Version 4 for Request Authentication support for S3

[gurvindersingh]

When running test to perform benchmarks against Amazon S3 or Minio (minio.io), I get this exception.

com.intel.cosbench.api.storage.StorageException: AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3, AWS Request ID: 3L137, AWS Error Code: InvalidRequest, AWS Error Message: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256., S3 Extended Request ID: 3L137

It seems cosbench needs to use V4 version signature as mentioned in https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html#specify-signature-version. I tested by adding the value as

-Dcom.amazonaws.services.s3.enableV4

in cosbench-start.sh but it still does not work. Any idea to fix this ?

[gurvindersingh]

If this helps here is the configuration used for testing

<workstage name="init">
  <work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
</workstage>

<workstage name="prepare">
  <work type="prepare" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,10);sizes=c(64)KB" />
</workstage>

<workstage name="main">
  <work name="main" workers="8" runtime="30">
    <operation type="read" ratio="80" config="cprefix=s3testqwer;containers=u(1,2);objects=u(1,10)" />
    <operation type="write" ratio="20" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(64)KB" />
  </work>
</workstage>

<workstage name="cleanup">
  <work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,20)" />
</workstage>

<workstage name="dispose">
  <work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
</workstage>

[threadfly]

hi,Did that solve？

[kisscelia]

Hi, any Solution?

[davidmgre]

I was able to get COSBench working with sigv4 by downloading the version from Hitachi.

https://github.com/HitachiDataSystems/cosbench/releases/
As of today, this works for me: https://github.com/HitachiDataSystems/cosbench/releases/download/v0.4.3.0/v0.4.3.0.tar

More info on their site here: https://community.hitachivantara.com/s/article/hcp-storage-adaptor-for-cosbench

I'm running on Amazon Linux (not Amazon Linux 2) and got up and running from a clean install with a single host using the commands below:

sudo yum -y update
sudo yum -y install nload wget git nc sysstat

mkdir ~/cosbench
cd ~/cosbench
wget https://github.com/HitachiDataSystems/cosbench/releases/download/v0.4.3.0/v0.4.3.0.tar
tar -xvf v0.4.3.0.tar
rm -f v0.4.3.0.tar
ln -s v0.4.3.0 cos
cd ~/cosbench/cos/
sh ./start-all.sh

[lkzz]

@davidmichaelgreen

no effect,is there something wrong?