add endpoint url parameter

[sattler]

I added a parameter to support custom endpoints.
The changes are minimal invasive and should not change any existing behavior or break backwards compatibility.

I tested these changes with a local minio deployment.

[ktrueda]

Thank you for your PR. Its' great.

I tried to use your code, but it didn't work for wildcard path on s3 like
parquet-tools csv -n 3 --endpoint-url https://s3.ap-northeast-1.amazonaws.com "s3://bucket-name/*".

Probably you have to modify util.py like

    def resolve_wildcard(self) -> List[ParquetFile]:
        list_res = self.aws_session.client('s3', endpoint_url=self.endpoint_url)\
            .list_objects_v2(
            Bucket=self.bucket,
            Prefix=self.key[:-1]  # remove *
        )
        if list_res['IsTruncated']:
            raise Exception(f'Too much file match s3://{self.bucket}/{self.key}')

        if list_res['KeyCount'] == 0:
            return []
        keys = [e['Key'] for e in list_res['Contents']]
        return sorted(
            [S3ParquetFile(aws_session=self.aws_session, bucket=self.bucket, key=key, endpoint_url=self.endpoint_url) for key in keys], // you have to add endpoint_url argment
            key=lambda x: x.key
        )

Could you confirm this?

[ktrueda]

S3ParquetFile is also used in https://github.com/ktrueda/parquet-tools/blob/master/tests/test_util.py .
You have to update this file and pass this test.

[sattler]

thank you for point this out. I will make the according changes!

[sattler]

I added a simple default value for the endpoint_url.
Therefore, no superfluous parameter definitions are needed for the test cases while still allowing to be set when needed.
Test are now also passing :)

[ktrueda]

Thank you.