Skip to content

Commit

Permalink
feat: Add ability to configure client TLS (#5241)
Browse files Browse the repository at this point in the history
  • Loading branch information
achetronic authored Oct 23, 2024
1 parent aa905e6 commit 3426d8d
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 0 deletions.
24 changes: 24 additions & 0 deletions cmd/format.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package cmd

import (
"bytes"
"crypto/tls"
"crypto/x509"
"encoding/pem"
"fmt"
Expand Down Expand Up @@ -216,6 +217,7 @@ func fixObjectSize(s uint64) uint64 {
}

func createStorage(format meta.Format) (object.ObjectStorage, error) {

if err := format.Decrypt(); err != nil {
return nil, fmt.Errorf("format decrypt: %s", err)
}
Expand All @@ -234,6 +236,28 @@ func createStorage(format meta.Format) (object.ObjectStorage, error) {
u.RawQuery = values.Encode()
format.Bucket = u.String()
}

// Configure client TLS when params are provided
if values.Get("ca-certs") != "" && values.Get("ssl-cert") != "" && values.Get("ssl-key") != "" {

clientTLSCert, err := tls.LoadX509KeyPair(values.Get("ssl-cert"), values.Get("ssl-key"))
if err != nil {
return nil, fmt.Errorf("error loading certificate and key file: %s", err.Error())
}

certPool := x509.NewCertPool()
caCertPEM, err := os.ReadFile(values.Get("ca-certs"))
if err != nil {
return nil, fmt.Errorf("error loading CA cert file: %s", err.Error())
}

if certAdded := certPool.AppendCertsFromPEM(caCertPEM); !certAdded {
return nil, fmt.Errorf("error appending CA cert to pool")
}

object.GetHttpClient().Transport.(*http.Transport).TLSClientConfig.RootCAs = certPool
object.GetHttpClient().Transport.(*http.Transport).TLSClientConfig.Certificates = []tls.Certificate{clientTLSCert}
}
}

if format.Shards > 1 {
Expand Down
3 changes: 3 additions & 0 deletions docs/en/reference/how_to_set_up_object_storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ juicefs format --storage s3 \

When executing the `juicefs format` or `juicefs mount` command, you can set some special options in the form of URL parameters in the `--bucket` option, such as `tls-insecure-skip-verify=true` in `https://myjuicefs.s3.us-east-2.amazonaws.com?tls-insecure-skip-verify=true` is to skip the certificate verification of HTTPS requests.

Client certificates are also supported as they are commonly used for mTLS connections, for example:
`https://myjuicefs.s3.us-east-2.amazonaws.com?ca-certs=./path/to/ca&ssl-cert=./path/to/cert&ssl-key=./path/to/privatekey`

## Enable data sharding {#enable-data-sharding}

When creating a file system, multiple buckets can be defined as the underlying storage of the file system through the [`--shards`](../reference/command_reference.mdx#format-data-format-options) option. In this way, the system will distribute the files to multiple buckets based on the hashed value of the file name. Data sharding technology can distribute the load of concurrent writing of large-scale data to multiple buckets, thereby improving the writing performance.
Expand Down
3 changes: 3 additions & 0 deletions docs/zh_cn/reference/how_to_set_up_object_storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ juicefs format --storage s3 \

在执行 `juicefs format``juicefs mount` 命令时,可以在 `--bucket` 选项中以 URL 参数的形式设置一些特别的选项,比如 `https://myjuicefs.s3.us-east-2.amazonaws.com?tls-insecure-skip-verify=true` 中的 `tls-insecure-skip-verify=true` 即为跳过 HTTPS 请求的证书验证环节。

客户端证书也受支持,因为它们通常用于 mTLS 连接,例如:
`https://myjuicefs.s3.us-east-2.amazonaws.com?ca-certs=./path/to/ca&ssl-cert=./path/to/cert&ssl-key=./path/to/privatekey`

## 配置数据分片(Sharding) {#enable-data-sharding}

创建文件系统时,可以通过 [`--shards`](../reference/command_reference.mdx#format-data-format-options) 选项定义多个 Bucket 作为文件系统的底层存储。这样一来,系统会根据文件名哈希值将文件分散到多个 Bucket 中。数据分片技术可以将大规模数据并发写的负载分散到多个 Bucket 中,从而提高写入性能。
Expand Down

0 comments on commit 3426d8d

Please sign in to comment.