README.md

Chorus

Chorus is vendor-agnostic S3-compatible backup, replication, and routing software. Once configured it can:

• sync existing buckets and objects from source to destination S3 storage
• capture live bucket/object changes from source S3
• propagate the changes to destination S3

Listed features can be configured per S3 user and per bucket with management CLI or webUI.

Components

Chorus S3 Proxy service responsible for routing s3 requests and capturing data change events. Chorus Agent can be used as an alternative solution for capturing events instead of proxy. Chorus Worker service does actual data replication with the help of RClone. Communication between Proxy/Agent and worker is done over work queue. Asynq with Redis is used as a work queue.

For more details, see:

• proxy
• worker
• agent
• standalone
• management CLI

Documentation

Documentation available at docs.clyso.com.

Run

From source

REQUIREMENTS:

• Go https://go.dev/doc/install

Run all-in-one standalone binry from Go:

go run ./cmd/chorus

Or run each service separately:

REQUIREMENTS:

• Go https://go.dev/doc/install
• Redis https://github.com/redis/redis

# run chorus worker
go run ./cmd/worker

# run chorus worker with a custom YAML config file
go run ./cmd/worker -config <path to worker yaml config>

# run chorus proxy with a custom YAML config file
go run ./cmd/proxy -config <path to proxy yaml config>

# run chorus agent with a custom YAML config file
go run ./cmd/agent -config <path to agent yaml config>

Standalone binary

See: service/standalone

Docker-compose

REQUIREMENTS:

• Docker

See: docker-compose

With Helm

REQUIREMENTS:

• K8s
• Helm

Install the Chorus Helm chart from the OCI registry:

helm install <release name> oci://harbor.clyso.com/chorus/chorus

See: deploy/chorus

Develop

test package contains e2e tests for replications between s3 storages. It runs:

• 3 embedded gofakes3 s3 storages
• embedded redis miniredis
• worker and proxy service

all listed tools are written in go so test can be run without external dependencies just by:

go test ./test/...