diff --git a/README.md b/README.md index 72ca7cf1..327d7952 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,7 @@ See the [ROADMAP](ROADMAP.md) for planned capabilities. For example scenarios, see +- [How To](https://docs.clusterless.io/guide/1.0-wip/howtos/index.html) - step-by-step guides - [clusterless-aws-example](https://github.com/ClusterlessHQ/clusterless-aws-examples) - simple examples to start with - [aws-s3-log-pipeline](https://github.com/ClusterlessHQ/aws-s3-log-pipeline) - end-to-end sample pipeline for processing AWS S3 access logs @@ -76,7 +77,7 @@ And can be implemented as: - Docker images (Python, Java, Node, etc) - Serverless functions (Python, Java, Node, etc) -- Native services (AWS SageMaker, AWS Athena, AWS Glue, etc) +- Native services (AWS SageMaker, AWS Athena, AWS Glue, etc.) Where the intent isn't to have functional parity across substrates, but to ease secure and reliable interoperability between them. @@ -85,82 +86,42 @@ Currently supported cloud substrates: - AWS -## Prerequisites +## Installing -_If you already have Node and npm, skip to the AWS CDK install._ - -Install [nvm](https://github.com/nvm-sh/nvm): - -> curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash - -Install node/npm: - -> nvm install --lts - -Install the AWS CDK: - -> npm install -g aws-cdk@2.89.0 - -after install, make sure to bootstrap your account and region: - -https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html#bootstrapping-howto - -> cdk bootstrap aws://ACCOUNT-NUMBER-1/REGION-1 aws://ACCOUNT-NUMBER-2/REGION-2 ... - -Install Java 17 via [sdkman](https://sdkman.io): - -> sdk install java 17.0.8-graalce - -## Installing Clusterless - -The Clusterless CLI app `cls` is downloadable from GitHub releases: - -- https://github.com/ClusterlessHQ/clusterless/releases - -The `bin` folder will have: - -- `cls` - the root CLI interface -- `cls-aws` - the AWS interface only used as an escape hatch and testing +See: https://docs.clusterless.io/guide/1.0-wip/install-quickstart.html ## Running > cls --help ```text -Usage: cls [-hVv] [-D=]... [-P=]... [COMMAND] +Usage: cls [-hVv] [--output=] [-D=]... + [-P=]... [COMMAND] -D, --property= - key=value properties, will be passed down - -h, --help Show this help message and exit. + Optional key=value properties, will be passed down. + -h, --help Show this help message and exit. + --output= Print results using given format. + Values: table, json, csv, tsv (default: table) -P, --providers= - provider substrates to target - -v, --verbose Specify multiple -v options to increase verbosity. - For example, `-v -v -v` or `-vvv` - -V, --version Print version information and exit. + Provider substrates to target. + -v, --verbose Specify multiple -v options to increase verbosity. + For example, `-v -v -v` or `-vvv` + -V, --version Print version information and exit. Commands: - help Display help information about the specified command. - config manage local and global configuration settings - show display details about providers, components, and project models - bootstrap initialize a cloud provider placement - deploy deploy a project into a declared placement - destroy destroy a project deployed a declared placement - diff compare local project changes with a deployed a declared placement - local support for executing workloads locally - verify verify project changes with a provider + help Display help information about the specified command. + config Manage local and global configuration settings. + show Display details about providers, components, and project models. + bootstrap Initialize a cloud provider placement. + deploy Deploy a project into a declared placement. + destroy Destroy a project deployed a declared placement. + diff Compare local project changes with a deployed a declared + placement. + local Support for executing workloads locally in a terminal. + verify Verify project changes with a provider. + placements List all visible placements with deployments. + projects List all deployed projects. ``` -Every region must be bootstrapped by the `cls` app before it can be used: - -> cls bootstrap help - -To bootstrap a region: - -> cls bootstrap --region us-east-1 --account 123456789012 --profile my-profile --stage DEV - -If any arguments are missing, `cls` will prompt for them. - -Note the `--stage` option creates a namespace for all resources, so you can have multiple environments in the same -account. - ## Building (optional) To build the CLI app locally from source, clone the repo and run: diff --git a/ROADMAP.md b/ROADMAP.md index ce07fbe0..712ce34d 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -21,15 +21,19 @@ Most things in the cli have yet to be implemented. 3. Custom Lambda based arc workloads 4. High frequency S3 listener boundary 1. For aggregating objects that arrive within a lot interval + 2. https://docs.clusterless.io/reference/1.0-wip/components/aws-core-s3-put-listener-boundary.html 5. Native resources and workloads 1. AWS Glue database and catalog updates + 1. https://docs.clusterless.io/reference/1.0-wip/components/aws-core-glue-database.html + 2. https://docs.clusterless.io/reference/1.0-wip/components/aws-core-glue-table.html + 3. https://docs.clusterless.io/reference/1.0-wip/components/aws-core-glue-add-partitions-arc.html 2. AWS Athena CTAS/INSERT INTO queries (for chaining SQL) 3. AWS Sagemaker training/validation 6. Common data processing workloads 1. Data reformatting (from text/json to binary/parquet) - 2. https://github.com/ClusterlessHQ/tessellate + 1https://github.com/ClusterlessHQ/tessellate 2. Dynamic data repartitioning (partitions based on data like timestamps) - 3. https://github.com/ClusterlessHQ/tessellate + 1. https://github.com/ClusterlessHQ/tessellate 3. Predicate/duplicate index creation and data filtering 7. Join Barrier implementations 8. Scheduled arc executions @@ -39,3 +43,7 @@ Most things in the cli have yet to be implemented. 10. Pluggable modules for providing third-party services 11. Localstack support for faster testing AWS scenarios 12. Alternate substrates/providers + 1. Azure + 2. GCP + 3. Digital Ocean + 4. OCI diff --git a/clusterless-main/src/main/java/clusterless/command/ProjectsCommand.java b/clusterless-main/src/main/java/clusterless/command/ProjectsCommand.java index 7876506c..89913eb9 100644 --- a/clusterless-main/src/main/java/clusterless/command/ProjectsCommand.java +++ b/clusterless-main/src/main/java/clusterless/command/ProjectsCommand.java @@ -14,7 +14,7 @@ @CommandLine.Command( name = "projects", - description = "list all deployed projects" + description = "List all deployed projects." ) public class ProjectsCommand extends CommandWrapper { public ProjectsCommand() {