Skip to content

Commit

Permalink
Fix: Data stacks documentation - 971 (#546)
Browse files Browse the repository at this point in the history 
* 917 - updates to data stacks documentation and fixed installation instructions for stacks-cli

* 1041 - updated requirements page for data stacks

* 917 - updates to data stacks documentation and fixed installation instructions for stacks-cli

* 1048 - changes to '1. Generate a data project' page

* Revert "917 - updates to data stacks documentation and fixed installation instructions for stacks-cli"

This reverts commit 4aaac07.

---------

Co-authored-by: Mehdi Kimakhe <[email protected]>
Co-authored-by: Jack Blower <[email protected]>
  • Loading branch information
@james-horrocks @mehdi-kimakhe-amido @ElvenSpellmaker
3 people authored Oct 29, 2024
1 parent 59f62a1 commit 94341ce
Show file tree
Hide file tree
Showing 5 changed files with 67 additions and 61 deletions.
4 changes: 2 additions & 2 deletions docs/stackscli/about.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ As the CLI is a single binary, the quickest way to install it is to download it
```bash
# Download the binary to a location in the PATH
## Mac OS
curl https://github.com/Ensono/stacks-cli/releases/download/v{stackscli_version}/stacks-cli-darwin-amd64-{stackscli_version} -o /usr/local/bin/stacks-cli
curl -L https://github.com/Ensono/stacks-cli/releases/download/v{stackscli_version}/stacks-cli-darwin-amd64-{stackscli_version} -o /usr/local/bin/stacks-cli

## Linux
curl https://github.com/Ensono/stacks-cli/releases/download/v{stackscli_version}/stacks-cli-linux-amd64-{stackscli_version} -o /usr/local/bin/stacks-cli
curl -L https://github.com/Ensono/stacks-cli/releases/download/v{stackscli_version}/stacks-cli-linux-amd64-{stackscli_version} -o /usr/local/bin/stacks-cli

## Ensure that the command is executable
chmod +x /usr/local/bin/stacks-cli
Expand Down
34 changes: 18 additions & 16 deletions docs/workloads/azure/data/getting_started/generate_project.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,12 @@ keywords:

This section provides an overview of scaffolding and generating a new data platform project using the [Ensono Stacks CLI](/docs/stackscli/about).

It assumes the following [requirements](./requirements_data_azure.md) are in place:
It assumes the following [pre-requisites](./requirements_data_azure.md) are in place:

* A [remote git repository](./requirements_data_azure.md#git-repository) for hosting the generated project
* [Terraform state storage](./requirements_data_azure.md#terraform-state-storage)
* A [storage account](./requirements_data_azure.md#terraform-state-storage) for the Terraform state

For more information on the pre-requisites, see [here](./requirements_data_azure.md).

## Step 1: Install the Ensono Stacks CLI

Expand All @@ -30,20 +32,20 @@ We will be using the `stacks-cli scaffold` command to generate a new data projec

A [sample data project config file](https://github.com/Ensono/stacks-azure-data/blob/main/stacks-cli/data-scaffold-example.yml) is provided. Prepare a copy of this file, and update the following entries as required for your new project:

| Config field | Example value | Description |
| ----- | ----- | ----- |
| directory.working | `stacks` | Target directory for the scaffolded project. |
| directory.export | `~` | Path to your Ensono Stacks CLI installation. |
| business.company | `mycompany` | Used for resource naming. |
| business.domain | `mydomain` | Used for environment & Terraform state key naming. |
| business.component | `data` | Used for resource naming. |
| project.name | `stacks-data-platform` | Name of project created & used for resource naming. |
| project.sourcecontrol.type | `github` | Remote repository type. |
| project.sourcecontrol.url | `https://github.com/mycompany/stacks-data-platform` | Used for setting up the remote repository - see [Git repository](./requirements_data_azure.md#git-repository). |
| project.cloud.region | `ukwest` | The Azure region you'll be deploying into. Using the Azure CLI, you can use `az account list-locations -o Table` to see available region names. |
| terraform.backend.storage | `tfstorage` | Storage account name for Terraform state - see [Terraform state storage](./requirements_data_azure.md#terraform-state-storage). |
| terraform.backend.group | `tfgroup` | Resource group account name for Terraform state. |
| terraform.backend.container | `tfcontainer` | Container name account name for Terraform state. |
| Config field | Example value | Description |
|-----------------------------|-----------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------|
| directory.working | `stacks` | Target directory for the scaffolded project. |
| directory.export | `~` | Path to your Ensono Stacks CLI installation. |
| business.company | `mycompany` | Used for resource naming. |
| business.domain | `mydomain` | Used for environment & Terraform state key naming. |
| business.component | `data` | Used for resource naming. |
| project.name | `stacks-data-platform` | Name of project created & used for resource naming. |
| project.sourcecontrol.type | `github` | Remote repository provider, e.g. GitHub or Azure DevOps. |
| project.sourcecontrol.url | `https://github.com/mycompany/stacks-data-platform` | Used for setting up the remote repository - see [Git repository](./requirements_data_azure.md#git-repository). |
| project.cloud.region | `ukwest` | The Azure region you'll be deploying into. Using the Azure CLI, you can use `az account list-locations -o Table` to see available region names. |
| terraform.backend.storage | `tfstorage` | Storage account name for Terraform state - see [Terraform state storage](./requirements_data_azure.md#terraform-state-storage). |
| terraform.backend.group | `tfgroup` | Resource group name for Terraform state. |
| terraform.backend.container | `tfcontainer` | Storage container name for Terraform state. |

All other values can be left as they are. For full documentation of all fields in the config file, refer to the Stacks CLI Manual.

Expand Down
17 changes: 9 additions & 8 deletions docs/workloads/azure/data/getting_started/getting_started.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@ A more [detailed workflow diagram](../architecture/architecture_data_azure.md#de

## Steps

1. [Generate a Data Project](./generate_project.md) - Generate a new data project.
2. [Infrastructure Deployment](./core_data_platform_deployment_azure.md) - Deploy the data platform infrastructure into your cloud environment.
3. [Local Development Quickstart](./dev_quickstart_data_azure.md) - Once your project has been generated, setup your local environment to start developing.
4. [Shared Resources Deployment](./shared_resources_deployment_azure.md) - Deploy common resources to be shared across data pipelines.
5. (Optional) [Example Data Source](./example_data_source.md) - To assist with the 'Getting Started' steps, you may wish to setup the Example Data Source.
6. [Data Ingest Pipeline Deployment](./ingest_pipeline_deployment_azure.md) - Generate and deploy a data ingest pipeline using the Datastacks CLI.
7. [Data Processing Pipeline Deployment](./processing_pipeline_deployment_azure.md) - Generate and deploy a data processing pipeline using the Datastacks CLI.
8. [Fabric Lakehouse Deployment](./fabric_deployment_guide.md) - Steps to implement a Microsoft Fabric Lakehouse over the data platform.
1. [Prerequisites](./requirements_data_azure.md) - Ensure you have the necessary tools and resources to get started.
2. [Generate a Data Project](./generate_project.md) - Generate a new data project.
3. [Infrastructure Deployment](./core_data_platform_deployment_azure.md) - Deploy the data platform infrastructure into your cloud environment.
4. [Local Development Quickstart](./dev_quickstart_data_azure.md) - Once your project has been generated, setup your local environment to start developing.
5. [Shared Resources Deployment](./shared_resources_deployment_azure.md) - Deploy common resources to be shared across data pipelines.
6. (Optional) [Example Data Source](./example_data_source.md) - To assist with the 'Getting Started' steps, you may wish to setup the Example Data Source.
7. [Data Ingest Pipeline Deployment](./ingest_pipeline_deployment_azure.md) - Generate and deploy a data ingest pipeline using the Datastacks CLI.
8. [Data Processing Pipeline Deployment](./processing_pipeline_deployment_azure.md) - Generate and deploy a data processing pipeline using the Datastacks CLI.
9. [Fabric Lakehouse Deployment](./fabric_deployment_guide.md) - Steps to implement a Microsoft Fabric Lakehouse over the data platform.
Loading

0 comments on commit 94341ce

Please sign in to comment.