Revamp onboarding docs

docs/getting_started/creating_a_flyte_project.md

@@ -0,0 +1,74 @@
+---
+jupytext:
+  formats: md:myst
+  text_representation:
+    extension: .md
+    format_name: myst
+---
+
+# Creating a Flyte project
+
+## About Flyte projects
+
+A Flyte project is a directory containing task and workflow code, internal Python source code, configuration files, and other artifacts structured according to software engineering best practices that enables you to package up your code so that it can be registered to a Flyte cluster.
+
+## Prerequisites
+
+* Follow the steps in {doc}`"Installing development tools" <installing_development_tools>`
+* Install [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+
+## Steps
+
+### 1. Activate your Python virtual environment
+
+If you are using conda or another Python virtual environment manager, first, activate the virtual environment you will use to manage dependencies for your Flyte project:
+
+```{prompt} bash $
+conda activate flyte-example
+```
+
+### 2. Initialize your Flyte project
+
+Next, initialize your Flyte project. The [flytekit-python-template GitHub repository](https://github.com/flyteorg/flytekit-python-template) contains Flyte project templates with sample code that you can run as is or modify to suit your needs.
+
+In this example, we will initialize the [basic-example-imagespec project template](https://github.com/flyteorg/flytekit-python-template/tree/main/basic-example-imagespec).
+
+```{prompt} bash $
+pyflyte init my_project
+```
+
+### 3. Install additional requirements
+
+After initializing your Flyte project, you will need to install requirements listed in `requirements.txt`:
+
+```{prompt} bash $
+pip install -r requirements.txt
+```
+
+### 4. (Optional) Version your Flyte project with git
+
+We highly recommend putting your Flyte project code under version control. To do so, initialize a git repository in the Flyte project directory:
+
+```{prompt} bash $
+cd my_project
+git init
+```
+
+```{note}
+If you are using a Dockerfile instead of ImageSpec, you will need to initialize a git repository and create at least one commit, since the commit hash is used to tag the image when it is built.
+```
+
+### 5. Run your workflow in a local Python environment
+
+To check that your Flyte project was set up correctly, run the workflow in a local Python environment:
+
+```{prompt} bash $
+cd my_project/workflows
+pyflyte run example.py wf
+```
+
+## Next steps
+
+To learn about the parts of a Flyte project, including tasks and workflows, see {doc}`"Flyte project components" <flyte_project_components>`.
+
+To run the workflow in your Flyte project in a local Flyte cluster, see the {ref}`"Running a workflow in a local cluster" <getting_started_running_workflow_local_cluster>` section of {doc}`"Running a workflow locally" <running_a_workflow_locally>`.

docs/getting_started/flyte_fundamentals.md

@@ -1,7 +1,8 @@
 ---
-# override the toc-determined page navigation order
-prev-page: index
-prev-page-title: Getting Started
+prev-page: getting_started/creating_and_running_a_flyte_launch_plan
+prev-page-title: Creating and running a Flyte Launch Plan
+next-page: getting_started/tasks_and_workflows
+next-page-title: Tasks, Workflows and LaunchPlans
 ---
 
 (getting_started_fundamentals)=
@@ -24,14 +25,11 @@ use cases.
 * - {doc}`🔀 Tasks, Workflows and LaunchPlans <tasks_and_workflows>`
   - Create tasks as building blocks, compose them into workflows, and schedule
     them with launchplans.
-* - {doc}`✨ Creating a Flyte Project <creating_flyte_project>`
-  - Build a Flyte project from scratch and learn about the recommended project
-    structure.
 * - {doc}`🗄 Registering Workflows <package_register>`
   - Develop and deploy workflows to a local Flyte demo cluster.
 * - {doc}`⏱ Running and Scheduling Workflows <run_schedule>`
   - Execute workflows programmatically and schedule them as cron jobs.
-* - {doc}`📊 Visualizing Artifacts <visualizing_artifacts>`
+* - {doc}`📊 Visualizing Task Input and Output <visualizing_task_input_and_output>`
   - Create rich, customizable static reports for increased visibility into tasks.
 * - {doc}`🏎 Optimizing Tasks <optimizing_tasks>`
   - Make tasks scalable, performant, and robust to unexpected failures.
@@ -52,10 +50,9 @@ cluster, see the {ref}`Deployment Guide <deployment>`.
 :hidden:
 
 tasks_and_workflows
-creating_flyte_project
 package_register
 run_schedule
-visualizing_artifacts
+visualizing_task_input_and_output
 optimizing_tasks
 extending_flyte
 ```

docs/getting_started/flyte_project_components.md

@@ -0,0 +1,90 @@
+---
+jupytext:
+  formats: md:myst
+  text_representation:
+    extension: .md
+    format_name: myst
+---
+
+# Flyte project components
+
+A Flyte project is a directory containing task and workflow code, internal Python source code, configuration files, and other artifacts needed to package up your code so that it can be run on a Flyte cluster.
+
+## Directory structure
+
+If you examine the project you created with `pyflyte init` in {doc}`"Creating a Flyte project" <creating_a_flyte_project>`, you'll see the following directory structure:
+
+```{code-block} bash
+my_project
+├── LICENSE
+├── README.md
+├── requirements.txt  # Python dependencies
+└── workflows
+    ├── __init__.py
+    └── example.py    # Example Flyte workflow code
+```
+
+## Configuration files
+
+Flyte projects contain a `requirements.txt` file that you can modify to suit the needs of your project.
+
+(getting_started_python_dependencies)=
+
+### `requirements.txt` Python dependencies
+
+You can specify pip-installable Python dependencies in your project by adding them to the
+`requirements.txt` file.
+
+```{note}
+We recommend using [pip-compile](https://pip-tools.readthedocs.io/en/latest/) to
+manage your project's Python requirements.
+```
+
+````{dropdown} See requirements.txt
+
+```{rli} https://raw.githubusercontent.com/flyteorg/flytekit-python-template/main/simple-example/%7B%7Bcookiecutter.project_name%7D%7D/requirements.txt
+:caption: requirements.txt
+```
+
+````
+
+(getting_started_workflow_code)=
+
+## Workflow code
+
+By default, Flyte projects contain a `workflows` directory, inside of which is a Python file that holds the workflow code for the application.
+
+The workflow code contains one or more task and workflow functions, decorated with the `@task` and `@workflow` decorators, respectively.
+
+* The @task and @workflow decorators can be parsed by Python provided that they are used only on functions at the top-level scope of the module.
+* Task and workflow function signatures must be type-annotated with Python type hints.
+* Tasks and workflows can be invoked like regular Python methods, and even imported and used in other Python modules or scripts.
+* Task and workflow functions must be invoked with keyword arguments.
+
+```{note}
+The workflow directory also contains an `__init__.py` file to indicate that the workflow code is part of a Python package. For more information, see the [Python documentation](https://docs.python.org/3/reference/import.html#regular-packages).
+```
+
+### @task decorator
+
+The @task decorator indicates a Python function that defines a task.
+
+* A task is a Python function that takes some inputs and produces an output.
+* Tasks are assembled into workflows.
+* When deployed to a Flyte cluster, each task runs in its own [Kubernetes Pod](https://kubernetes.io/docs/concepts/workloads/pods/), where Flyte orchestrates what task runs at what time in the context of a workflow.
+
+### @workflow decorator
+
+The @workflow decorator indicates a function-esque construct that defines a workflow.
+
+* Workflows specify the flow of data between tasks, and the dependencies between tasks.
+* A workflow appears to be a Python function but is actually a [domain-specific language (DSL)](https://en.wikipedia.org/wiki/Domain-specific_language) that only supports a subset of Python syntax and semantics.
+* When deployed to a Flyte cluster, the workflow function is "compiled" to construct the directed acyclic graph (DAG) of tasks, defining the order of execution of task pods and the data flow dependencies between them.
+
+### ImageSpec
+
+The workflow code in the basic example also includes an optional ImageSpec configuration. ImageSpec is a Flyte feature that enables you to build a custom container image without having to write a Dockerfile. To learn more, see the [ImageSpec documentation](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/customizing_dependencies/image_spec.html#image-spec-example)
+
+```{note}
+If you need to use a Dockerfile instead of ImageSpec, you will need to add a Dockerfile and a `docker_build.sh` script to the top-level directory of your project, and either remove any ImageSpec configurations from the workflow code file or leave them commented out.
+```

docs/getting_started/getting_started_with_workflow_development.md

@@ -0,0 +1,34 @@
+---
+# override the toc-determined page navigation order
+prev-page: getting_started/quickstart_guide
+prev-page-title: Quickstart guide
+---
+
+(getting_started_workflow_development)=
+# Getting started with workflow development
+
+Machine learning engineers, data engineers, and data analysts represent the processes that consume, transform, and output data with directed acyclic graphs (DAGs). In this section, you will learn how to create a Flyte project to contain the workflow code that implements your DAG, as well as the configuration files needed to package the code to run on a local or remote Flyte cluster.
+
+```{list-table}
+:header-rows: 0
+:widths: 20 30
+
+* - {doc}`Installing development tools <installing_development_tools>`
+  - Install the tools needed to create Flyte projects and run workflows and tasks.
+* - {doc}`Creating a Flyte project <creating_a_flyte_project>`
+  - Create a Flyte project that contains workflow code and essential configuration files.
+* - {doc}`Flyte project components <flyte_project_components>`
+  - Understand the directory structure, configuration files, and code in a Flyte project.
+* - {doc}`Running a workflow locally <running_a_workflow_locally>`
+  - Execute a workflow in a local Python environment or in a local Flyte cluster.
+```
+
+```{toctree}
+:maxdepth: -1
+:hidden:
+
+installing_development_tools
+creating_a_flyte_project
+flyte_project_components
+running_a_workflow_locally
+```