Revamp onboarding docs

docs/getting_started/creating_a_flyte_project.md

@@ -0,0 +1,81 @@
+---
+jupytext:
+  formats: md:myst
+  text_representation:
+    extension: .md
+    format_name: myst
+---
+
+# Creating a Flyte project
+
+## About Flyte projects
+
+A Flyte project is a directory containing task and workflow code, internal Python source code, configuration files, and other artifacts structured according to software engineering best practices that enables you to package up your code so that it can be registered to a Flyte cluster.
+
+## Prerequisites
+
+* Follow the steps in {doc}`"Installing development tools" <installing_development_tools>`
+* (Optional, but recommended) Install [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+
+## Steps
+
+### 1. Activate your Python virtual environment
+
+If you are using conda or another Python virtual environment manager, first, activate the virtual environment you will use to manage dependencies for your Flyte project:
+
+```{prompt} bash $
+conda activate flyte-example
+```
+
+### 2. Initialize your Flyte project
+
+Next, initialize your Flyte project. The [flytekit-python-template GitHub repository](https://github.com/flyteorg/flytekit-python-template) contains Flyte project templates with sample code that you can run as is or modify to suit your needs.
+
+In this example, we will initialize the [basic-example-imagespec project template](https://github.com/flyteorg/flytekit-python-template/tree/main/basic-example-imagespec).
+
+```{prompt} bash $
+pyflyte init my_project
+```
+
+:::{note}
+
+To initialize a Flyte project with a different template, use the `--template` parameter:
+
+`pyflyte init --template hello-world hello-world`
+:::
+
+### 3. Install additional requirements
+
+After initializing your Flyte project, you will need to install requirements listed in `requirements.txt`:
+
+```{prompt} bash $
+cd my_project
+pip install -r requirements.txt
+```
+
+### 4. (Optional) Version your Flyte project with git
+
+We highly recommend putting your Flyte project code under version control. To do so, initialize a git repository in the Flyte project directory:
+
+```{prompt} bash $
+git init
+```
+
+```{note}
+If you are using a Dockerfile instead of ImageSpec, you will need to initialize a git repository and create at least one commit, since the commit hash is used to tag the image when it is built.
+```
+
+### 5. Run your workflow in a local Python environment
+
+To check that your Flyte project was set up correctly, run the workflow in a local Python environment:
+
+```{prompt} bash $
+cd workflows
+pyflyte run example.py wf
+```
+
+## Next steps
+
+To learn about the parts of a Flyte project, including tasks and workflows, see {doc}`"Flyte project components" <flyte_project_components>`.
+
+To run the workflow in your Flyte project in a local Flyte cluster, see the {ref}`"Running a workflow in a local cluster" <getting_started_running_workflow_local_cluster>` section of {doc}`"Running a workflow locally" <running_a_workflow_locally>`.

docs/getting_started/flyte_fundamentals.md

@@ -1,7 +1,8 @@
 ---
-# override the toc-determined page navigation order
-prev-page: index
-prev-page-title: Getting Started
+prev-page: getting_started/creating_and_running_a_flyte_launch_plan
+prev-page-title: Creating and running a Flyte Launch Plan
+next-page: getting_started/tasks_and_workflows
+next-page-title: Tasks, Workflows and LaunchPlans
 ---
 
 (getting_started_fundamentals)=
@@ -24,14 +25,11 @@ use cases.
 * - {doc}`🔀 Tasks, Workflows and LaunchPlans <tasks_and_workflows>`
   - Create tasks as building blocks, compose them into workflows, and schedule
     them with launchplans.
-* - {doc}`✨ Creating a Flyte Project <creating_flyte_project>`
-  - Build a Flyte project from scratch and learn about the recommended project
-    structure.
 * - {doc}`🗄 Registering Workflows <package_register>`
   - Develop and deploy workflows to a local Flyte demo cluster.
 * - {doc}`⏱ Running and Scheduling Workflows <run_schedule>`
   - Execute workflows programmatically and schedule them as cron jobs.
-* - {doc}`📊 Visualizing Artifacts <visualizing_artifacts>`
+* - {doc}`📊 Visualizing Task Input and Output <visualizing_task_input_and_output>`
   - Create rich, customizable static reports for increased visibility into tasks.
 * - {doc}`🏎 Optimizing Tasks <optimizing_tasks>`
   - Make tasks scalable, performant, and robust to unexpected failures.
@@ -52,10 +50,9 @@ cluster, see the {ref}`Deployment Guide <deployment>`.
 :hidden:
 
 tasks_and_workflows
-creating_flyte_project
 package_register
 run_schedule
-visualizing_artifacts
+visualizing_task_input_and_output
 optimizing_tasks
 extending_flyte
 ```

docs/getting_started/flyte_project_components.md

@@ -0,0 +1,118 @@
+---
+jupytext:
+  formats: md:myst
+  text_representation:
+    extension: .md
+    format_name: myst
+---
+
+# Flyte project components
+
+A Flyte project is a directory containing task and workflow code, internal Python source code, configuration files, and other artifacts needed to package up your code so that it can be run on a Flyte cluster.
+
+## Directory structure
+
+If you look at the project you created with `pyflyte init` in {doc}`"Creating a Flyte project" <creating_a_flyte_project>`, you'll see the following directory structure:
+
+```{code-block} bash
+my_project
+├── LICENSE
+├── README.md
+├── requirements.txt  # Python dependencies
+└── workflows
+    ├── __init__.py
+    └── example.py    # Example Flyte workflow code
+```
+
+(getting_started_python_dependencies)=
+
+## `requirements.txt` Python dependencies
+
+Most Flyte projects contain a `requirements.txt` file that you can modify to suit the needs of your project.
+
+You can specify pip-installable Python dependencies in your project by adding them to the
+`requirements.txt` file.
+
+```{note}
+We recommend using [pip-compile](https://pip-tools.readthedocs.io/en/latest/) to
+manage your project's Python requirements.
+```
+
+````{dropdown} See requirements.txt
+
+```{rli} https://raw.githubusercontent.com/flyteorg/flytekit-python-template/main/simple-example/%7B%7Bcookiecutter.project_name%7D%7D/requirements.txt
+:caption: requirements.txt
+```
+
+````
+
+(getting_started_workflow_code)=
+
+## `example.py` workflow code
+
+Flyte projects initialized with `pyflyte init` contain a `workflows` directory, inside of which is a Python file that holds the workflow code for the application.
+
+The workflow code may contain ImageSpec configurations, and one or more task and workflow functions, decorated with the `@task` and `@workflow` decorators, respectively.
+
+```{note}
+The workflow directory also contains an `__init__.py` file to indicate that the workflow code is part of a Python package. For more information, see the [Python documentation](https://docs.python.org/3/reference/import.html#regular-packages).
+```
+
+### ImageSpec
+
+The workflow code file in the basic template includes an optional ImageSpec configuration. ImageSpec is a Flyte feature that enables you to build a custom container image without having to write a Dockerfile. To learn more, see the [ImageSpec documentation](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/customizing_dependencies/image_spec.html#image-spec-example)
+
+```python
+# basic_image = ImageSpec(
+#    name="flytekit",  # rename this to your docker image name
+#    base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2",
+#    # the base image that flytekit will use to build your image
+#    packages=["example-package"],  # packages to add to the base image
+#    # remove "example-package" before using.
+#    registry="ghcr.io/unionai-oss",
+#    # the registry your image will be pushed to
+#    python_version="3.11"
+#    # the python version; optional if not different from the base image
+# )
+```
+
+```{note}
+If you need to use a Dockerfile instead of ImageSpec, you will need to add a Dockerfile and a `docker_build.sh` script to the top-level directory of your project, and either remove any ImageSpec configurations from the workflow code file or leave them commented out.
+```
+
+### The `@task` and `@workflow` decorators
+
+* The `@task` and `@workflow` decorators can be parsed by Python provided that they are used only on functions at the top-level scope of the module.
+* Task and workflow function signatures must be type-annotated with Python type hints.
+* Tasks and workflows can be invoked like regular Python methods, and even imported and used in other Python modules or scripts.
+* Task and workflow functions must be invoked with keyword arguments.
+
+#### `@task`
+
+The `@task` decorator indicates a Python function that defines a task.
+
+* A task is a Python function that takes some inputs and produces an output.
+* Tasks are assembled into workflows.
+* When deployed to a Flyte cluster, each task runs in its own [Kubernetes Pod](https://kubernetes.io/docs/concepts/workloads/pods/), where Flyte orchestrates what task runs at what time in the context of a workflow.
+
+```python
+@task()
+def say_hello(name: str) -> str:
+    return f"Hello, {name}!"
+```
+
+#### `@workflow`
+
+The `@workflow` decorator indicates a function-esque construct that defines a workflow.
+
+* Workflows specify the flow of data between tasks, and the dependencies between tasks.
+* A workflow appears to be a Python function but is actually a [domain-specific language (DSL)](https://en.wikipedia.org/wiki/Domain-specific_language) that only supports a subset of Python syntax and semantics.
+* When deployed to a Flyte cluster, the workflow function is "compiled" to construct the directed acyclic graph (DAG) of tasks, defining the order of execution of task pods and the data flow dependencies between them.
+
+```python
+@workflow
+def wf(name: str = "world") -> typing.Tuple[str, int]:
+    greeting = say_hello(name=name)
+    greeting_len = greeting_length(greeting=greeting)
+    return greeting, greeting_len
+```