[Docs] Monodocs build - a build system to consolidate Flyte docs

[cosmicBboy]

Description

Since 1.10.0, the majority of the Go components of Flyte have been moved to the flyte monorepo (see release notes).

In the spirit of this effort, we want to do something similar for the Flyte documentation. Currently, the Flyte docs are composed of 5 different static docs sites for: flyte, flytesnacks, flytekit, flyteidl, and flytectl. Now that flyteidl has been merged into the flyte repo, the purpose of this task is to create a simple and transparent sphinx build system that consolidates all of the docs sites into a single static website.

The benefits of this are:

• Simplicity of having a single static website
• Ease of navigation across the different components of Flyte.
• Versioning the docs, anchored to Flyte release

The downsides are:

• Introducing maintenance overhead of the build system
• Making sure CI/CD for flytekit, flytesnacks, and flytectl use the flyte monorepo to make sure PRs don't cause breaking changes.

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[thomasjpfan]

I do not see this as a downside and can be done independently of a mono-repo:

Making sure CI/CD for flytekit, flytesnacks, and flytectl use the flyte monorepo to make sure PRs don't cause breaking changes.

This is typically done by having a nightly job that builds against the upstream dependency. Concretely, flytesnacks will have a nightly CI job that builds the docs and runs its test with flyte's master branch. If a job fails, it will open an issue (or update an existing one) that links to the failure.

Looking over #4347, the docs will remain in their individual repos. The proposal is to have flyteorg/flyte pull them all in and build one static site. flyteidl's docs is the only one migrating into this repo. Is that correct?

[cosmicBboy]

The proposal is to have flyteorg/flyte pull them all in and build one static site

Yep!

flyteidl's docs is the only one migrating into this repo. Is that correct?

Correct, flytectl will be migrated into the flyteorg/flyte repo too eventually, not sure what the timeline is for that @eapolinario