[Epic] Maintain and generate provider installation instructions in-provider

[guineveresaenger]

This effort has two major parts:

1. Keep installation_configuration.md and index.md files in-provider. As of today this is mainly a migration from pulumi/registry to $provider/docs.

2. Consume upstream upstream index.md file to $provider/docs
   a) Transform the content to match Pulumi standards.

• Migrate pulumi-owned package installation docs to live in provider repos registry#4740
• Emit upstream provider overview to $provider/docs folder #2082
• Upstream installation instructions: Docs Edits and Transformations #2078 (this is a tracking issue with a checklist of necessary transformations).

[guineveresaenger]

Update, 9/30:

During the rollout, we gathered important information on exactly which transformations different providers will need for this landing page to build well and be usable.

Overall, it's been a bit slower to roll out, because there are so many potentially different situations presenting in upstream docs - but the good news is that for the most part, these situations can be generalized and applied globally in the bridge.

The following themes were discovered and implemented in addition to the sub-issues in #2078:

• Translation and display of the Pulumi.yaml file to show config examples
  • Translate installation examples #2214
• Docs edits transforming TF verbiage to Pulumi language
  • Schema header translates to Configuration Reference #2350
  • Add more regex filters to installation docsgen #2421
  • Additional replacements for installation docs #2431
• Ordering of docs edits
  • Add option to filter edit rules by location in docs conversion pipeline and apply for installation docs #2435
• Addition of an end-to-end docs transformation test for the bridge
  • Add option to filter edit rules by location in docs conversion pipeline and apply for installation docs #2435
• Moving toward AST based Markdown parsing
  • fix: strip upstream frontmatter #2361
  • Skip sections by header with goldmark #2292
• Multiple small docs edits to smooth along the process for providers more globally
  • Add option to filter edit rules by location in docs conversion pipeline and apply for installation docs #2435
  • Add more regex filters to installation docsgen #2421
  • Schema header translates to Configuration Reference #2350
  • Add more section skipper defaults #2342
  • Make regex link matching more strict #2344
• Strip TF version references
  • Add default patterns to strip TF version references and tests #2349
• The section skipping functionality is implemented as an exported docsEdit function that providers can use for one-off section skipping.
• All new work is under test, and missing tests were implemented for some of the existing functionality.
• We have rolled out 2/3 of lower-tier providers, and believe most of the remainder of the rollout will be simplified by the work listed in this comment.

We also encountered a few challenges with our Markdown renderer:

• Ensure Markdown renderer preserves newline whitespace #2420
• Ensure all sections are marked for skipping by ast.Walk #2335
• Add a goldmark parser extension for first class sections #2306

Overall, the outcome here is optimistic for future streamlining of docs transformations, making automation more robust and usable. The things we learned may ultimately benefit the registry docs in general as well -see #2251 and #2248.

[guineveresaenger]

This issue is complete.

[pulumi-bot]

Cannot close issue:

• does not have required labels: resolution/

Please fix these problems and try again.