[Prototype] Migrate docs from Hugo to Astro 🚀

[julien-deramond]

Introduction

This PR is a Work In Progress to migrate our documentation from Hugo to Astro, see if it's worth it and gather feedback from the team regarding such a potential migration.

A preview of the current state of the migration deployed by Netlify is available here.

This PR is co-authored by @HiDeoo.

Why?

Please note that the various points below are not exhaustive and are mostly based on the current implementation and usage of Hugo in Bootstrap, and the same goes for Astro.

Hugo's limitations

Hardcoded version number

The current version number of Bootstrap is still hardcoded a lot in the current documentation website in various places:

• In the config.yml file.
• In some content frontmatter and in the content itself.
• In some folder names.

Bumping the version number requires to change all these places manually, which can be error-prone and time-consuming. In this migration attempt, we took the opportunity to refactor this and use a single source of truth for the version number, which is defined in the config.yml file and is used everywhere else, even to generate dynamic routes like /docs/5.3/getting-started/introduction/. This means that bumping the version number in the documentation website will only require to change a single file.

Extensibility

According to the discussion that happened when switching to Hugo and this open Bootstrap issue, a lot of back and forth had to happen between Bootstrap team and the Hugo main developer to make the website work as expected. Even today, it still feels like there are some places where we're limited to what Hugo can do and we lack some flexibility, one example coming to mind is the way the ToC is generated. This discussion, even if the issue wasn't finally a real issue, shows that we are really dependent on the Hugo implementation and choices.

The fact that Hugo is written in Go makes it a bit difficult to contribute to it if we ever have to, especially considering the languages mostly used by the Bootstrap community. It also means that we're limited to what the Hugo team is willing to ship in its binary (we are relying on hugo-bin for this) and we can't really extend it to our needs easily.

Astro is written in TypeScript/JavaScript and is a lot more flexible in this regard. It is also a lot more extensible and this migration attempt has shown this. In order to reduce the number of changes, we tried to stick to the current way of doing things as much as possible, even if this means not 100% doing things the Astro way and using all the features it provides. Even with that restriction, we were able to implement ourselves all the features without needing to contribute to Astro itself, which is a good sign. It even goes as far as controlling how Markdown is parsed and rendered.

Documentation

This is only a personal opinion but I'm often lost in the documentation where I don't always understand how the features are really working and how to finally exploit all of Hugo's possibilities and power with autonomy. It feels like you need to know exactly what you are looking for to properly find it in the documentation.

Dynamic routes

While working on a pull request to add a glossary for all our classes, I stumbled upon a limitation of Hugo regarding generating dynamic routes. Building upon this new glossary feature, I want to add pages in the documentation to preview the changes between 2 versions of Bootstrap.

Using a new process, I generate a list of classes shipped in each version of Bootstrap and then use these lists to generate diffs between 2 versions. Using all these data, I want to generate various routes to access these diffs, e.g. /docs/5.3/changes/5.0.2...5.3.0/. According to this issue, it is currently not possible in Hugo.

Workarounds exist, but they are not really satisfying as they require using another process to dynamically create pages at build time in the correct folder and with proper naming, for Hugo to be able to pick them up and remove them at the end of the build process.

Astro provides a way to generate any dynamic routes directly through TypeScript/JavaScript which is a lot more flexible and easier to use.

Syntax

Hugo uses Go Templates, provides a primer documentation about it, and also links to the Go documentation for more details. This last link can be overwhelming for someone who doesn't know Go and is not familiar with the syntax.

The Go templating syntax definitely has a learning curve, can be very confusing and hard to read for many people at first sight which can make it hard to contribute to the documentation website, e.g. the following syntax examples:

{{ else if and (eq .Layout "single") .Content }}

{{ template "header" (dict "Section" . "CurrentPage" $.CurrentPage) }}

Another point that can be annoying with Go Templates is when rendering plain text (e.g. an attribute value) using ranges, conditions, etc. Due to the fact that, unlike HTML, white spaces are significant in text, it can be difficult to make templates that both outputs the correct format and are easy to read. For example, the following code:

{{ if and conditional1 conditional2 }}
  value1
{{ else }}
  value2
{{ end }}

will output:

    value1

To get value1 without the leading and trailing white spaces, we either need to inline the condition, which can make the code hard to read if there are multiple conditions, or use the different {{- and -}} syntax, which is not very intuitive and can be hard to remember.

Astro's limitations/disadvantages

Astro's cons

Build time

Hugo is blazingly fast. There is no denying that. Astro is not as fast as Hugo, but we are still far from being slow and close to the build time we had when using Jekyll. This basic benchmark proves that for a production build of the entire website:

Although Astro is not as fast as Hugo, it is important to note that the process used in this migration is doing a lot more things than the current ones. All the logic is written in a type-safe language (TypeScript) so this requires more work, and a lot of effort has been put into adding more confidence to not ship a broken website by validating and typing a lot of data.

For example, all the configuration from config.yml is validated before building the website, e.g. ensuring a field containing a URL actually has a valid URL, that a version number is semver compliant, etc. The same goes for the data but also frontmatters of all pages, e.g. if a documentation page expects to have a frontmatter field title which is a string, it will fail to build if it's not the case or it is missing.

Syntax

Even if the syntax is most of the time simpler than Hugo's, it is still a new syntax to learn for the Bootstrap team and there is a learning curve. Astro component syntax is a superset of HTML and is meant to feel familiar to anyone with experience writing HTML or JSX. Regarding content, this migration uses MDX (.mdx) files which bring added features like support for JavaScript expressions and components in Markdown content.

{{< callout danger >}}
{{< partial "callouts/danger-async-methods.md" >}}
{{< /callout >}}

<Callout name="danger-async-methods" type="danger" />

Altho, in the effort to reduce changes with the current website and stick to some concept used with Hugo, some bits can definitely feels more complex and verbose. These cases can probably be simplified and refactored in the future if we decide to go with Astro and adopt a more Astro way of doing things but this was not the goal of this migration attempt.

{{< example >}}

<ul class="list-group">
  <li class="list-group-item">A simple default list group item</li>
{{< list.inline >}}
{{- range (index $.Site.Data "theme-colors") }}
  <li class="list-group-item list-group-item-{{ .name }}">A simple {{ .name }} list group item</li>
{{- end -}}
{{< /list.inline >}}
</ul>
{{< /example >}}

<Example code={[
  `<div class="list-group">
  <a href="#" class="list-group-item list-group-item-action">A simple default list group item</a>
  `,
  ...getData('theme-colors').map((themeColor) => `  <a href="#" class="list-group-item list-group-item-action list-group-item-${themeColor.name}">A simple ${themeColor.name} list group item</a>`),
  `</div>`
]} />

To support some of these concepts, we also had to extend Astro, and even tho it was fairly easy, it is some extra code to maintain.

TypeScript

The logic and code are written in TypeScript, which has benefits but also some drawbacks. It is a bit more verbose than JavaScript, can feel a bit more complex to write, and adds some overhead to the build process. The choice to use TypeScript was made to give more confidence and reduce the risk of shipping a broken website and also increase the developer experience, especially with auto-completion or inline documentation in code editors but it should be noted that this is not a requirement for Astro and it can be used with JavaScript as well so we could decide to switch to JavaScript if we feel like it's too much of a burden.

Astro's pros

Some of the pros of Astro have already been mentioned in the previous section but here are some more.

Language/platform/ecosystem

Astro is written in TypeScript/JavaScript running on Node.js which is close to the tools and languages used by Bootstrap. The Bootstrap community is also way more familiar with JavaScript than Go if we ever need to deep dive into the codebase of the build process to either fix a bug or add a feature.

This also means that any library published on npm can be used in Astro, which can be a huge advantage when it comes to adding more complex features to the website. Markdown processing is built around Remark and Rehype which have a huge ecosystem of plugins and tools to extend the Markdown syntax and the HTML output.

Community

Astro is still a young project but it is growing fast and has a very active community. The Discord server is very active and the core team is very responsive and helpful. By relying on a more modern and trendy tool, we might be able to attract new contributors to the project, and also seems like a good way to communicate that Bootstrap is compatible with modern tools and technologies.

Potential enhancements

This pull request is already huge and as stated in the beginning, the goal was to migrate the website to Astro without changing the current website too much by porting some concepts from Hugo to Astro. Using Astro and its components-based approach, there should be a lot of things that should be easier to do later on, like:

• More easily avoid some code duplication, e.g. this draft PR to add a StackBlitz edit button for examples.
• Easily embed demos of Bootstrap usage with different modern UI frameworks, e.g. Alpine.js, Lit, Preact, React, SolidJS, Svelte, Vue, etc. using Astro's official integrations using the concept of Astro Islands to render in isolation interactive UI component on an otherwise static page of HTML.
• Display a better and more accessible tree for the content of Bootstrap by hooking into the markdown processing and adding a custom component to render the tree.

How to test

To test this PR locally, after checking out the branch, running npm install && npm run astro-dev should start a local development server on http://localhost:3000 with the new website.

To test a production version locally, run npm run astro-build && npm run astro-preview and point your browser to http://localhost:3000.

Online preview is also available for this PR on Netlify which is the hosting solution currently used by Bootstrap for PRs.

GitHub Pages preview is not provided here to avoid dealing with the setup of GitHub pages on a fork and setting up a custom base href which would be different from the one used in the main repo. The easiest way is to run npm run astro-build and copy the content of site-new/dist like we would do it for a release.

Implementation details

Check implementations details

Folder structure

Everything related to the new website is in the site-new directory at the moment. This is temporary and should be renamed if we decide to go with Astro. Here is a list of the most important directories:

• site-new/src contains the source of the website that will be built by Astro.
• site-new/src/assets contains assets that are handled by Astro, e.g. bundled, minified, etc. like some JavaScript code used by the website, examples and their associated assets, etc.
• site-new/src/content contains Astro Content Collections representing various content of the website that can either be Markdown or MDX files.
  • site-new/src/content/callouts contains all the reusable callouts used in the documentation.
  • site-new/src/content/docs contains the documentation content.
• site-new/static contains files that are copied as-is to the final build directory.

Configuration and data

In order to simplify the migration, the config.yml file has been kept around but simplified. It now only contains configuration related to Bootstrap (and not to the technology used by the website, e.g. Hugo or Astro which is configured in site-new/astro.config.ts), can be edited without any chance to break the website, and everything in it is accessible, validated and type-safe.

Data files have also been kept around and are now validated against a schema to ensure all required values are present and all values are of the correct type.

Configuration and data values are easily accessible from Markdown files (even frontmatter) or TypeScript files with a syntax similar to Hugo's {{ .Site.Params.foo }}:

---
extra_js:
  - src: "[[docsref:/assets/js/validate-forms.js]]"
    async: true
---

See something wrong or out of date here? Please [open an issue on GitHub]([[config:repo]]/issues/new/choose).

getConfig().params.cdn.css_rtl_hash;
getData("breakpoints");

Having everything type-safe and validated also means that we get auto-completion or inline documentation in code editors, e.g. in VS Code:

Layouts

Layouts are Astro components used to provide a reusable UI structure, such as a page template. Located in site-new/src/layouts, they mimic the current Hugo layouts:

• BaseLayout.astro is the base layout for all pages.
• DocsLayout.astro is the layout for all documentation pages.
• ExamplesLayout.astro is the layout for all examples pages.
• RedirectLayout.astro is the layout used to redirect to another page.
• SingleLayout.astro is the layout used for pages that don't need a specific layout.

Shortcodes

Still in the idea of not introducing too many changes at once, the concept of shortcodes has been kept around. They are now Astro components (everything is pretty much components in Astro) located in site-new/src/components/shortcodes. They are automatically loaded by Astro and can be used in Markdown files like this:

## CSS

### Variables

<AddedIn version="5.2.0" />

As part of Bootstrap's evolving CSS variables approach, accordions now use local CSS variables on `.accordion` for enhanced real-time customization. Values for the CSS variables are set via Sass, so Sass customization is still supported, too.

Setup

Editor

Astro works with any code editor. The team is maintaining an official Astro VS Code Extension that unlocks several key features and developer experience improvements for Astro projects like syntax highlight, TypeScript type information, and Intellisense for .astro files.

Prettier

Prettier, a popular formatter for JavaScript, HTML, CSS, and more, is used in the new website for Astro components and TypeScript files to ensure a consistent code style but it should be noted that this is a choice and totally optional as Astro doesn't require it.

If using the official Astro VS Code Extension or the Astro language server within another editor, code formatting with Prettier is included out of the box.

Linting

The linting process for the new website is currently split into various parts:

• Prettier formatting in relevant files is checked (this is only a check, nothing is written to the files automatically).
• astro check that runs Astro-related diagnostics (such as type-checking within .astro files).
• We run the TypeScript compiler (without emitting any files) to check for any type-related errors.

ESLint is not yet part of this combo yet but will be easily added in the future for the documentation website but also for code snippets like it is done in the current website.

Deployments

Regarding deployments, there are 2 different targets to take into account: Netlify for previews deployments and GitHub Pages for the main website.

Netlify

This PR is already deployed to Netlify and a preview is available here.

Regarding settings, only 2 settings need to be defined in the Netlify UI:

The publish directory is where we build the new website, it is site-new/dist at the moment.

The build command is the command to build the new website, it is npm run astro-netlify at the moment and mimics what the netlify npm script was doing.

GitHub Pages

Running npm run astro-build will create site-new/dist.

Some files are generated which are not in Hugo _site directory but are present in gh-pages at the root level:

• 404.html
• index.html
• robots.txt
• sitemap-index.xml + sitemap-0.xml (sitemap.xml in gh-pages)

The other files are the same as Hugo _site:

• apple-touch-icon.png
• CNAME
• favicon.ico
• sw.js

An extra _astro directory is created.

All the "redirection" directories and their index.html exist with the same content:

• about
• components
• examples
• getting-started
• migration

docs/5.3 contains the same structure of folders:

• about
• assets
• components
• content
• customize
• dist
• examples
• forms
• getting-started
• helpers
• index.html
• layout
• migration
• utilities

The main difference we can observe is the loading of _astro/*.js that will handle the assets which is done differently in Hugo.

Some discrepancies can be observed in the HTML files but don't change the behavior.

Closing remarks

Surprisingly, it only took a few weeks with a few hours here and there to completely rewrite all the documentation in Astro.
Some tricky things were not planned but we managed to quickly find a workaround each time without having to create an issue or a discussion upstream. Autonomy is the first word that comes to my mind.

Even after a few years working on Bootstrap (and Orange fork), I still find Hugo's syntax challenging and sometimes even magical (see twbs#38308). In a few days, I felt more confident with Astro's syntax, closer to what web developers are used to.

Types definitely provide a safety net and give a lot more confidence in not shipping a broken website.

I can't guarantee it but I got the feeling that we might have more contributors by having an Astro-based documentation. Moreover, a lot of plugins are already in place and we could benefit from a lot of them in the future to have a more modern documentation without too much effort.

I really liked this experience with Astro.

It is also important to note that even tho this PR might not be merged, it was a good exercise to learn more about Astro and see how it could be used to build the Bootstrap website. It also helped us to identify some issues in the current website that we will or were able to fix and contribute to the project.

Boostrap upstream fixes

• Fix code indentation in homepage JavaScript plugins section twbs/bootstrap#38112
• Docs: fix highlight border radius twbs/bootstrap#38114
• Docs: fix highlight border radius twbs/bootstrap#38114 (comment)
• Drop unused bootstrap-social-logo.png twbs/bootstrap#38164
• It's possible to remove metaDataFormat from the config, not used anymore since 2019: Remove metaDataFormat setting gohugoio/hugoDocs#908
• The group frontmatter property could be entirely removed from all pages.
• Fix placeholder shortcode color and background params when 'img' markup twbs/bootstrap#38263