Enhancing Content Collection Structure in Astro

[scriptogre]

Body

Issue Description

Background

In my Astro project, I'm developing a course structure with chapters and lessons, each having its own metadata. I'm aiming to organize the course content in a way that logically groups lessons under their respective chapters, with both chapters and lessons maintaining individual metadata. However, Astro's current content collection system poses challenges in achieving this clear, logical, and maintainable folder structure.

Attempted Solutions & Challenges

1. (Current) Splitting Content into Separate Collections:

   • Structure: Course content divided into two collections: course_chapters (YAML) and course_lessons (Markdown):

     content
     ├── course_chapters  // Data collection (.yaml)
     │   ├── introduction.yaml
     │   └── foundations.yaml
     ├── course_lessons  // Content collection (.md)
     │   ├── introduction
     │   │   ├── overview.md
     │   │   └── objectives.md
     │   └── foundations
     │       ├── basics.md
     │       └── principles.md
     └── blog
     

   • Config Setup:

     import { z, defineCollection } from "astro:content";
     
     const courseChapters = defineCollection({
         type: 'data',
         schema: z.object({
             title: z.string(),
             sortOrder: z.number()
         })
     });
     
     const courseLessons = defineCollection({
         type: 'content',
         schema: z.object({
             title: z.string(),
             sortOrder: z.number(),
             chapter: z.string()
         })
     });
     
     export const collections = { 
       course_chapters: courseChapters,
       course_lessons: courseLessons
     };

   • Challenges: Leads to a fragmented organization, separating related chapters and lessons.

2. (Proposal) Nesting Collections under Parent Folder:

   • Structure: Course content divided into two collections: chapters (YAML) and lessons (Markdown) under a parent course folder (which is NOT a collection):

     src
     ├── content
     │   ├── course
     │   │   ├── chapters
     │   │   │   ├── introduction.yaml
     │   │   │   └── foundations.yaml
     │   │   └── lessons
     │   │       ├── introduction
     │   │       │   ├── overview.md
     │   │       │   └── objectives.md
     │   │       └── foundations
     │   │           ├── basics.md
     │   │           └── principles.md
     │   └── blog
     

   • Config Setup:

     import { z, defineCollection } from "astro:content";
     
     const courseChapters = defineCollection({
         type: 'data',
         schema: z.object({
             title: z.string(),
             sortOrder: z.number(),
         })
     });
     
     const courseLessons = defineCollection({
         type: 'content',
         schema: z.object({
             title: z.string(),
             sortOrder: z.number(),
             chapter: z.string(),
         })
     });
     
     export const collections = { 
       'course/chapters': courseChapters,
       'course/lessons': courseLessons
     };

   • Challenges: Although chapters and lessons are still separated, they are unified under a parent folder.
   • Limitation: Current Astro setup does not support this nested collection structure, as collections are expected to be placed in the root content folder.

3. (Hypothetical) Unified Mixed Collection:

   • Concept: Combining courseChapters and courseLessons under a mixed collection, maintaining separate schemas but unified under one umbrella.
   • Structure:

     content
      └── course
          ├── introduction
          │   ├── what_is_this.md  // 'introduction' chapter lesson
          │   ├── who_is_it_for.md  // 'introduction' chapter lesson
          │   └── meta.yaml  // Contains 'introduction' chapter's metadata
          └── foundations
              ├── getting_started.md  // 'foundations' chapter lesson
              ├── core_concepts.md  // 'foundations' chapter lesson
              └── meta.yaml  // Contains 'foundations' chapter's metadata
     

   • Config Setup (Hypothetical):

     import { z, defineCollection } from "astro:content";
     
     // Define the schema for chapters (data collection)
     const ChapterSchema = z.object({
         title: z.string(),
         sortOrder: z.number()
     });
     
     // Define the schema for lessons (content collection)
     const LessonSchema = z.object({
         title: z.string(),
         sortOrder: z.number(),
         chapter: z.string() // Linking to a chapter
     });
     
     // Hypothetical mixed content collection definition
     const courseContent = defineCollection({
         type: 'mixed', // Hypothetical mixed content type
         schemas: {
             chapter: {
                 type: 'data',
                 schema: ChapterSchema
             },
             lesson: {
                 type: 'content',
                 schema: LessonSchema
             }
         }
     });
     
     // Specifying the path in the export statement
     export const collections = { 
       course: courseContent // 'content' refers to the directory path
     };

   • Concerns: While this approach might provide a more cohesive structure, I have reservations about its feasibility and potential complexity. It's not clear if this would be the best approach, and I am curious about what the Astro community thinks.

Conclusion

These enhancements would greatly benefit projects requiring complex content organization and provide more versatility in managing diverse content types in Astro. I am eager to hear the community's thoughts on these proposals, especially the third option. Thank you for considering this feature request.

[bholmesdev]

Thanks for sharing this detailed example! We're exploring deeper changes to content collections right now to break out of the "top-level directory" requirement. Noted

[scriptogre]

That's great to hear! Can't wait to see what solution you guys come up with :)

[danielbayley80]

This would be great. I'm migrating from a Hugo site and I have a structure with

/en/collection-1/collection-2/
/fr/collection-1/collection-2/

Having my translations like this is tidier and I have multi level collections.

It would be nice just to set the collection in the frontmatter, then I can almost ignore the folder structure. Given I can set the layout, it would make sense that I could set the collection and inherit the layout from this.

[bholmesdev]

Totally. @danielbayley80 we've now opened content collections to accept globs for any pattern you choose. Check out the 5.0 Beta to try the new Content Layer and give us feedback! https://5-0-0-beta.docs.astro.build/en/guides/content-collections/

[danielbayley80]

Awesome! This looks like a really nice way of doing things. I'll dig a little deeper when I get time.

[bholmesdev]

Thanks for sharing this @scriptogre! Happy to share we overhauled collection configs for the new Content Layer. We introduced "loaders" to unlock total flexibility over where your content lives. We include a glob() utility out-of-the-box to describe nested folder structures, which maps pretty well to the course > chapters > lessons schema you shared here. You can also store content outside of the src/content/ directory to simplify further.

I created an example project showing how this might work. I will note that the generated id for chapters will include the /about suffix since I used about.yaml files to track course information. If this isn't to your liking, you could opt for a different naming structure or implement a custom loader to define IDs yourself.

Check out the example on Stackblitz

View the repository

I'm also marking this discussion as "closed." If you have any feedback for us, feel free to share on discord or start a new discussion. Appreciate the solutions everyone!