Autogenerated sidebar item from subfolder don't respect index.md at the subfolder root

[fabriziocucci]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Prerequisites

• I'm using the latest version of Docusaurus.
• I have tried the npm run clear or yarn clear command.
• I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
• I have tried creating a repro with https://new.docusaurus.io.
• I have read the console error message carefully (if applicable).

Description

It seems that autogenerated items for subfolders containing an index.md file don't show up as sections.

For instance, on the sample website from the the 5m tutorial, I would have expected this:

const sidebars = {
  tutorialSidebar: [
    "intro",
    {type: 'autogenerated', dirName: 'tutorial-basics'},
    {type: 'autogenerated', dirName: 'tutorial-extras'},
  ],
}

to create two sections in the sidebar because they both contain a _category_.json file but everything is actually flattened at the root. Replacing both _category_.json with an index.md leads to the same results.

Is this expected?

Reproducible demo

CodeSandbox

Steps to reproduce

1. npm init docusaurus@latest my-website classic
2. Update sidebars.js from:

tutorialSidebar: [{type: 'autogenerated', dirName: '.'}],

to

const sidebars = {
  tutorialSidebar: [
    "intro",
    {type: 'autogenerated', dirName: 'tutorial-basics'},
    {type: 'autogenerated', dirName: 'tutorial-extras'},
  ],
}

Expected behavior

I would expect two sections to appear in the sidebar since both subfolders contain a _category_.json.

Actual behavior

The content of the two sections is actually flattened at the root.

NOTE: replacing the _category_.json files with an index.md files leads to the same results.

Your environment

• Docusaurus version used: 2.4.1

Self-service

• I'd be willing to fix this bug myself.

[caramelmelmel]

Try this configuration:https://docusaurus.io/docs/sidebar/autogenerated you can't simply replace with an index.md. you have to autogenerate using the docs dirname here in docusaurus

[fabriziocucci]

@caramelmelmel I'm using that configuration, check out the Steps to reproduce section or the sandbox.

More generally, assuming this structure:

docs/
  foo/
    index.md
    Bar.md
    Baz.md

I would expect this:

const sidebars = {
  mySidebar: [
    {type: 'autogenerated', dirName: 'foo'},
  ],
}

to translate in:

Foo ↓
  Bar
  Baz

because an index.md exists in the folder foo for which we are requesting to autogenerate the sidebar items.

[caramelmelmel]

no you cannot assume that index.md would directly translate to index.html here as docusaurus already automatically maps to docs in its route. Here's how you can do it
in sidebars.js:

/** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */
const sidebars = {
  // By default, Docusaurus generates a sidebar from the docs folder structure
  tutorialSidebar: [{type: 'autogenerated', dirName: '.'}],
};

module.exports = sidebars;

Create this folder structure:

You will get:

[caramelmelmel]

if you want to manipulate the keys you cannot just touch the docusaurus.config.js there are other repositories that use the component page routing from the config to perform something like this. Let me know if I did not answer your question. You have to have the _category_.json for this

[fabriziocucci]

Thanks @caramelmelmel, I think the use case I was trying to come across is not entirely clear, let me expand a bit.

Imagine you want to have some part of the sidebar being autogenerated and some part being manually managed:

docs/
  manual-doc/
    index.md
    manual1.md
    manual2.md
  auto-doc/
    index.md
    auto1.md
    auto2.md

The corresponding sidebar config I would think it should suffice is:

const sidebars = {
  tutorialSidebar: [
    {
      label: "Manual Doc",
      link: {type: 'doc', id: 'manual-doc/index'},
      type: "category",
      items: [ ... ],
    },
    {
      dirName: 'auto-doc',
      type: 'autogenerated',
    }
  ],
};

but that would lead to:

Manual Doc ↓
  Manual 1
  Manual 2
Auto Doc
Auto 1
Auto 2

reather than:

Manual Doc ↓
  Manual 1
  Manual 2
Auto Doc ↓
  Auto 1
  Auto 2

---

Taking your example, this:

const sidebars = {
  tutorialSidebar: [{type: 'autogenerated', dirName: '.'}],
};

would still not work as I would expect for:

docs/
  index.md  <-- this index will NOT generate a section
  foo/
    index.md <-- this index will generate a section
    Bar.md
    Baz.md

leading to:

Docs Index
Foo Index ↓
  Bar
  Baz

rather than:

Docs Index ↓
  Foo Index ↓
    Bar
    Baz

I guess, regardless by the underlying implementation, I don't fully understand why the index.md in a folder for which we are requesting an autogenerated sidebar should be treated differently from any other index.md in the descendant folders.

[caramelmelmel]

now I understand your use case here is to create more than one layer of nested structure of folders am I right for a single side bar for tutorial?

Something like this:

[caramelmelmel]

but as far as I know the auto and manual configurations cannot mix together as auto means I take from the folder itself based on the mapped Id for that one folder.

[caramelmelmel]

Instead, this could not be a bug. You might want to ask about this in Discord.

[fabriziocucci]

Posted on Discord but didn't have much traction.

My biggest concern still stands so I'll leave this open:

I guess, regardless by the underlying implementation, I don't fully understand why the index.md in a folder for which we are requesting an autogenerated sidebar should be treated differently from any other index.md in the descendant folders.

I believe this ultimately makes mixing manual and autogenerated sidebars harder than it should be.

[Josh-Cena]

You generate an entire sidebar from a folder; you don't just generate one category. index.md within a folder becomes the category index page only if the folder becomes a category, but the root folder doesn't become a category—it becomes a "sidebar slice".

[fabriziocucci]

Thanks @Josh-Cena!

...but the root folder doesn't become a category—it becomes a "sidebar slice".

Out curiosity, would there be any downside to make the root folder become a category if there is an index.md inside it and just be a "sidebar slice" if not? I've the feeling this would improve the overall composability but I'm pretty sure if it wasn't designed to work this way, there is some gotcha I'm not considering.

[Josh-Cena]

I think a new sidebar item called autogenerated-category sounds good. @slorber WDYT?

[caramelmelmel]

@Josh-Cena Just to clarify, I think an autogenerated-category works by enabling a subfolder in docs to become an auto-generated category if an index.md is added? If I get you correctly? else, I would really want to work on this issue as I face the same problem of having index.md not appearing as a subfolder as I would like on my sidebar

[slorber]

@fabriziocucci the current behavior is expected. The autogenerated item generates a sidebar slice and not a category.

const sidebars = {
  tutorialSidebar: [
    "intro",
    {type: 'autogenerated', dirName: 'tutorial-basics'},
    {type: 'autogenerated', dirName: 'tutorial-extras'},
  ],
}

This config is not supposed to use ./tutorial-basics/index.md and infers that it should create a tutorial-basics category. You requested to generate a slice from this folder and this is exactly what you got.

Similarly if you to:

const sidebars = {
  tutorialSidebar: [
    {type: 'autogenerated', dirName: '.'},
  ],
}

Then we don't use a ./docs/index.md, to create a Docs Root category or whatever, so do we want to add magic custom logic for only the subfolder case? I don't know.

---

What you really want is something like this:

const sidebars = {
  tutorialSidebar: [
    "intro",
    {
      type: "category",
      label: "Tutorial basics",
      link: {type: 'doc', id: 'tutorial-basics/index'},
      items: [{type: 'autogenerated', dirName: 'tutorial-basics'},],
    },
    {
      type: "category",
      label: "Tutorial Extras",
      link: {type: 'doc', id: 'tutorial-extras/index'},
      items: [{type: 'autogenerated', dirName: 'tutorial-extras'},],
    }
  ],
}

And yes I admit this is not as straightforward as what you thought would work, but it's a generic/low-level/flexible way to support many patterns without too much magic.

---

Note in #5689 we would already like to support include/exclude attributes like this:

const sidebars = {
  tutorialSidebar: [
    "intro",
    {type: 'autogenerated', dirName: '.', include: ["tutorial-basics", "tutorial-extras"]},
  ],
}

Or this:

const sidebars = {
  tutorialSidebar: [
    "intro",
    {type: 'autogenerated', dirName: '.', include: ["tutorial-basics"]},
    {type: 'autogenerated', dirName: '.', include: ["tutorial-extras"]},
  ],
}

So if we support this, the behavior you want becomes less needed. We can keep something based on the concept of "sidebar slice" (without magic nor edge-case exceptions), and still have a concise API.

---

I think a new sidebar item called autogenerated-category sounds good. @slorber WDYT?

@Josh-Cena I would prefer to first introduce include and see if we really need this afterward, as it would only be a "shortcut" to the above somehow.

[caramelmelmel]

@slorber what I really think about including index.md in the subfolder is that the user is able to customise the introduction page however he or she likes instead of using a category.json so by providing the /index.md functionality, and then autogenerating the sub bars in a sidebar, based on the presence of this index.md helps to provide this functionality of customisation here.

[Josh-Cena]

Neither the presence of index.md nor _category_.json can affect whether a folder becomes a category or a sidebar slice—that would be a huge behavior inconsistency. These files are picked up when and only when the folder is actually destined to become a category.

[Josh-Cena]

I don't think there's anything wrong with Docusaurus' behavior—I agree with @slorber that we can try #5689 first, although I think autogenerated-category will still be very intuitive and convenient, but that would be another feature request :)

[fabriziocucci]

Thanks @slorber and @Josh-Cena for the comments.

@slorber, regarding:

What you really want is something like this:

const sidebars = {
  tutorialSidebar: [
    "intro",
    {
      type: "category",
      label: "Tutorial basics",
      link: {type: 'doc', id: 'tutorial-basics/index'},
      items: [{type: 'autogenerated', dirName: 'tutorial-basics'},],
    },
    {
      type: "category",
      label: "Tutorial Extras",
      link: {type: 'doc', id: 'tutorial-extras/index'},
      items: [{type: 'autogenerated', dirName: 'tutorial-extras'},],
    }
  ],
}

I was going to say, that's not gonna fully work without a way to exclude tutorial-basics/index from the auto-generated items. I remember we tried this and, besides not being straightforward as you also pointed out, leads to a duplication of the index file that will then appear as both a category and a document inside the category.

Being able to exclude files like in #5689 will definitely help, looking forward to it!

@Josh-Cena, regarding:

Neither the presence of index.md nor category.json can affect whether a folder becomes a category or a sidebar slice—that would be a huge behavior inconsistency.

For some reason, from a user prospective without any clue about the implementation details, I had exactly this expectation! I agree that autogenerated-category would be more intuitive and convenient, not sure about the gain of having it separate from the already existing autogenerated type when the mere convention of adding an index.md could do the trick (similarly to the category index convention). Regardless, I'd love to see autogenerated-category come to life.

I appreciate you guys taking the time to even evaluate this, keep up the good work! 🙂

[slorber]

I was going to say, that's not gonna fully work without a way to exclude tutorial-basics/index from the auto-generated items. I remember we tried this and, besides not being straightforward as you also pointed out, leads to a duplication of the index file that will then appear as both a category and a document inside the category.

    {
      type: "category",
      label: "Tutorial basics",
      link: {type: 'doc', id: 'tutorial-basics/index'},
      items: [{type: 'autogenerated', dirName: 'tutorial-basics'},],
    },

As far as I remember (not 100% sure) we have logic so that if it's already a category link, index is excluded from the sidebar slice to prevent this.

---

We'll add the include/exclude feature first and eventually see if autogenerated-category is needed.

not sure about the gain of having it separate from the already existing autogenerated type when the mere convention of adding an index.md could do the trick

If we implement this as a convention, then it becomes impossible to generate a sidebar slice including the index. By implementing your use case we might break another user use case, with no easy workaround to opt-out apart renaming a file that is maybe named "index" on purpose.

I'd rather keep the ability to be able to support both if possible, even if it makes the API slightly less intuitive.

I think your convention could make sense, but it's more like a shortcut and I'd prefer to delay implementing it until we are sure it's really needed.

[rodcloutier]

As far as I remember (not 100% sure) we have logic so that if it's already a category link, index is excluded from the sidebar slice to prevent this.

It actually highlight both links at the same time (category and page) but it does not remove the actual index page from the page list in the category.

[robbieaverill]

Another side effect of this is that when you're viewing the index category page, the pagination's "next" link is the same page. The pagination next link becomes unusable here as it links to itself.

After following all of these threads, I decided to simply do an entire sidebar as autogenerated instead of trying to achieve this. It's too janky without being able to exclude the index file from the autogenerated child items.