Collect and parse external markdown files outside of dir.input?

[jevets]

Not sure if 11ty can do this or if anyone has ideas on a better approach... or if it's a worthy use-case for 11ty.

I have a folder of markdown files that shouldn't be treated as templates; they don't map to resulting .html files. They do have frontmatter, though. I'd like to use them as collections within template files, parsed and ready to go.

For example:

colors/
  blue.md
  red.md
  yellow.md
src/
  index.pug

// colors/blue.md
---
title: Blue
hex: #00f;
---

I'd like to be able to

eleventyConfig.dir = {
  input: 'src',
  output: '_site'
}

eleventyConfig.addCollection('colors', function (collection) {
  return collection.getFilteredByGlob('colors/*.md')
})

So I could

// index.pug
each color in collections.colors
  p= color.data.title
  p= color.data.hex

Which builds to:

_site/
  index.html

One idea is to use a separate task to parse the external markdown files and generate .json to be used as _data, but 11ty can already parse markdown files and frontmatter; would be great to tap into that instead of having to reinvent.

Another idea is to force them into the dir.input then rm -rf the resulting _site/colors dir, but that seems just kinda wrong.

[kleinfreund]

You likely want to use data files. Have a look at the docs. I can't look it up right now because I'm not at a computer.

[jevets]

Thanks @kleinfreund

My use-case is a bit different, though.

The example colors directory comes from an external source (an external git repo) that doesn't belong to my 11ty project.

It seems that 11ty only parses markdown files if they exist within the dir.input tree.

I'm wondering if there's a way to get 11ty to read and parse external markdown files (outside of config dir.input) and provide them for use as collections to actual 11ty template files — the markdown files themselves aren't actual templates and don't map to built .html files.

Sounds like I may have to create a separate task that reads the markdown files and generates .json files for use as global data...

[kleinfreund]

You should move the data you require to a location where Eleventy can work with it. Eleventy being able to touch anything outside of its root directory would be catastrophic. I certainly don’t want it to attempt building a static site out of my whole repo directory.

Anyway, if you’re keen on having Eleventy process data from outside its directory, you can try working with JS data files and pull in the data with Node.js to set dynamic collections.

[zachleat]

Option 1

Are you open to the idea of using input: '.', but stopping templates in the colors directory from being included in the output?

If so, there are feature requests related to this: #61, #188

Option 2

Are you only using front matter in these markdown files? If so, it sounds like you might be asking for the option to use markdown front matter as an option for data files, which I think makes sense to me: https://www.11ty.io/docs/data-global/ (currently we only support .js and .json)

If so, you could implement something yourself today (without waiting on our feature queue) using lodash.merge and gray-matter with JavaScript Data Files, I think: https://www.11ty.io/docs/data-js/

[jevets]

Thanks @zachleat

I see value in both options above. I have another separate case where permalink: false would be very useful. (In this particular "colors" case, I can't modify the color markdown files, as they come from another repo entirely.)

I'm going to take take a stab at implementing directly using js data files and gray-matter. Should be simple enough.

I think allowing markdown as data files is also a great idea. Might be worth supporting .yaml/.yml files, too? But .js data files w/ some custom parsing are probably sufficient for these kinds of cases.

[zachleat]

@jevets note that when #61 is implemented you should be able to use a Directory specific data file (https://www.11ty.io/docs/data-template-dir/) to create a colors/colors.11tydata.json file that could apply {"permalink": false} to all templates in the directory.

I personally think #61 has very high value at this point so it’ll probably be implemented soon.

One more note: if you’d like additional data types to be supported in data files, please file a enhancement request so that it can go into the voting queue!

Thanks!