Additional language plugin system

[kennethormandy]

Opened at @sintaxi’s request after a conversation with @heath about LiveScript.

Harp needs to have a language system. We’re thinking any preprocessors beyond the ones we’ve already committed to would probably not be added to Harp’s core. Instead, we’d like it to have a language system for anyone to add preprocessors.

I believe there’s an API in mind, but no implementation started yet. It would probably stem from terraform, which is Harp’s asset pipeline.

Potentially still in core:

• .sass polyfill (Support the Sass indented syntax (.sass) #185)
• Enhanced EJS (Include Enhanced EJS Template Plugins #162)
• Handlebars (should support Handlebars #60)
• Yoke

Requested so far:

• Ruby implementation of Sass (Support the Sass indented syntax (.sass) #185)
• MultiMarkdown (As an option for front matter)
• TraceurScript (Support traceur script. terraform#17)
• LiveScript (Add LiveScript support based on its introduction into Terraform (closes ... #111, Add LiveScript support. terraform#10)
• Myth (Support for Myth #199) +1
• Wisp (Support for Wisp (JS pre-processor) #110) +1
• LitCoffee (Serve .litcoffee-files as both html and js.. #156) +2
• OJ (Feature Request: OJ templating support for .oj and .ojc file types #154) +1
• Razor (Add support for @razor templates/syntax #141) +1
• ClojureScript (Add support for ClojureScript #86) +2
• ECO (Add support for ECO #49) +1
• TypeScript (should support TypeScript #61) +3

[rashad612]

+1
In our case we needed Dust.js to be served by Harp. It would be nice to have simple API, so that we can hook any plugin easily.

[trodrigues]

+1
I also think this would be great. I'd like to have JS file concatenation and browserify, and pretty sure these would be fairly easy to implement as plugins. Everyone has different needs, and a pluggable plugin system seems to be the best way to move forward.

[kennethormandy]

@trodrigues This would be to add preprocessors, so I don’t think Browserify or concatenation would necissarly be covered by it unless you wrote a JS preprocessors with those added. That said, we are looking at build-in solutions to both of those in other issues.

[trodrigues]

@kennethormandy I haven't yet looked at how Harp does this internally, but is this pre processing just doing the job of pre processing each file to something else?

I kind of assumed that the whole process of preprocessing and building into one file would all be part of that same task.

I think at the end of the day, what would be very ideal would be to have a plugin system where plugins could be used to put together a pipeline, where maybe with different categories of plugins, these would separately do the work of preprocessing and building.

[kennethormandy]

All that stuff is in Terraform. I think I misunderstood, and we’re on the same page after all. I just wanted to be clear that it wouldn’t necessarily be a plugin system for “anything,” just for the languages in the asset pipeline.

[edrex]

another use case: supporting Ruby SASS via a plugin (libsass is lagging behind in features, and some newer SASS components don't work with it)

[kennethormandy]

It’s on the list :)
That will also be the most reliable way to support the indented syntax, although I’m hoping we can possibly use sass2scss for that with libsass.

Any other thoughts about language system, @edrex?

[torgeir]

👍

[evanmoran]

As the author of OJ (ojjs.org) I'm exited about this plugin idea=). For OJ integration it should be very easy to integrate with a simple compileDirectory or compilePath notion. The OJ node module already can compile directories recursively and will turn .oj files into .html/.css/.js files and output them to a directory of your choosing while preserving the folder structure it found. So literally one .oj file = one web page.

One plus is OJ doesn't care what other files are in the directory. It only notices .oj (javascript) and .ojc (coffeescript) files so it can be done safely in Harp directories as is. So if your system is in node-land just call require("oj").compilePath(...) or if it is a commandline process use the OJ commandline tool like so: oj dir_path. Let me know if you need help=).

[kevinSuttle]

+1 for Dust

[4d47]

What's the status of this issue ?

[kevinsimper]

We need a way to declare what plugins you want to use for this to work.

It could either be global or local to the folder you are working in.

Method one:
the harp.json could contain a property that harp will try to require and put the content through.
Could be made easier by doing $ harp enable sass markdown gulp-imagemin
Benefits:

• declarative
• easy
  Downsides:
• Have to do it everytime

Method two:
A proposed this to @sintaxi, but the idea is that there will be modules like harp-sass, and then harp will look at plugins installed that has harp- as a prefix and then load those. This will make it work global. The plugin would reqister an fileending and a function for the content to be passed though.
Benefits:

• Easy
• Fast
  Downsides:
• Version Incompatibles

[4d47]

May I propose another option, something à la Consolidate.js where all preprocessors integration would still be in Terraform (mapping error to TerraformError etc) but activate only if module is actually installed (eg. you run npm install LiveScript in your project if you want LiveScript).

Benefit would be to work either per project or globally. It would also relief Terraform the need to choose/manage preprocessors versions.

There would still have the need to moderate modules that compete for file extensions (eg. different implementations of the same language) but that would still be an issue with other methods.

[kevinsimper]

@4d47 Seems like the best of those two, I think you are on the right path!

But would it make it a hub that slows down the process? Is there a need for a central source to control what plugins can control the different file endings. I can see a lot of people trying to get their fileending with their favorite module.

npm install harp-sass

harp would look to see if modules listet with harp prefix and then load the plugin that would then register .scss and .sass as fileendings. This way if you do not like harp-sass, you can you make another plugin or even just fork harp-sass and then do a npm install kevinsimper/harp-sass --save and change the plugin to fit my needs.

[Munter]

This project looks like it fits with the goals of https://github.com/jenius/accord

Have you thought about using that?