Bundler Story: Babel plugin?

[mAAdhaTTah]

Right now, we don't have a very good "story" for bundlers (webpack, browserify, et al.). We're mostly sending people to the download page on prismjs.com itself, which can easily be served through a script tag or concatenated w/ your project's JS. In Node, we've implemented loadLanguages, which handles this in Node.

However, there are a number of issues related to bundler usage that have popped up, as making this work is a bit tedious. Currently, you have to import / require prismjs, then import / require each of the languages, as well as their dependencies, in the correct order, in order to make the resulting file compile and run correctly.

I think we may be able to solve this with a babel plugin. Most people using a bundler are using babel as well. The babel plugin would allow you to import / require Prism like you normally would, and the babel plugin will rewrite that import / require to also import all the languages / plugins you want to include.

So you'd configure it somewhat like this:

// .babelrc
{
  "plugins": [
    ["prismjs", {
      "languages": ["handlebars"]
      "plugins": ["line-numbers"],
      "theme": "twilight"
    }]
  ]
}

and in your bundle, you could just write:

import Prism from 'prismjs'

...and it'll rewrite it thus:

import Prism from 'prismjs'
import 'prismjs/themes/prism-twilight.css'
import 'prismjs/components/prism-markup.js'
import 'prismjs/components/prism-markup-templating.js'
import 'prismjs/components/prism-handlebars.js'

...allowing the bundler to properly compile Prism into your bundle with its correct dependencies and such. Obviously, the above assumes your bundler handles .css files; omitting theme would avoid injecting the css file.

I wanted to open this issue to get some feedback on whether this approach seems reasonable to those of your using a bundler, and what other options we may need and problems to solve with this.

Comments / thoughts / suggestions?

[mAAdhaTTah]

This has been created and published here: https://www.npmjs.com/package/babel-plugin-prismjs

Leaving this issue open to remind me to add docs to prismjs. If the babel plugin works, we'll migrate it over to the PrismJS organization.

[Golmote]

Yay! 🎉 Nice job!

[adi518]

I spent couple hours figuring out Prism will not work fully without this plugin. How does it differ from importing Prism and its plugins? I mean considering I couldn't get it to load anything but the default languages until I used this plugin it's obviously different, but how? it seems odd to have to use a Babel plugin for this package to work. Babel is used for transforming new spec code to old spec, but Prism isn't doing anything 'new'. I'd love to know the reasoning/motivation for this. Btw, it's worth documenting that any of the extra themes cannot be loaded from the plugin (you have to manually import its CSS). I also think it should be part of the Prism organization.

[mAAdhaTTah]

Simply put, it doesn't. You can manually import Prism and it's plugins. The complexity is some languages and plugins have dependencies on other languages and plugins. If you were to import things manually, you have to make sure those dependencies are imported in the correct order.

To make this easier, you can instead define the languages and plugins you want to use in the babel plugin configuration, and the plugin will ensure everything is imported in the right order.

[adi518]

Right, that explains it. However, it means Prism cannot be shipped as a package. For example, a React component that depends on a markdown parser and Prism. Consumers will always have to deal with this configuration. I'm guessing it's because Prism is not written with modules? I mean, is it possible to just write an ES module that imports all dependencies instead of a Babel plugin? otherwise, it sounds like Prism will have to go through a refactor to actually become bundleable.

Edit: Actually no, I'm wrong. You can build a package that bundles Prism with everything. The real issue is just how Prism is offered as a standalone package.

[mAAdhaTTah]

Consumers will always have to deal with this configuration.

Not necessarily. If the React component package runs babel over its code, to import just the Prism components it needs, then the React consumer wouldn't need to worry about it.

This is going to be a complexity to any syntax highlighter, because you don't want to bundle all of the potential languages if you're not going to use them, so all of them need some mechanism for this.

[adi518]

Yes, that corresponds with my edit in the previous comment.

Why is it specific to syntax highlighters? similar to Lodash, I'd expect to have named exports I can import and plug into the core package before using it. I'm merely guessing it's a bit legacy and wasn't built around modules. Refactoring that would probably be too painful, haha.

[mAAdhaTTah]

Part of it is every syntax highlighter will need a way of importing & registering extensions to support languages without loading all of them (so how do you do this without overburdening the user?), but part of it Prism's design philosophy. Prism works by allowing a user to just drop a script tag into their site, add their HTML for the code snippet, and have it just work, without configuring anything in JS. We provide a download page you can build it easily and drop it into your site, and you can choose which languages you need there and download the script & CSS.

No matter what we build on top of that, we need to enable that "plug-and-play" use case for non-developers. We provide an autoloader plugin, if you want to decide the language(s) at runtime, or the babel plugin for those who want it at build time, so we have ways of making it work, but not at the expense of that basic philsophy.

And yes, we do support IE11, so we can't refactor to modern syntax generally. See #1578.

[LeaVerou]

I know this thread is closed, but one idea is to use ES6 modules and rollup. Rollup works client-side as well, so our download page could use it.
However, this rewrite would be a fair amount of work...

[RunDevelopment]

Rollup works via static import declarations but these don't support the optional dependencies we have between files. How do we make that work?

[LeaVerou]

Client side, we can generate the static import declarations!

[danielepolencic]

I'd consider helping if the fair amount of work is the only issue.

I use Prismjs server-side, and bundling the js isn't straightforward.