mapping workflow, first attempt

[guybedford]

This is a rough draft of a mapping workflow.

The idea is that when I am writing a package, say thispackage, I may want to have custom maps set up within that package that will transfer when it is published.

The way we can handle this is with the package.json map property, which is supported by jspm:

{
  "map": {
    "custom-path": "value"
  }
}

This PR propogates the above map configuration into the configuration file, supporting syncing between the package.json and config file values.

[theefer]

Would that allow overriding any mapping including *, github:*, etc? I'm looking for a way to persist custom mappings (to replace the default that normally come in config.js).

[guybedford]

What type of custom mappings do you mean? Like the github:* mappings, or single path mappings?

If it is the endpoint mappings, is this configuration project-specific? Currently endpoint mappings are global jspm configuration.

[theefer]

I'm talking about both endpoint mappings (github:*, npm:*) and fallback default (*). The problem we have is that the default are based on local filesystem paths, which may not be reflected in how the directories are exposed on the web by the server.

To give a more concrete example, we have a Play app that has the following structure:

.
├── app (Play)
├── package.json
└── public
    ├── config.js
    ├── js
    ├── jspm_packages
    └── stylesheets

with package.json configured to make json write to public:

{
  "jspm": {
    "directories": {
      "jspmPackages": "public/jspm_packages"
    },
    "configFile": "public/config.js",
    "dependencies": { ... }
}

This results in the default public/config.js containing a config like this:

System.config({
  "paths": {
    "*": "*.js",
    "npm:*": "public/jspm_packages/npm/*.js",
    "github:*": "public/jspm_packages/github/*.js"
  }
});

However, our own modules are actually within public/js and the standard in Play is to serve the public directory as /assets online, so the paths don't resolve correctly and we have to override them (currently in the main HTML file after loading config.js):

<script src="@routes.Assets.at("config.js")"></script>
<script>
  System.config({
    baseURL: "/assets",
    paths: {
      "*":        "js/*.js",
      "npm:*":    "jspm_packages/npm/*.js",
      "github:*": "jspm_packages/github/*.js"
     }
  });

  System.import("app");
</script>

So it feels there are at least three things happening here:

• Use of baseURL to make paths simpler
• Override default catchall path (*) to match where the sources really are
• The local filesystem paths may not match the paths exposed by the web server

Have we missed some clever tricks to make this configuration possible within the jspm setup (which would make bundling easier etc)?

It'd be great if jspm didn't have to enforce any particular directory structure and it could be configured to match whatever people used.

Thanks!

[guybedford]

directories.lib can set the default baseURL.

For setting the server base path, perhaps we need a new directories entry for this so that all paths in the config file are relative to this.

How about directories.public?

[theefer]

I found the doc for directories.lib but I'm not sure what it's supposed to do or what I'm meant to set it to in the example above. Would it be "js" (the directory prefix for the * default mapping)?

Running jspm install after having set it doesn't seem to cause any change to the config.js file however; is it supported in the current jspm? Some of its uses in config.js seem to be disabled?

If I understand you correctly, the "server base path" would be the baseUrl config, so in the example above directories.public would be "/assets" (or "assets")?

[guybedford]

Ahh, there might have been some changes in this - directories.lib is supposed to correspond to the *.js path actually, effectively being the baseURL. Note jspm init will do the same thing as jspm install in resyncing config.

Then what I mean by directories.public is that say you have:

{
  "jspm": {
    "directories": {
      "jspmPackages": "public/jspm_packages",
      "public": "public",
      "lib": "public/js"
    },
    "configFile": "public/config.js",
    "dependencies": { ... }
}

Then the config.js paths should look like:

System.config({
  paths: {
    '*': 'js/*.js',
    'npm:*': 'jspm_packages/npm/*.js', ...
  }
});

Let me know if that sounds sensible?

[guybedford]

Maybe the paths should even have a leading / so that the current page URL doesn't affect things either?

[jamesgorrie]

I think the requirements are to have a path that is referred to on the filesystem by JSPM and one that is referred to on the web by SystemJS.

{
  "jspm": {
    "baseURL": "/assets",
    "directories": {
      "jspmPackages": "public/jspm_packages",
      "public": "public",
      "lib": "public/js"
    },
    "configFile": "public/config.js",
    "dependencies": { ... }
}

Which then gives you:

System.config({
  baseURL: '/assets'
  paths: {
    '*': 'js/*.js',
    'npm:*': 'jspm_packages/npm/*.js', ...
  }
});

This would also require JSPM to not look at the baseURL when bundling but use directories.public.

In short directories.public is for JSPM and baseURL is for SystemJS.

How's that sound?

[guybedford]

Note that the * path is identical to a baseURL in concept. Why do we need a baseURL as well here?

[theefer]

The baseURL we're talking about is the path to the directory that contains all relative endpoints. Note that it is a path on the web server, not the local filesystem.

So in theory we could remove the need for baseURL by prepending it to all relative endpoints, e.g.

System.config({
  paths: {
    '*': '/assets/js/*.js',
    'npm:*': '/assets/jspm_packages/npm/*.js', ...
  }
});

which let SystemJS resolve paths to https://example.com/assets/js/... for instance, but that of course would break jspm locally as it will try to find files in /assets/js/....

So @jamesgorrie's solution relies on there being a different base path for jspm (directories.public) and SystemJS (baseURL), while the relative paths can be appended to either of these base paths.

Unless we're still missing something?

[theefer]

I wonder if the known working use case for jspm and SystemJS so far is one where both base paths (local and remote) were coincidentally identical?

[guybedford]

Right ok I get you, that makes sense. So the baseURL here be different between development and production? What do you think about providing the baseURL manually?

[theefer]

It's not really the difference between development and production, but more between the local paths and the paths served by the web server.

To reuse the same example, the following local directory structure on the filesystem:

.
├── app (Play)
├── package.json
└── public
    ├── config.js
    ├── js
    |   └── app.js
    ├── jspm_packages
    └── stylesheets

results in all the files in public being served under a /assets prefix online, for instance:

https://example.com/assets/config.js
https://example.com/assets/js/app.js
https://example.com/assets/jspm_packages/[email protected]
... etc ...

jspm works at the level of the local filesystem to install, bundle, etc so as far as it's concerned, the stuff is in ./public (relatively to package.json). That could be configured via directories.public in package.json.

SystemJS works at the level of files served on the web, so the stuff is in /assets (relatively to the domain of the page that invokes it, https://example.com/). That could be configured via baseURL in package.json (or directly in config.js, although I'm a bit reluctant to make people edit this mostly auto-generated file?).

In terms of providing the baseURL, you're right it'd have to be provided manually, just like all the other paths. And we'd also need to make sure jspm doesn't pick it up and uses directories.public instead.

Does that make sense?

[guybedford]

Ok I see. Thanks for explaining. So directories.public should override the baseURL when using SystemJS on the server or bundling. From an isometric perspective I just worry a little about messing with the baseURL in config.js as it means I can't just do:

var System = require('systemjs').System;
require('config');
System.import('x');

And expect that to work, which would be good to maintain.

Perhaps if you set baseURL in the HTML page itself, would that be manageable?

[theefer]

Alright, fair enough on also allowing SystemJS to resolve paths locally. Setting baseURL in the HTML file seems like an acceptable compromise for now.

So maybe let me try to recap what we've arrived to.

First we add directories.lib to point to the directory containing all the modules, and directories.public to point to the base directory.

{
  "jspm": {
    "directories": {
      "jspmPackages": "public/jspm_packages",
      "public": "public",
      "lib": "public/js"
    },
    "configFile": "public/config.js",
    "dependencies": { ... }
}

jspm would then produce the following config.js:

System.config({
  baseURL: 'public' // <- directories.public
  paths: {
    '*': 'js/*.js', // <- directories.lib relative to directories.public (see question below)
    'npm:*': 'jspm_packages/npm/*.js', ... // <- directories.jspmPackages relative to directories.public
  }
});

which should give the correct local paths to jspm (bundle, etc) and SystemJS when used locally, e.g.

var System = require('systemjs').System;
require('config');
System.import('app');

In order to get SystemJS to use the correct paths on the web though, we'd need to override the baseURL in the HTML like so:

<script src="@routes.Assets.at("config.js")"></script>
<script>
  System.config({
    baseURL: "/assets"
  });

  System.import("app");
</script>

We don't need to override the paths because they resolve correctly once we've set the correct baseURL.

A few questions:

• Should directories.jspmPackages, directories.lib and probably also configFile be paths relative to the root (dirname package.json), or to the directories.public? It may feel nice not to have them coupled, but on the other hand, if they were relative to directories.public, all you'd need to do is set directories.public; default values for the others would resolve all paths correctly, rather than having to prefix all other paths with directories.public (which we then need to use as base path to strip it again later).
• Is directories.public the best name, especially if it becomes more of a base directory for all? Would baseURL be more confusing or more logical, seeing as that's what it actually maps to in config.js?

I think we're getting close!

[guybedford]

I wouldn't actually set the baseURL at all, and rather let this be handled externally. The node use case would require setting the cwd to the public dir, or just ensuring the config file is in the same folder (for a Node app I think that is an adequate assumption). So you could actually just set your own baseURL in System config if you like, I just wouldn't recommend it generally.

I like keeping the directories all in the same relative root to the package.json for consistency despite the duplication. Unless you see a strong argument against that?

I'm open to names other than directories.public. directories.base may well be more logical.

[guybedford]

Posted a new issue to track at #151, would be better to keep discussion going there, and discussion here for custom map workflows.

[theefer]

I like keeping the directories all in the same relative root to the package.json for consistency despite the duplication. Unless you see a strong argument against that?

I guess that's fine too!

directories.base does sound more logical to me too.

[guybedford]

This has been included in the latest beta (npm install jspm@beta / npm install [email protected]).