Mount common JS config files into /assets/_jsconfig and add these filenames to the JS environment #7656

bep · 2020-09-10T06:42:14Z

This issue talks about using PostCSS with the TailwindCSS plugin in different situations in Hugo, from the main project, from the theme in /themes, from modules, from vendored modules ... But this is also a general issue which is mostly relevant to these JS tools living outside of Hugo.

I created this local "test project" to illustrate the problem for myself:

├── mod
│   ├── babel.config.js
│   ├── package-lock.json
│   ├── package.json
│   ├── postcss.config.js
│   └── tailwind.config.js
└── project
    ├── out.css
    ├── package-lock.json
    ├── package.json
    ├── run.sh
    └── styles.css

So, when running postcss from project with the configuration living in mod, I can do:

postcss  --config ../mod/postcss.config.js -o out.css styles.css;

This is in essence what Hugo does today, and it breaks in, at least, the Tailwind setup above. There are some other "gotchas" to the current situation (one being vendored modules), but I'm not going into details about those.

The main problem with the above is that it fails to find the TailwindCSS configuration. So I could do:

module.exports = {
        plugins: [ require('tailwindcss')('/Users/bep/mod/tailwind.config.js') ]
};

Which works, but how do I, running in Hugo, know where to find that file?

I have thought long and hard about this, and jumping to my conclusion, I thought we could do something ala this:

console.warn('In postcss.config.js, root.');

let tailwindConfig = process.env.HUGO_FILE_TAILWIND_CONFIG_JS || './tailwind.config.js';

module.exports = {
	plugins: [ require('tailwindcss')(tailwindConfig) ]
};

We allow mounting of single files in the root of a project/module. For the project I think we can mount the *.config.js files by default (if no other root files are mounted)
We add these filenames to the OS (and Node) environment mapping them to their absolute filename, normalized on the form tailwind.config.js => HUGO_FILE_TAILWIND_CONFIG_JS. In the example above, we could be even more specific doing:

let tailwindConfig = process.env.HUGO_MY_TAILWIND_CONFIG_JS || './tailwind.config.js';

Thoughts? Any better ideas?

/cc @regisphilibert @onedrawingperday @digitalcraftsman @jmooring @coliff and gang.

Also see #7635 and some others ....

The text was updated successfully, but these errors were encountered:

Fixes gohugoio#7644 Fixes gohugoio#7656

richtera · 2020-09-11T16:32:20Z

Question: is this using ESBuild or the normal JS pipeline. Some of this seems to work in ESBuild which is why I am asking.

bep · 2020-09-11T16:45:35Z

@richtera Not relevant for ESBuild. Main issue I want to solve here is PostCSS/Tailwind living in a Hugo Module.

What I would have loved to solve on the ESBuild side is to allow some custom "import resolver", so Hugo could have a way to say react => /some/path/react etc. This is how we handle imports that live outside of the current tree in the SCSS (using LIbSass) function. That concept works pretty fantastic, where you can mount different SCSS/SASS libraries into the same tree structure.

ESBuild have some internal abstractions that would enable this, it's just a matter of convincing the author that it's a fantastic idea, and seeing how busy he has been, I have not wanted to push that idea, yet ...

richtera · 2020-09-12T02:11:01Z

I think I was able to actually get this to work. I am using two theme modules layered and I was able to put a jsconfig.json on each layer to optional search for js files in the other theme layer. Although it's a bit tricky to setup.
NOTE: Add an identical matching entry to just "react" only if "react/index.js" is not the right file to import by default.

{
  "compilerOptions": {
     "paths": {
         "react/*": ["/some/path/react/*"],
         "react": ["/some/path/react/main"]
     }
  }
}

This should work although it was not quite obvious where to put the jsconfig.json file. I put it parallel to the file I used as source for the resource.Get path. I did find that if I had the file in theme/*/assets/js and there was a assets/js folder it would no longer find the file even though there was no assets/js/index.js file. So the ESBuild will assume an index.js file exists as long as it finds the folder. But that was a minor issue for me.

Fixes gohugoio#7644 Fixes gohugoio#7656

@assets

Map @assets to search through root and theme layer assets. Map @js to search through root and theme layer assets for a js folder with the file. Read original ts/js config and amend settings to temporary tsconfig file. Ref gohugoio#7656

richtera · 2020-09-12T13:58:23Z

Although the js.Build PR solves the resolution of js files within the js.Build transform in the pipeline, it would need to dynamically render the js from assets in order to inject the dependency into the postcss pipeline, so it's related. I am not sure how easy it would be to launch the postcss transform from the output of a js.Build pipeline.

Fixes gohugoio#7644 Fixes gohugoio#7656

This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into: ``` assets/_jsconfig ´`` These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename. But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific. Fixes gohugoio#7644 Fixes gohugoio#7656

bep · 2020-09-12T16:17:36Z

@richtera I have modified the description of this issue somewhat, and I don't think this and ESBuild overlaps. This is about running an external program (PostCSS) and how that program resolves its configuration, there is no "Javascript building" involved.

richtera · 2020-09-12T16:19:15Z

Ok, true. I was just thinking in theory if we were to "build" the tailwinds config using js.Build and then execute the output file from postcss it would resolve the correct paths.

This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into: ``` assets/_jsconfig ´`` These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename. But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific. Fixes gohugoio#7644 Fixes gohugoio#7656

bep · 2020-09-12T17:38:15Z

OK, so my initial testing of this seems to have been flawed, so I got to get back to the drawing board. These JS/Node path issues are fairly frustrating.

bep · 2020-09-12T17:58:46Z

OK, I think I figured it out ... And that was about to ruin my Saturday evening ... Now, a glass of wine.

bep · 2020-09-12T18:01:31Z

@richtera oh, and your PR/research around jsconfig vs ESBuild looks very promising, I will look into it in a couple of days.

richtera · 2020-09-12T18:28:56Z

Cool, thanks. I have been using it locally and if I find anything I'll amend the PR. Have a great night!

richtera · 2020-09-12T20:33:16Z

I just thought of a crazy idea. why not setup NODE_PATH and PATH environment variable inside postcss.go to be something like.

PATH=./node_modules/.bin:./themes/THEMEMOD1/node_modules/.bin:./themes/THEMEMOD2/node_modules/.bin:$PATH
NODE_PATH=./node_modules:./themes/THEMEMOD1/node_modules:./themes/THEMEMOD2/node_modules

Sounds crazy, but in theory, it should do what is required for postcss to find the main modules. You'd then only need to resolve the config file by looking up the same paths. Hmmm.

bep · 2020-09-12T20:43:25Z

So, the "I think I got it" comment above is rooted in setting the NODE_PATH (to at least) include the main dir (e.g. the Hugo project).
One could think that one could expand that idea to the different submodules, and that would probably work if you could (or wanted) do a npm install in all of them (or they had node_modules in Git)
For the problem in the previous bullet I have made a hugo mod npm pack command that makes a top level package.json -- which should also little "in the face" what you bring in from the JS world.

bep · 2020-09-12T21:53:23Z

... also, the NODE_PATH does not solve the primary problem in this issue: That Tailwind resolves its config file relatively to where it lives.

richtera · 2020-09-13T01:14:32Z

I like the hugo mod idea to install modules, but if those modules are pulled automatically then I think npm install should be run automatically if there is a package.json within them. It should use yarn is there is a yarn.lock or npm if there is a package-lock.json. It's a bit tricky for sure and this would cause things to be a little fragile. There would be a lot of moving parts and if there was babel going on as well, then this is a lot of things. Hmmm.

This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into: ``` assets/_jsconfig ´`` These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename. But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific. Fixes gohugoio#7644 Fixes gohugoio#7656

This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into: ``` assets/_jsconfig ´`` These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename. But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific. This commit also adds adds `workDir/node_modules` to NODE_PATH if it exists and `HUGO_WORKDIR` to the env when running the JS tools above. Fixes gohugoio#7644 Fixes gohugoio#7656 Fixes gohugoio#7675

This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into: ``` assets/_jsconfig ´`` These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename. But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific. This commit also adds adds `workDir/node_modules` to `NODE_PATH` and `HUGO_WORKDIR` to the env when running the JS tools above. Fixes gohugoio#7644 Fixes gohugoio#7656 Fixes gohugoio#7675

bep · 2020-09-13T15:22:19Z

I like the hugo mod idea to install modules, but if those modules are pulled automatically then I think npm install should be run automatically

So, I'm complicating this further at this moment. What I add now works seamlessly with Netlify and other build tools. In your local setup you will currently still have to do a npm install.

This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into: ``` assets/_jsconfig ´`` These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename. But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific. This commit also adds adds `workDir/node_modules` to `NODE_PATH` and `HUGO_WORKDIR` to the env when running the JS tools above. Fixes gohugoio#7644 Fixes gohugoio#7656 Fixes gohugoio#7675

This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into: ``` assets/_jsconfig ´`` These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename. But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific. This commit also adds adds `workDir/node_modules` to `NODE_PATH` and `HUGO_WORKDIR` to the env when running the JS tools above. Fixes #7644 Fixes #7656 Fixes #7675

@assets

Map @assets to search through root and theme layer assets. Map @js to search through root and theme layer assets for a js folder with the file. Read original ts/js config and amend settings to temporary tsconfig file. Ref gohugoio#7656

@assets

Map @assets to search through root and theme layer assets. Map @js to search through root and theme layer assets for a js folder with the file. Read original ts/js config and amend settings to temporary tsconfig file. Ref gohugoio#7656

github-actions · 2022-01-28T01:50:30Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

bep added the Proposal label Sep 10, 2020

bep added this to the v0.75 milestone Sep 10, 2020

bep self-assigned this Sep 10, 2020

bep added a commit to bep/hugo that referenced this issue Sep 11, 2020

Add "hugo mod npm package"

6acd990

Fixes gohugoio#7644 Fixes gohugoio#7656

bep mentioned this issue Sep 11, 2020

Add "hugo mod npm package" #7664

Merged

bep added a commit to bep/hugo that referenced this issue Sep 11, 2020

Add "hugo mod npm package"

215226e

Fixes gohugoio#7644 Fixes gohugoio#7656

bep added a commit to bep/hugo that referenced this issue Sep 11, 2020

Add "hugo mod npm package"

b782549

Fixes gohugoio#7644 Fixes gohugoio#7656

bep added a commit to bep/hugo that referenced this issue Sep 12, 2020

Add "hugo mod npm package"

7c2cc22

Fixes gohugoio#7644 Fixes gohugoio#7656

richtera mentioned this issue Sep 12, 2020

js.Build: Add support to js.Build to automatically generate tsconfig files to search assets and js folders #7670

Closed

bep added a commit to bep/hugo that referenced this issue Sep 12, 2020

Add "hugo mod npm package"

1f80e85

Fixes gohugoio#7644 Fixes gohugoio#7656

bep added Enhancement and removed Proposal labels Sep 12, 2020

bep changed the title ~~Add a root filesystem (files only) and add these filenames to the JS environment~~ Mount common JS config files into /assets/_jsconfig and add these filenames to the JS environment Sep 12, 2020

richtera mentioned this issue Sep 13, 2020

postcss: Mockup simple test to setup NODE_PATH and PATH #7672

Closed

bep mentioned this issue Sep 13, 2020

Add workDir/node_modules to NODE_PATH if it exists #7675

Closed

bep closed this as completed in #7664 Sep 13, 2020

fmotrifork mentioned this issue Sep 14, 2020

hugo 0.75.0 fishworks/fish-food#948

Merged

github-actions bot added the Outdated label Jan 28, 2022

github-actions bot locked as resolved and limited conversation to collaborators Jan 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mount common JS config files into /assets/_jsconfig and add these filenames to the JS environment #7656

Mount common JS config files into /assets/_jsconfig and add these filenames to the JS environment #7656

bep commented Sep 10, 2020 •

edited

Loading

richtera commented Sep 11, 2020

bep commented Sep 11, 2020 •

edited

Loading

richtera commented Sep 12, 2020

richtera commented Sep 12, 2020

bep commented Sep 12, 2020

richtera commented Sep 12, 2020

bep commented Sep 12, 2020

bep commented Sep 12, 2020

bep commented Sep 12, 2020

richtera commented Sep 12, 2020

richtera commented Sep 12, 2020

bep commented Sep 12, 2020

bep commented Sep 12, 2020

richtera commented Sep 13, 2020

bep commented Sep 13, 2020

github-actions bot commented Jan 28, 2022

Mount common JS config files into /assets/_jsconfig and add these filenames to the JS environment #7656

Mount common JS config files into /assets/_jsconfig and add these filenames to the JS environment #7656

Comments

bep commented Sep 10, 2020 • edited Loading

richtera commented Sep 11, 2020

bep commented Sep 11, 2020 • edited Loading

richtera commented Sep 12, 2020

richtera commented Sep 12, 2020

bep commented Sep 12, 2020

richtera commented Sep 12, 2020

bep commented Sep 12, 2020

bep commented Sep 12, 2020

bep commented Sep 12, 2020

richtera commented Sep 12, 2020

richtera commented Sep 12, 2020

bep commented Sep 12, 2020

bep commented Sep 12, 2020

richtera commented Sep 13, 2020

bep commented Sep 13, 2020

github-actions bot commented Jan 28, 2022

bep commented Sep 10, 2020 •

edited

Loading

bep commented Sep 11, 2020 •

edited

Loading