Manage frontend dependencies with npm/yarn and webpack #1072

ArthurHoaro · 2018-01-31T14:41:04Z

Relates to #755

move default theme assets to /assets/default
asset folders in default template are now git ignored
handle assets dependencies with yarn (and fixed version with yarn.lock)
handle assets compilation with webpack 2

JS:

use ES6 syntax
compile it with babel
minify it using babel-minify-webpack-plugin
initiate splitting JS files into separate modules (base/picwall/[todo])
dependencies are now bundled with Shaarli's JS file (awesomplete and blazy)
Use JS linting

CSS:

move Shaarli's pure CSS to SASS
dependencies are now bundled with Shaarli's CSS file (awesomplete/purecss/fontawesome)
CSS is now minified

Images:

~~compressed at compilation by image-webpack-loader~~

Usage with npm and yarn installed:

yarn install
yarn run build

TODO in this PR:

TODO list for later:

~~JSLint: fix 1k JS style errors and add it to Travis build~~
CSSLint: fix CSS style erroors and add it to Travis build
Split JS code into dedicated module files
JS unit tests
Use SASS features such as variables for colors
???
Profit!

ArthurHoaro · 2018-02-04T13:21:59Z

I don't think I can build the armhf architecture Docker image.

virtualtam · 2018-02-05T17:33:17Z

docker/alpine/Dockerfile.master

@@ -1,4 +1,4 @@
-FROM alpine:3.6
+FROM mhart/alpine-node:9


at this point, it would be worth migrating to a Docker multi-stage build to avoid providing an image embedding development resources (Composer and NodeJS) that are not needed to run the application

I can work on this in a separate issue/PR, as there's a bit of Docker involved in the next release with #1010

Oh! I've never heard of multi-stage build, it looks great! I'll try to play with it in another PR.

I've started to work on this to use the composer image to download the Shaarli archive and resolve PHP deps :)

virtualtam · 2018-02-05T17:37:27Z

webpack.config.js

+
+const extractCssVintage = new ExtractTextPlugin({
+    filename: "../css/[name].css",
+    publicPath: 'tpl/default/css/',


s/default/vintage/

virtualtam · 2018-02-05T17:38:41Z

webpack.config.js

+    {
+        entry: {
+            shaarli: './assets/vintage/js/base.js',
+            picwall: './assets/default/js/picwall.js',


s/default/vintage/ (not sure about this one... are resources from default needed by vintage?)

They are. But maybe I should add a common folder instead?

yes, I think it would make things clearer and help isolating template-specific resources

virtualtam · 2018-02-05T17:48:15Z

assets/default/js/base.js

+import Awesomplete from 'awesomplete';
+
+// CSS
+import '../scss/shaarli.scss';


referring to SASS / assets from JS code should not be necessary; asset copy and generation should be handled by Webpack

virtualtam · 2018-02-05T18:07:58Z

Nice!

This is a great step towards more flexible front-end generation and asset management :)

The diff is quite huge though, maybe we could start with commits dealing with the low hanging fruits? E.g.:

JS cleanup: var, let, const
asset location update

ArthurHoaro · 2018-02-06T21:07:05Z

Thanks for the early review. I only had experience with webpack on POCs and such. I'll probably try to redo everything step by step to get a clean commit log. The global diff will probably be huge anyway. Note that yarn.lock takes 4.6k line of the +5k lines here.

I'll also use real ES6 syntax here and run eslint on travis in this PR.

virtualtam · 2018-02-14T21:46:19Z

Yep, Webpack has an awful lot of dependencies. Do we actually need it for our use case?

It seems very powerful to create full-fledged frontend application bundles (as in Angular/React/Vue + Node + SASS/SCSS), but quite overkill to package a handful of dependencies.

For similar use cases (backend serving static pages with 3rd-party CSS/JS), I went back to using:

npm/yarn to resolve dependencies
npm tasks to copy bundled dependencies from node_modules to the app's asset directories
Gulp tasks to render SCSS/SASS to CSS and minify assets

Ping @nicolasdanelon and @tcitworld who contributed to the discussion in #755 :)

Relates to shaarli#755 Relates to shaarli#1072 See: - https://docs.docker.com/develop/develop-images/multistage-build/ - https://hub.docker.com/r/library/composer/ - https://github.com/composer/docker - https://github.com/docker-library/docs/tree/master/composer Signed-off-by: VirtualTam <[email protected]>

tcitworld · 2018-02-14T22:16:47Z

Yep, Webpack has an awful lot of dependencies. Do we actually need it for our use case?

These are devDependencies, not the project dependencies.

virtualtam · 2018-02-14T23:14:03Z

Our needs here are quite basic:

resolve frontend dependencies
compile SASS to CSS
compile ES6 modules to plain JS
minify and bundle assets

Dependencies (and overhead) are worth keeping to a minimum, so we can easily maintain our build toolchain while avoiding headaches 🤕

ArthurHoaro · 2018-02-15T18:08:18Z

Well, who can do more can do less. I'm sure we can do everything using other tools packaged in a single Makefile target, but once configured webpack does everything easily and has a lot of resources since it has more or less became a standard.

Regarding our build toolchain and dev tools, I would tend to say that the ease of use overcomes their weight.

ArthurHoaro · 2018-02-24T18:00:41Z

Ok I think I'm mostly done here. I've entirely rewritten the commit log to make everything more understandable, even though the JS rewrite diff isn't really readable.

Also I was pleasantly surprise to see that yarn is already installed by default in Travis PHP image.

virtualtam · 2018-02-26T22:22:31Z

.eslintrc.js

+  "rules": {
+    "no-param-reassign": 0, // manipulate DOM style properties
+    "no-restricted-globals": 0, // currently Shaarli uses alert/confirm, could be be improved later
+    "no-alert": 0, // currently Shaarli uses alert/confirm, could be be improved later


time to raise some issues then :)

I'm about to do the same for the TODOs concerning usage of die() and exit() in PHP code

Issue raised in #1095

virtualtam · 2018-02-26T22:27:16Z

.travis.yml

 before_script:
  - PATH=${PATH//:\.\/node_modules\/\.bin/}
 script:
  - make clean
  - make check_permissions
+  - make eslint


frontend lint & tests should be run in a dedicated JS / Node matrix to avoid cluttering the PHP test environments

this will be a bit tricky so it's OK to address this point in a separate PR

virtualtam

Tested on my local development environment, works fine :)

@ArthurHoaro do you want to wait until a PR addressing Docker builds is submitted before merging?

ArthurHoaro · 2018-03-14T17:18:27Z

I think that we should wait, otherwise the master image will be broken.

nodiscc · 2018-03-14T18:25:57Z

package.json

+    "font-awesome": "^4.7.0",
+    "pure-extras": "^1.0.0",
+    "purecss": "^1.0.0"
+  },


Should we setup some automation to warn us about new upstream releases of the dependencies? (or can yarndo that for us?)

Great patch! It adds requirements for yarn+node sources.lists + packages in the build environment, but I can live with that :) Looks definitely more manageable.

https://yarnpkg.com/lang/en/docs/package-json/

https://stackoverflow.com/questions/22343224/whats-the-difference-between-tilde-and-caret-in-package-json#22345808

@nodiscc Github provides such a feature:

https://help.github.com/articles/listing-the-packages-that-a-repository-depends-on/

https://help.github.com/articles/listing-the-projects-that-depend-on-a-repository/

https://help.github.com/articles/viewing-and-updating-vulnerable-dependencies-in-your-repository/

The list of supported languages and tools is growing :)

… to assets/

Relates to shaarli#1072 Changed: - update paths to resource files (assets, images) Removed: - references to resources now resolved through NPM - licenses corresponding to the aforementioned resources Signed-off-by: VirtualTam <[email protected]>

ArthurHoaro added enhancement in progress documentation tools developer tools template HTML rendering 3rd party interoperability with third-party platforms javascript client-side rendering labels Jan 31, 2018

ArthurHoaro added this to the 0.10.0 milestone Jan 31, 2018

ArthurHoaro self-assigned this Jan 31, 2018

virtualtam reviewed Feb 5, 2018

View reviewed changes

virtualtam mentioned this pull request Feb 14, 2018

docker: introduce multi-stage image build (master, latest) #1085

Merged

ArthurHoaro force-pushed the feature/modern-front-end branch from 29a5b47 to d05d751 Compare February 24, 2018 17:50

ArthurHoaro changed the title ~~WIP: Manage frontend dependencies with npm/yarn and webpack~~ Manage frontend dependencies with npm/yarn and webpack Feb 24, 2018

ArthurHoaro added in review and removed in progress labels Feb 24, 2018

ArthurHoaro force-pushed the feature/modern-front-end branch from d05d751 to 003118c Compare February 24, 2018 17:57

ArthurHoaro requested review from virtualtam and nodiscc February 24, 2018 18:02

ArthurHoaro added the please test label Feb 24, 2018

ArthurHoaro force-pushed the feature/modern-front-end branch 2 times, most recently from 7dcf0b5 to 5c7caf4 Compare February 26, 2018 20:23

virtualtam reviewed Feb 26, 2018

View reviewed changes

ArthurHoaro mentioned this pull request Feb 28, 2018

Use bootbox like frontend lib instead of JS alert/confirm? #1095

Open

virtualtam approved these changes Mar 14, 2018

View reviewed changes

ArthurHoaro removed the please test label Mar 14, 2018

nodiscc approved these changes Mar 14, 2018

View reviewed changes

ArthurHoaro mentioned this pull request Mar 26, 2018

Docker: build frontend dependencies with node and yarn #1113

Merged

ArthurHoaro added 5 commits March 28, 2018 19:01

Webpack / Remove frontend dependencies from tpl/ & inc/ and move them…

b3375c7

… to assets/

Webpack / Rewrite all JS to ES6 Syntax

a33c565

Webpack / Update front paths in template files

7e9bd97

Webpack / Configure webpack, ESLint, Travis, Makefile, npm/yarn and git

47978e8

Webpack / Documentation update

d7eb06b

ArthurHoaro force-pushed the feature/modern-front-end branch from 5c7caf4 to d7eb06b Compare March 28, 2018 17:04

ArthurHoaro merged commit c81f1af into shaarli:master Mar 28, 2018

ArthurHoaro mentioned this pull request Mar 31, 2018

Use Travis stages to run JS tests separately #1116

Merged

virtualtam mentioned this pull request Apr 3, 2018

Build front-end dependencies locally #755

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Manage frontend dependencies with npm/yarn and webpack #1072

Manage frontend dependencies with npm/yarn and webpack #1072

ArthurHoaro commented Jan 31, 2018 •

edited by virtualtam

Loading

ArthurHoaro commented Feb 4, 2018

virtualtam Feb 5, 2018

ArthurHoaro Feb 6, 2018

virtualtam Feb 14, 2018

virtualtam Feb 5, 2018

virtualtam Feb 5, 2018

ArthurHoaro Feb 24, 2018

virtualtam Feb 24, 2018

virtualtam Feb 5, 2018

virtualtam commented Feb 5, 2018

ArthurHoaro commented Feb 6, 2018

virtualtam commented Feb 14, 2018

tcitworld commented Feb 14, 2018

virtualtam commented Feb 14, 2018

ArthurHoaro commented Feb 15, 2018

ArthurHoaro commented Feb 24, 2018

virtualtam Feb 26, 2018

ArthurHoaro Feb 28, 2018

virtualtam Feb 26, 2018

virtualtam left a comment

ArthurHoaro commented Mar 14, 2018

nodiscc Mar 14, 2018

virtualtam Mar 15, 2018

Manage frontend dependencies with npm/yarn and webpack #1072

Manage frontend dependencies with npm/yarn and webpack #1072

Conversation

ArthurHoaro commented Jan 31, 2018 • edited by virtualtam Loading

ArthurHoaro commented Feb 4, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

virtualtam commented Feb 5, 2018

ArthurHoaro commented Feb 6, 2018

virtualtam commented Feb 14, 2018

tcitworld commented Feb 14, 2018

virtualtam commented Feb 14, 2018

ArthurHoaro commented Feb 15, 2018

ArthurHoaro commented Feb 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

virtualtam left a comment

Choose a reason for hiding this comment

ArthurHoaro commented Mar 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ArthurHoaro commented Jan 31, 2018 •

edited by virtualtam

Loading