-
Notifications
You must be signed in to change notification settings - Fork 5
Pipeline Architecture
The DocumentCloud frontend is built in Svelte and Webpack. It is based on the SuperSvelte template and supports Sass, single-page routing, unit testing, environment variable management, and inlining SVG assets.
This document intends to outline exactly how the compile and build pipeline works.
Everything Webpack-related starts with webpack.config.js
. This file exports two other Webpack configs:
-
webpack.app.config.js
: the main Webpack config for the frontend application. This config outputs the main chunked JavaScript bundle for the entire frontend application, along with the CSS and other needed assets. -
webpack.embed.config.js
: the Webpack config that exports static JavaScript files for legacy embeds. In the past, DocumentCloud embeds relied on static JavaScript files that inject the embed into an element — this config outputs the JavaScript bundles to accomplish those needs.
These two Webpack configs each in turn inherit from webpack.base.config.js
, which handles the common Webpack functionality between the two configs. The base config has the following features:
- minimizes the JavaScript bundles
- aliases
@
in include paths to resolve to thesrc/
directory - links in the Svelte preprocessor (
preprocess.config.js
), which makes@
work in Sass files (.scss
) and also imports the variables Sass file globally in all other Sass snippets (seesrc/style/variables.scss
) - links in the SVG inliner
- hooks in
.env
environment variable files (along with.env.staging
and.env.production
) - analyzes the imports and warns about circular dependencies
Docker and Docker Compose are used to build and run this application. There are two top-level Docker Compose files that handle building and running the application:
This is the main Docker Compose file, and it primarily handles running the web application, but it also defines several other services which are useful. The full services offered are:
-
documentcloud_frontend
: compiles and runs the main web application in dev mode (compiles quicker and supports live-reloading), along with compiling and running the embed loaders (the embed bundles are auto-compiled with each file change as well). This effectively just runs these two services concurrently:-
documentcloud_frontend_app
: compiles and runs the main web application -
documentcloud_frontend_embed
: compiles and runs the embed loaders
-
-
documentcloud_frontend_build
: compiles and runs the main web application in production mode (compile time is slower but bundle sizes are lower) -
documentcloud_frontend_analyze
: compiles the main web application and serves a page at the normal app URL (www.dev.documentcloud.org
) that provides an interactive treemap view of dependency sizes
This is the main Docker file for running ephemeral npm commands, like build scripts and so on. It defines the following services:
-
install
: effectivelynpm install
— it installs all the Node dependencies -
test
: runs the unit testing suite -
test-watch
: runs the unit testing suite in watch mode -
build
: compiles the frontend web application into a static bundle for production (unlike inlocal.yml
, the web application is not served in this mode) -
build_staging
: compiles the frontend web application into a static bundle for staging
There is a central Makefile that effectively aliases Docker Compose commands for ease of development. You can run any of them by executing make [cmd]
in your terminal in the root directory of this repo. The commands mostly correspond to the Docker services outlined above. Here are some notable ones:
-
make install
: install all the Node modules -
make build
: compile a static bundle for production -
make build-staging
: compile a static bundle for staging -
make dev
: run the frontend web application in dev mode -
make test
: run the unit testing suite -
make test-watch
: run the unit testing suite in watch mode
dotenv
simplifies environment variable management and is run by Webpack. There are three primary environment variable files:
-
.env
: defines common environment variables, such as the API URLs along with lots of constants -
.env.staging
: overrides and defines environment variables just for the staging environment -
.env.production
: overrides and defines environment variables just for the production environment
The main difference between building for staging and building for production is the environment file that's applied. The variables defined in this file are accessible in the main JavaScript codebase via process.env.[VARIABLE]
.
Unit testing is handled by Jest. There are a few files to be aware of here:
-
jest.config.js
: the main configuration file for Jest, which mainly handles the@
aliasing tosrc
and hooking indotenv
-
babel.config.js
: this defines the Babel transformations needed to get Jest to work. This file is not used for anything else
Code formatting is handled by Prettier. The .prettierrc
file defines the formatting conventions for this repo. Ensure that your editor has the Prettier plugin installed to enable auto-formatting.
This file defines code editor integration settings for use with the Svelte plugin.