Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker compose #859

Merged
merged 26 commits into from
Mar 19, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
d509fc6
add Dockerfile and make dependency adjustments
sharkySharks Dec 14, 2018
6d9eabd
add configurable port to webpack dev server and refactor Dockerfile
sharkySharks Dec 18, 2018
d55102a
add docs draft
sharkySharks Dec 19, 2018
c9b72a5
add docker-compose file with nginx proxy and https enabled
sharkySharks Jan 11, 2019
95d0fd6
add environment variables and setup script
sharkySharks Jan 14, 2019
bc9f8ed
add command to setup script to add host to hostfile
sharkySharks Jan 18, 2019
f1048ad
Convert build to yarn/babel and fix webpack publicPath build output
sharkySharks Feb 6, 2019
cc541f1
add automation for certificate generation to the docker setup script
sharkySharks Feb 6, 2019
7a35e6c
prettier and linting fixes
sharkySharks Feb 6, 2019
92d5153
add docs and minor file clean up
sharkySharks Feb 6, 2019
c2f8ac1
refactor env vars in webpack.config and pwadevserver files
sharkySharks Feb 8, 2019
3f972f1
update custom domain to reserved DNS label .localhost
sharkySharks Feb 8, 2019
a0e9854
adjust docker setup script to include environment clean up
sharkySharks Feb 13, 2019
8cb13b1
Adding setup script support for Windows users
sharkySharks Feb 14, 2019
abb8029
Add command line argument support to bypass setup prompts
sharkySharks Feb 15, 2019
be6242e
error handle if the build fails then docker should not start
sharkySharks Feb 22, 2019
35c4c17
resolve issue with publicPath and public asset serving through nginx …
sharkySharks Feb 26, 2019
ac1769b
give adduser group full permissions of app directory
sharkySharks Mar 1, 2019
d16490f
add volume mounts to edit files on host with hot reloading
sharkySharks Mar 1, 2019
6bc962e
move cert creation to devcert in separate script
sharkySharks Mar 5, 2019
0325d14
add polling to watch options for guaranteed recompile
sharkySharks Mar 5, 2019
bc91d42
refactor PWADevServer configuration
sharkySharks Mar 5, 2019
328f694
update docs and disable service workers by default
sharkySharks Mar 6, 2019
70feb69
replace hard coded host name and .env file path with variables
sharkySharks Mar 7, 2019
77e3b4d
add polling to ensure hot reloading
sharkySharks Mar 8, 2019
47389e9
update configs to include changes to validate-magento-pwa-queries script
sharkySharks Mar 19, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
packages/**/node_modules
packages/**/dist
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ lerna-debug.log
yarn-error.log
# Packages that build partially transpiled ES modules put them here
packages/*/esm/*
docker/certs
58 changes: 58 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
##############################################################
# This file is intended to be used with ./docker-compose.yml #
##############################################################

FROM node:10.14.1-alpine as build
# working directory
WORKDIR /usr/src/app

# global environment setup : yarn + dependencies needed to support node-gyp
RUN apk --no-cache --virtual add \
python \
make \
g++ \
yarn

# copy just the dependency files and configs needed for install
COPY packages/peregrine/package.json ./packages/peregrine/package.json
COPY packages/pwa-buildpack/package.json ./packages/pwa-buildpack/package.json
COPY packages/upward-js/package.json ./packages/upward-js/package.json
COPY packages/upward-spec/package.json ./packages/upward-spec/package.json
COPY packages/venia-concept/package.json ./packages/venia-concept/package.json
COPY package.json yarn.lock babel.config.js browserslist.js magento-compatibility.js ./

# install dependencies with yarn
RUN yarn install

This comment was marked as resolved.

This comment was marked as resolved.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to run "yarn install" and copy these files in Dockerfile build phase at all? Perhaps at least for development environment this could be done on entrypoint script, so that image itself does not need to contain the application files.

Usually application files are only "burned" to the image when final deployable production image is created.

Copy link
Contributor Author

@sharkySharks sharkySharks Mar 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in order to build you need to yarn install. without the build dependencies then it would throw errors when running yarn run build. above I have copied only the build dependencies, package.json and relevant files needed for the install. Below on line 28 I am copying over the remainder of the package files before building. All files in the repo are needed to create a build that can then be spun up with the CMD.

If you look at what yarn run build is executing, it runs the build commands of all the directories in the workspace under packages/, therefore it is needed to run yarn install to get all relevant dependencies needed for the build scripts to execute, as well as copy over all the contents of each package.

# copy over the rest of the package files
COPY packages ./packages

# set in docker-compose
ARG ENVFILEPATH
# copy configuration env file from host file system to venia-concept .env for build
COPY ${ENVFILEPATH} ./packages/venia-concept/.env

# build the app
RUN yarn run build

#######################################################################################
# UNCOMMENT FOR PRODUCTION BUILD - not as necessary for dev env to have non-root user #
#######################################################################################
# # MULTI-STAGE BUILD
# FROM node:10.14.1-alpine
# # working directory
# WORKDIR /usr/src/app
# # copy build from previous stage
# COPY --from=build /usr/src/app .
# # create and set non-root USER
# RUN addgroup -g 1001 appuser && \
# adduser -S -u 1001 -G appuser appuser
# RUN chown -R appuser:appuser /usr/src/app && \
# chmod 755 /usr/src/app
# USER appuser
#######################################################################################

# Pass the `WEBPACK_HOST` arg from docker-compose args and set it to the HOST
ARG WEBPACK_HOST
# command to run application
CMD [ "yarn", "workspace", "@magento/venia-concept", "run", "watch", "-- --host ${WEBPACK_HOST}"]
48 changes: 48 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
version: '3'
services:
# proxy service that creates a reverse proxy to the pwa container - https://bit.ly/2HpZJDI
# the reverse proxy connects the SSL/TLS certs created on the local file system into the nginx container
# this allows the pwa container to be accessible over HTTPS
# it also proxies requests to the configured domain running at the configured port for the pwa container
nginx-proxy:
image: jwilder/nginx-proxy:alpine
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I agree with using jwilder/nginx-proxy. It seems to work in quite non-standard way of reading docker containers and automatically exposing them. While this is nice, it is slightly unexpected behaviour, and can take while to understand how it works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on the unexpected behavior you are experiencing? Spinning up these services together automatically puts them in the same network, thereby exposing them to one another. links also connect the containers in a way that is natural to docker.

I did not add any comments to this docker-compose file, I can add some here to give some more context on what is going on.

Copy link
Contributor Author

@sharkySharks sharkySharks Mar 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One other thing that I like about this image is that it allows you to set up as many proxies as you like. You can even move the proxy up a layer in your dev environment and have it running on its own and whenever you have a project you need a proxy for then you can add it to the network and proxy as many projects through it at once. For the purposes of PWA, without changing the current setup (You need to remove the nginx proxy from this docker-compose and have it in its own file and connect as an external docker network to run multiple storefronts concurrently), you can use this proxy to run multiple PWA storefronts through the proxy, essentially have as many PWA domains as you like that all have their own service workers assigned and running without collision because they are running in different containers and have different domains, etc. You just add it as another service in this file and run it with a new .env file.

hostname: nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- ./docker/certs:/etc/nginx/certs
- /var/run/docker.sock:/tmp/docker.sock:ro
restart: unless-stopped

pwa:
hostname: ${PWA_STUDIO_PUBLIC_PATH}
# build pwa using the Dockerfile from the PWD
build:
context: .
dockerfile: Dockerfile
args:
WEBPACK_HOST: ${PWA_STUDIO_PUBLIC_PATH}
ENVFILEPATH: ${ENVFILEPATH}
# list of directories and files on the host system to volume mount into the container
# changes made to files in the container and on the host file system are mapped to one another
# this enables hot reloading from the container to detect changes made on the host file system
volumes:
- ./packages/peregrine/.storybook:/usr/src/app/packages/peregrine/.storybook:rw
- ./packages/peregrine/esm:/usr/src/app/packages/peregrine/esm:rw
- ./packages/peregrine/scripts:/usr/src/app/packages/peregrine/scripts:rw
- ./packages/peregrine/src:/usr/src/app/packages/peregrine/src:rw
- ./packages/pwa-buildpack/src:/usr/src/app/packages/pwa-buildpack/src:rw
- ./packages/upward-js/lib:/usr/src/app/packages/upward-js/lib:rw
- ./packages/venia-concept/.storybook:/usr/src/app/packages/venia-concept/.storybook:rw
- ./packages/venia-concept/esm:/usr/src/app/packages/venia-concept/esm:rw
- ./packages/venia-concept/src:/usr/src/app/packages/venia-concept/src:rw
- ./packages/venia-concept/static:/usr/src/app/packages/venia-concept/static:rw
links:
- nginx-proxy
environment:
# environment variables consumed by the nginx-proxy service
VIRTUAL_HOST: ${PWA_STUDIO_PUBLIC_PATH}
VIRTUAL_PORT: ${PWA_STUDIO_PORTS_DEVELOPMENT}
expose:
- ${PWA_STUDIO_PORTS_DEVELOPMENT}
16 changes: 16 additions & 0 deletions docker/.env.docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
########## .env.docker ########################################################
#
# See packages/venia-concept/.env.dist file for full option details
#
###############################################################################

PWA_STUDIO_PUBLIC_PATH=pwa-docker.localhost
PWA_STUDIO_PORTS_DEVELOPMENT=8080
ENABLE_SERVICE_WORKER_DEBUGGING=0
PWA_STUDIO_HOT_RELOAD_WITH_POLLING=0
MAGENTO_BACKEND_URL=https://release-dev-231-npzdaky-zddsyhrdimyra.us-4.magentosite.cloud/
MAGENTO_BUILDPACK_PROVIDE_SECURE_HOST=0
UPWARD_JS_UPWARD_PATH=venia-upward.yml
UPWARD_JS_BIND_LOCAL=1
UPWARD_JS_LOG_URL=1
BRAINTREE_TOKEN=sandbox_8yrzsvtm_s2bg8fs563crhqzk
39 changes: 39 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Docker Directory

This directory stores certificates for local SSL/TLS, created by running `run-docker` script, as well as configuration for docker environment setup.

Run the `docker/run-docker` script from the root of the repository to create a container running PWA with a secure https protocol.

This script will:

* copy the `.env.docker` environment variables to `.env` to be easily consumable by `docker-compose`
* add a custom domain, configured in `.env.docker`
* generate a self-signed ssl/tls certificate and trust the certificate using `devcert` in the `makeHostAndCert.js` script
* run `docker-compose build` to build the container network
* run `docker-compose up` to start the container running PWA at the custom domain with https

After `docker/run-docker` is executed from the root of the repository, the default configuration will have the PWA application running at `https://pwa-docker.localhost`.

## Configure a custom domain

The domain is configurable. Just set `PWA_STUDIO_PUBLIC_PATH` key to the new domain under `docker/.env.docker`, or pass a custom .env file with the `PWA_STUDIO_PUBLIC_PATH` key set. All required fields can be found in `docker/.env.docker`. See how to pass the custom .env file below.

## Pass custom .env file configuration through cli args (optional)

To use a custom .env file for configuration, pass it to the `run-docker` script like so: `docker/run-docker -e path-from-project-root`. This file will take the place of the default `.env.docker` file.

## Service Workers and Hot Reloading

Service workers are disabled by default when running the `docker/run-docker` script, but they can easily be turned on by changing the default value of `ENABLE_SERVICE_WORKER_DEBUGGING=0` to `ENABLE_SERVICE_WORKER_DEBUGGING=1` in `.env.docker`.

Hot reloading is enabled by default when running the `docker/run-docker` script and automatically refreshes the browser on changes made in the container as well as on the host machine, ie your local file system.

If service workers are enabled during development, then service worker caching will affect the hot reloading and will require a manual refresh after the cached assets have fully reloaded.

In order to avoid manual page refreshing and have hot reloading work as expected with service workers enabled, it is recommended for developers to click the `Update on reload` checkbox in the `Service Workers` panel in Chrome developer tools. This feature in Chrome is helpful when developing with service workers because it ensures that the service worker is updated on every page reload and you will see changes immediately, avoiding the service worker cache.

For more details check out the [dev tools docs](https://bit.ly/2tTGWc0).

### Hot Reloading is not working

If you find that hot reloading is not working for you the webpack docs recommend using [polling](https://webpack.js.org/configuration/watch/#watchoptionspoll) as watching does not work with network file systems and machines in VirtualBox. To enable polling, set `PWA_STUDIO_HOT_RELOAD_WITH_POLLING=1` in `.env.docker`.
16 changes: 16 additions & 0 deletions docker/makeHostAndCert.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
const devcert = require('devcert');
const domain = process.argv[2];
const fs = require('fs');

const makeFile = (name, content) => {
fs.writeFile(name, content, err => {
if (err) throw err;
console.log(`${name} CREATED SUCCESSFULLY`);
});
};
const ssl = async function (domain) {
const s = await devcert.certificateFor(domain);
makeFile(`./docker/certs/${domain}.crt`, s.cert);
makeFile(`./docker/certs/${domain}.key`, s.key);
};
ssl(domain);
10 changes: 10 additions & 0 deletions docker/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"name": "pwa-docker",
"version": "1.0.0",
"description": "run pwa in a container with https",
"author": "Kayden Althen",
"license": "ISC",
"dependencies": {
"devcert": "~1.0.0"
}
}
92 changes: 92 additions & 0 deletions docker/run-docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
#!/usr/bin/env sh

####################################################################################
# Run this file from the root of the repository to build and run a PWA container #
####################################################################################
CONFIG_ENV_FILE=./docker/.env.docker
STATUS=0

# accepts -e <envfile_path> to set a separate environment variable file configuration
while getopts ":e" opt; do
case ${opt} in
e )
# validate that the file path passed is valid
if [ -f $2 ]; then
CONFIG_ENV_FILE="$2"
echo "Configuration using environment variables from: $2"
else
echo -e "The env file you provided does not exist at this path: $2\nPlease provide the relative path to your environment file.\nExample: $CONFIG_ENV_FILE"
exit 1
fi
;;
esac
done

main () {
env_setup
create_certificate
# If cert setup failed then don't start docker
if [ "$STATUS" == 0 ]; then
start_docker
else
message "An error occurred during setup."
clean_up_env & # clean up the environment in the background
exit 1
fi
}

env_setup () {
ENVFILE=./.env
message "Adding env vars from $CONFIG_ENV_FILE to $ENVFILE for docker setup."
cp $CONFIG_ENV_FILE $ENVFILE
echo "ENVFILEPATH=$CONFIG_ENV_FILE" >> $ENVFILE
cat $ENVFILE
. $ENVFILE
DOMAIN=$PWA_STUDIO_PUBLIC_PATH
}

create_certificate () {
message "Creating SSL/TLS certificate"
# make docker/certs folder if one does not already exist
[ -d ./docker/certs ] || mkdir ./docker/certs
# install devcert dependency
cd ./docker && yarn install && cd -
node ./docker/makeHostAndCert $DOMAIN || STATUS=$?
}

start_docker () {
message "Building PWA image"
BUILD_STATUS=0
docker-compose build || BUILD_STATUS=$?

clean_up_env & # clean up the environment in the background

if [ "$BUILD_STATUS" == 0 ]; then
message "Starting Docker network and containers"
docker-compose up
else
message "Build failed. See output for details."
fi
}

clean_up_env () {

This comment was marked as resolved.

# the docker-compose command needs the .env file to be present in order to run
# but we want to remove it after execution so it doesn't pollute the dev environment
# when switching git branches or running the application outside of docker
if [ -f .env ]; then

This comment was marked as resolved.

sleep 20
rm .env
fi
}

message () {
echo ""
echo "==========================================================================="
echo ""
echo -e " " "$1"
echo ""
echo "==========================================================================="
echo ""
}

main
Loading