Skip to content

Latest commit

 

History

History
513 lines (384 loc) · 19 KB

getting-started.md

File metadata and controls

513 lines (384 loc) · 19 KB
Raw

Getting Started

Table of Contents

Overview

This document provides a step-by-step walkthrough on how to setup Lighthouse CI on your repository. After this guide, your build system will be running Lighthouse on your project's URLs on every commit, automatically asserting that important Lighthouse audits pass, and uploading the reports for manual inspection.

Important Note Before Starting

Introducing performance measurement to a project for the first time can have a fairly steep learning curve. The Lighthouse team recommends starting slowly. Refer to the diagram below for recommended stopping points based on the project's maturity and the team's familiarity with Lighthouse or performance measurement. Once the team is more comfortable with Lighthouse and familiar with its results, feel free to continue on to the next stages of setup.

recommended setup flow

Prerequisites

Before starting, your project should meet the following requirements:

  1. Source code is managed with git (GitHub, GitLab, Bitbucket, etc).
  2. Branches/pull requests are gated on the results of a continuous integration build process (Travis CI, CircleCI, Jenkins, AppVeyor, GitHub Actions, etc). If you aren't using a CI provider yet, start with our Introduction to CI documentation.
  3. Your CI process can build your project into production assets (typically provided as an npm run build command by most modern JavaScript frameworks).
  4. Your project either: A) has a command that runs a web server with production-like assets. B) is a static site.

Collect Lighthouse Results

In this section, we'll configure Lighthouse CI to automatically find your project's static assets, run Lighthouse 3 times on each HTML file, and upload the reports to temporary public storage where they'll be accessible to anyone with the URL.

NOTE: As the name implies, this is temporary and public storage. If you're uncomfortable with the idea of your Lighthouse reports being stored on a public URL for anyone to see, skip to the add assertions or Lighthouse CI server steps. Please read the full terms of service and privacy policy before deciding to upload your reports.

Configure Lighthouse CI

A config file called lighthouserc.js at the root of your repo controls the options for Lighthouse CI. For advanced users who'd prefer to use CLI flags or keep the configuration file in another location, refer to the configuration documentation.

The simple configuration file below is all you need to get started collecting Lighthouse reports to temporary public storage.

lighthouserc.js

module.exports = {
  ci: {
    upload: {
      target: 'temporary-public-storage',
    },
  },
};

Configure Your CI Provider

Next we need to configure the CI provider to run Lighthouse using the lhci autorun command. autorun will automatically execute a number of commands behind the scenes and infer sensible defaults for us. For more advanced use cases where you'd like to control the exact sequence of Lighthouse commands, refer to the configuration documentation.

GitHub Actions

.github/workflows/ci.yml

name: CI
on: [push]
jobs:
  lhci:
    name: Lighthouse
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Use Node.js 12.x
        uses: actions/setup-node@v1
        with:
          node-version: 12.x
      - name: npm install, build
        run: |
          npm install
          npm run build
      - name: run Lighthouse CI
        run: |
          npm install -g @lhci/[email protected]
          lhci autorun
Travis CI

.travis.yml

language: node_js
node_js: v12
addons:
  chrome: stable
before_install:
  - npm install -g @lhci/[email protected]
script:
  - npm run build
  - lhci autorun
Circle CI

.circleci/config.yml

version: 2.1
orbs:
  browser-tools: circleci/[email protected]
jobs:
  build:
    docker:
      - image: cimg/node:16.13-browsers
    working_directory: ~/your-project
    steps:
      - checkout
      - browser-tools/install-chrome
      - run: npm install
      - run: npm run build
      - run: sudo npm install -g @lhci/[email protected]
      - run: lhci autorun
GitLab CI

NOTE: Learn more about the security tradeoffs behind use of the --no-sandbox Chrome option before proceeding.

.lighthouserc.js

module.exports = {
  ci: {
    collect: {
      settings: {chromeFlags: '--no-sandbox'},
    },
    upload: {
      target: 'temporary-public-storage',
    },
  },
};

.gitlab-ci.yml

image: cypress/browsers:node14.15.0-chrome86-ff82
lhci:
  script:
    - npm install
    - npm run build
    - npm install -g @lhci/[email protected]
    - lhci autorun --upload.target=temporary-public-storage --collect.settings.chromeFlags="--no-sandbox" || echo "LHCI failed!"
Jenkins (Ubuntu-based)

machine-setup.sh

#!/bin/bash

set -euxo pipefail

# Add Chrome's apt-key
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" | sudo tee -a /etc/apt/sources.list.d/google.list
wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | sudo apt-key add -

# Add Node's apt-key
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -

# Install NodeJS and Google Chrome
sudo apt-get update
sudo apt-get install -y nodejs google-chrome-stable

job.sh

#!/bin/bash

set -euxo pipefail

npm install
npm run build

export CHROME_PATH=$(which google-chrome-stable)
export LHCI_BUILD_CONTEXT__EXTERNAL_BUILD_URL="$BUILD_URL"

npm install -g @lhci/[email protected]
lhci autorun
Google Cloud Build

NOTE: Learn more about the security tradeoffs behind use of the --no-sandbox Chrome option before proceeding.

.lighthouserc.js

module.exports = {
  ci: {
    collect: {
      settings: {chromeFlags: '--no-sandbox'},
    },
    upload: {
      target: 'temporary-public-storage',
    },
  },
};

Notes

  • LHCI_BUILD_CONTEXT__CURRENT_BRANCH doesn't pick up the right variables in Cloud Build so passing through $BRANCH_NAME will fix this.
  • machineType defines the machine you can pick. The bigger the machine the more stable the performance results will be (see Lighthouse variability documentation). WARNING: Look through the Cloud Build machine pricing before deciding on a machine type.

cloudbuild.yml

steps:
  - id: 'install'
    args: ['npm', 'ci']
    name: node:14-alpine

  - id: 'build'
    waitFor: ['install']
    name: node:14-alpine
    args: ['npm', 'run', 'build']

  - id: 'lighthouse'
    waitFor: ['build']
    name: cypress/browsers:node14.15.0-chrome86-ff82
    entrypoint: '/bin/sh'
    args: ['-c', 'npm install -g @lhci/[email protected] && lhci autorun --failOnUploadFailure']
    env:
      - 'LHCI_BUILD_CONTEXT__CURRENT_BRANCH=$BRANCH_NAME'

options:
  machineType: 'N1_HIGHCPU_8'

That's it! With this in place, you'll have Lighthouse reports collected and uploaded with links to each report on every push.

Temporary public storage provides access to individual reports, but not historical data, report diffing, or build failures. Read on to find out how to add assertions, configure the Lighthouse CI server for report diffs and timeseries charts, and enable GitHub status checks.

Modifications for Sites without a Build Step

If your site does not use a package.json-based build step, you might receive strange error messages in the build logs of the above. If your site is static and doesn't require a build step, you'll need to make 2 changes to the configurations above.

  1. Modify lighthouserc.js to configure the location of your HTML files using the staticDistDir property. If your HTML files are located in the root of your repo, just use ./.
module.exports = {
  ci: {
    collect: {
      staticDistDir: './',
    },
    upload: {
      target: 'temporary-public-storage',
    },
  },
};
  1. Modify your CI provider's configuration to remove the npm install and npm run build commands. IMPORTANT: Leave the npm install -g @lhci/cli step.

Refer to the configuration documentation for more common examples and the options available to you.

Modifications for Sites with a Custom Server

If your site is not static and requires the use of a custom server, you'll need to teach Lighthouse CI how to start your server.

Modify lighthouserc.js to configure the command used to start your server using the startServerCommand property. You'll also need to configure the url property with the URLs that you'd like to audit.

module.exports = {
  ci: {
    collect: {
      url: ['http://localhost:3000/'],
      startServerCommand: 'rails server -e production',
    },
    upload: {
      target: 'temporary-public-storage',
    },
  },
};

Refer to the configuration documentation for more common examples and the options available to you.

GitHub Status Checks

The setup so far will run Lighthouse through your CI provider, but there's no differentiation between the build failing because of Lighthouse CI compared to your other tests. Links to the Lighthouse report are also tucked away in your build logs.

GitHub status checks add additional granularity to your build reporting and direct links to uploaded reports within the GitHub UI!

screenshot of GitHub status checks for Lighthouse CI

GitHub App Method (Recommended)

NOTE: Before installing the GitHub App, refer to the terms of service.

To enable GitHub status checks via the official GitHub app, install and authorize the app with the owner of the target repo. If the repo is within an organization, organization approval might be necessary. Copy the app token provided on the authorization confirmation page and add it to your build environment as LHCI_GITHUB_APP_TOKEN. The next time your lhci autorun command runs it will also set the results as GitHub status checks!

Be sure to keep this token secret. Anyone in possession of this token will be able to set status checks on your repository.

Alternative: Personal Access Token Method

If you don't want to use the GitHub App, you can also enable this via a personal access token. The only difference is that your user account (and its avatar) will post a status check. Create a personal access token with the repo:status scope and add it to your environment as LHCI_GITHUB_TOKEN.

Be sure to keep this token secret. Anyone in possession of this token will be able to set status checks on your repository.

Additional configuration for GitHub Actions as CI Provider

Make sure you define the LHCI_GITHUB_APP_TOKEN as environment variable on the workflow and ensure that the git history is available too.

name: CI
on: [push]
jobs:
  lhci:
    name: Lighthouse
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          ref: ${{ github.event.pull_request.head.sha }}
      - name: Use Node.js 10.x
        uses: actions/setup-node@v1
        with:
          node-version: 10.x
      - name: npm install, build
        run: |
          npm install
          npm run build
      - name: run Lighthouse CI
        run: |
          npm install -g @lhci/[email protected]
          lhci autorun
        env:
          LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}

Add Assertions

NOTE: If you're new to performance measurement or your site has a lot of room for improvement, we recommend skipping this step for now and revisiting once your Lighthouse scores are a little higher.

While Lighthouse reports at your fingertips is great, failing the build based on the audit results is even better! Add an assert object to your configuration with the preset to get started with assertions.

lighthouserc.js

module.exports = {
  ci: {
    // ...
    assert: {
      preset: 'lighthouse:recommended',
    },
    // ...
  },
};

The setup so far will automatically assert the Lighthouse team's recommended set of audits, but your project might have a bit of work to go before hitting straight 100s! Fear not, the assertions are completely configurable and you can disable as many audits as you need.

To get back to passing, inspect the log output of the build and disable the audits that failed. Treat this as a burndown list of your team of issues to fix!

For example if you had the build output:

  ✘  uses-rel-preload failure for minScore assertion
       Preload key requests
       https://web.dev/uses-rel-preload

        expected: >=1
           found: 0.46
      all values: 0.46, 0.46

  ✘  uses-rel-preconnect failure for maxLength assertion
       Preconnect to required origins
       https://web.dev/uses-rel-preconnect

        expected: <=0
           found: 1
      all values: 1, 1

You would change your configuration to disable those two audits:

module.exports = {
  ci: {
    // ...
    assert: {
      preset: 'lighthouse:recommended',
      assertions: {
        'uses-rel-preload': 'off',
        'uses-rel-preconnect': 'off',
      },
    },
    // ...
  },
};

Read more about what's possible in configuration with the assertions format.

The Lighthouse CI Server

Screenshot of the Lighthouse CI server dashboard UI

The Lighthouse CI server is an open source node server that can be setup on AWS, GCP, heroku or even your local machine. Think of the CI server as your personal storage and visualization center for the history of Lighthouse reports in your project.

You can use the CI server to...

  • Store Lighthouse reports for more than a few days
  • Keep Lighthouse reports private on your infrastructure
  • Generate graphs of improved category scores over time to impress your boss
  • Identify regressions in category score or metric performance to specific commits
  • View detailed diffs between any two Lighthouse reports to find the root cause of a regression

The process for setting up the server involves commands across a couple different machines and will vary depending on your specific infrastructure setup.

Deployment

To deploy the server to cloud infrastructure, refer to our deployment guides. Note that by default anyone with HTTP access to the server will be able to view and create data, see the documentation on server security to learn about security options.

Project Creation

Once the server is set up, on your local laptop or desktop, make sure you can connect to the server, install the Lighthouse wizard with npm, and create a new project:

$ curl https://your-lhci-server.example.com/version # Make sure you can connect to your server.
0.x.x
$ npm install -g @lhci/[email protected] # Install the Lighthouse CI CLI.
Installing...
$ lhci wizard # Use the wizard to create a project.
? Which wizard do you want to run? new-project
? What is the URL of your LHCI server? https://your-lhci-server.example.com/
? What would you like to name the project? My Favorite Project
? Where is the project's code hosted? https://github.com/GoogleChrome/lighthouse-ci

Created project My Favorite Project (XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX)!
Use build token XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX to connect.
Use admin token XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX to manage the project.

The output should contain both a build token and an admin token. Save both of these values somewhere safe for later.

The build token is used by CI to upload new data to your server. It's not really very secret, especially if you'd like to evaluate the performance of external contributor contributions.

The admin token is used to edit or delete data within the project. KEEP THIS SECRET - Anyone possessing the admin token can delete your entire project and all of the data within it.

If your lhci server is behind basic auth, you can provide the username and password via the --basicAuth.password and --basicAuth.username command line flags.

Configuration

With your server setup and the project created, all that's left to do is update your Lighthouse CI configuration file with the new URL and token. If you'd like to keep the token secret from contributors, you can use the LHCI_TOKEN environment variable instead of putting the token in plain text in your code.

lighthouserc.js

module.exports = {
  ci: {
    upload: {
      target: 'lhci',
      serverBaseUrl: 'https://your-lhci-server-url.example.com',
      token: 'Your *build token* goes here', // could also use LHCI_TOKEN variable instead
    },
  },
};

NOTE: If you run a matrix of environments in your tests, make sure you only run lhci autorun ONCE per build. The Lighthouse CI server will only accept a single upload per hash and future attempts to upload data for that hash will be rejected.