Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[changelog skip] Use CircleCI over Travis #76

Merged
merged 5 commits into from
Apr 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
version: 2
references:
unit: &unit
run:
name: Run test suite
command: bundle exec parallel_test test/hatchet -n 11
hatchet_setup: &hatchet_setup
run:
name: Hatchet setup
command: |
bundle exec hatchet ci:setup
bundle: &bundle
run:
name: install dependencies
command: |
bundle install --jobs=4 --retry=3 --path vendor/bundle
jobs:
"ruby-2.5":
docker:
- image: circleci/ruby:2.5
steps:
- checkout
- <<: *bundle
- <<: *hatchet_setup
- <<: *unit
"ruby-2.6":
docker:
- image: circleci/ruby:2.6
steps:
- checkout
- <<: *bundle
- <<: *hatchet_setup
- <<: *unit
"ruby-2.7":
docker:
- image: circleci/ruby:2.7
steps:
- checkout
- <<: *bundle
- <<: *hatchet_setup
- <<: *unit

workflows:
version: 2
build:
jobs:
- "ruby-2.5"
- "ruby-2.6"
- "ruby-2.7"
1 change: 0 additions & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,3 @@ source "http://rubygems.org"

gemspec

gem 'm'
81 changes: 12 additions & 69 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,62 +4,13 @@

Hatchet is a an integration testing library for developing Heroku buildpacks.

[![Build Status](https://travis-ci.org/heroku/hatchet.svg?branch=master)](https://travis-ci.org/heroku/hatchet)

## Install

First run:

$ bundle install

This library uses the heroku CLI and API. You will need to make your API key available to the system. If you're running on a CI platform, you'll need to generate an OAuth token and make it available on the system you're running on.

To get a token, install the https://github.com/heroku/heroku-cli-oauth#creating plugin. Then run:

```sh
$ heroku authorizations:create --description "For Travis"
Creating OAuth Authorization... done
Client: <none>
ID: <id value>
Description: For Travis
Scope: global
Token: <token>
```

You'll set the `<token>` value to the `HEROKU_API_KEY` env var. For example, you could add it on Travis like this:

```sh
$ travis encrypt HEROKU_API_KEY=<token> --add
```

You'll also need an email address that goes with your token:

```sh
$ travis encrypt HEROKU_API_USER=<[email protected]> --add
```

If you're running locally, your system credentials will be pulled from `heroku auth:token`.

You'll also need to trigger a "setup" step for CI tasks. You can do it on Travis CI like this:

```
# .travis.yml
before_script: bundle exec hatchet ci:setup
```

and on Heroku CI like this:

```json
{
"environments": {
"test": {
"scripts": {
"test-setup": "bundle exec hatchet ci:setup",
}
}
}
}
```
This library uses the heroku CLI and API. You will need to make your API key available to the system. If you're running on a CI platform, you'll need to generate an OAuth token and make it available on the system you're running on see the "CI" section below.

## Run the Tests

Expand All @@ -83,7 +34,7 @@ Whatever testing framework you chose, we recommend using a parallel test runner

If you're unfamiliar with the ruby testing eco-system or want some help, start by looking at existing projects.

*Spoilers: There is a section below on getting Hatchet to work on Travis.
*Spoilers*: There is a section below on getting Hatchet to work on CI

## Testing a Buildpack

Expand Down Expand Up @@ -152,7 +103,7 @@ HATCHET_BUILDPACK_BASE=https://github.com/heroku/heroku-buildpack-ruby.git
HATCHET_BUILDPACK_BRANCH=master
```

If you do not specify `HATCHET_BUILDPACK_URL` the default Ruby buildpack will be used. If you do not specify a `HATCHET_BUILDPACK_BRANCH` the current branch you are on will be used. This is how the Ruby buildpack runs tests on branches on travis (by leaving `HATCHET_BUILDPACK_BRANCH` blank).
If you do not specify `HATCHET_BUILDPACK_URL` the default Ruby buildpack will be used. If you do not specify a `HATCHET_BUILDPACK_BRANCH` the current branch you are on will be used. This is how the Ruby buildpack runs tests on branches on CI (by leaving `HATCHET_BUILDPACK_BRANCH` blank).

If the `ENV['HATCHET_RETRIES']` is set to a number, deploys are expected to work and automatically retry that number of times. Due to testing using a network and random failures, setting this value to `3` retries seems to work well. If an app cannot be deployed within its allotted number of retries, an error will be raised.

Expand Down Expand Up @@ -391,31 +342,23 @@ Hatchet::Runner.new("rails5_ruby_schema_format", buildpacks: buildpacks).run_ci
end
```

## Testing on Travis

Once you've got your tests working locally, you'll likely want to get them running on Travis because a) CI is awesome, and b) you can use pull requests to run your all your tests in parallel without having to kill your network connection.

Set the `HATCHET_DEPLOY_STRATEGY` to `git`.
## Testing on CI

To run on travis, you will need to configure your `.travis.yml` to run the appropriate commands and to set up encrypted data so you can run tests against a valid heroku user.
Once you've got your tests working locally, you'll likely want to get them running on CI. For reference see the [Circle CI config from this repo](https://github.com/heroku/hatchet/blob/master/.circleci/config.yml) and the [Heroku CI config from the ruby buildpack](https://github.com/heroku/heroku-buildpack-ruby/blob/master/app.json).

For reference see the `.travis.yml` from [hatchet](https://github.com/heroku/hatchet/blob/master/.travis.yml) and the [heroku-ruby-buildpack](https://github.com/heroku/heroku-buildpack-ruby/blob/master/.travis.yml). To make running on travis easier, there is a rake task in Hatchet that can be run before your tests are executed
To make running on CI easier, there is a setup script in Hatchet that can be run before your tests are executed:

```yml
before_script: bundle exec hatchet ci:setup
bundle exec hatchet ci:setup
```

I recommend signing up for a new heroku account for running your tests on travis to prevent exceding your API limit. Once you have the new api token, you can use this technique to [securely send travis the data](http://docs.travis-ci.com/user/environment-variables/#Secure-Variables).
If you're a Heroku employee see [private instructions for setting up test users](https://github.com/heroku/languages-team/blob/master/guides/create_test_users_for_ci.md) to generate a user a grab the API token. Once you have an API token you'll want to set up these env vars with your CI provider:

```sh
$ travis encrypt HEROKU_API_KEY=<token> --add
```

If your Travis tests are containerized, you may need sudo to complete this successfully. In that case, add the following:

```yml
before_script: bundle exec hatchet ci:setup
sudo: required
HATCHET_APP_LIMIT=100
HATCHET_RETRIES=3
HEROKU_API_KEY=<redacted>
HEROKU_API_USER=<[email protected]>
```

## Extra App Commands
Expand Down
14 changes: 8 additions & 6 deletions etc/ci_setup.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,13 @@
File.open(netrc_file, 'w') do |file|
file.write <<-EOF
machine git.heroku.com
login #{ENV.fetch('HEROKU_API_USER')}
password #{ENV.fetch('HEROKU_API_KEY')}
login #{ENV.fetch('HEROKU_API_USER')}
password #{ENV.fetch('HEROKU_API_KEY')}
machine api.heroku.com
login #{ENV.fetch('HEROKU_API_USER')}
password #{ENV.fetch('HEROKU_API_KEY')}
EOF
`chmod 0600 "$HOME/.netrc"`
end
end

Expand All @@ -21,9 +25,7 @@
"git config --get user.name > /dev/null || git config --global user.name 'BuildpackTester'",
].each do |command|
puts "== Running: #{command}"
Bundler.with_clean_env do
result = `#{command}`
raise "Command failed: #{command.inspect}\nResult: #{result}" unless $?.success?
end
result = `#{command}`
raise "Command failed: #{command.inspect}\nResult: #{result}" unless $?.success?
end
puts "== Done =="
1 change: 1 addition & 0 deletions hatchet.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,5 @@ Gem::Specification.new do |gem|
gem.add_development_dependency "mocha", ">= 1"
gem.add_development_dependency "parallel_tests", ">= 2"
gem.add_development_dependency "travis", ">= 1"
gem.add_development_dependency "m"
end