From 132cf359699204dbfa1c0423ed92008abee7f2a7 Mon Sep 17 00:00:00 2001 From: schneems Date: Wed, 15 Apr 2020 11:22:08 -0500 Subject: [PATCH 1/5] Use CircleCI over Travis --- .circleci/config.yml | 49 ++++++++++++++++++++++++++++++++++++++++++++ Gemfile | 1 - hatchet.gemspec | 1 + 3 files changed, 50 insertions(+), 1 deletion(-) create mode 100644 .circleci/config.yml diff --git a/.circleci/config.yml b/.circleci/config.yml new file mode 100644 index 0000000..dd0e1dd --- /dev/null +++ b/.circleci/config.yml @@ -0,0 +1,49 @@ +version: 2 +references: + unit: &unit + run: + name: Run test suite + command: bundle exec parallel_test test/hatchet -n 11 + hatchet_setup: &hatchet_setup + run: + name: Hatchet setup + command: | + bundle exec hatchet ci:setup + bundle: &bundle + run: + name: install dependencies + command: | + bundle install --jobs=4 --retry=3 --path vendor/bundle +jobs: + "ruby-2.5": + docker: + - image: circleci/ruby:2.5 + steps: + - checkout + - <<: *bundle + - <<: *hatchet_setup + - <<: *unit + "ruby-2.6": + docker: + - image: circleci/ruby:2.6 + steps: + - checkout + - <<: *bundle + - <<: *hatchet_setup + - <<: *unit + "ruby-2.7": + docker: + - image: circleci/ruby:2.7 + steps: + - checkout + - <<: *bundle + - <<: *hatchet_setup + - <<: *unit + +workflows: + version: 2 + build: + jobs: + - "ruby-2.5" + - "ruby-2.6" + - "ruby-2.7" diff --git a/Gemfile b/Gemfile index 75ef332..1c86af9 100644 --- a/Gemfile +++ b/Gemfile @@ -2,4 +2,3 @@ source "http://rubygems.org" gemspec -gem 'm' diff --git a/hatchet.gemspec b/hatchet.gemspec index 332717e..bfcefab 100644 --- a/hatchet.gemspec +++ b/hatchet.gemspec @@ -31,4 +31,5 @@ Gem::Specification.new do |gem| gem.add_development_dependency "mocha", ">= 1" gem.add_development_dependency "parallel_tests", ">= 2" gem.add_development_dependency "travis", ">= 1" + gem.add_development_dependency "m" end From 8ff9ec6b6a5b6e984125e7250d7cb04559f24678 Mon Sep 17 00:00:00 2001 From: schneems Date: Thu, 16 Apr 2020 10:55:15 -0500 Subject: [PATCH 2/5] Bundle context is needed to find `hatchet` executable Hatchet is installed as a gem, without `bundle exec` it won't be found. The directive `Bundler.with_clean_env` is how the environment is set up before `bundle install`. I don't know how this ever worked before unless travis was putting hatchet on the path or `bin` was already in the path somehow. --- etc/ci_setup.rb | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/etc/ci_setup.rb b/etc/ci_setup.rb index 63b027d..ad18adf 100755 --- a/etc/ci_setup.rb +++ b/etc/ci_setup.rb @@ -21,9 +21,7 @@ "git config --get user.name > /dev/null || git config --global user.name 'BuildpackTester'", ].each do |command| puts "== Running: #{command}" - Bundler.with_clean_env do - result = `#{command}` - raise "Command failed: #{command.inspect}\nResult: #{result}" unless $?.success? - end + result = `#{command}` + raise "Command failed: #{command.inspect}\nResult: #{result}" unless $?.success? end puts "== Done ==" From 7b51b4f2d02c0d69726a2330582e47796cea1152 Mon Sep 17 00:00:00 2001 From: schneems Date: Thu, 16 Apr 2020 11:14:03 -0500 Subject: [PATCH 3/5] Update readme with better setup instructions --- README.md | 81 +++++++++---------------------------------------------- 1 file changed, 12 insertions(+), 69 deletions(-) diff --git a/README.md b/README.md index 303beb4..8a9e6cf 100644 --- a/README.md +++ b/README.md @@ -4,62 +4,13 @@ Hatchet is a an integration testing library for developing Heroku buildpacks. -[![Build Status](https://travis-ci.org/heroku/hatchet.svg?branch=master)](https://travis-ci.org/heroku/hatchet) - ## Install First run: $ bundle install -This library uses the heroku CLI and API. You will need to make your API key available to the system. If you're running on a CI platform, you'll need to generate an OAuth token and make it available on the system you're running on. - -To get a token, install the https://github.com/heroku/heroku-cli-oauth#creating plugin. Then run: - -```sh -$ heroku authorizations:create --description "For Travis" -Creating OAuth Authorization... done -Client: -ID: -Description: For Travis -Scope: global -Token: -``` - -You'll set the `` value to the `HEROKU_API_KEY` env var. For example, you could add it on Travis like this: - -```sh -$ travis encrypt HEROKU_API_KEY= --add -``` - -You'll also need an email address that goes with your token: - -```sh -$ travis encrypt HEROKU_API_USER= --add -``` - -If you're running locally, your system credentials will be pulled from `heroku auth:token`. - -You'll also need to trigger a "setup" step for CI tasks. You can do it on Travis CI like this: - -``` -# .travis.yml -before_script: bundle exec hatchet ci:setup -``` - -and on Heroku CI like this: - -```json -{ - "environments": { - "test": { - "scripts": { - "test-setup": "bundle exec hatchet ci:setup", - } - } - } -} -``` +This library uses the heroku CLI and API. You will need to make your API key available to the system. If you're running on a CI platform, you'll need to generate an OAuth token and make it available on the system you're running on see the "CI" section below. ## Run the Tests @@ -83,7 +34,7 @@ Whatever testing framework you chose, we recommend using a parallel test runner If you're unfamiliar with the ruby testing eco-system or want some help, start by looking at existing projects. -*Spoilers: There is a section below on getting Hatchet to work on Travis. +*Spoilers*: There is a section below on getting Hatchet to work on CI ## Testing a Buildpack @@ -152,7 +103,7 @@ HATCHET_BUILDPACK_BASE=https://github.com/heroku/heroku-buildpack-ruby.git HATCHET_BUILDPACK_BRANCH=master ``` -If you do not specify `HATCHET_BUILDPACK_URL` the default Ruby buildpack will be used. If you do not specify a `HATCHET_BUILDPACK_BRANCH` the current branch you are on will be used. This is how the Ruby buildpack runs tests on branches on travis (by leaving `HATCHET_BUILDPACK_BRANCH` blank). +If you do not specify `HATCHET_BUILDPACK_URL` the default Ruby buildpack will be used. If you do not specify a `HATCHET_BUILDPACK_BRANCH` the current branch you are on will be used. This is how the Ruby buildpack runs tests on branches on CI (by leaving `HATCHET_BUILDPACK_BRANCH` blank). If the `ENV['HATCHET_RETRIES']` is set to a number, deploys are expected to work and automatically retry that number of times. Due to testing using a network and random failures, setting this value to `3` retries seems to work well. If an app cannot be deployed within its allotted number of retries, an error will be raised. @@ -391,31 +342,23 @@ Hatchet::Runner.new("rails5_ruby_schema_format", buildpacks: buildpacks).run_ci end ``` -## Testing on Travis - -Once you've got your tests working locally, you'll likely want to get them running on Travis because a) CI is awesome, and b) you can use pull requests to run your all your tests in parallel without having to kill your network connection. - -Set the `HATCHET_DEPLOY_STRATEGY` to `git`. +## Testing on CI -To run on travis, you will need to configure your `.travis.yml` to run the appropriate commands and to set up encrypted data so you can run tests against a valid heroku user. +Once you've got your tests working locally, you'll likely want to get them running on CI. For reference see the [Circle CI config from this repo](https://github.com/heroku/hatchet/blob/master/.circleci/config.yml) and the [Heroku CI config from the ruby buildpack](https://github.com/heroku/heroku-buildpack-ruby/blob/master/app.json). -For reference see the `.travis.yml` from [hatchet](https://github.com/heroku/hatchet/blob/master/.travis.yml) and the [heroku-ruby-buildpack](https://github.com/heroku/heroku-buildpack-ruby/blob/master/.travis.yml). To make running on travis easier, there is a rake task in Hatchet that can be run before your tests are executed +To make running on CI easier, there is a setup script in Hatchet that can be run before your tests are executed: ```yml -before_script: bundle exec hatchet ci:setup +bundle exec hatchet ci:setup ``` -I recommend signing up for a new heroku account for running your tests on travis to prevent exceding your API limit. Once you have the new api token, you can use this technique to [securely send travis the data](http://docs.travis-ci.com/user/environment-variables/#Secure-Variables). +If you're a Heroku employee see [private instructions for setting up test users](https://github.com/heroku/languages-team/blob/master/guides/create_test_users_for_ci.md) to generate a user a grab the API token. Once you have an API token you'll want to set up these env vars with your CI provider: -```sh -$ travis encrypt HEROKU_API_KEY= --add ``` - -If your Travis tests are containerized, you may need sudo to complete this successfully. In that case, add the following: - -```yml -before_script: bundle exec hatchet ci:setup -sudo: required +HATCHET_APP_LIMIT=100 +HATCHET_RETRIES=3 +HEROKU_API_KEY= +HEROKU_API_USER= ``` ## Extra App Commands From 8e05072699068f01c654f9a7443ed6572c2d32d3 Mon Sep 17 00:00:00 2001 From: schneems Date: Thu, 16 Apr 2020 13:28:27 -0500 Subject: [PATCH 4/5] Force .netrc to correct permission --- etc/ci_setup.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/etc/ci_setup.rb b/etc/ci_setup.rb index ad18adf..ac604c7 100755 --- a/etc/ci_setup.rb +++ b/etc/ci_setup.rb @@ -11,6 +11,7 @@ login #{ENV.fetch('HEROKU_API_USER')} password #{ENV.fetch('HEROKU_API_KEY')} EOF + `chmod 0600 "$HOME/.netrc"` end end From 09d358e2daebdc171f7a0c6ab991fdd16f18b9bd Mon Sep 17 00:00:00 2001 From: schneems Date: Thu, 16 Apr 2020 13:34:10 -0500 Subject: [PATCH 5/5] api.heroku.com entry in .netrc --- etc/ci_setup.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/etc/ci_setup.rb b/etc/ci_setup.rb index ac604c7..987cfaf 100755 --- a/etc/ci_setup.rb +++ b/etc/ci_setup.rb @@ -8,8 +8,11 @@ File.open(netrc_file, 'w') do |file| file.write <<-EOF machine git.heroku.com -login #{ENV.fetch('HEROKU_API_USER')} -password #{ENV.fetch('HEROKU_API_KEY')} + login #{ENV.fetch('HEROKU_API_USER')} + password #{ENV.fetch('HEROKU_API_KEY')} +machine api.heroku.com + login #{ENV.fetch('HEROKU_API_USER')} + password #{ENV.fetch('HEROKU_API_KEY')} EOF `chmod 0600 "$HOME/.netrc"` end