[fix] atlantis autoplan when modules change #304
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This will autoplan all components when any module changes. This is brute force approach, but should catch all changes. In the future we can try to calculate a depenency graph and only trigger plans that are needed, but that will require some thought and planning.
ryanking
changed the title
Codecov Report
@@ Coverage Diff @@
## master #304 +/- ##
=======================================
Coverage 81.38% 81.38%
=======================================
Files 18 18
Lines 1590 1590
=======================================
Hits 1294 1294
Misses 172 172
Partials 124 124Continue to review full report at Codecov.
|
edulop91
approved these changes
Jun 22, 2019
echanakira
pushed a commit
that referenced
this pull request
Jun 24, 2019
[fix] atlantis autoplan when modules changeThis configures auto-planning when anything in modules changes. In the future we may want to build a dependency graph between modules and components (including data dependencies), but for now the brute force approach will ensure that we don't miss changes. ### Test Plan * CI * ran on 1 internal repo ### References * https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#configuring-autoplanning
echanakira
added a commit
that referenced
this pull request
Jun 26, 2019
…nfig to yaml (#298) * Adding yaml_migrate files to branch * Created yaml_migrate command, currently creates a alphabetized fogg.yml * added yaml support in v1/config.go and init.go supports creation of .json and .yml files * Added yaml tags to all config structs, minor issue with plugins * Fixed component aws_provider spelling mistake * Resolved all initial code review comments * Added yaml support to all commands; however, unit tests are not yet updated to match the changes * Fixed all unit tests to work with current json, and added missing yaml tags * Uncommented golden-files, but YamlIntegration not working * Fixed potential PR problems * Changed apply_test.go to work with yaml * Fixed apply, plan, and golden test files to use yaml * Passing all tests, fixed: aws-config, plan, apply, and config.go * Removed code duplication and added a json test for apply_tests * Passing test apply_test.go and user can now specify any file name, instead of fogg.yml of fogg.json * fixed config structure within apply_test.go * Added yaml tests to plan and created new yaml folders in testdata * Added yaml tests for v2/config and created new yaml folders in testdata * Fixed: PR Comments, Added: os.IsNotExist to check if error is caused by a non-existent file * Fixed: added yaml support to all of v2/config * Fixed trivial PR comments * Added more config variables to test data * Fixed: PR comments * Removed resolveTfLint tests * Moved file closing * added (*)json.Number to v1/config.go * Removed unnecessary flag and consolidated yamlMigrate * Removed unnecessary comment, and moved file close for clarity * Removed extra whitespace and moved file close for clarity * Removed unnecessary comments and commented plan config structs * Moved yaml-migrate from exp to cmd * Moved yaml_migrate from exp to util, and renamed to yaml * Fixed: package import * removed extra whtiespace and continued make file space consistency * re added config flag * Simplified migrate/yaml.go, fixed marshaling, created migrate folder for future upgrade features * Removed yaml support from v1, tests and helper methods * fogg v1 minimally supports yaml * Upgrade now converts to yaml * Updated: ReadConfig and detectVersion to parse based on file extension * [feature] atlantis integration (#303) [feature] atlantis integrationThis PR adds Atlantis (<runatlantis.io>) integration to fogg. The goal is to be able to use fogg to generate an atlantis.yaml configuration file and all related tools, scripts etc. Happy to take feedback on both the approach and the implementation. ### Approach The basic approach is, in no particular order– * To the degree possible it would be great to be able to make changes to how Atlantis works without touching the Atlantis instances, so as much as possible is put into the per-repo config file. * We allow configuring Atlantis on a per-component basis. I think this is important as I would expect teams to adopt it incrementally. * We do a 2-phase plan - first . plan for each component (so that we can set a var in each) and then walk those plans and generate a plan for the atlantis.yaml and aws config * We generate an aws config for atlantis. Because we rely on aws profiles for authentication, we need to find a way to get the Atlantis instance configured with all the right profiles. In the future we may want to think about doing this for other modes. * If atlantis is enabled for a component, `make apply` will by default fail. You have to add `FORCE=1` to override. This should nudge people in the right direction while allowing a fall-back to running locally (important for adoption phase). * we add a `make setup` that will install fogg and run `fogg setup` (you can run this from components, not just top-level) * we no longer run `terraform get` since that is redundant with `terrraform init` and we **don't run -update=true anymore** which means that modules will not get updated. People should be using stable versions anyway and this makes it much faster * tfenv is now installed inside the repo, rather than homedir. This makes it so that we can use tfenv in atlantis (rather than atlantis' own process for getting terraform versions) --- Note there are also a handful of stylistic changes in this diff. Tried to keep those to a minimum, but also wanted to make some improvements, especially to our makefiles. ## Test Plan * unit tests * ran on shared-infra repo for 1 component ## References * <https://www.runatlantis.io/docs/> * Fixes #290 * Fixes #289 * Fixes #278 * release version 0.24.0 * [fix] atlantis autoplan when modules change (#304) [fix] atlantis autoplan when modules changeThis configures auto-planning when anything in modules changes. In the future we may want to build a dependency graph between modules and components (including data dependencies), but for now the brute force approach will ensure that we don't miss changes. ### Test Plan * CI * ran on 1 internal repo ### References * https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#configuring-autoplanning * release version 0.24.1 * [feature] TravisCI now allows per component configuration + allows to set a specific make command to run (#305) [feature] TravisCI now allows per component configuration + allows to set a specific make command to run### Summary - Mostly lifted from the Atlantis implementation ### Test Plan - unittest - run in existing repo * release version 0.25.0 * Fixed struct type merge conflicts * Update config/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Update config/v1/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Update config/v1/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Added .yaml to switch and standardized err name to e * Update config/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Standardized ReadConfig and error * Changed WrapUser to allow format variables * Fixed spacing * Fixed duplicate key in Plan struct * Elaborate v2/config.go description Co-Authored-By: Ryan King <[email protected]> * Update test name fog v1/config_test.go Co-Authored-By: Ryan King <[email protected]> * Rename ReadConfig func description Co-Authored-By: Ryan King <[email protected]> * Update plan/plan.go Co-Authored-By: Ryan King <[email protected]> * Refactor Print description Co-Authored-By: Eduardo Lopez <[email protected]> * updated fogg init to use v2 and made config.go error more detailed * removed extra whitespace in v1 and v2 configs * Added Dynamo table to init and fixed init test to match v2 struct * Removed unnecessary fmt * remove duplication of InitConfig Co-Authored-By: Ryan King <[email protected]> * fixed v1/config project tag * Removed TODO and added description to FindAndReadConfig
echanakira
added a commit
that referenced
this pull request
Jul 1, 2019
…nfig to yaml (#298) * Adding yaml_migrate files to branch * Created yaml_migrate command, currently creates a alphabetized fogg.yml * added yaml support in v1/config.go and init.go supports creation of .json and .yml files * Added yaml tags to all config structs, minor issue with plugins * Fixed component aws_provider spelling mistake * Resolved all initial code review comments * Added yaml support to all commands; however, unit tests are not yet updated to match the changes * Fixed all unit tests to work with current json, and added missing yaml tags * Uncommented golden-files, but YamlIntegration not working * Fixed potential PR problems * Changed apply_test.go to work with yaml * Fixed apply, plan, and golden test files to use yaml * Passing all tests, fixed: aws-config, plan, apply, and config.go * Removed code duplication and added a json test for apply_tests * Passing test apply_test.go and user can now specify any file name, instead of fogg.yml of fogg.json * fixed config structure within apply_test.go * Added yaml tests to plan and created new yaml folders in testdata * Added yaml tests for v2/config and created new yaml folders in testdata * Fixed: PR Comments, Added: os.IsNotExist to check if error is caused by a non-existent file * Fixed: added yaml support to all of v2/config * Fixed trivial PR comments * Added more config variables to test data * Fixed: PR comments * Removed resolveTfLint tests * Moved file closing * added (*)json.Number to v1/config.go * Removed unnecessary flag and consolidated yamlMigrate * Removed unnecessary comment, and moved file close for clarity * Removed extra whitespace and moved file close for clarity * Removed unnecessary comments and commented plan config structs * Moved yaml-migrate from exp to cmd * Moved yaml_migrate from exp to util, and renamed to yaml * Fixed: package import * removed extra whtiespace and continued make file space consistency * re added config flag * Simplified migrate/yaml.go, fixed marshaling, created migrate folder for future upgrade features * Removed yaml support from v1, tests and helper methods * fogg v1 minimally supports yaml * Upgrade now converts to yaml * Updated: ReadConfig and detectVersion to parse based on file extension * [feature] atlantis integration (#303) [feature] atlantis integrationThis PR adds Atlantis (<runatlantis.io>) integration to fogg. The goal is to be able to use fogg to generate an atlantis.yaml configuration file and all related tools, scripts etc. Happy to take feedback on both the approach and the implementation. ### Approach The basic approach is, in no particular order– * To the degree possible it would be great to be able to make changes to how Atlantis works without touching the Atlantis instances, so as much as possible is put into the per-repo config file. * We allow configuring Atlantis on a per-component basis. I think this is important as I would expect teams to adopt it incrementally. * We do a 2-phase plan - first . plan for each component (so that we can set a var in each) and then walk those plans and generate a plan for the atlantis.yaml and aws config * We generate an aws config for atlantis. Because we rely on aws profiles for authentication, we need to find a way to get the Atlantis instance configured with all the right profiles. In the future we may want to think about doing this for other modes. * If atlantis is enabled for a component, `make apply` will by default fail. You have to add `FORCE=1` to override. This should nudge people in the right direction while allowing a fall-back to running locally (important for adoption phase). * we add a `make setup` that will install fogg and run `fogg setup` (you can run this from components, not just top-level) * we no longer run `terraform get` since that is redundant with `terrraform init` and we **don't run -update=true anymore** which means that modules will not get updated. People should be using stable versions anyway and this makes it much faster * tfenv is now installed inside the repo, rather than homedir. This makes it so that we can use tfenv in atlantis (rather than atlantis' own process for getting terraform versions) --- Note there are also a handful of stylistic changes in this diff. Tried to keep those to a minimum, but also wanted to make some improvements, especially to our makefiles. ## Test Plan * unit tests * ran on shared-infra repo for 1 component ## References * <https://www.runatlantis.io/docs/> * Fixes #290 * Fixes #289 * Fixes #278 * release version 0.24.0 * [fix] atlantis autoplan when modules change (#304) [fix] atlantis autoplan when modules changeThis configures auto-planning when anything in modules changes. In the future we may want to build a dependency graph between modules and components (including data dependencies), but for now the brute force approach will ensure that we don't miss changes. ### Test Plan * CI * ran on 1 internal repo ### References * https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#configuring-autoplanning * release version 0.24.1 * [feature] TravisCI now allows per component configuration + allows to set a specific make command to run (#305) [feature] TravisCI now allows per component configuration + allows to set a specific make command to run### Summary - Mostly lifted from the Atlantis implementation ### Test Plan - unittest - run in existing repo * release version 0.25.0 * Fixed struct type merge conflicts * Update config/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Update config/v1/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Update config/v1/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Added .yaml to switch and standardized err name to e * Update config/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Standardized ReadConfig and error * Changed WrapUser to allow format variables * Fixed spacing * Fixed duplicate key in Plan struct * Elaborate v2/config.go description Co-Authored-By: Ryan King <[email protected]> * Update test name fog v1/config_test.go Co-Authored-By: Ryan King <[email protected]> * Rename ReadConfig func description Co-Authored-By: Ryan King <[email protected]> * Update plan/plan.go Co-Authored-By: Ryan King <[email protected]> * Refactor Print description Co-Authored-By: Eduardo Lopez <[email protected]> * updated fogg init to use v2 and made config.go error more detailed * removed extra whitespace in v1 and v2 configs * Added Dynamo table to init and fixed init test to match v2 struct * Removed unnecessary fmt * remove duplication of InitConfig Co-Authored-By: Ryan King <[email protected]> * fixed v1/config project tag * Removed TODO and added description to FindAndReadConfig
palasha
pushed a commit
to palasha/fogg
that referenced
this pull request
Apr 7, 2020
[fix] atlantis autoplan when modules changeThis configures auto-planning when anything in modules changes. In the future we may want to build a dependency graph between modules and components (including data dependencies), but for now the brute force approach will ensure that we don't miss changes. ### Test Plan * CI * ran on 1 internal repo ### References * https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#configuring-autoplanning
palasha
pushed a commit
to palasha/fogg
that referenced
this pull request
Apr 7, 2020
…nfig to yaml (chanzuckerberg#298) * Adding yaml_migrate files to branch * Created yaml_migrate command, currently creates a alphabetized fogg.yml * added yaml support in v1/config.go and init.go supports creation of .json and .yml files * Added yaml tags to all config structs, minor issue with plugins * Fixed component aws_provider spelling mistake * Resolved all initial code review comments * Added yaml support to all commands; however, unit tests are not yet updated to match the changes * Fixed all unit tests to work with current json, and added missing yaml tags * Uncommented golden-files, but YamlIntegration not working * Fixed potential PR problems * Changed apply_test.go to work with yaml * Fixed apply, plan, and golden test files to use yaml * Passing all tests, fixed: aws-config, plan, apply, and config.go * Removed code duplication and added a json test for apply_tests * Passing test apply_test.go and user can now specify any file name, instead of fogg.yml of fogg.json * fixed config structure within apply_test.go * Added yaml tests to plan and created new yaml folders in testdata * Added yaml tests for v2/config and created new yaml folders in testdata * Fixed: PR Comments, Added: os.IsNotExist to check if error is caused by a non-existent file * Fixed: added yaml support to all of v2/config * Fixed trivial PR comments * Added more config variables to test data * Fixed: PR comments * Removed resolveTfLint tests * Moved file closing * added (*)json.Number to v1/config.go * Removed unnecessary flag and consolidated yamlMigrate * Removed unnecessary comment, and moved file close for clarity * Removed extra whitespace and moved file close for clarity * Removed unnecessary comments and commented plan config structs * Moved yaml-migrate from exp to cmd * Moved yaml_migrate from exp to util, and renamed to yaml * Fixed: package import * removed extra whtiespace and continued make file space consistency * re added config flag * Simplified migrate/yaml.go, fixed marshaling, created migrate folder for future upgrade features * Removed yaml support from v1, tests and helper methods * fogg v1 minimally supports yaml * Upgrade now converts to yaml * Updated: ReadConfig and detectVersion to parse based on file extension * [feature] atlantis integration (chanzuckerberg#303) [feature] atlantis integrationThis PR adds Atlantis (<runatlantis.io>) integration to fogg. The goal is to be able to use fogg to generate an atlantis.yaml configuration file and all related tools, scripts etc. Happy to take feedback on both the approach and the implementation. ### Approach The basic approach is, in no particular order– * To the degree possible it would be great to be able to make changes to how Atlantis works without touching the Atlantis instances, so as much as possible is put into the per-repo config file. * We allow configuring Atlantis on a per-component basis. I think this is important as I would expect teams to adopt it incrementally. * We do a 2-phase plan - first . plan for each component (so that we can set a var in each) and then walk those plans and generate a plan for the atlantis.yaml and aws config * We generate an aws config for atlantis. Because we rely on aws profiles for authentication, we need to find a way to get the Atlantis instance configured with all the right profiles. In the future we may want to think about doing this for other modes. * If atlantis is enabled for a component, `make apply` will by default fail. You have to add `FORCE=1` to override. This should nudge people in the right direction while allowing a fall-back to running locally (important for adoption phase). * we add a `make setup` that will install fogg and run `fogg setup` (you can run this from components, not just top-level) * we no longer run `terraform get` since that is redundant with `terrraform init` and we **don't run -update=true anymore** which means that modules will not get updated. People should be using stable versions anyway and this makes it much faster * tfenv is now installed inside the repo, rather than homedir. This makes it so that we can use tfenv in atlantis (rather than atlantis' own process for getting terraform versions) --- Note there are also a handful of stylistic changes in this diff. Tried to keep those to a minimum, but also wanted to make some improvements, especially to our makefiles. ## Test Plan * unit tests * ran on shared-infra repo for 1 component ## References * <https://www.runatlantis.io/docs/> * Fixes chanzuckerberg#290 * Fixes chanzuckerberg#289 * Fixes chanzuckerberg#278 * release version 0.24.0 * [fix] atlantis autoplan when modules change (chanzuckerberg#304) [fix] atlantis autoplan when modules changeThis configures auto-planning when anything in modules changes. In the future we may want to build a dependency graph between modules and components (including data dependencies), but for now the brute force approach will ensure that we don't miss changes. ### Test Plan * CI * ran on 1 internal repo ### References * https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#configuring-autoplanning * release version 0.24.1 * [feature] TravisCI now allows per component configuration + allows to set a specific make command to run (chanzuckerberg#305) [feature] TravisCI now allows per component configuration + allows to set a specific make command to run### Summary - Mostly lifted from the Atlantis implementation ### Test Plan - unittest - run in existing repo * release version 0.25.0 * Fixed struct type merge conflicts * Update config/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Update config/v1/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Update config/v1/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Added .yaml to switch and standardized err name to e * Update config/config.go Co-Authored-By: Eduardo Lopez <[email protected]> * Standardized ReadConfig and error * Changed WrapUser to allow format variables * Fixed spacing * Fixed duplicate key in Plan struct * Elaborate v2/config.go description Co-Authored-By: Ryan King <[email protected]> * Update test name fog v1/config_test.go Co-Authored-By: Ryan King <[email protected]> * Rename ReadConfig func description Co-Authored-By: Ryan King <[email protected]> * Update plan/plan.go Co-Authored-By: Ryan King <[email protected]> * Refactor Print description Co-Authored-By: Eduardo Lopez <[email protected]> * updated fogg init to use v2 and made config.go error more detailed * removed extra whitespace in v1 and v2 configs * Added Dynamo table to init and fixed init test to match v2 struct * Removed unnecessary fmt * remove duplication of InitConfig Co-Authored-By: Ryan King <[email protected]> * fixed v1/config project tag * Removed TODO and added description to FindAndReadConfig
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This configures auto-planning when anything in modules changes. In the future we may want to build a dependency graph between modules and components (including data dependencies), but for now the brute force approach will ensure that we don't miss changes.
Test Plan
References