Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: global repo params #3379

Merged
merged 20 commits into from
Jul 7, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions cmd/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ const (
AllowRepoConfigFlag = "allow-repo-config"
AtlantisURLFlag = "atlantis-url"
AutomergeFlag = "automerge"
ParallelPlanFlag = "parallel-plan"
ParallelApplyFlag = "parallel-apply"
AutoplanModules = "autoplan-modules"
AutoplanModulesFromProjects = "autoplan-modules-from-projects"
AutoplanFileListFlag = "autoplan-file-list"
Expand Down Expand Up @@ -462,6 +464,14 @@ var boolFlags = map[string]boolFlag{
"VCS support is limited to: GitHub.",
defaultValue: false,
},
ParallelPlanFlag: {
description: "Run plan operations in parallel.",
defaultValue: false,
},
ParallelApplyFlag: {
description: "Run apply operations in parallel.",
defaultValue: false,
},
QuietPolicyChecks: {
description: "Exclude policy check comments from pull requests unless there's an actual error from conftest. This also excludes warnings.",
defaultValue: false,
Expand Down
2 changes: 2 additions & 0 deletions cmd/server_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ var testFlags = map[string]interface{}{
AllowDraftPRs: true,
PortFlag: 8181,
ParallelPoolSize: 100,
ParallelPlanFlag: true,
ParallelApplyFlag: true,
RepoAllowlistFlag: "github.com/runatlantis/atlantis",
RequireApprovalFlag: true,
RequireMergeableFlag: true,
Expand Down
3 changes: 3 additions & 0 deletions runatlantis.io/docs/repo-level-atlantis-yaml.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,9 @@ Use the `--parallel-pool-size` to configure the max number of plans and applies

Parallel plans and applies work across both multiple directories and multiple workspaces.

:::Note: When `--parallel-plan` or `--parallel-apply` is set at the server config level, it will be enabled for all repos. This behavior is similar to `automerge` and `--automerge`.
pseudomorph marked this conversation as resolved.
Show resolved Hide resolved
:::

### Configuring Planning

Given the directory structure:
Expand Down
16 changes: 16 additions & 0 deletions runatlantis.io/docs/server-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -656,6 +656,22 @@ This is useful when you have many projects and want to keep the pull request cle
```
Max size of the wait group that runs parallel plans and applies (if enabled). Defaults to `15`

### `--parallel-plan`
```bash
atlantis server --parallel-plan
# or
ATLANTIS_PARALLEL_PLAN=true
```
Whether to run plan operations in parallel. Defaults to `false`. When enabled at this level, it overrides the `parallel_plan` setting in the repo config.

### `--parallel-apply`
```bash
atlantis server --parallel-apply
# or
ATLANTIS_PARALLEL_APPLY=true
```
Whether to run apply operations in parallel. Defaults to `false`. When enabled at this level, it overrides the `parallel_apply` setting in the repo config.
pseudomorph marked this conversation as resolved.
Show resolved Hide resolved

### `--port`
```bash
atlantis server --port=4141
Expand Down
2 changes: 2 additions & 0 deletions server/controllers/events/events_controller_e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1194,6 +1194,8 @@ func setupE2E(t *testing.T, repoDir string, opt setupOption) (events_controllers
commentParser,
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down
42 changes: 25 additions & 17 deletions server/events/project_command_builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,6 @@ const (
DefaultWorkspace = "default"
// DefaultAutomergeEnabled is the default for the automerge setting.
DefaultAutomergeEnabled = false
// DefaultParallelApplyEnabled is the default for the parallel apply setting.
DefaultParallelApplyEnabled = false
// DefaultParallelPlanEnabled is the default for the parallel plan setting.
DefaultParallelPlanEnabled = false
// DefaultDeleteSourceBranchOnMerge being false is the default setting whether or not to remove a source branch on merge
DefaultDeleteSourceBranchOnMerge = false
)
Expand All @@ -50,6 +46,8 @@ func NewInstrumentedProjectCommandBuilder(
commentBuilder CommentBuilder,
skipCloneNoChanges bool,
EnableRegExpCmd bool,
EnableParallelPlan bool,
EnableParallelApply bool,
AutoDetectModuleFiles string,
AutoplanFileList string,
RestrictFileList bool,
Expand Down Expand Up @@ -77,6 +75,8 @@ func NewInstrumentedProjectCommandBuilder(
commentBuilder,
skipCloneNoChanges,
EnableRegExpCmd,
EnableParallelPlan,
EnableParallelApply,
AutoDetectModuleFiles,
AutoplanFileList,
RestrictFileList,
Expand All @@ -102,6 +102,8 @@ func NewProjectCommandBuilder(
commentBuilder CommentBuilder,
skipCloneNoChanges bool,
EnableRegExpCmd bool,
EnableParallelPlan bool,
EnableParallelApply bool,
AutoDetectModuleFiles string,
AutoplanFileList string,
RestrictFileList bool,
Expand All @@ -120,6 +122,8 @@ func NewProjectCommandBuilder(
PendingPlanFinder: pendingPlanFinder,
SkipCloneNoChanges: skipCloneNoChanges,
EnableRegExpCmd: EnableRegExpCmd,
EnableParallelPlan: EnableParallelPlan,
EnableParallelApply: EnableParallelApply,
AutoDetectModuleFiles: AutoDetectModuleFiles,
AutoplanFileList: AutoplanFileList,
RestrictFileList: RestrictFileList,
Expand Down Expand Up @@ -205,6 +209,8 @@ type DefaultProjectCommandBuilder struct {
AutoDetectModuleFiles string
AutoplanFileList string
EnableDiffMarkdownFormat bool
EnableParallelPlan bool
EnableParallelApply bool
RestrictFileList bool
SilenceNoProjects bool
TerraformExecutor terraform.Client
Expand Down Expand Up @@ -361,6 +367,15 @@ func (p *DefaultProjectCommandBuilder) buildAllCommandsByCfg(ctx *command.Contex
}
ctx.Log.Debug("moduleInfo for %s (matching %q) = %v", repoDir, p.AutoDetectModuleFiles, moduleInfo)

automerge := DefaultAutomergeEnabled
parallelApply := p.EnableParallelApply
parallelPlan := p.EnableParallelPlan
if hasRepoCfg {
automerge = automerge || repoCfg.Automerge
parallelApply = parallelApply || repoCfg.ParallelApply
parallelPlan = parallelPlan || repoCfg.ParallelPlan
}

if len(repoCfg.Projects) > 0 {
matchingProjects, err := p.ProjectFinder.DetermineProjectsViaConfig(ctx.Log, modifiedFiles, repoCfg, repoDir, moduleInfo)
if err != nil {
Expand All @@ -380,9 +395,9 @@ func (p *DefaultProjectCommandBuilder) buildAllCommandsByCfg(ctx *command.Contex
mergedCfg,
commentFlags,
repoDir,
repoCfg.Automerge,
repoCfg.ParallelApply,
repoCfg.ParallelPlan,
automerge,
parallelApply,
parallelPlan,
verbose,
p.TerraformExecutor,
)...)
Expand All @@ -404,14 +419,7 @@ func (p *DefaultProjectCommandBuilder) buildAllCommandsByCfg(ctx *command.Contex
if err != nil {
return nil, errors.Wrapf(err, "looking for Terraform Cloud workspace from configuration %s", repoDir)
}
automerge := DefaultAutomergeEnabled
parallelApply := DefaultParallelApplyEnabled
parallelPlan := DefaultParallelPlanEnabled
if hasRepoCfg {
automerge = repoCfg.Automerge
parallelApply = repoCfg.ParallelApply
parallelPlan = repoCfg.ParallelPlan
}

pCfg := p.GlobalCfg.DefaultProjCfg(ctx.Log, ctx.Pull.BaseRepo.ID(), mp.Path, pWorkspace)

projCtxs = append(projCtxs,
Expand Down Expand Up @@ -698,8 +706,8 @@ func (p *DefaultProjectCommandBuilder) buildProjectCommandCtx(ctx *command.Conte
var projCtxs []command.ProjectContext
var projCfg valid.MergedProjectCfg
automerge := DefaultAutomergeEnabled
parallelApply := DefaultParallelApplyEnabled
parallelPlan := DefaultParallelPlanEnabled
parallelApply := p.EnableParallelApply
parallelPlan := p.EnableParallelPlan
if repoCfgPtr != nil {
automerge = repoCfgPtr.Automerge
parallelApply = repoCfgPtr.ParallelApply
Expand Down
8 changes: 8 additions & 0 deletions server/events/project_command_builder_internal_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -667,6 +667,8 @@ projects:
&CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -878,6 +880,8 @@ projects:
&CommentParser{ExecutableName: "atlantis"},
false,
true,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -1122,6 +1126,8 @@ workflows:
&CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -1274,6 +1280,8 @@ projects:
&CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down
22 changes: 22 additions & 0 deletions server/events/project_command_builder_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,8 @@ projects:
&events.CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -454,6 +456,8 @@ projects:
&events.CommentParser{ExecutableName: "atlantis"},
false,
true,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -636,6 +640,8 @@ projects:
&events.CommentParser{ExecutableName: "atlantis"},
false,
true,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
true,
Expand Down Expand Up @@ -822,6 +828,8 @@ projects:
&events.CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -921,6 +929,8 @@ func TestDefaultProjectCommandBuilder_BuildMultiApply(t *testing.T) {
&events.CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -1011,6 +1021,8 @@ projects:
&events.CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -1096,6 +1108,8 @@ func TestDefaultProjectCommandBuilder_EscapeArgs(t *testing.T) {
&events.CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -1263,6 +1277,8 @@ projects:
&events.CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -1357,6 +1373,8 @@ parallel_plan: true`,
&events.CommentParser{ExecutableName: "atlantis"},
true,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -1421,6 +1439,8 @@ func TestDefaultProjectCommandBuilder_WithPolicyCheckEnabled_BuildAutoplanComman
&events.CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down Expand Up @@ -1508,6 +1528,8 @@ func TestDefaultProjectCommandBuilder_BuildVersionCommand(t *testing.T) {
&events.CommentParser{ExecutableName: "atlantis"},
false,
false,
false,
false,
"",
"**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
false,
Expand Down
2 changes: 2 additions & 0 deletions server/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -581,6 +581,8 @@ func NewServer(userConfig UserConfig, config Config) (*Server, error) {
commentParser,
userConfig.SkipCloneNoChanges,
userConfig.EnableRegExpCmd,
userConfig.ParallelPlan,
userConfig.ParallelApply,
userConfig.AutoplanModulesFromProjects,
userConfig.AutoplanFileList,
userConfig.RestrictFileList,
Expand Down
2 changes: 2 additions & 0 deletions server/user_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ type UserConfig struct {
LogLevel string `mapstructure:"log-level"`
MarkdownTemplateOverridesDir string `mapstructure:"markdown-template-overrides-dir"`
ParallelPoolSize int `mapstructure:"parallel-pool-size"`
ParallelPlan bool `mapstructure:"parallel-plan"`
ParallelApply bool `mapstructure:"parallel-apply"`
StatsNamespace string `mapstructure:"stats-namespace"`
PlanDrafts bool `mapstructure:"allow-draft-prs"`
Port int `mapstructure:"port"`
Expand Down