Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: streaming terraform logs in real-time #1937

Merged
merged 31 commits into from
Dec 30, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
1d59a6a
Current progress (#86)
msarvar Jul 26, 2021
a9e1712
Orca 845 (#89)
msarvar Aug 2, 2021
3401510
Revert "Orca 845 (#89)"
Aayyush Dec 10, 2021
b91001e
Fixing merge
msarvar Aug 2, 2021
b516c11
Channel Refactoring (#91)
isatasan Aug 17, 2021
8761857
Fixing errors
Aayyush Dec 10, 2021
56fd6c4
Update the PR with a log streaming URL (#98)
isatasan Aug 31, 2021
3c4ceaa
Log Streaming Bug Fix and Refactor (#101)
Aayyush Sep 9, 2021
f422fc5
Orca 935 move log streaming url to status metadata (#106)
msarvar Sep 15, 2021
ca9b851
Removing tf show output when streaming logs (#104)
Aayyush Sep 16, 2021
d2c22ea
Adding logic to clear terminal screen when running new plan (#105)
Aayyush Sep 16, 2021
9d67480
Do not display non essential tf commands (#108)
msarvar Sep 16, 2021
9c88cd0
Remove blocking call from within Rlock. (#119)
nishkrishnan Sep 27, 2021
d574dcb
Use communication channel to notify about the websocket closed (#131)
msarvar Sep 28, 2021
bcd5a5e
increasing bufio buffer size to 10mb (#133)
msarvar Sep 28, 2021
34323fb
Remove no-color args and strip ansi for PRs. (#136)
nishkrishnan Oct 6, 2021
0c71715
Make log-streaming terminal UI full screen (#134)
Aayyush Oct 6, 2021
6276c72
Add error checking to job url generator. (#137)
nishkrishnan Oct 13, 2021
91f23a7
Making terminal scrollback to 15000 (#138)
Aayyush Oct 13, 2021
2b5fa72
Support Log Streaming URLs for project without a project name (#140)
Aayyush Oct 21, 2021
b8ca55e
Generating handler mock
Aayyush Dec 13, 2021
cf2812e
Create NoopProjectCommandOutputHandler to disable log streaming if TF…
msarvar Nov 10, 2021
bfbd2a3
Refactor job controller and websocket logic. (#150)
nishkrishnan Dec 2, 2021
e48aba0
Fixing test
Aayyush Dec 15, 2021
ce64ed2
Fixing e2e tests
Aayyush Dec 15, 2021
781ab37
Reverting test fixtures to match master
Aayyush Dec 15, 2021
e40b984
Revert "Reverting test fixtures to match master"
Aayyush Dec 28, 2021
a59dd4b
Final pass
Aayyush Dec 28, 2021
b3cbc81
Merge branch 'master' into log-streaming-upstream
Aayyush Dec 28, 2021
670ec04
Fix unused variable error
Aayyush Dec 28, 2021
009a246
retrigger checks
Aayyush Dec 29, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ require (
github.com/googleapis/gax-go/v2 v2.1.1 // indirect
github.com/gorilla/css v1.0.0 // indirect
github.com/gorilla/mux v1.8.0
github.com/gorilla/websocket v1.4.2 // indirect
github.com/gorilla/websocket v1.4.2
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-getter v1.5.9
github.com/hashicorp/go-retryablehttp v0.6.8 // indirect
Expand Down
18 changes: 9 additions & 9 deletions runatlantis.io/docs/custom-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,19 +128,19 @@ workflows:
myworkflow:
plan:
steps:
- run: terraform init -input=false -no-color
- run: terraform init -input=false

# If you're using workspaces you need to select the workspace using the
# $WORKSPACE environment variable.
- run: terraform workspace select -no-color $WORKSPACE
- run: terraform workspace select $WORKSPACE

# You MUST output the plan using -out $PLANFILE because Atlantis expects
# plans to be in a specific location.
- run: terraform plan -input=false -refresh -no-color -out $PLANFILE
- run: terraform plan -input=false -refresh -out $PLANFILE
apply:
steps:
# Again, you must use the $PLANFILE environment variable.
- run: terraform apply -no-color $PLANFILE
- run: terraform apply $PLANFILE
```

### Terragrunt
Expand Down Expand Up @@ -176,14 +176,14 @@ workflows:
- env:
name: TERRAGRUNT_TFPATH
command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
- run: terragrunt plan -no-color -out=$PLANFILE
- run: terragrunt show -no-color -json $PLANFILE > $SHOWFILE
- run: terragrunt plan -out=$PLANFILE
- run: terragrunt show -json $PLANFILE > $SHOWFILE
apply:
steps:
- env:
name: TERRAGRUNT_TFPATH
command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
- run: terragrunt apply -no-color $PLANFILE
- run: terragrunt apply $PLANFILE
```

If using the repo's `atlantis.yaml` file you would use the following config:
Expand All @@ -201,13 +201,13 @@ workflows:
- env:
name: TERRAGRUNT_TFPATH
command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
- run: terragrunt plan -no-color -out $PLANFILE
- run: terragrunt plan -out $PLANFILE
apply:
steps:
- env:
name: TERRAGRUNT_TFPATH
command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
- run: terragrunt apply -no-color $PLANFILE
- run: terragrunt apply $PLANFILE
```

**NOTE:** If using the repo's `atlantis.yaml` file, you will need to specify each directory that is a Terragrunt project.
Expand Down
14 changes: 9 additions & 5 deletions server/controllers/events/events_controller_e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import (
"github.com/runatlantis/atlantis/server/events/webhooks"
"github.com/runatlantis/atlantis/server/events/yaml"
"github.com/runatlantis/atlantis/server/events/yaml/valid"
handlermocks "github.com/runatlantis/atlantis/server/handlers/mocks"
"github.com/runatlantis/atlantis/server/logging"
. "github.com/runatlantis/atlantis/testing"
)
Expand Down Expand Up @@ -816,6 +817,7 @@ func setupE2E(t *testing.T, repoDir string) (events_controllers.VCSEventsControl
e2eStatusUpdater := &events.DefaultCommitStatusUpdater{Client: e2eVCSClient, TitleBuilder: vcs.StatusTitleBuilder{TitlePrefix: "atlantis"}}
e2eGithubGetter := mocks.NewMockGithubPullGetter()
e2eGitlabGetter := mocks.NewMockGitlabMergeRequestGetter()
projectCmdOutputHandler := handlermocks.NewMockProjectCommandOutputHandler()

// Real dependencies.
logger := logging.NewNoopLogger(t)
Expand All @@ -830,7 +832,7 @@ func setupE2E(t *testing.T, repoDir string) (events_controllers.VCSEventsControl
GithubUser: "github-user",
GitlabUser: "gitlab-user",
}
terraformClient, err := terraform.NewClient(logger, binDir, cacheDir, "", "", "", "default-tf-version", "https://releases.hashicorp.com", &NoopTFDownloader{}, false)
terraformClient, err := terraform.NewClient(logger, binDir, cacheDir, "", "", "", "default-tf-version", "https://releases.hashicorp.com", &NoopTFDownloader{}, false, projectCmdOutputHandler)
Ok(t, err)
boltdb, err := db.New(dataDir)
Ok(t, err)
Expand Down Expand Up @@ -1057,10 +1059,12 @@ func setupE2E(t *testing.T, repoDir string) (events_controllers.VCSEventsControl
TestingMode: true,
CommandRunner: commandRunner,
PullCleaner: &events.PullClosedExecutor{
Locker: lockingClient,
VCSClient: e2eVCSClient,
WorkingDir: workingDir,
DB: boltdb,
Locker: lockingClient,
VCSClient: e2eVCSClient,
WorkingDir: workingDir,
DB: boltdb,
PullClosedTemplate: &events.PullClosedEventTemplate{},
LogStreamResourceCleaner: projectCmdOutputHandler,
},
Logger: logger,
Parser: eventParser,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@ null_resource.automerge[0]: Creation complete after *s [id=*******************]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.


```

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@ null_resource.automerge[0]: Creation complete after *s [id=*******************]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.


```

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Terraform will perform the following actions:

Plan: 1 to add, 0 to change, 0 to destroy.


```

* :arrow_forward: To **apply** this plan, comment:
Expand Down Expand Up @@ -50,6 +51,7 @@ Terraform will perform the following actions:

Plan: 1 to add, 0 to change, 0 to destroy.


```

* :arrow_forward: To **apply** this plan, comment:
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Loading