Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

file out writer implementation #1229

Merged
merged 8 commits into from
May 4, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=accurics_terrascan&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=accurics_terrascan)
[![AUR package](https://repology.org/badge/version-for-repo/aur/terrascan.svg)](https://repology.org/project/terrascan/versions)
[![codecov](https://codecov.io/gh/accurics/terrascan/branch/master/graph/badge.svg)](https://codecov.io/gh/accurics/terrascan)
[![Documentation Status](https://readthedocs.com/projects/accurics-terrascan/badge/?version=latest)](https://docs.accurics.com/projects/accurics-terrascan/en/latest/?badge=latest)
[![Documentation Status](https://readthedocs.com/projects/accurics-terrascan/badge/?version=latest)](https://runterrascan.io/)
[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](code_of_conduct.md)

## Introduction
Expand Down
4 changes: 2 additions & 2 deletions deploy/helm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Validating Webhook as well, that'll use the terrascan server as its backend.
In server mode, terrascan will act both as an API server for
performing remote scans of IAC, as well as a validating admission
webhook for a Kubernetes cluster. Further details can be found in
the [main documentation](https://docs.accurics.com/projects/accurics-terrascan/en/latest/).
the [main documentation](https://runterrascan.io/).

## Usage
### Set up TLS certificates
Expand All @@ -25,7 +25,7 @@ will be named `terrascan` and hosted in `terrascan` namespace. You'll have to ma

### Terrascan configuration file
This chart will look for a [terrascan configuration
file](https://docs.accurics.com/projects/accurics-terrascan/en/latest/usage/#config-file)
file](https://runterrascan.io/docs/usage/config_options/)
at `data/config.toml`. If that file exists before running `helm
install`, it's contents will be loaded into a configMap and provided
to the terrascan server.
Expand Down
24 changes: 12 additions & 12 deletions deploy/kustomize/README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
## INSTALLING TERRASCAN IN A KUBERNETES CLUSTER USING KUSTOMIZE
## Installing terrascan in a Kubernetes cluster using Kustomize

This guide will help you install terrascan server inside your kubernetes cluster.
We have covered the following use cases in the sections below.

- #####[Deploying Terrascan Server](deploying-terrascan-server)
- [Deploying Terrascan Server](#deploying-terrascan-server)
Terrascan operating in http server mode.

- ####[Deploying Terrascan Server in TLS Mode](deploying-terrascan-server-in-tls-mode)
- [Deploying Terrascan Server in TLS Mode](#deploying-terrascan-server-in-tls-mode)
Terrascan operating in https server mode. This deployment is also a foundation for the terrascan webhook setup.

- ####[Deploying Terrascan Server for Remote Repository Scan](deploying-terrascan-server-for-remote-repository-scan)
- [Deploying Terrascan Server for Remote Repository Scan](#deploying-terrascan-server-for-private-remote-repository-scan)
Terrascan in https server mode installed with ssh capabilities, to scan ***private*** remote repositories. For remote
scanning public repos, deploying `Terrascan Server in TLS Mode` is sufficient.
This deployment can be handy for use-cases like an argocd pre-sync hook that sends remote repository scan requests to the server.

- ####[Setting Up Terrascan Webhook](setting-up-terrascan-webhook)
- [Setting Up Terrascan Webhook](#setting-up-terrascan-webhook)
A Kubernetes Validating Webhook, that safeguards your cluster by denying the creation of kubernetes resources that
can cause potential security violations.

- ####[Clean Up](clean-up)
- [Clean Up](#clean-up)

###PRE-REQUISITE
### Pre-requisite
1. Make sure you have required access on the kubernetes cluster to create and update the following resources:

- Secrets
Expand Down Expand Up @@ -63,7 +63,7 @@ terrascan server. The string replacement will be required in the following files
kubectl create namespace terrascan
```

###Deploying Terrascan Server
### Deploying Terrascan Server

Deploy terrascan in server mode operating in plain HTTP mode.

Expand All @@ -78,7 +78,7 @@ Deploy terrascan in server mode operating in plain HTTP mode.
kustomize build server/ | kubectl apply -f -
```

###Deploying Terrascan Server in TLS Mode
### Deploying Terrascan Server in TLS Mode

Deploy terrascan in server mode operating in HTTPS mode.

Expand Down Expand Up @@ -126,7 +126,7 @@ Deploy terrascan in server mode operating in HTTPS mode.
kustomize build server-tls/ | kubectl apply -f -
```

###Deploying Terrascan Server For Private Remote Repository Scan
### Deploying Terrascan Server For Private Remote Repository Scan

For scanning ***Private*** remote IaC file repositories, Terrascan must be provided with the required SSH keys to connect and clone the
repository locally to scan it. The following steps will help in setting up for that.
Expand Down Expand Up @@ -170,7 +170,7 @@ repository locally to scan it. The following steps will help in setting up for t
kustomize build server-remote-repo-scan/ | kubectl apply -f -
```

###Setting Up Terrascan Webhook
### Setting Up Terrascan Webhook
If you want to setup a Validating Webhook that scans your incoming kubernetes resources using terrascan,
follow the steps below.

Expand Down Expand Up @@ -245,7 +245,7 @@ follow the steps below.
kubectl apply -f webhook/validating-webhook.yaml
```

###Clean Up
### Clean Up

Deleting the namespace that you used, will delete all the resources itself.
```bash
Expand Down
2 changes: 1 addition & 1 deletion docs/integrations/argocd-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ Host github.com

After making changes to the webhook deployment file, apply this yaml in your cluster.

You can also run terrascan admission controller server outside cluster, for more information and instructions on configuring terrascan as an admission controller webhook, see https://docs.accurics.com/projects/accurics-terrascan/en/latest/integrations/admission-controller-webhooks-usage.
You can also run terrascan admission controller server outside cluster, for more information and instructions on configuring terrascan as an admission controller webhook, see https://runterrascan.io/docs/integrations/k8s/.

#### Step 2: Create a Dockerfile

Expand Down
27 changes: 15 additions & 12 deletions docs/usage/command_line_mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ $ terrascan
Terrascan

nasir-rabbani marked this conversation as resolved.
Show resolved Hide resolved
Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.
For more information, please visit https://docs.accurics.com
For more information, please visit https://runterrascan.io/

Usage:
terrascan [command]
Expand All @@ -34,12 +34,13 @@ Available Commands:
version Shows the Terrascan version you are currently using.

Flags:
-c, --config-path string config file path
-h, --help help for terrascan
-l, --log-level string log level (debug, info, warn, error, panic, fatal) (default "info")
-x, --log-type string log output type (console, json) (default "console")
-o, --output string output type (human, json, yaml, xml) (default "human")
--temp-dir string temporary directory path to download remote repository,module and templates
-c, --config-path string config file path
-h, --help help for terrascan
-l, --log-level string log level (debug, info, warn, error, panic, fatal) (default "info")
--log-output-dir string directory path to write the log and output files
-x, --log-type string log output type (console, json) (default "console")
-o, --output string output type (human, json, yaml, xml, junit-xml, sarif, github-sarif) (default "human")
--temp-dir string temporary directory path to download remote repository,module and templates

Use "terrascan [command] --help" for more information about a command.
```
Expand Down Expand Up @@ -290,6 +291,7 @@ aws_ecr_repository:
| ----------- | ----------- |------------|
| -c | Use this to specify config file settings | Format supported is `*.TOML` |
| -l | Use this to specify what log settings | debug, **info**, warn, error, panic, fatal |
| --log-output-dir | Use this to specify the directory path for writing scan logs and output files along with console output. In case the directory could not be resolved, the scan logs and results will be printed on console only. |
| -x | Use this to specify the log file format | **console**, json |
| -o | Use this to specify the scan output type | **human**, json, yaml, xml, junit-xml, sarif, github-sarif |
| --temp-dir | Use this to specify temporary directory path to download remote repository,module and templates |
Expand Down Expand Up @@ -335,9 +337,10 @@ Flags:
-v, --verbose will show violations with details (applicable for default output)

Global Flags:
-c, --config-path string config file path
-l, --log-level string log level (debug, info, warn, error, panic, fatal) (default "info")
-x, --log-type string log output type (console, json) (default "console")
-o, --output string output type (human, json, yaml, xml, junit-xml, sarif, github-sarif) (default "human")
--temp-dir string temporary directory path to download remote repository,module and templates
-c, --config-path string config file path
-l, --log-level string log level (debug, info, warn, error, panic, fatal) (default "info")
--log-output-dir string directory path to write the log and output files
-x, --log-type string log output type (console, json) (default "console")
-o, --output string output type (human, json, yaml, xml, junit-xml, sarif, github-sarif) (default "human")
--temp-dir string temporary directory path to download remote repository,module and templates
```
9 changes: 5 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,13 @@ require (
github.com/zclconf/go-cty v1.9.1
go.uber.org/zap v1.16.0
golang.org/x/sys v0.0.0-20211205182925-97ca703d548d
golang.org/x/tools v0.1.8 // indirect
golang.org/x/tools v0.1.11-0.20220316014157-77aa08bb151a // indirect
google.golang.org/api v0.34.0
google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a
gopkg.in/src-d/go-git.v4 v4.13.1
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
helm.sh/helm/v3 v3.6.1
honnef.co/go/tools v0.2.2 // indirect
honnef.co/go/tools v0.3.1 // indirect
k8s.io/api v0.21.0
k8s.io/apimachinery v0.21.0
k8s.io/client-go v10.0.0+incompatible
Expand Down Expand Up @@ -175,9 +175,10 @@ require (
go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 // indirect
go.uber.org/atomic v1.6.0 // indirect
go.uber.org/multierr v1.5.0 // indirect
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 // indirect
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 // indirect
golang.org/x/exp/typeparams v0.0.0-20220218215828-6cf2b201936e // indirect
golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect
golang.org/x/mod v0.5.1 // indirect
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 // indirect
golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f // indirect
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43 // indirect
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
Expand Down
11 changes: 11 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1510,6 +1510,8 @@ golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWP
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 h1:It14KIkyBFYkHkwZ7k45minvA9aorojkyjGk9KJ5B/w=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 h1:7I4JAnoQBe7ZtJcBaYHi5UtiO8tQHbUSXxL+pnGRANg=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190125153040-c74c464bbbf2/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
Expand All @@ -1521,7 +1523,10 @@ golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u0
golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6 h1:QE6XYQK6naiK1EPAe1g/ILLxN5RBoH5xkJk3CqlMI/Y=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/exp/typeparams v0.0.0-20220218215828-6cf2b201936e h1:qyrTQ++p1afMkO4DPEeLGq/3oTsdlvdH4vqZUBWzUKM=
golang.org/x/exp/typeparams v0.0.0-20220218215828-6cf2b201936e/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
Expand Down Expand Up @@ -1549,6 +1554,8 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.5.1 h1:OJxoQ/rynoF0dcCdI7cLPktw/hR2cueqYfjm43oqK38=
golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 h1:kQgndtyPBW/JIYERgdxfwMYh3AVStj88WQTlNDi2a+o=
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY=
golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180530234432-1e491301e022/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
Expand Down Expand Up @@ -1836,6 +1843,8 @@ golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4f
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.1.8 h1:P1HhGGuLW4aAclzjtmJdf0mJOjVUZUzOTqkAkWL+l6w=
golang.org/x/tools v0.1.8/go.mod h1:nABZi5QlRsZVlzPpHl034qft6wpY4eDcsTt5AaioBiU=
golang.org/x/tools v0.1.11-0.20220316014157-77aa08bb151a h1:ofrrl6c6NG5/IOSx/R1cyiQxxjqlur0h/TvbUhkH0II=
golang.org/x/tools v0.1.11-0.20220316014157-77aa08bb151a/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
Expand Down Expand Up @@ -2029,6 +2038,8 @@ honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9
honnef.co/go/tools v0.0.1-2020.1.5/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.2.2 h1:MNh1AVMyVX23VUHE2O27jm6lNj3vjO5DexS4A1xvnzk=
honnef.co/go/tools v0.2.2/go.mod h1:lPVVZ2BS5TfnjLyizF7o7hv7j9/L+8cZY2hLyjP9cGY=
honnef.co/go/tools v0.3.1 h1:1kJlrWJLkaGXgcaeosRXViwviqjI7nkBvU2+sZW0AYc=
honnef.co/go/tools v0.3.1/go.mod h1:vlRD9XErLMGT+mDuofSr0mMMquscM/1nQqtRSsh6m70=
k8s.io/api v0.19.0 h1:XyrFIJqTYZJ2DU7FBE/bSPz7b1HvbVBuBf07oeo6eTc=
k8s.io/api v0.19.0/go.mod h1:I1K45XlvTrDjmj5LoM5LuP/KYrhWbjUKT/SoPG0qTjw=
k8s.io/apiextensions-apiserver v0.21.0 h1:Nd4uBuweg6ImzbxkC1W7xUNZcCV/8Vt10iTdTIVF3hw=
Expand Down
35 changes: 35 additions & 0 deletions pkg/cli/output_writer.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@ package cli
import (
"io"
"os"
"path/filepath"

"github.com/accurics/terrascan/pkg/termcolor"
"go.uber.org/zap"
)

// NewOutputWriter gets a new io.Writer based on os.Stdout.
Expand All @@ -17,3 +19,36 @@ func NewOutputWriter(useColors bool) io.Writer {
}
return os.Stdout
}

// NewFileWriter gets a new io.Writer based on file output and closing function.
// It returns `nil nil` if the value of dir is empty or if the file can't be opened
func NewFileWriter(dir string, outputType string) (io.Writer, func() error) {

// if no directory resolved
if dir == "" {
return nil, nil
}

fileName := "scan-result.txt"

// decide the file extension/type
switch outputType {
case "json":
fileName = "scan-result.json"
case "yaml":
fileName = "scan-result.yaml"
case "xml", "junit-xml":
fileName = "scan-result.xml"
case "sarif", "github-sarif":
fileName = "scan-result.sarif"
}

filePath := filepath.Join(dir, fileName)
f, err := os.OpenFile(filePath, os.O_RDWR|os.O_CREATE, 0755)
if err != nil {
zap.S().Warn("unable to open file: %s, error:%w", filePath, err)
return nil, nil
}

return f, f.Close
}
14 changes: 12 additions & 2 deletions pkg/cli/register.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,24 @@ func Execute() {
rootCmd.PersistentFlags().StringVarP(&OutputType, "output", "o", "human", "output type (human, json, yaml, xml, junit-xml, sarif, github-sarif)")
rootCmd.PersistentFlags().StringVarP(&ConfigFile, "config-path", "c", "", "config file path")
rootCmd.PersistentFlags().StringVarP(&CustomTempDir, "temp-dir", "", "", "temporary directory path to download remote repository,module and templates")
rootCmd.PersistentFlags().StringVarP(&LogOutputDir, "log-output-dir", "", "", "directory path to write the log and output files")

//Added init here in case flag parsing failed we should log which flag was incorrect.
logging.Init(LogType, LogLevel)
logging.Init(LogType, LogLevel, LogOutputDir)

// Function to execute before processing commands
cobra.OnInitialize(func() {
// making sure the LogOutputDir Exist
if LogOutputDir != "" {
err := os.MkdirAll(LogOutputDir, 0755)
if err != nil {
zap.S().Warnf("failed to resolve the log output directory: %s", LogOutputDir)
LogOutputDir = ""
}
}

// Set up the logger
logging.Init(LogType, LogLevel)
logging.Init(LogType, LogLevel, LogOutputDir)

if len(ConfigFile) == 0 {
ConfigFile = os.Getenv(config.ConfigEnvvarName)
Expand Down
5 changes: 4 additions & 1 deletion pkg/cli/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ var (

// CustomTempDir Temporary directory path to download remote repository,module and templates
CustomTempDir string

// LogOutputDir Directory to write scan logs and result files
LogOutputDir string
)

var rootCmd = &cobra.Command{
Expand All @@ -43,6 +46,6 @@ var rootCmd = &cobra.Command{
Long: `Terrascan

Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.
For more information, please visit https://docs.accurics.com
For more information, please visit https://runterrascan.io/
`,
}
Loading