Allow plugins located on github.com and GHES to be installed at once as an authenticated request

[wata727]

Introduction

TFLint plugins are primarily intended to be hosted on github.com and is installed using the GitHub API.
Currently, we support authenticated requests with GITHUB_TOKEN for the following purposes:

• Avoid rate limits
• Install from private repositories

https://github.com/terraform-linters/tflint/blob/v0.50.3/docs/user-guide/plugins.md#avoiding-rate-limiting

Apart from this, plugins can also be hosted on GitHub Enterprise Server (GHES). See #1751

The problem is that since both github.com and GHES use the same GITHUB_TOKEN, there is no way to install a plugin hosted on both at once as an authenticated request. See also #2004

Proposal

Add access_token_env as an attribute of the plugin block.

plugin "custom" {
  source  = "github.example.com/example/tflint-ruleset-example"
  version = "0.1.0"
  enabled = true

  access_token_env = "GITHUB_ENT_TOKEN"
}

Plugins with access_token_env set refer to the declared environment variable instead of GITHUB_TOKEN when sending authenticated requests. We'll probably need to change the code below:

tflint/plugin/install.go

Lines 253 to 259 in 9bad280

	if t := os.Getenv("GITHUB_TOKEN"); t != "" {
	log.Printf("[DEBUG] GITHUB_TOKEN set, plugin requests to the GitHub API will be authenticated")
	hc = oauth2.NewClient(ctx, oauth2.StaticTokenSource(&oauth2.Token{
	AccessToken: t,
	}))
	}

Perhaps this is a minimal change, but I'm still not sure if this is the ideal interface. We may need to extend the source attribute to object, or new concepts like the connection block, or an authentication mechanism other than environment variables.

---

EDIT: @bendrucker suggests a way to provide per-host environment variables like TFE #2005 (comment). I prefer this suggestion as it is better than extending the existing syntax.

We may be able to use something like terraform-svchost to normalize hostnames.
https://github.com/hashicorp/terraform/blob/v1.5.7/internal/command/cliconfig/credentials.go#L120

References

• Plugins from GHE and github.com private repo, possible? #2004
• https://github.com/terraform-linters/tflint/blob/v0.50.3/docs/user-guide/plugins.md#avoiding-rate-limiting
• https://github.com/terraform-linters/tflint/blob/v0.50.3/plugin/install.go#L253-L259

[bendrucker]

Adding configuration to the plugin block creates a need for repetition and an opportunity for divergence. The real use case here is to provide different tokens per plugin host, not necessarily per plugin.

Terraform does this with environment variables where the variable name is suffixed with the domain, with . replaced with _.

https://developer.hashicorp.com/terraform/cli/config/config-file#environment-variable-credentials

This should work here without having to touch any interfaces. If an env var isn't set for GITHUB_TOKEN_${sanitized_host} then it can fall back to plain GITHUB_TOKEN. So for example GITHUB_TOKEN_github_com would be equivalent to setting GITHUB_TOKEN for github.com users.

[wata727]

Great suggestion. I was going to look up how other services solved similar issues, but I couldn't find it, so thank you for your help.
I also think it's better to be able to set per-host environment variables than to extend the existing syntax.