TF lint initialization is failing. Noticed an tailing ":" at the end of the aws plugin download during tflint init

[chilukurib]

Describe the bug
TF lint initialization is failing. Noticed an tailing ":" at the end of the aws plugin download during tflint init which is giving not-found message when opened in browser

To Reproduce
Steps to reproduce the behavior:
exexcute linting on terraform code

Expected behavior
megalinter should initialize the tf_lint

ERROR LOG

❌ Linted [TERRAFORM] files with [tflint]: Found 0 error(s) - (0.33s) (expand for details)
  - Using [tflint v0.47.0] https://megalinter.io/7.3.0/descriptors/terraform_tflint
  - MegaLinter key: [TERRAFORM_TFLINT]
  - Rules config: [.tflint.hcl]
  [Pre][TERRAFORM_TFLINT] run: [tflint --init --config /action/lib/.automation/.tflint.hcl] in cwd [/github/workspace]
  [Pre][TERRAFORM_TFLINT] error:
  Installing `aws` plugin...
  Failed to install a plugin; Failed to fetch GitHub releases: GET https://api.github.com/repos/terraform-linters/tflint-ruleset-aws/releases/tags/v0.23.1: 401 Bad credentials []
  
  --Error detail:
  error: Failed to initialize plugins; Plugin `aws` not found. Did you run `tflint --init`?
  
  error: 0 errors emitted

I see an tailing colon in the url from the log https://api.github.com/repos/terraform-linters/tflint-ruleset-aws/releases/tags/v0.23.1: which is giving message not-found

[nvuillam]

@chilukurib did you override your GITHUB_TOKEN ?
Of are you use GitHub enterprise ?

tflint downloads aws plugin on Github, so the "bad credentials" could be from that

You can also try

[priorax]

I'm currently using GitHub Enterprise and finding similar quirks, largely due to setting GITHUB_TOKEN to be able to use reporters.

[nvuillam]

@priorax maybe try to define a Personal Access Token then force its value in environment variable GITHUB_TOKEN ?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[codezninja]

So we are having this issue. If we do add a personal pat for github.com wouldn't this mean it would fail to post back to the PR the results? I'm assuming it was using the GITHUB_TOKEN to post back the results of the PR

[nvuillam]

@codezninja if you PAT has the rights to post PR comments, they will be posted :)

[codezninja]

@codezninja if you PAT has the rights to post PR comments, they will be posted :)

Sorry I should've been more clear. I meant if we use a PAT for GitHub.com to fix the tflint init but this action is running on GitHub Enterprise it might fix that but it won't have permissions to post back to the PR

[nvuillam]

@codezninja indeed :o

[nvuillam]

GITHUB_TOKEN is used for tflint --init (not PAT)
But the reporter checks for PAT existence before checking for GITHUB_TOKEN existence

So you might hack by defining:
GITHUB_TOKEN = YOUR_GITHUB_DOT_COM_PAT
PAT = YOUR_GITHUB_ENTERPRISE_TOKEN (= GITHUB_TOKEN provided by default by your ghe workflow)

It's ugly but it might work ^^

[codezninja]

We'll try that and see how that works.

[jared-bloomer]

@codezninja and I work together. I just tested this out and I have mixed results to share

I updated our Github Actions on our GitHub Enterprise server to use

- name: MegaLinter
  uses: oxsecurity/megalinter@v8
  id: ml
  env:
     VALIDATE_ALL_CODEBASE: true
     PAT: ${{ secrets.GITHUB_TOKEN }}
     GITHUB_TOKEN: ${{ secrets.TOKEN_GITHUB_COM }}

This fixed the issues we were having with tflint, however it seems to have broke other linters that reference things in repos on our GitHub Enterprise Server.

Some Examples:

✅ Linted [REPOSITORY] files with [checkov]: Found 7 non blocking error(s) - (12.96s) (expand for details)
  - Using [checkov v3.2.298] https://megalinter.io/8.2.0/descriptors/repository_checkov
  - MegaLinter key: [REPOSITORY_CHECKOV]
  - Rules config: [/github/workspace/.github/linters/.checkov.yml]
  --Error detail:
  2024-11-20 14:00:51,069 [MainThread  ] [WARNI]  Failed to download module git::ssh://[email protected]/org/tf_aurora_cluster.git?ref=v2.1.0:None (for external modules, the --download-external-modules flag is required)
  2024-11-20 14:00:51,069 [MainThread  ] [WARNI]  Failed to download module git::ssh://[email protected]/org2/tf_rds//modules/vanity_url?ref=v2.6.0:None (for external modules, the --download-external-modules flag is required)
  2024-11-20 14:00:51,070 [MainThread  ] [WARNI]  Failed to download module git::ssh://[email protected]/org/tf_tags.git?ref=v2.2.0:None (for external modules, the --download-external-modules flag is required)

Terrascan results in several

<nil>: Failed to read module directory; Module directory  does not exist or cannot be read.

✅ Linted [MARKDOWN] files with [markdown-link-check]: Found 13 non blocking error(s) - (2.6s) (expand for details)
  - Using [markdown-link-check v3.12.2] https://megalinter.io/8.2.0/descriptors/markdown_markdown_link_check
  - MegaLinter key: [MARKDOWN_MARKDOWN_LINK_CHECK]
  - Rules config: [.markdown-link-check.json]
  - Number of files analyzed: [2]
  --Error detail:
  
    ERROR: 13 dead links found in README.md !
    [✖] #requirement_terraform → Status: 404
    [✖] #requirement_sso → Status: 404
    [✖] #requirement_aws → Status: 404
    [✖] #provider_aws → Status: 404
    [✖] #provider_random → Status: 404
    [✖] #module_aurora_serverless → Status: 404
    [✖] mailto:[email protected] → Status: 400
    [✖] #module_tags → Status: 404
    [✖] #module_vanity_url → Status: 404
    [✖] #input_ci-id → Status: 404
    [✖] #input_component → Status: 404
    [✖] #input_environment → Status: 404
    [✖] #input_tags → Status: 404

There are others, but I think this is enough example to get the point across.

[nvuillam]

Damn, i get it ^^
I'll make an update to all to add another variable just for TFLINT :)

[jared-bloomer]

@nvuillam Thank You Sir

[nvuillam]

@jared-bloomer you can check with beta in a few minutes :)

Try with

TERRAFORM_TFLINT_UNSECURED_ENV_VARIABLES:
  - GITHUB_TOKEN
  - PAT_GITHUB_COM

Also define a secret variable PAT_GITHUB_COM with a github.com Personal Access Token :)

[jared-bloomer]

@jared-bloomer you can check with beta in a few minutes :)

Try with

TERRAFORM_TFLINT_UNSECURED_ENV_VARIABLES:
  - GITHUB_TOKEN
  - PAT_GITHUB_COM

Also define a secret variable PAT_GITHUB_COM with a github.com Personal Access Token :)

I will check the next time I am at work. Thanks

[jared-bloomer]

@nvuillam we are getting closer. Now we are getting host key verification errors on our GHE server.

2024-12-02T19:01:43.653Z	error	downloader/getter.go:105	failed to download "git::ssh://[email protected]/MyOrg/tf_aurora_cluster.git?ref=v2.1.0". error: 'error downloading 'ssh://[email protected]/MyOrg/tf_aurora_cluster.git?ref=v2.1.0': /usr/bin/git exited with 128: Cloning into '/tmp/odgne5'...
  Host key verification failed.
  fatal: Could not read from remote repository.

I think we need to load in our GHE Known host. I am not entirely sure how to go about that.

[echoix]

Is there a "permissions" key at the workflow level that doesn't allow to use "content: read" to allow checkout when not in GitHub.com?

(If you were using PATs instead, then I would be surprised it would make a difference)

[jared-bloomer]

I think it is because we have terraform trying to grab modules over ssh and that's where the host keys are coming from. On Dec 2, 2024 at 17:24, Edouard Choinière ***@***.******@***.***>> wrote: Is there a "permissions" key at the workflow level that doesn't allow to use "content: read" to allow checkout when not in GitHub.com? (If you were using PATs instead, then I would be surprised it would make a difference) — Reply to this email directly, view it on GitHub<#2947 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFZNGBM5IQULIME4SDDMGYD2DTM2BAVCNFSM6AAAAABSDEIBZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJTGA4DGMZXGU>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[nvuillam]

This seems to not he a tflint init issue anymore, please can you open another issue ? :)

[jared-bloomer]

This seems to not he a tflint init issue anymore, please can you open another issue ? :)

@nvuillam Issue #4343 has been created