Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiline secrets stored in json files not picked up due to regexes not matching escaped newlines #2504

Closed
kobus-v-schoor opened this issue Jul 13, 2022 · 3 comments · Fixed by #2532
Closed

Multiline secrets stored in json files not picked up due to regexes not matching escaped newlines #2504

kobus-v-schoor opened this issue Jul 13, 2022 · 3 comments · Fixed by #2532
Assignees
@DmitriyLewen
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@kobus-v-schoor
Copy link
Contributor

Description

When a multi-line secret is stored inside a json file, its newline characters are escaped e.g.

{"rsa-key": "first line\nsecond key"}

The current secret regexes doesn't permit the backslashes so they don't match any multi-line secrets stored in json files. Weirdly enough, this wasn't an issue in versions before the v29 release. I've set up a test image demonstrating the issue at kobusvschoor/trivy-test:secrets

Version 0.29.2:

$ trivy --debug image -f json --security-checks secret kobusvschoor/trivy-test:secrets

2022-07-13T11:06:36.220+0200	DEBUG	Severities: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
2022-07-13T11:06:36.356+0200	DEBUG	cache dir:  /home/kobus/.cache/trivy
2022-07-13T11:06:36.356+0200	INFO	Secret scanning is enabled
2022-07-13T11:06:36.356+0200	INFO	If your scanning is slow, please try '--security-checks vuln' to disable secret scanning
2022-07-13T11:06:36.356+0200	INFO	Please see also https://aquasecurity.github.io/trivy/v0.29.2/docs/secret/scanning/#recommendation for faster secret detection
2022-07-13T11:06:39.001+0200	DEBUG	No secret config detected: trivy-secret.yaml
2022-07-13T11:06:39.356+0200	DEBUG	Image ID: sha256:ede28ed3cc8469905df97047794fe0d103f61ae13ba460f6ba0cff2a5812c17d
2022-07-13T11:06:39.356+0200	DEBUG	Diff IDs: [sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d sha256:75a530eec90ca9f2c87d1c209f7ac58e9a444b9fa8423fa09091da315b594149 sha256:bf295afd7126c745f5eb4c8c7a0e42efc389181e4924601bdf6f49a7f22f53b1]
2022-07-13T11:06:39.356+0200	DEBUG	Base Layers: [sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d]
2022-07-13T11:06:39.356+0200	DEBUG	Missing image ID in cache: sha256:ede28ed3cc8469905df97047794fe0d103f61ae13ba460f6ba0cff2a5812c17d
2022-07-13T11:06:39.356+0200	DEBUG	Missing diff ID in cache: sha256:bf295afd7126c745f5eb4c8c7a0e42efc389181e4924601bdf6f49a7f22f53b1
2022-07-13T11:06:39.356+0200	DEBUG	Missing diff ID in cache: sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d
2022-07-13T11:06:39.356+0200	DEBUG	Missing diff ID in cache: sha256:75a530eec90ca9f2c87d1c209f7ac58e9a444b9fa8423fa09091da315b594149
2022-07-13T11:06:40.874+0200	DEBUG	Secret file: /key.pem
{
  "SchemaVersion": 2,
  "ArtifactName": "kobusvschoor/trivy-test:secrets",
  "ArtifactType": "container_image",
  "Metadata": {
    "OS": {
      "Family": "alpine",
      "Name": "3.16.0"
    },
    "ImageID": "sha256:ede28ed3cc8469905df97047794fe0d103f61ae13ba460f6ba0cff2a5812c17d",
    "DiffIDs": [
      "sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d",
      "sha256:75a530eec90ca9f2c87d1c209f7ac58e9a444b9fa8423fa09091da315b594149",
      "sha256:bf295afd7126c745f5eb4c8c7a0e42efc389181e4924601bdf6f49a7f22f53b1"
    ],
    "RepoTags": [
      "kobusvschoor/trivy-test:secrets"
    ],
    "RepoDigests": [
      "kobusvschoor/trivy-test@sha256:4e43712a5bc61aded177924d1e0b9d69906a33c5aa38ed288f68b86e3ea18249"
    ],
    "ImageConfig": {
      "architecture": "amd64",
      "created": "2022-07-13T09:02:05.939543781Z",
      "docker_version": "20.10.12",
      "history": [
        {
          "created": "2022-05-23T19:19:30.413290187Z",
          "created_by": "/bin/sh -c #(nop) ADD file:8e81116368669ed3dd361bc898d61bff249f524139a239fdaf3ec46869a39921 in / "
        },
        {
          "created": "2022-05-23T19:19:31.970967174Z",
          "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
          "empty_layer": true
        },
        {
          "created": "2022-07-13T09:01:46.66108384Z",
          "created_by": "/bin/sh -c #(nop) COPY file:4f5a648b283986d703b8d840e4df56fa96c0b96d91f3ea52bfb5791e88cc5f76 in /key.pem "
        },
        {
          "created": "2022-07-13T09:02:05.939543781Z",
          "created_by": "/bin/sh -c #(nop) COPY file:923f01359a087b7c057227defcbc25ceeeabf4037af1399e97f02d3d9ca31aa5 in /key.json "
        }
      ],
      "os": "linux",
      "rootfs": {
        "type": "layers",
        "diff_ids": [
          "sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d",
          "sha256:75a530eec90ca9f2c87d1c209f7ac58e9a444b9fa8423fa09091da315b594149",
          "sha256:bf295afd7126c745f5eb4c8c7a0e42efc389181e4924601bdf6f49a7f22f53b1"
        ]
      },
      "config": {
        "Cmd": [
          "/bin/sh"
        ],
        "Env": [
          "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "Image": "sha256:b71b99a347265f93f36a092e27ea73b952eb64543c31d61abe98b92a462c4d9b"
      }
    }
  },
  "Results": [
    {
      "Target": "/key.pem",
      "Class": "secret",
      "Secrets": [
        {
          "RuleID": "private-key",
          "Category": "AsymmetricPrivateKey",
          "Severity": "HIGH",
          "Title": "Asymmetric Private Key",
          "StartLine": 1,
          "EndLine": 1,
          "Match": "-----BEGIN RSA PRIVATE KEY-----"
        }
      ]
    }
  ]
}

Version 0.28.1:

$ trivy --debug image -f json --security-checks secret kobusvschoor/trivy-test:secrets

2022-07-13T11:09:25.791+0200	DEBUG	Severities: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
2022-07-13T11:09:25.950+0200	DEBUG	cache dir:  /home/kobus/.cache/trivy
2022-07-13T11:09:25.950+0200	DEBUG	Vulnerability type:  [os library]
2022-07-13T11:09:28.984+0200	DEBUG	No secret config detected: trivy-secret.yaml
2022-07-13T11:09:29.378+0200	DEBUG	Image ID: sha256:ede28ed3cc8469905df97047794fe0d103f61ae13ba460f6ba0cff2a5812c17d
2022-07-13T11:09:29.378+0200	DEBUG	Diff IDs: [sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d sha256:75a530eec90ca9f2c87d1c209f7ac58e9a444b9fa8423fa09091da315b594149 sha256:bf295afd7126c745f5eb4c8c7a0e42efc389181e4924601bdf6f49a7f22f53b1]
2022-07-13T11:09:29.378+0200	DEBUG	Base Layers: [sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d]
2022-07-13T11:09:29.378+0200	DEBUG	Missing image ID in cache: sha256:ede28ed3cc8469905df97047794fe0d103f61ae13ba460f6ba0cff2a5812c17d
2022-07-13T11:09:29.378+0200	DEBUG	Missing diff ID in cache: sha256:bf295afd7126c745f5eb4c8c7a0e42efc389181e4924601bdf6f49a7f22f53b1
2022-07-13T11:09:29.378+0200	DEBUG	Missing diff ID in cache: sha256:75a530eec90ca9f2c87d1c209f7ac58e9a444b9fa8423fa09091da315b594149
2022-07-13T11:09:29.378+0200	DEBUG	Missing diff ID in cache: sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d
2022-07-13T11:09:31.062+0200	DEBUG	Secret file: key.pem
2022-07-13T11:09:31.062+0200	DEBUG	Secret file: key.json
{
  "SchemaVersion": 2,
  "ArtifactName": "kobusvschoor/trivy-test:secrets",
  "ArtifactType": "container_image",
  "Metadata": {
    "OS": {
      "Family": "alpine",
      "Name": "3.16.0"
    },
    "ImageID": "sha256:ede28ed3cc8469905df97047794fe0d103f61ae13ba460f6ba0cff2a5812c17d",
    "DiffIDs": [
      "sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d",
      "sha256:75a530eec90ca9f2c87d1c209f7ac58e9a444b9fa8423fa09091da315b594149",
      "sha256:bf295afd7126c745f5eb4c8c7a0e42efc389181e4924601bdf6f49a7f22f53b1"
    ],
    "RepoTags": [
      "kobusvschoor/trivy-test:secrets"
    ],
    "RepoDigests": [
      "kobusvschoor/trivy-test@sha256:4e43712a5bc61aded177924d1e0b9d69906a33c5aa38ed288f68b86e3ea18249"
    ],
    "ImageConfig": {
      "architecture": "amd64",
      "created": "2022-07-13T09:02:05.939543781Z",
      "docker_version": "20.10.12",
      "history": [
        {
          "created": "2022-05-23T19:19:30.413290187Z",
          "created_by": "/bin/sh -c #(nop) ADD file:8e81116368669ed3dd361bc898d61bff249f524139a239fdaf3ec46869a39921 in / "
        },
        {
          "created": "2022-05-23T19:19:31.970967174Z",
          "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
          "empty_layer": true
        },
        {
          "created": "2022-07-13T09:01:46.66108384Z",
          "created_by": "/bin/sh -c #(nop) COPY file:4f5a648b283986d703b8d840e4df56fa96c0b96d91f3ea52bfb5791e88cc5f76 in /key.pem "
        },
        {
          "created": "2022-07-13T09:02:05.939543781Z",
          "created_by": "/bin/sh -c #(nop) COPY file:923f01359a087b7c057227defcbc25ceeeabf4037af1399e97f02d3d9ca31aa5 in /key.json "
        }
      ],
      "os": "linux",
      "rootfs": {
        "type": "layers",
        "diff_ids": [
          "sha256:24302eb7d9085da80f016e7e4ae55417e412fb7e0a8021e95e3b60c67cde557d",
          "sha256:75a530eec90ca9f2c87d1c209f7ac58e9a444b9fa8423fa09091da315b594149",
          "sha256:bf295afd7126c745f5eb4c8c7a0e42efc389181e4924601bdf6f49a7f22f53b1"
        ]
      },
      "config": {
        "Cmd": [
          "/bin/sh"
        ],
        "Env": [
          "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "Image": "sha256:b71b99a347265f93f36a092e27ea73b952eb64543c31d61abe98b92a462c4d9b"
      }
    }
  },
  "Results": [
    {
      "Target": "key.pem",
      "Class": "secret",
      "Secrets": [
        {
          "RuleID": "private-key",
          "Category": "AsymmetricPrivateKey",
          "Severity": "HIGH",
          "Title": "Asymmetric Private Key",
          "StartLine": 1,
          "EndLine": 1,
          "Match": "-----BEGIN RSA PRIVATE KEY-----"
        }
      ]
    },
    {
      "Target": "key.json",
      "Class": "secret",
      "Secrets": [
        {
          "RuleID": "private-key",
          "Category": "AsymmetricPrivateKey",
          "Severity": "HIGH",
          "Title": "Asymmetric Private Key",
          "StartLine": 1,
          "EndLine": 1,
          "Match": "----BEGIN RSA PRIVATE KEY-----*****----END RSA PRIVATE"
        }
      ]
    }
  ]
}
@kobus-v-schoor kobus-v-schoor added the kind/bug Categorizes issue or PR as related to a bug. label Jul 13, 2022
@DmitriyLewen
Copy link
Contributor

Hello @kobus-v-schoor

Thanks for your report!
I see this problem and we will fix it.

Regards, Dmitriy

@DmitriyLewen DmitriyLewen mentioned this issue Jul 18, 2022
Merged
6 tasks
@DmitriyLewen
Copy link
Contributor

Hello @kobus-v-schoor

Thanks for waiting!

We have fixed your issue.
Fix will be included to next release.

Also we have canary binary and image. You can use it.

Regards, Dmitriy

@kobus-v-schoor
Copy link
Contributor Author

kobus-v-schoor commented Aug 1, 2022 •
edited
Loading

Thanks @DmitriyLewen - I'll check out the canary image and give it a test, thanks for looking at the issue!

EDIT: Tested it out, looks like it's working 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DmitriyLewen DmitriyLewen

Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(secret): add newline escaping for asymmetric private key aquasecurity/trivy
2 participants
@kobus-v-schoor @DmitriyLewen