Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RawData fails for some resources #957

Closed
czunker opened this issue Feb 22, 2023 · 8 comments · Fixed by #1456
Closed

RawData fails for some resources #957

czunker opened this issue Feb 22, 2023 · 8 comments · Fixed by #1456
Assignees

Comments

@czunker
Copy link
Contributor

czunker commented Feb 22, 2023

Describe the bug
When I try to get the RawData of a resource/field, the data isn't converted because of typing problems. But the resource/field has a type.

I came across this problem while exposing query pack data:

{
              "name": "date",
              "description": "End-of-Life date",
              "path": "platform.eol.date",
              "value": "{\"error\":\"cannot convert primitive with NO type information\"}",
              "type": "time",
              "numElements": 1
            }

This happens here:

return &RawData{Error: errors.New("cannot convert primitive with NO type information")}

Strange is, that it works for other fields of the same type:

{
              "name": "uptime",
              "description": "Current uptime",
              "path": "os.uptime",
              "value": "\"0000-01-05T04:29:31.423622144+00:53\"",
              "type": "time",
              "numElements": 1
            },

To Reproduce
Have to find out, how to isolate this.

  1. Go to '...'
  2. Select '....'
  3. Scroll down to '....'
  4. Note the error

Expected behavior
The data should get converted.

Screenshots or CLI Output
Some more examples:

{
              "name": "vulnerabilityReport",
              "description": "Full advisory & vulnerability report",
              "path": "platform.vulnerabilityReport",
              "value": "{\"error\":\"cannot convert primitive with NO type information\"}",
              "type": "dict",
              "numElements": 1
            },
{
              "name": "jobEnvironment",
              "description": "Returns the agent execution environment",
              "path": "mondoo.jobEnvironment",
              "value": "{\"error\":\"cannot convert primitive with NO type information\"}",
              "type": "dict",
              "numElements": 1
            }
{
              "name": "params",
              "description": "Configuration values of this SSH server",
              "path": "sshd.config.params",
              "value": "{\"error\":\"cannot convert primitive with NO type information\"}",
              "type": "map[string]string",
              "numElements": 1
            }
@czunker
Copy link
Contributor Author

czunker commented Feb 23, 2023

This also ends up in the UI:

image

@czunker
Copy link
Contributor Author

czunker commented Feb 23, 2023

I added a query for platform.eol.date to the query pack. Now I see data and no error message.

So to me, two questions remain:

  • How did the query get there before actually fetching it?
  • Why did I see this strange error message, when it perhaps should only state that there is no data?

@czunker
Copy link
Contributor Author

czunker commented Feb 23, 2023

I can no longer reproduce this in my local dev environment. It seems to be a strange combination of scanning an asset against a policy and an inventory pack, when not all backend data is present.

I assume, I scanned the asset via cnspec, when no package and advisory data was present. Later I scanned it against an inventory pack. So I had the combination of queries from both scans, but with partially missing data.

Perhaps I can reproduce this, when I reset my local DB, do a scan with cnspec and then cnquery.

@benr
Copy link
Member

benr commented May 10, 2023

This is happening for me on v8.8.0 on Ubuntu Linux.

What is strange is that the same host is scanning with another config to another space and its fine. I created a new space for a project and scan the host and hit this problem.

I'm seeing it in the Console:

image

But the problem doesn't happen locally:

benr@swell:~/mondoo/soc2$ sudo cnspec scan --config mondoo_config.yaml ^C
benr@swell:~/mondoo/soc2$ sudo cnspec shell --config mondoo_config.yaml 
→ no provider specified, defaulting to local.
  Use --help for a list of available providers.
→ loaded configuration from mondoo_config.yaml using source --config
→ discover related assets for 1 asset(s)
→ resolved assets resolved-assets=1
  ___ _ __  ___ _ __   ___  ___ 
 / __| '_ \/ __| '_ \ / _ \/ __|
| (__| | | \__ \ |_) |  __/ (__ 
 \___|_| |_|___/ .__/ \___|\___|
   mondoo™     |_|              
cnspec> platform.cves.cvss.score
platform.cves.cvss.score: 10
cnspec> score(100 - platform.cves.cvss.score * 10)
score: 0

If I delete the asset, create a new config and rescan the problem still happens.

If I find an old space ("default" in my org) and scan to it, no problems.

If I create a new org and new space and scan it, and I hit the problem:

image

@czunker czunker self-assigned this May 17, 2023
@czunker
Copy link
Contributor Author

czunker commented May 17, 2023

I gave it another try with cnspec v8.10.0 and locally running latest backend and UI, but couldn't reproduce it. Seems like it git fixed in the meantime.

I tried an already existing org with an existing space: worked
... existing org with new space: worked
... new org with new space: worked

I checked different controls in the UI and also the resources tab. I scanned my local Linux system and also an older Ubuntu container image.

Here are some screenshots:
Screenshot from 2023-05-17 15-47-03
Screenshot from 2023-05-17 15-46-02
Screenshot from 2023-05-17 15-44-58
Screenshot from 2023-05-17 15-44-39

@czunker
Copy link
Contributor Author

czunker commented May 22, 2023

We close this for now, as we cannot reproduce it.

@czunker czunker closed this as completed May 22, 2023
@czunker
Copy link
Contributor Author

czunker commented Jun 21, 2023

I see the error again on my local system:

{
          "name": "package",
          "type": "package",
          "path": "package",
          "numElements": 1,
          "nestedContent": false,
          "fields": [
            {
              "name": "installed",
              "path": "package.installed",
              "value": "{\"error\":\"cannot convert primitive with NO type information\"}",
              "type": "bool",
              "nestedContent": false
            }
          ]
        },

@czunker czunker reopened this Jun 21, 2023
czunker added a commit to mondoohq/cnspec-policies that referenced this issue Aug 4, 2023
Before, scanning a user repo with this policy resulted in two errors:

```
! resolver.db> failed to store data, types don't match asset=//policy.api.mondoo.com/assets/2TWAyduY7PG2mz6cThAiw6izaBH checksum=ykt2Zc3Oo80XLuytqsUSE6fkGlJZvq7kE0pZrR8YX027tEtyd8TYI7RxI8EWAuL8wTzhWljY4P3OYN3mdmohOA== data={"type":"\u001bgithub.organization"} expected=block received=github.organization
x failed to send datapoints error="1 error occurred:\n\t* failed to store data for \"ykt2Zc3Oo80XLuytqsUSE6fkGlJZvq7kE0pZrR8YX027tEtyd8TYI7RxI8EWAuL8wTzhWljY4P3OYN3mdmohOA==\", types don't match: expected block, got github.organization\n\n"
```

and

```
cannot convert primitive with NO type information
```

related to mondoohq/cnquery#957

Signed-off-by: Christian Zunker <[email protected]>
@czunker
Copy link
Contributor Author

czunker commented Aug 4, 2023

I can reproduce it on my local linux system with this query: https://github.com/mondoohq/cnspec-policies/blob/main/core/mondoo-linux-security.mql.yaml#L1084

It results in:

                {
                  "error": "cannot convert primitive with NO type information"
                },
                {
                  "error": "1 error occurred:\n\t* file '/boot/grub/grub.cfg' does not exist: open /boot/grub/grub.cfg: permission denied\n\n"
                },
                {
                  "error": "cannot convert primitive with NO type information"
                },
                {
                  "error": "cannot convert primitive with NO type information"
                }

The files checked in the 1st, 3rd, and 4th if aren't present on my machine.

czunker added a commit that referenced this issue Aug 7, 2023
In case we have a resource without data, it also does not have a type.
We need to catch this to prevent the error: `cannot convert primitive with NO type information`

This happens, e.g., when we test for the exists of a file before doing anything else with the resource.

Partially-Fixes #957

Signed-off-by: Christian Zunker <[email protected]>
czunker added a commit that referenced this issue Aug 8, 2023
In case we have a resource without data, it also does not have a type.
We need to catch this to prevent the error: `cannot convert primitive
with NO type information`

This happens, e.g., when we test for the exists of a file before doing
anything else with the resource.

Partially-Fixes #957

Signed-off-by: Christian Zunker <[email protected]>
czunker added a commit that referenced this issue Aug 8, 2023
In case we have a resource without data, it also does not have a type.
We need to catch this to prevent the error: `cannot convert primitive
with NO type information`

This happens, e.g., when we test for the exists of a file before doing
anything else with the resource.

Partially-Fixes #957

Signed-off-by: Christian Zunker <[email protected]>
czunker added a commit that referenced this issue Aug 8, 2023
In case we have a resource without data, it also does not have a type.
We need to catch this to prevent the error: `cannot convert primitive
with NO type information`

This happens, e.g., when we test for the exists of a file before doing
anything else with the resource.

Backport from main.

Partially-Fixes #957

Signed-off-by: Christian Zunker <[email protected]>
czunker added a commit to mondoohq/cnspec-policies that referenced this issue Aug 14, 2023
Before, scanning a user repo with this policy resulted in two errors:

```
! resolver.db> failed to store data, types don't match asset=//policy.api.mondoo.com/assets/2TWAyduY7PG2mz6cThAiw6izaBH checksum=ykt2Zc3Oo80XLuytqsUSE6fkGlJZvq7kE0pZrR8YX027tEtyd8TYI7RxI8EWAuL8wTzhWljY4P3OYN3mdmohOA== data={"type":"\u001bgithub.organization"} expected=block received=github.organization
x failed to send datapoints error="1 error occurred:\n\t* failed to store data for \"ykt2Zc3Oo80XLuytqsUSE6fkGlJZvq7kE0pZrR8YX027tEtyd8TYI7RxI8EWAuL8wTzhWljY4P3OYN3mdmohOA==\", types don't match: expected block, got github.organization\n\n"
```

and

```
cannot convert primitive with NO type information
```

related to mondoohq/cnquery#957

Signed-off-by: Christian Zunker <[email protected]>
czunker added a commit to mondoohq/cnspec-policies that referenced this issue Aug 14, 2023
Before, scanning a user repo with this policy resulted in two errors:

```
! resolver.db> failed to store data, types don't match asset=//policy.api.mondoo.com/assets/*** checksum=*** data={"type":"\u001bgithub.organization"} expected=block received=github.organization
x failed to send datapoints error="1 error occurred:\n\t* failed to store data for \"***\", types don't match: expected block, got github.organization\n\n"
```

and

```
cannot convert primitive with NO type information
```

related to mondoohq/cnquery#957

---------

Signed-off-by: Christian Zunker <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants