Skip to content
This repository has been archived by the owner on Dec 1, 2024. It is now read-only.

cvecache panics on eviction #154

Closed
s-guedon opened this issue Jul 22, 2020 · 1 comment
Closed

cvecache panics on eviction #154

s-guedon opened this issue Jul 22, 2020 · 1 comment

Comments

@s-guedon
Copy link

Description

When using cpe2cve with a positive -cache_size (0 and negative work fine) a panic occurs:

panic: attempted to evict non-existent record

goroutine 12 [running]:
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).evict(0xc0046dedc0, 0x0)
	/home/s/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:238 +0x12a
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).Get(0xc0046dedc0, 0xc00000e5a0, 0x1, 0x1, 0x1, 0xc0059d4110, 0x10)
	/home/s/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:171 +0x425
main.processAll(0xc0059d2600, 0xc0059d2660, 0xc0046c4d20, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/s/nvdtools/cmd/cpe2cve/cpe2cve.go:73 +0xd43
main.processInput.func1(0xc0059d2600, 0xc0059d2660, 0xc0046c4d20, 0xc00008c0f0, 0xc0059d40d8, 0xc0059d40e0)
	/home/s/nvdtools/cmd/cpe2cve/cpe2cve.go:139 +0xcc
created by main.processInput
	/home/s/nvdtools/cmd/cpe2cve/cpe2cve.go:138 +0x309

To reproduce

echo "cpe:/a:foo:bar:*" | ./cpe2cve -cpe 1 -e 1 -cve 1 -cache_size 1 ./nvdcve-1.1-2002.json.gz
This bug doesn't happen when the cache size is large enough (432 for the 2002 feed).
skogtwin added a commit to skogtwin/nvdtools that referenced this issue Jul 22, 2020
@skogtwin
Fix facebookincubator#154.
3f12633 
    Cache eviction was trying to evict the records until the new blob fits w/o checking if any records are aactually cached.

    This could happen when a record does not fit into cache:

```
    dvl@dvl-mbp cpe2cve % echo "cpe:/a:foo:bar:*" | go run . -cpe 1 -e 1 -cve 1 -cache_size 1 ~/feeds/cve_20200424000000_nvd.json.gz
    panic: attempted to evict non-existent record

    goroutine 14 [running]:
    github.com/facebookincubator/nvdtools/cvefeed.(*Cache).evict(0xc05df0b9f0, 0x0)
            /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:238 +0x140
    github.com/facebookincubator/nvdtools/cvefeed.(*Cache).Get(0xc05df0b9f0, 0xc00008e008, 0x1, 0x1, 0x1, 0xc07bb2bd70, 0x10)
            /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:171 +0x425
    main.processAll(0xc07bb45b60, 0xc07bb45bc0, 0xc05e3e20f0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
            /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:73 +0xd6a
    main.processInput.func1(0xc07bb45b60, 0xc07bb45bc0, 0xc05e3e20f0, 0xc0001340f0, 0xc07bb2bd38, 0xc07bb2bd40)
            /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:139 +0xcc
    created by main.processInput
            /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:138 +0x309
    exit status 2
```

    After the fix:
```
    dvl@dvl-mbp cpe2cve % echo "cpe:/a:foo:bar:*" | go run . -cpe 1 -e 1 -cve 1 -cache_size 1 ~/feeds/cve_20200424000000_nvd.json.gz
    dvl@dvl-mbp cpe2cve %
```
skogtwin added a commit to skogtwin/nvdtools that referenced this issue Jul 22, 2020
@skogtwin
Fix facebookincubator#154.
295f206 
Cache eviction was trying to evict the records until the new blob fits w/o checking if any records are actually cached.

This could happen when a record does not fit into cache:

```
dvl@dvl-mbp cpe2cve % echo "cpe:/a:foo:bar:*" | go run . -cpe 1 -e 1 -cve 1 -cache_size 1 ~/feeds/cve_20200424000000_nvd.json.gz
panic: attempted to evict non-existent record

goroutine 14 [running]:
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).evict(0xc05df0b9f0, 0x0)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:238 +0x140
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).Get(0xc05df0b9f0, 0xc00008e008, 0x1, 0x1, 0x1, 0xc07bb2bd70, 0x10)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:171 +0x425
main.processAll(0xc07bb45b60, 0xc07bb45bc0, 0xc05e3e20f0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:73 +0xd6a
main.processInput.func1(0xc07bb45b60, 0xc07bb45bc0, 0xc05e3e20f0, 0xc0001340f0, 0xc07bb2bd38, 0xc07bb2bd40)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:139 +0xcc
created by main.processInput
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:138 +0x309
exit status 2
```

After the fix:
```
dvl@dvl-mbp cpe2cve % echo "cpe:/a:foo:bar:*" | go run . -cpe 1 -e 1 -cve 1 -cache_size 1 ~/feeds/cve_20200424000000_nvd.json.gz
dvl@dvl-mbp cpe2cve %
```
skogtwin added a commit to skogtwin/nvdtools that referenced this issue Jul 22, 2020
@skogtwin
Fix facebookincubator#154.
fef866f 
Cache eviction was trying to evict the records until the new blob fits w/o checking if any records are actually cached.

This could happen when a record does not fit into cache:

```
dvl@dvl-mbp cpe2cve % echo "cpe:/a:foo:bar:*" | go run . -cpe 1 -e 1 -cve 1 -cache_size 1 ~/feeds/cve_20200424000000_nvd.json.gz
panic: attempted to evict non-existent record

goroutine 14 [running]:
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).evict(0xc05df0b9f0, 0x0)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:238 +0x140
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).Get(0xc05df0b9f0, 0xc00008e008, 0x1, 0x1, 0x1, 0xc07bb2bd70, 0x10)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:171 +0x425
main.processAll(0xc07bb45b60, 0xc07bb45bc0, 0xc05e3e20f0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:73 +0xd6a
main.processInput.func1(0xc07bb45b60, 0xc07bb45bc0, 0xc05e3e20f0, 0xc0001340f0, 0xc07bb2bd38, 0xc07bb2bd40)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:139 +0xcc
created by main.processInput
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:138 +0x309
exit status 2
```

After the fix:
```
dvl@dvl-mbp cpe2cve % echo "cpe:/a:foo:bar:*" | go run . -cpe 1 -e 1 -cve 1 -cache_size 1 ~/feeds/cve_20200424000000_nvd.json.gz
dvl@dvl-mbp cpe2cve %
```
skogtwin added a commit to skogtwin/nvdtools that referenced this issue Jul 22, 2020
@skogtwin
Fix facebookincubator#154.
3124fcf 
Cache eviction was trying to evict the records until the new blob fits w/o checking if any records are actually cached.

This could happen when a record does not fit into cache:

```
dvl@dvl-mbp cpe2cve % echo "cpe:/a:foo:bar:*" | go run . -cpe 1 -e 1 -cve 1 -cache_size 1 ~/feeds/cve_20200424000000_nvd.json.gz
panic: attempted to evict non-existent record

goroutine 14 [running]:
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).evict(0xc05df0b9f0, 0x0)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:238 +0x140
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).Get(0xc05df0b9f0, 0xc00008e008, 0x1, 0x1, 0x1, 0xc07bb2bd70, 0x10)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:171 +0x425
main.processAll(0xc07bb45b60, 0xc07bb45bc0, 0xc05e3e20f0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:73 +0xd6a
main.processInput.func1(0xc07bb45b60, 0xc07bb45bc0, 0xc05e3e20f0, 0xc0001340f0, 0xc07bb2bd38, 0xc07bb2bd40)
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:139 +0xcc
created by main.processInput
        /Users/dvl/go/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:138 +0x309
exit status 2
```

After the fix:
```
dvl@dvl-mbp cpe2cve % echo "cpe:/a:foo:bar:*" | go run . -cpe 1 -e 1 -cve 1 -cache_size 1 ~/feeds/cve_20200424000000_nvd.json.gz
dvl@dvl-mbp cpe2cve %
```
scottcunningham added a commit that referenced this issue Jul 23, 2020
@scottcunningham
Merge pull request #155 from skogtwin/fixcvecache
33c4eb7 
Fix #154.
@s-guedon
Copy link
Author

Thanks

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@s-guedon