Add prometheus metrics test into E2E #5673
Conversation
Co-Authored-By: Preston Van Loon <[email protected]>
endtoend/evaluators/metrics.go
Outdated
| return -1, fmt.Errorf("did not find requested text %s in %s", topic, pageContent) | ||
| } | ||
| endOfTopic := startIdx + len(topic) | ||
| fmt.Println(pageContent[startIdx:endOfTopic]) |
There was a problem hiding this comment.
is fmt.Println here intended or just for debug?
| { | ||
| name: "memory usage", | ||
| topic: "go_memstats_alloc_bytes", | ||
| value: 100000000, // 100 Mb |
There was a problem hiding this comment.
100MB only? I guess since it's only a few validators
There was a problem hiding this comment.
how do you get 100mb ? is there a better way to derive expected memory usage other than hardcoding it.
There was a problem hiding this comment.
I ran it and it seemed to be around 40mb normally. And yeah this is for only 64 validators so thats why its slow low.
endtoend/endtoend_test.go
Outdated
| @@ -71,7 +71,8 @@ func runEndToEndTest(t *testing.T, config *types.E2EConfig) { | |||
| } | |||
| // Small offset so evaluators perform in the middle of an epoch. | |||
| epochSeconds := params.BeaconConfig().SecondsPerSlot * params.BeaconConfig().SlotsPerEpoch | |||
| genesisTime := time.Unix(genesis.GenesisTime.Seconds+int64(epochSeconds/2), 0) | |||
| // Adding a half slot here to ensure the requests are in the middle of an epoch. | |||
| genesisTime := time.Unix(genesis.GenesisTime.Seconds+int64(epochSeconds/2+(params.BeaconConfig().SecondsPerSlot/2)), 0) | |||
There was a problem hiding this comment.
this is genesis time + half_epoch + half_slot . Can you clarify why you are doing this , this way ?
There was a problem hiding this comment.
Added clearer comments.
endtoend/long_minimal_e2e_test.go
Outdated
| @@ -16,7 +16,7 @@ func TestEndToEnd_Long_MinimalConfig(t *testing.T) { | |||
| testutil.ResetCache() | |||
| params.UseMinimalConfig() | |||
|
|
|||
| epochsToRun := 20 | |||
| epochsToRun := 100 | |||
There was a problem hiding this comment.
This doesn't matter right? It gets set from ENV variable
| epochsToRun := 100 | |
| var epochsToRun int |
There was a problem hiding this comment.
Ah my bad was for testing
What type of PR is this?
Improvement
What does this PR do? Why is it needed?
This PR improves E2E testing by adding metric checks, to ensure breaking changes aren't added that cause an increased amount of ssz topic errors, and memory usage.
Which issues(s) does this PR fix?
Fixes #5635