Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf(filter): add cache for pattern matching handler #1038

Merged
merged 11 commits into from
Jun 26, 2024
Merged

perf(filter): add cache for pattern matching handler #1038

merged 11 commits into from
Jun 26, 2024

Conversation

almostinf
Copy link
Member

@almostinf almostinf commented Jun 21, 2024 •
edited
Loading

Add pattern matching cache

Added pattern matching handler caching to make GC life a little easier. Also added Refresh pattern storage benchmarks

On prod data with the right cache size:

benchmark                                      old ns/op     new ns/op     delta
BenchmarkPatternStorageProd/small_cache-12     167404983     171102503     +2.21%
BenchmarkPatternStorageProd/big_cache-12       167404983     122361363     -26.91%

benchmark                                      old allocs     new allocs     delta
BenchmarkPatternStorageProd/small_cache-12     740377         759268         +2.55%
BenchmarkPatternStorageProd/big_cache-12       740377         353399         -52.27%

benchmark                                      old bytes     new bytes     delta
BenchmarkPatternStorageProd/small_cache-12     59553850      60456515      +1.52%
BenchmarkPatternStorageProd/big_cache-12       59553850      29123259      -51.10%

On test data:

> go test -v -benchmem -memprofile=refresh.mem.out -cpuprofile=refresh.cpu.out -count=1 -benchtime=10s -bench=^BenchmarkPatternStorage . -run=^$ . | tee refresh_bench.txt

goos: linux
goarch: amd64
pkg: github.com/moira-alert/moira/perfomance_tests/filter
cpu: AMD Ryzen 5 5500U with Radeon Graphics         
BenchmarkPatternStorageRefresh
BenchmarkPatternStorageRefresh/small_cache
BenchmarkPatternStorageRefresh/small_cache-12               1140          10600212 ns/op         3580941 B/op   27984 allocs/op
BenchmarkPatternStorageRefresh/big_cache
BenchmarkPatternStorageRefresh/big_cache-12                 2172           5350007 ns/op          415106 B/op    4913 allocs/op
PASS
ok      github.com/moira-alert/moira/perfomance_tests/filter    25.613s

You can see that the cache increases speed and also greatly reduces the number of allocations

@almostinf almostinf marked this pull request as ready for review June 24, 2024 13:09
@almostinf almostinf requested a review from a team as a code owner June 24, 2024 13:09
@almostinf almostinf changed the title feat(filter): add cache regex for tags feat(filter): add cache for pattern matching handler Jun 24, 2024
@almostinf almostinf changed the title feat(filter): add cache for pattern matching handler perf(filter): add cache for pattern matching handler Jun 24, 2024
kissken
kissken previously approved these changes Jun 24, 2024
View reviewed changes
almostinf and others added 3 commits June 25, 2024 11:02
@almostinf
Merge branch 'master' into feat/cache-regex-for-tags
f6d324d
@dependabot
build(deps): bump github.com/hashicorp/go-retryablehttp (#1040)
9ba9ef1 
Bumps [github.com/hashicorp/go-retryablehttp](https://github.com/hashicorp/go-retryablehttp) from 0.7.1 to 0.7.7.
- [Changelog](https://github.com/hashicorp/go-retryablehttp/blob/main/CHANGELOG.md)
- [Commits](hashicorp/go-retryablehttp@v0.7.1...v0.7.7)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-retryablehttp
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@almostinf
fix godoc
7108e0b
@almostinf
Copy link
Member Author

/build

@almostinf
Copy link
Member Author

/build

@almostinf almostinf merged commit 4a2b7a5 into master Jun 26, 2024
7 checks passed
@almostinf almostinf deleted the feat/cache-regex-for-tags branch June 26, 2024 10:24
@github-actions GitHub Actions
Copy link

Build and push Docker images with tag: 2024-06-26.4a2b7a5

2 similar comments
@github-actions GitHub Actions
Copy link

Build and push Docker images with tag: 2024-06-26.4a2b7a5

@github-actions GitHub Actions
Copy link

Build and push Docker images with tag: 2024-06-26.4a2b7a5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kissken kissken kissken approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@almostinf @kissken