Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backports for Activitylog and Reporting 1.12.x #21133

Merged
merged 26 commits into from
Jun 14, 2023

Conversation

mpalmi
Copy link
Contributor

@mpalmi mpalmi commented Jun 12, 2023

This PR encapsulates the backports for Census and ActivityLog changes to get the release branches in sync and should resolve VAULT-14702 and VAULT-14765 along with the backports for the new client count testing library.

It was easier to backport the changes en masse, since they're all intertwined. Teasing apart feature-specific changes would have been more difficult.

Commits were prepared using (with fish shell, $() is just ()):

git cherry-pick -v -x (git log origin/release/1.12.x..origin/main --oneline --reverse -- "*activity*" | awk '{ print $1 }')

and editorialized/reworded using interactive rebase. The ACME-related changes (including testing of the clientType proto field via the client count testing library) were not backported.

Apologies for the large PR. It may be easiest to review by comparing each backport commit with the original. I can split this up if others think it's worthwhile.

Diff between backport and main

## NOTE: This diff does not include changes to core. 
git diff origin/activitylog-backports-1.12.x..origin/main -- "*activity*" "*census*" ':!ui/*'

https://gist.github.com/mpalmi/8cbb375f8616a8b1237e5189490ff23e

@mpalmi mpalmi force-pushed the activitylog-backports-1.12.x branch from 4552efd to bc283b0 Compare June 12, 2023 18:15
@mpalmi mpalmi requested review from raskchanky, banks, miagilepner, hghaf099 and a team June 12, 2023 19:18
@mpalmi mpalmi marked this pull request as ready for review June 12, 2023 19:20
@mpalmi mpalmi changed the title Activitylog backports 1.12.x backports for Activitylog and Reporting 1.12.x Jun 12, 2023
@mpalmi mpalmi force-pushed the activitylog-backports-1.12.x branch from bc283b0 to 3daf087 Compare June 13, 2023 18:09
Copy link
Contributor

@raskchanky raskchanky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot here to go through, especially for a backport. I did my best to sort through it, and to my eyes, it looks correct.

@mpalmi
Copy link
Contributor Author

mpalmi commented Jun 14, 2023

While going through the backport, there were 4 classes of commits (marked below):

❌ empty commits, which had already been backported but weren't attributed by git
❌ superfluous changes which rejected the entire commit (these got the 'ol --skip during the cherry-pick)
[no modifier] changes that ported over (sometimes with amendment)
[no modifier] clean cherry-picks, which ported over with no modifications

50a1f9e fix off by one err in current month client count computation (#17457)
0423ffb update protoc version to 3.21.7 oss (#17499)
483ef19 Eslint prefer-const (#17864)
840abfb Fix activity log end time (#17856)
87aa644 Don't return a 204 if there's no historical data (#17935)
186ee31 Link OSS (#18228)
b8de2c2 De-duplicate namespaces when historical and current month data are mixed (#18452)
cb61488 Account for mount counts when de-duplicating current and historical month data (#18598)
d641bbc Add new clients into the monthly breakdown (#18629)
2a5a07e Revert "Add new clients into the monthly breakdown (#18629)" (#18726)
771bd8b Add new clients into the monthly breakdown (#18766)
46b9921 Allow Token Create Requests To Be Replicated (#18689)
fbed961 UI: combine current + history client count tabs into one dashboard (#17575)
b5d7d47 VAULT-13061: Fix mount path discrepancy in activity log (#18916)
a9e17c2 VAULT-13763 normalize activity log mount paths (#19343)
7ef7297 UI: Use specific date in clients activity test (#19419)
9f7f8d5 VAULT-13729 activity log test godocs (#19433)
e55c18e adding copyright header (#19555)
e3c5977 Add no-op CensusAgent (#19625)
e9d6dbc activitylog: Fix pq.Get trace logger output (#19650)
bf2f1a8 UI: Fix flaky time-related tests (#19521)
b4fab6a VAULT-13191: OSS changes (#19891)
54904e4 VAULT-14733: Refactor processClientRecord in activity log (#19933)
d70c17f VAULT-14733: SegmentReader interface for reading activity log segments (#19934)
4b6ec40 Require activity log retention months at least the minimum (#20078)
05ba6bb api: Add reporting fields to activitylog config endpoint (#20086)
c95d4fb VAULT-14734: activity log write endpoint (#20019)
75c903f openapi: Add display attributes for /sys (p2) (#19707)
002a59a Add minimum_retention_months to config endpoint (#20150)
77f83d9 Refactor reporter for unseal setup (#20296)
730d0e2 VAULT-14733: Split logic of precomputedQueryWorker (#20073)
d234111 Start counting ACME certificate issuance as client activity (#20520)
35e2c16 VAULT-15703: Reload automated reporting (#20680)
6d95f8c Add client_type field to EntityRecord protobuf (#20626)
810d504 Add current_billing_period activity endpoint param (#20694)
5b23dd5 VAULT-14735: generate mock clients for activity log (#20252)
018ea84 VAULT-15395: Support mocking time functions in the activity log (#20720)
541f18e VAULT-14735: repeated and segmented activity log clients (#20699)
b4e2751 VAULT-14735: write mock activity log entity files (#20702)
dc5dd71 Deflake TestActivityLog_MultipleFragmentsAndSegments (#20930)
5002489 VAULT-15394: Generate activity log precomputed queries (#20778)

@mpalmi
Copy link
Contributor Author

mpalmi commented Jun 14, 2023

Here's a bit more insight into the diff between the 1.12 backports and main:

https://gist.github.com/mpalmi/817616ae97ddd5b2a5b90004bf766c23

@mpalmi mpalmi force-pushed the activitylog-backports-1.12.x branch from 47d9c63 to 10a4726 Compare June 14, 2023 20:02
@mpalmi mpalmi added this to the 1.12.8 milestone Jun 14, 2023
@mpalmi mpalmi force-pushed the activitylog-backports-1.12.x branch from 10a4726 to 2a315f5 Compare June 14, 2023 20:10
@mpalmi mpalmi force-pushed the activitylog-backports-1.12.x branch from 2a315f5 to 89247a2 Compare June 14, 2023 20:52
@mpalmi mpalmi merged commit e81c1f2 into release/1.12.x Jun 14, 2023
@mpalmi mpalmi deleted the activitylog-backports-1.12.x branch June 14, 2023 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants