Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CBG-3379 add missing context logging #6408

Merged
merged 11 commits into from
Sep 13, 2023
Merged

CBG-3379 add missing context logging #6408

merged 11 commits into from
Sep 13, 2023

Conversation

torcolvin
Copy link
Collaborator

@torcolvin torcolvin commented Sep 11, 2023

  • add contexts through functions
  • does not include any places needed to make modifications to sg-bucket (will come in a separate PR)

Performed a search on context.Background() and context.TODO() and followed them through.

There are a few cases I didn't change:

  • generally things should come from context.Background() in the start of main, but there were some cases we use logging in init
  • there are a few cases we intentionally reset context during replication and ISGR, but these will need to be tagged with a database

I think the way to review this is to see where I've changed context.Background() to something that is passed in. Sometimes this could be a problem

Pre-review checklist

  • Removed debug logging (fmt.Print, log.Print, ...)
  • Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
  • Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Integration Tests

- add contexts through functions
- does not include any places needed to make modifications to sg-bucket
  (will come in a separate PR)

Performed a search on context.Background() and context.TODO() and
followed them through.

There are a few cases I didn't change:

- generally things should come from context.Background() in the start of
  main, but there were some cases we use logging in init
- there are a few cases we intentionally reset context during
  replication and ISGR, but these will need to be tagged with a database
@torcolvin torcolvin marked this pull request as ready for review September 11, 2023 14:41
@torcolvin torcolvin requested a review from bbrks September 11, 2023 16:20
Copy link
Member

@bbrks bbrks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - some comments inline for specifics about:

  • Consistency on interface methods (all or none?)
  • Structs that have embedded contexts AND parameterised context
  • bh.loggingCtx 3.1.0 regression for default collection

Comment on lines +416 to +417
ctx := context.TODO() // fix in sg-bucket
WarnfCtx(ctx, "Unable to obtain gocbcore.Agent while retrieving expiry:%v", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arguably this should be a wrapped error rather than a log but agree with TODO for now to coordinate other sg-bucket API changes.

Comment on lines +523 to +524
func (c *changeCache) Remove(ctx context.Context, collectionID uint32, docIDs []string, startTime time.Time) (count int) {
return c.channelCache.Remove(ctx, collectionID, docIDs, startTime)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a pattern/rule to follow for usage of parameter context vs. c.logCtx? Usage-dependent?

Having context defined only in some methods makes me wonder how we're going to maintain this with any consistency going forwards.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I largely didn't change the existing code - but I think writing new code we should try to pass a context in. There are some places this is impractical:

  • DCP related code (especially but not limited to cbgt) with specific callbacks we have to match interfaces with

I was thinking in main branch we have the opportunity to clean this up but not for log streaming.

Comment on lines 59 to +64
// Clear reinitializes the cache to an empty state
Clear()

// Size of the the largest individual channel cache, invoked for stats reporting
// // TODO: let the cache manage its own stats internally (maybe take an updateStats call)
MaxCacheSize() int
MaxCacheSize(context.Context) int
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we make context consistent across the full interface? Even if the current implementation doesn't actually use context in all methods?

@@ -242,7 +242,7 @@ func (bh *blipHandler) handleSetCheckpoint(rq *blip.Message) error {
func (bh *blipHandler) handleSubChanges(rq *blip.Message) error {
defaultSince := CreateZeroSinceValue()
latestSeq := func() (SequenceID, error) {
seq, err := bh.collection.LastSequence()
seq, err := bh.collection.LastSequence(bh.loggingCtx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned via Slack - What looks like a 3.1.0 regression - bh.loggingCtx isn't being set properly for default collection replications, resulting in us losing context ID for a lot of logging called via BLIP.

bbrks
bbrks previously approved these changes Sep 13, 2023
@bbrks bbrks enabled auto-merge (squash) September 13, 2023 16:42
@bbrks bbrks merged commit 2da8f9c into master Sep 13, 2023
26 checks passed
@bbrks bbrks deleted the CBG-3379 branch September 13, 2023 17:03
torcolvin added a commit that referenced this pull request Sep 13, 2023
bbrks added a commit that referenced this pull request Sep 15, 2023
bbrks added a commit that referenced this pull request Mar 28, 2024
mohammed-madi pushed a commit that referenced this pull request Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants