Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reporter: allow extending samples with extra attributes #237

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

athre0z
Copy link
Member

@athre0z athre0z commented Nov 12, 2024

This PR does two things:

Replace addProfileAttributes and attrKeyValue[T] with AttrTableManager

I found the previous design with the generic on addProfileAttributes rather inelegant. Firstly, there's no covariance on Go generics, so it would take two addProfileAttributes calls and an append to add both string and int attributes:

sample.Attributes = append(addProfileAttributes(profile, []attrKeyValue[string]{
{key: string(semconv.ContainerIDKey), value: traceKey.containerID},
{key: string(semconv.ThreadNameKey), value: traceKey.comm},
{key: string(semconv.ServiceNameKey), value: traceKey.apmServiceName},
}, attributeMap), addProfileAttributes(profile, []attrKeyValue[int64]{
{key: string(semconv.ProcessPIDKey), value: traceKey.pid},
}, attributeMap)...)

It also required reflection in addProfileAttributes for no good reason. IMHO this isn't really a good use-case for generics. In AttrTableManager this is replaced with two separate methods that each add just one attribute.

Secondly, a proper type for this results in a much cleaner interface for the second change discussed below.

Introduce SampleAttrProducer interface

This allows agent implementations to add custom sample attributes without maintaining a fork of the reporter implementation. It can gather extra meta-data when the trace is initially added and then emit extra attributes when the sample is sent out.

@athre0z athre0z added the enhancement New feature or request label Nov 12, 2024
@athre0z athre0z self-assigned this Nov 12, 2024
expectedIndices [][]uint64
expectedAttributeTable []*common.KeyValue
}{
"empty": {
profile: &profiles.Profile{},
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why this was in the struct: it seems that simply making this a local temporary variable in the loop below is easier. I reworked it accordingly, but perhaps I'm missing something on why this was done like this originally.

@@ -29,7 +27,6 @@ func TestGetSampleAttributes(t *testing.T) {
pid: 0,
},
},
attributeMap: make(map[string]uint64),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

@rockdaboot
Copy link
Contributor

Overall looks better than the generics approach 👍

@athre0z athre0z marked this pull request as ready for review November 12, 2024 18:24
@athre0z athre0z requested review from a team as code owners November 12, 2024 18:24
reporter/attrmgr.go Outdated Show resolved Hide resolved
reporter/config.go Show resolved Hide resolved
profile *profiles.Profile
}

func NewAttrTableManager(profile *profiles.Profile) *AttrTableManager {
Copy link
Contributor

@florianl florianl Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*profiles.Profile is internal to the package reporter. How are users of the package supposed to create it?
How should the integration look like, for a different kind of reporter, like for the OTel collector? see #208

From the updated testcase (TestGetSampleAttributes), where this API is used, it looks more like this API builds a parallel *profiles.Profile on top of an existing one just with attributes but without traces.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this private? It's a public type from the OTEL proto lib?

From the updated testcase (TestGetSampleAttributes), where this API is used, it looks more like this API builds a parallel *profiles.Profile on top of an existing one just with attributes but without traces.

No, it's just populating the attribute table portion. I'll apply the change that Tim suggested here to make this clearer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I looked through #208 now, and I think what you are referring to is that it will be private in the future, replaced by the pdata type? I hadn't followed up with the collector reporter impl -- still catching up after ~1 month mostly away from this repo here -- apologies! Fortunately the changes that Tim suggested (and that I have applied now) should allow the type to generalize to allow using it in the new reporter implementation as well.

reporter/attrmgr.go Outdated Show resolved Hide resolved
reporter/attrmgr.go Outdated Show resolved Hide resolved
reporter/attrmgr.go Outdated Show resolved Hide resolved
reporter/attrmgr.go Outdated Show resolved Hide resolved
attrTable *[]*common.KeyValue
}

func NewAttrTableManager(attrTable *[]*common.KeyValue) *AttrTableManager {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively we could not take that any argument, and rename the type to AttrTableBuilder. In that case I'd then add a new method GetAttrTable() that returns the final built table. That'd change the workflow for the caller to:

  • Create empty instance
  • Populate it with AddXXX calls
  • Take the final table with GetAttrTable() and manually assign it to the attribute table field

Perhaps more obvious than mutating the slice field in place through a pointer? No strong preference here.

Copy link
Contributor

@rockdaboot rockdaboot Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about this as well (named it Get() to not repeat AttrTable), but it comes with the downside that you need to call this function when done. And this can be forgotten, while the solution in this PR is safer from accidental failure.

My not very strong preference would be to avoid an extra Get function call.

type SampleAttrProducer interface {
// CollectExtraSampleMeta gathers extra sample meta-info and returns it as
// a pointer to a **hashable** struct.
CollectExtraSampleMeta(trace *libpf.Trace, meta *TraceEventMeta) any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the returned any replaced with a (generic) type?
Maybe generic type parameters could help here? My worry is, that any might introduce subtle bugs by not using this API correctly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not elegantly. It'd require us to also propagate the type parameter to Config and OTLPReporter structs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants