[pkg/ottl] Express context cache as a map[string]any.

[kentquirk]

Component(s)

pkg/ottl

Is your feature request related to a problem? Please describe.

Separate ticket from #22010

Describe the solution you'd like

We need to update all the cache accessors to be able to handle getting and setting time.Time and time.Duration.

More info to come.

Describe alternatives you've considered

No response

Additional context

@fchikwekwe is intending to work on this. Further comment expected from @TylerHelmuth .

[TylerHelmuth]

@evan-bradley with the new Now() function we'll want this functionality so that we can save off a Now() result that could be used for each statement in the list.

This change will impact complex indexing, so we'll have to be careful.

[TylerHelmuth]

We started working on this and the impact to complex indexing is non-trivial.

[TylerHelmuth]

Related to this problem, I was just shown the scenario

set(resource.attributes["my.environment.2"], Split(resource.attributes["host.name"],"-")[1])

that produces an error:

{"kind": "processor", "name": "transform", "pipeline": "logs", "error": "type, []string, does not support int indexing", "statement": "set(resource.attributes[\"my.environment.2\"], Split(resource.attributes[\"host.name\"],\"-\")[1])"}

Although the language does support indexing the return value of the Converter, it only supports indexing pcommon.Slice and []any, but Split returns a []string. We need to make indexing slices more generic. This might need to be its own issue. Related code:

opentelemetry-collector-contrib/pkg/ottl/expression.go

Lines 98 to 112 in 60430e1

	case k.Int != nil:
	switch r := result.(type) {
	case pcommon.Slice:
	if int(*k.Int) >= r.Len() || int(*k.Int) < 0 {
	return nil, fmt.Errorf("index %v out of bounds", *k.Int)
	}
	result = ottlcommon.GetValue(r.At(int(*k.Int)))
	case []any:
	if int(*k.Int) >= len(r) || int(*k.Int) < 0 {
	return nil, fmt.Errorf("index %v out of bounds", *k.Int)
	}
	result = r[*k.Int]
	default:
	return nil, fmt.Errorf("type, %T, does not support int indexing", result)
	}

[evan-bradley]

I think this is a separate issue, but I agree that we should be able to index any slice type that is supported in OTTL.

[TylerHelmuth]

I've started thinking about this problem again and I keep getting stuck on how much we lose my making cache a map[string]any instead of a pcommon.Map. For most scenarios it is very beneficial to have a pcommon.Map since we are already working with the data in pdata format. To switch to a map[string]any means translating back and forth, which should not be necessary most of the time.

I've come up with a couple alternative solutions I'd like to discuss.

1. Leave cache as it is and add a new accessor for storing non-pcommon.Value types such as time.Time and time.Duration. This has the benefit of performance, but I think there could be confusion with which accessor to use, cache or this other one.

2. Continue using cache as the path accessor, but for non-pcommon.Value types, store the value in a different map behind the scenes. This add increased complexity in the Context implementation of the accessor, but hides those details from the user. I think there is probably a way while setting a value to record that this key is not a pdata value and then save the value off in some other map[string]any. Then when getting we'd always check if the key is for a non-pcommon.Value type, which is at least a constant-time check.

[TylerHelmuth]

Thinking about this issue some more:

If the primary goal is to be able to store time.Time in the cache, then one solution is to convert time.Time to int64 using UnixSeconds and then convert back to a time.Time using Unix.

I believe all other types can be expressed in the pcommon.Map via a pcommon.Value. If this is true then we could potential close this issue until more non-pdata types come up.

[TylerHelmuth]

With the release of the Unix function in v0.98.0 I am going to close this issue for now. I believe there are sufficient solutions for storing a timestamp if needed and the efficiency we're gaining from using a pcommon.Map is one of the reasons we chose to allow OTTL to closely tie itself to pdata.

If this issue comes up again we can reopen this issue.