Syslog structured_data field loses structure in Attributes field

[ttomsu]

Component(s)

receiver/syslog

What happened?

Description

When the syslog receiver uses RFC5424 structured data, that data get stuffed into the Attributes here. Unfortunately, this map[string]map[string]string type is not recognized by the converter, and the resulting map loses all structure as it gets stringified by the default block.

Steps to Reproduce

You can use the same github.com/influxdata/go-syslog/v3/rfc5424 package used internally to create a SyslogMessage with structured data that gets squashed:

msg.SetPriority(100 /* Severity Warn and Facility 12*/)
  .SetMessage(contentStr)
  .SetTimestamp(ts.Format(time.RFC3339))
  .SetVersion(1)
  .SetParameter("customID", "customTag", "customValue")
			

Expected Result

I expected logRecord.Attributes().Get("structured_data") to return a pcommon.ValueTypeMap - though I can see why, because I think according to the spec attributes are string -> string.

Because of this, I also thought maybe I could change the parse_to field like in other receivers, but it looks like that's baked in.

Actual Result

It returns a ValueTypeStr

Collector version

0.85.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/syslog: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[djaglowski]

@ttomsu, thanks for reporting this, and especially for finding the relevant code. I agree it's not handled correctly.

You can use the same github.com/influxdata/go-syslog/v3/rfc5424 package used internally to create a SyslogMessage with structured data that gets squashed:

msg.SetPriority(100 /* Severity Warn and Facility 12*/)
  .SetMessage(contentStr)
  .SetTimestamp(ts.Format(time.RFC3339))
  .SetVersion(1)
  .SetParameter("customID", "customTag", "customValue")		

I'm not understanding what you are suggesting here. Can you clarify how this would be used in our code?

---

thought maybe I could change the parse_to field like in other receivers, but it looks like that's baked in.

I think perhaps a version of this which applies specifically to structured data makes sense. e.g. parse_structured_data_to This could be a non-breaking change if the default is attributes.structured_data.

As much as I'd like to avoid stingified maps as a default, I think we are in a tough spot here because we need to respect some general expectations:

• attributes should be flat
• body should be the original

---

Another option would be that we flatten the structured data string into a dot-notation. e.g.

{
  "foo": {
    "hello": "world",
    "one": "two"
  },
  "bar": {
    "three": "four"
  }
}

is flattened into the following attributes:

• foo.hello: "world"
• foo.one: "two"
• bar.three: "four"

Again, if we went this route, I think we should make it opt-in since it's somewhat abnormal behavior.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/syslog: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/syslog: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.