Add parse_ints config in json parser to support parsing int or float properly

.chloggen/json_parser_number_data_type.yaml

@@ -0,0 +1,27 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: pkg/stanza
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: expose json iterator config in json parser
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [33696]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:
+
+# If your change doesn't affect end users or the exported elements of any package,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: []

pkg/stanza/docs/operators/json_parser.md

@@ -4,21 +4,43 @@ The `json_parser` operator parses the string-type field selected by `parse_from`
 
 ### Configuration Fields
 
-| Field         | Default          | Description |
-| ---           | ---              | ---         |
-| `id`          | `json_parser`    | A unique identifier for the operator. |
-| `output`      | Next in pipeline | The connected operator(s) that will receive all outbound entries. |
-| `parse_from`  | `body`           | The [field](../types/field.md) from which the value will be parsed. |
-| `parse_to`    | `attributes`     | The [field](../types/field.md) to which the value will be parsed. |
-| `on_error`    | `send`           | The behavior of the operator if it encounters an error. See [on_error](../types/on_error.md). |
-| `if`          |                  | An [expression](../types/expression.md) that, when set, will be evaluated to determine whether this operator should be used for the given entry. This allows you to do easy conditional parsing without branching logic with routers. |
-| `timestamp`   | `nil`            | An optional [timestamp](../types/timestamp.md) block which will parse a timestamp field before passing the entry to the output operator. |
-| `severity`    | `nil`            | An optional [severity](../types/severity.md) block which will parse a severity field before passing the entry to the output operator. |
+| Field             | Default          | Description |
+| ---               | ---              | ---         |
+| `id`              | `json_parser`    | A unique identifier for the operator. |
+| `output`          | Next in pipeline | The connected operator(s) that will receive all outbound entries. |
+| `parse_from`      | `body`           | The [field](../types/field.md) from which the value will be parsed. |
+| `parse_to`        | `attributes`     | The [field](../types/field.md) to which the value will be parsed. |
+| `on_error`        | `send`           | The behavior of the operator if it encounters an error. See [on_error](../types/on_error.md). |
+| `if`              |                  | An [expression](../types/expression.md) that, when set, will be evaluated to determine whether this operator should be used for the given entry. This allows you to do easy conditional parsing without branching logic with routers. |
+| `timestamp`       | `nil`            | An optional [timestamp](../types/timestamp.md) block which will parse a timestamp field before passing the entry to the output operator. |
+| `severity`        | `nil`            | An optional [severity](../types/severity.md) block which will parse a severity field before passing the entry to the output operator. |
+| `jsontier_config` | `nil`            | An optional jsontier config block. See below for details. |
 
 ### Embedded Operations
 
 The `json_parser` can be configured to embed certain operations such as timestamp and severity parsing. For more information, see [complex parsers](../types/parsers.md#complex-parsers).
 
+### `jsoniter_config` Configuration
+
+The `json_parser` operator uses the [json-iterator](https://github.com/json-iterator/go) as the underlying json parser, the default config is [ConfigFastest](https://pkg.go.dev/github.com/json-iterator/go#pkg-variables).
+
+In additional, this `jsoniter_config` block allows you to configure the json parser with a custom configuration. Here are available fields that map to the corresponding fields in [json-iterator Config](https://pkg.go.dev/github.com/json-iterator/go#Config):
+
+| Field                                | Default | Description                                        |
+|--------------------------------------|---------|----------------------------------------------------|
+| `indention_step`                     | 0       | json-iterator.Config.IndentionStep                 |
+| `marshal_float_with_6_digits`        | `false` | json-iterator.Config.MarshalFloatWith6Digits       |
+| `escape_html`                        | `false` | json-iterator.Config.EscapeHTML                    |
+| `sort_map_keys`                      | `false` | json-iterator.Config.SortMapKeys                   |
+| `use_number`                         | `false` | json-iterator.Config.UseNumber                     |
+| `disallow_unknown_fields`            | `false` | json-iterator.Config.DisallowUnknownFields         |
+| `tag_key`                            | ``      | json-iterator.Config.TagKey                        |
+| `only_tagged_field`                  | `false` | json-iterator.Config.OnlyTaggedField               |
+| `validate_json_raw_message`          | `false` | json-iterator.Config.ValidateJsonRawMessage        |
+| `object_field_must_be_simple_string` | `false` | json-iterator.Config.ObjectFieldMustBeSimpleString |
+| `case_sensitive`                     | `false` | json-iterator.Config.CaseSensitive                 |
+
+numbers like `int` and `float` are parsed as `float64` by default, when `use_number` is enabled, numbers are parsed as `json.Number` and then coverted to `int64` or `float64` based on the value.
 
 ### Example Configurations
 

pkg/stanza/operator/parser/json/config.go

@@ -32,6 +32,38 @@ func NewConfigWithID(operatorID string) *Config {
 // Config is the configuration of a JSON parser operator.
 type Config struct {
 	helper.ParserConfig `mapstructure:",squash"`
+
+	*JsoniterConfig `mapstructure:"jsoniter_config,omitempty"`
+}
+
+type JsoniterConfig struct {
+	IndentionStep                 int    `mapstructure:"indention_step"`
+	MarshalFloatWith6Digits       bool   `mapstructure:"marshal_float_with_6_digits"`
+	EscapeHTML                    bool   `mapstructure:"escape_html"`
+	SortMapKeys                   bool   `mapstructure:"sort_map_keys"`
+	UseNumber                     bool   `mapstructure:"use_number"`
+	DisallowUnknownFields         bool   `mapstructure:"disallow_unknown_fields"`
+	TagKey                        string `mapstructure:"tag_key"`
+	OnlyTaggedField               bool   `mapstructure:"only_tagged_field"`
+	ValidateJsonRawMessage        bool   `mapstructure:"validate_json_raw_message"`
+	ObjectFieldMustBeSimpleString bool   `mapstructure:"object_field_must_be_simple_string"`
+	CaseSensitive                 bool   `mapstructure:"case_sensitive"`
+}
+
+func (jc JsoniterConfig) toJsoniterAPI() jsoniter.API {
+	return jsoniter.Config{
+		IndentionStep:                 jc.IndentionStep,
+		MarshalFloatWith6Digits:       jc.MarshalFloatWith6Digits,
+		EscapeHTML:                    jc.EscapeHTML,
+		SortMapKeys:                   jc.SortMapKeys,
+		UseNumber:                     jc.UseNumber,
+		DisallowUnknownFields:         jc.DisallowUnknownFields,
+		TagKey:                        jc.TagKey,
+		OnlyTaggedField:               jc.OnlyTaggedField,
+		ValidateJsonRawMessage:        jc.ValidateJsonRawMessage,
+		ObjectFieldMustBeSimpleString: jc.ObjectFieldMustBeSimpleString,
+		CaseSensitive:                 jc.CaseSensitive,
+	}.Froze()
 }
 
 // Build will build a JSON parser operator.
@@ -41,8 +73,18 @@ func (c Config) Build(set component.TelemetrySettings) (operator.Operator, error
 		return nil, err
 	}
 
+	var json jsoniter.API
+	var convertNumber bool
+	if c.JsoniterConfig != nil {
+		json = c.JsoniterConfig.toJsoniterAPI()
+		convertNumber = c.JsoniterConfig.UseNumber
+	} else {
+		json = jsoniter.ConfigFastest
+	}
+
 	return &Parser{
 		ParserOperator: parserOperator,
-		json:           jsoniter.ConfigFastest,
+		json:           json,
+		useNumber:      convertNumber,
 	}, nil
 }

pkg/stanza/operator/parser/json/config_test.go

@@ -110,6 +110,16 @@ func TestConfig(t *testing.T) {
 					return p
 				}(),
 			},
+			{
+				Name: "use_number",
+				Expect: func() *Config {
+					p := NewConfig()
+					p.JsoniterConfig = &JsoniterConfig{
+						UseNumber: true,
+					}
+					return p
+				}(),
+			},
 		},
 	}.Run(t)
 }

pkg/stanza/operator/parser/json/parser.go

@@ -5,6 +5,7 @@ package json // import "github.com/open-telemetry/opentelemetry-collector-contri
 
 import (
 	"context"
+	"encoding/json"
 	"fmt"
 
 	jsoniter "github.com/json-iterator/go"
@@ -16,7 +17,8 @@ import (
 // Parser is an operator that parses JSON.
 type Parser struct {
 	helper.ParserOperator
-	json jsoniter.API
+	json      jsoniter.API
+	useNumber bool
 }
 
 // Process will parse an entry for JSON.
@@ -36,5 +38,47 @@ func (p *Parser) parse(value any) (any, error) {
 	default:
 		return nil, fmt.Errorf("type %T cannot be parsed as JSON", value)
 	}
+
+	if p.useNumber {
+		p.convertNumbers(parsedValue)
+	}
 	return parsedValue, nil
 }
+
+func (p *Parser) convertNumbers(parsedValue map[string]any) {
+	for k, v := range parsedValue {
+		switch t := v.(type) {
+		case json.Number:
+			parsedValue[k] = p.convertNumber(t)
+		case map[string]any:
+			p.convertNumbers(t)
+		case []any:
+			p.convertNumbersArray(t)
+		}
+	}
+}
+
+func (p *Parser) convertNumbersArray(arr []any) {
+	for i, v := range arr {
+		switch t := v.(type) {
+		case json.Number:
+			arr[i] = p.convertNumber(t)
+		case map[string]any:
+			p.convertNumbers(t)
+		case []any:
+			p.convertNumbersArray(t)
+		}
+	}
+}
+
+func (p *Parser) convertNumber(value json.Number) any {
+	i64, err := value.Int64()
+	if err == nil {
+		return i64
+	}
+	f64, err := value.Float64()
+	if err == nil {
+		return f64
+	}
+	return value.String()
+}

pkg/stanza/operator/parser/json/parser_test.go

@@ -139,6 +139,102 @@ func TestParser(t *testing.T) {
 				ScopeName: "logger",
 			},
 		},
+		{
+			"use_number_simple",
+			func(p *Config) {
+				p.JsoniterConfig = &JsoniterConfig{UseNumber: true}
+			},
+			&entry.Entry{
+				Body: `{"int":1,"float":1.0}`,
+			},
+			&entry.Entry{
+				Attributes: map[string]any{
+					"int":   int64(1),
+					"float": float64(1),
+				},
+				Body: `{"int":1,"float":1.0}`,
+			},
+		},
+		{
+			"use_number_nested",
+			func(p *Config) {
+				p.JsoniterConfig = &JsoniterConfig{UseNumber: true}
+			},
+			&entry.Entry{
+				Body: `{"int":1,"float":1.0,"nested":{"int":2,"float":2.0}}`,
+			},
+			&entry.Entry{
+				Attributes: map[string]any{
+					"int":   int64(1),
+					"float": float64(1),
+					"nested": map[string]any{
+						"int":   int64(2),
+						"float": float64(2),
+					},
+				},
+				Body: `{"int":1,"float":1.0,"nested":{"int":2,"float":2.0}}`,
+			},
+		},
+		{
+			"use_number_arrays",
+			func(p *Config) {
+				p.JsoniterConfig = &JsoniterConfig{UseNumber: true}
+			},
+			&entry.Entry{
+				Body: `{"int":1,"float":1.0,"nested":{"int":2,"float":2.0},"array":[1,2]}`,
+			},
+			&entry.Entry{
+				Attributes: map[string]any{
+					"int":   int64(1),
+					"float": float64(1),
+					"nested": map[string]any{
+						"int":   int64(2),
+						"float": float64(2),
+					},
+					"array": []any{int64(1), int64(2)},
+				},
+				Body: `{"int":1,"float":1.0,"nested":{"int":2,"float":2.0},"array":[1,2]}`,
+			},
+		},
+		{
+			"use_number_mixed_arrays",
+			func(p *Config) {
+				p.JsoniterConfig = &JsoniterConfig{UseNumber: true}
+			},
+			&entry.Entry{
+				Body: `{"int":1,"float":1.0,"mixed_array":[1,1.5,2]}`,
+			},
+			&entry.Entry{
+				Attributes: map[string]any{
+					"int":         int64(1),
+					"float":       float64(1),
+					"mixed_array": []any{int64(1), float64(1.5), int64(2)},
+				},
+				Body: `{"int":1,"float":1.0,"mixed_array":[1,1.5,2]}`,
+			},
+		},
+		{
+			"use_number_nested_arrays",
+			func(p *Config) {
+				p.JsoniterConfig = &JsoniterConfig{UseNumber: true}
+			},
+			&entry.Entry{
+				Body: `{"int":1,"float":1.0,"nested":{"int":2,"float":2.0,"array":[1,2]},"array":[3,4]}`,
+			},
+			&entry.Entry{
+				Attributes: map[string]any{
+					"int":   int64(1),
+					"float": float64(1),
+					"nested": map[string]any{
+						"int":   int64(2),
+						"float": float64(2),
+						"array": []any{int64(1), int64(2)},
+					},
+					"array": []any{int64(3), int64(4)},
+				},
+				Body: `{"int":1,"float":1.0,"nested":{"int":2,"float":2.0,"array":[1,2]},"array":[3,4]}`,
+			},
+		},
 	}
 
 	for _, tc := range cases {

pkg/stanza/operator/parser/json/testdata/config.yaml

@@ -37,3 +37,7 @@ timestamp:
     parse_from: body.timestamp_field
     layout_type: strptime
     layout: '%Y-%m-%d'
+use_number:
+  type: json_parser
+  jsoniter_config:
+    use_number: true