Fileexporter rotation breaks up log lines

[peterzandbergen]

Component(s)

exporter/file

What happened?

Description

When rotation is configured the rotation does not occur on line ends.

Steps to Reproduce

• Configure the file exporter with rotation and set max_megabytes: 1
• produce enough signals, e.g. using the filelog receiver to trigger a rotation
• tail the rotated file. This shows that the end of the file breaks up the json

Expected Result

The logfiles produced by the file exporter should contain valid json structs

Actual Result

The last line ends in the middle of the last json struct

Collector version

85a618f

Environment information

Environment

Distributor ID: Pop
Description: Pop!_OS 22.04 LTS
Release: 22.04
Codename: jammy

Compiler(if manually compiled): (e.g., "go 14.2")
go version go1.20.4 linux/amd64

OpenTelemetry Collector configuration

# File log receiver
extensions:
  file_storage/filelog_status:
    directory: ./filelog_status

receivers:
  otlp:
    protocols:
      grpc:
      http:
  filelog/test.log:
    include:
      - ./app_logs/*.log
    operators:
      - type: json_parser
        # parse_from: key
    storage: file_storage/filelog_status

exporters:
  file/stdout:
    path: /dev/stdout
  file/test-out:
    rotation:
      max_megabytes: 1
    path: ./logs/test.out
    flush_interval: 1s

service:
  # telemetry:
  #   logs:
  #     encoding: json
  #   metrics:
  #     address: 0.0.0.0:18888

  extensions:
    - file_storage/filelog_status
    
  pipelines:
    traces:
      receivers:
        - otlp
      processors: []
      exporters:
        - file/stdout
        - file/test-out
    logs:
      receivers:
        - filelog/test.log
        - otlp
      processors: []
      exporters:
        - file/stdout
        - file/test-out

Log output

No response

Additional context

I have analyzed the error and it was introduced by my contribution for flushing the file exporter.
I have also managed to fix it by not wrapping the buffered writer around the lumberjack logger.

diff --git a/exporter/fileexporter/factory.go b/exporter/fileexporter/factory.go
index baecdc4701..246a61f5f0 100644
--- a/exporter/fileexporter/factory.go
+++ b/exporter/fileexporter/factory.go
@@ -142,13 +142,13 @@ func buildFileWriter(cfg *Config) (io.WriteCloser, error) {
                }
                return newBufferedWriteCloser(f), nil
        }
-       return newBufferedWriteCloser(&lumberjack.Logger{
+       return &lumberjack.Logger{
                Filename:   cfg.Path,
                MaxSize:    cfg.Rotation.MaxMegabytes,
                MaxAge:     cfg.Rotation.MaxDays,
                MaxBackups: cfg.Rotation.MaxBackups,
                LocalTime:  cfg.Rotation.LocalTime,
-       }), nil
+       }, nil
 }
 

[github-actions]

Pinging code owners:

• exporter/file: @atingchen

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atingchen]

@peterzandbergen hello,Peter.Are you interested in submitting a pull request to fix this bug?

[peterzandbergen]

Hi Luke,
Sure. I broke it so I am more than willing to fix it 😁

[atingchen]

Hi Luke,
Sure. I broke it so I am more than willing to fix it 😁

Thank you very much for your contribution～

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/file: @atingchen

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.