Failover does not respect priority-levels

[ioannispl41]

Component(s)

connector/failover

What happened?

Hello,

I came across a weird behavior …

It seems am unable to redirect traffic back to priority level one.

Here’s my current configuration:

config:
  connectors:
    failover:
      priority_levels:
        - [traces/first]
        - [traces/second]
      retry_interval: 1m
      retry_gap: 10s
      max_retries: 0 # Unlimited retries
  exporters:
    # Data sources: traces, metrics, logs
    otlp/1:
      endpoint: endpointA
      retry_on_failure:
        enabled: false
      sending_queue:
        enabled: false
      tls:
        insecure: true
    otlp/2:
      endpoint: endpointB
      retry_on_failure:
        enabled: false
      sending_queue:
        enabled: false
      tls:
        insecure: true
  service:
    pipelines:
      traces:
        receivers: [otlp]
        exporters: [failover]
      traces/first:
        receivers: [failover]
        exporters: [otlp/1]
      traces/second:
        receivers: [failover]
        exporters: [otlp/2]

When changed the max-retries value to 1000, it worked as expected: when otlp/1 stopped, traffic flowed through otlp2, and when otlp1 was restarted, it went through otlp1 again.

I believe that the issue is related to this line: not allowing connector to switch to a higher priority-level when max-retries==0.

Collector version

v0.110.0

Environment information

No response

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• connector/failover: @akats7 @djaglowski @fatsheep9146

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[akats7]

Hi @ioannispl41,

Thanks for bringing this up, I was able to recreate the issue and am pushing up a PR with a fix.