Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[extension/sigv4authextension] Auth failed without sts_region field #14573

Closed
bryan-aguilar opened this issue Sep 28, 2022 · 2 comments
Closed
Assignees
Labels
bug Something isn't working extension/sigv4auth priority:p2 Medium

Comments

@bryan-aguilar
Copy link
Contributor

bryan-aguilar commented Sep 28, 2022

What happened?

Description

Sigv4 auth extension fails initialization inside an EKS-Anywhere cluster without sts_region value. The cluster is setup with IRSA and the service account is attached.

Steps to Reproduce

  1. Setup cluster
  2. Setup IRSA
  3. Add service account
  4. Ensure service account is attached to deployment manifest
  5. Run collector with attached config

Expected Result

collector to start

Actual Result

Collector fails to start due to sigv4 error

Collector version

ADOT Collector v0.21.1-6b7244d. Uses upstream collector version v0.60.0

Environment information

EKS-Anywhere cluster using vSphere provider, os Bottlerocket, Kubernetes 1.21

OpenTelemetry Collector configuration

exporters:
      logging:
        logLevel: info
      prometheusremotewrite:
        auth:
          authenticator: sigv4auth
        endpoint: https://aps-workspaces.us-west-2.amazonaws.com/workspaces/<redacted-namespace>/v1/api/remote_write
    extensions:
      health_check: {}
      memory_ballast:
        size_mib: "819"
      sigv4auth:
        region: us-west-2
        service: aps
    processors:
      batch: {}
      memory_limiter:
        check_interval: 5s
        limit_mib: 1638
        spike_limit_mib: 512
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
          http:
            endpoint: 0.0.0.0:4318
      prometheus:
        config:
          scrape_configs:
          - job_name: opentelemetry-collector
            scrape_interval: 10s
            static_configs:
            - targets:
              - ${MY_POD_IP}:8888
    service:
      extensions:
      - health_check
      - memory_ballast
      - sigv4auth
      pipelines:
        metrics:
          exporters:
          - logging
          - prometheusremotewrite
          processors:
          - batch
          receivers:
          - prometheus
      telemetry:
        metrics:
          address: 0.0.0.0:8888

Log output

2022/09/28 16:37:19 ADOT Collector version: v0.21.1
2022/09/28 16:37:19 found no extra config, skip it, err: open /opt/aws/aws-otel-collector/etc/extracfg.txt: no such file or directory
Error: failed to get config: invalid configuration: extension "sigv4auth" has invalid configuration: could not retrieve credential provider: failed to refresh cached credentials, failed to retrieve credentials, operation error STS: AssumeRoleWithWebIdentity, failed to resolve service endpoint, an AWS region is required, but was not found
2022/09/28 16:37:19 application run finished with error: failed to get config: invalid configuration: extension "sigv4auth" has invalid configuration: could not retrieve credential provider: failed to refresh cached credentials, failed to retrieve credentials, operation error STS: AssumeRoleWithWebIdentity, failed to resolve service endpoint, an AWS region is required, but was not found

Additional context

Ideally, if sts_region isn't provided then the default region or aws_region value is used.

@bryan-aguilar bryan-aguilar added bug Something isn't working needs triage New item requiring triage labels Sep 28, 2022
@evan-bradley evan-bradley added priority:p2 Medium extension/sigv4auth and removed needs triage New item requiring triage labels Sep 29, 2022
@github-actions
Copy link
Contributor

Pinging code owners: @Aneurysm9 @erichsueh3. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@erichsueh3
Copy link
Contributor

Resolved by #14630.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working extension/sigv4auth priority:p2 Medium
Projects
None yet
Development

No branches or pull requests

4 participants