Monolithic deployment with s3 storage fails with "mkdir /var/tempo/wal: permission denied"

[HendrikLevering]

Describe the bug
I want to deploy tempo monolithic with was s3 storage on an on premises k8 (k3s) cluster. The pod fails with:

level=info ts=2024-08-13T16:09:28.028083561Z caller=main.go:225 msg="initialising OpenTracing tracer" 
level=info ts=2024-08-13T16:09:28.049284798Z caller=main.go:118 msg="Starting Tempo" version="(version=2.5.0, branch=HEAD, revision=46dad3411)"  
level=info msg="server listening on addresses" http=:3101 grpc=:33549            
level=info ts=2024-08-13T16:09:28.148624944Z caller=cache.go:55 msg="caches available to storage backend" footer=false bloom=false offset_idx=false column_idx=false trace_id_idx=false page=false 
level=error ts=2024-08-13T16:09:28.149841871Z caller=main.go:121 msg="error running Tempo" err="failed to init module services: error initialising module: store: failed to create store: mkdir /var/tempo/wal: permission denied"

To Reproduce
Steps to reproduce the behavior:

tempo resource

apiVersion: tempo.grafana.com/v1alpha1
kind: TempoMonolithic
metadata:
  name: tempo
  namespace: tempo
spec:
  storage:
    traces:
      backend: "s3"
      s3:
        secret: "tempo-secret"
  resources:
    limits:
      memory: 2Gi
      cpu: 2000m

secret

  apiVersion: v1
  kind: Secret
  metadata:
    name: tempo-secret
    namespace: tempo
  stringData:
    endpoint: "https://s3.eu-central-1.amazonaws.com"
    bucket: my-bucket
    access_key_id: RETRACTED
    access_key_secret:  RETRACTED
  type: Opaque

Expected behavior
the pod starts succesfully

Environment:

• Infrastructure: kubernetes v1.29.7+k3s1, running on Hetzner VPS
• Deployment tool: kubectl

Additional Context

[HendrikLevering]

ok. It seems that this is a bug with the tempo operator. It fails with operator 0.11.0, but works with operator version 0.10. Related issue in the operator repo:
grafana/tempo-operator#996

[github-actions]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days.
The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity.
Please apply keepalive label to exempt this Issue.