Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not possible to access init container logs #745

Open
1 of 3 tasks
iamdanthedev opened this issue May 10, 2023 · 8 comments
Open
1 of 3 tasks

Not possible to access init container logs #745

iamdanthedev opened this issue May 10, 2023 · 8 comments
Assignees
Labels
investigating currently looking into the issue observability related to observability Provisioning Related to deployment issues, revision provisioning, etc.

Comments

@iamdanthedev
Copy link

Please provide us with the following information:

This issue is a: (mark with an x)

  • bug report -> please search issues before submitting
  • documentation issue or request
  • regression (a behavior that used to work and stopped in a new release)

Issue description

A clear and concise description of the observed issue.

Steps to reproduce

  1. Configure initContainers similar to
      initContainers: [
        {
          name: 'migration-up-runner'
          image: '${crName}.azurecr.io/migration_up:${dockerTag}'
          env: dbEnvVars
          resources: {
            cpu: '0.25'
            memory: '0.5Gi'
          }
        }
      ]
  1. The container probably throws an error during execution
  2. Go to System logs / Console logs
  3. No execution logs appear

To be fair, I did see the logs 1 time out of 20 deployments, and I am not sure what made it work. I tried all different combinations of ENTRYPOINT, command, command + args

Expected behavior [What you expected to happen.]
I expect to go to Console Logs and see init container execution logs in order to debug the problem

Actual behavior [What actually happened.]
The logs are nowhere to be found

Screenshots
If applicable, add screenshots to help explain your problem.

@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs: triage 🔍 Pending a first pass to read, tag, and assign label May 10, 2023
@SophCarp SophCarp added observability related to observability Provisioning Related to deployment issues, revision provisioning, etc. Needs: Validation Bug needs to be replicated. Community members welcome to validate and share! Needs: Investigation Issue has been assigned and tagged, pending them to process it investigating currently looking into the issue and removed Needs: triage 🔍 Pending a first pass to read, tag, and assign Needs: Validation Bug needs to be replicated. Community members welcome to validate and share! Needs: Investigation Issue has been assigned and tagged, pending them to process it labels May 16, 2023
@alicyn
Copy link

alicyn commented May 18, 2023

I am having the same issue, cannot see logs from init containers at all.

I tried:
az containerapp logs show -n <containerapp_name> -g <resource_group> --follow --revision <revision_name> --container <init_container_name> --type console

result:
Could not find container

@vinisoto
Copy link
Collaborator

Hi, we are working on a fix for this issue. Will update here when the fix is completed.

@ractando
Copy link

ractando commented Sep 4, 2023

@vinisoto is the fix applied for this issue?

@kobeyy
Copy link

kobeyy commented Nov 29, 2023

We are experiencing exactly the same issue. Our database migration script fails but we are not able to view the logs which makes debugging really hard.

@vinisoto any update?

We are in the progress of migrating ~30 backend services from AKS to container apps. Would really appreciate an update regarding this issue. Depending on the timeframe we might need to delay the roll out.

@iamdanthedev
Copy link
Author

By the way for us this problem got resolved somehow by itself somewhere around early autumn. I can see log messages coming from the init container in the regular Console Logs section.

@vinisoto
Copy link
Collaborator

vinisoto commented Dec 5, 2023

@kobeyy - this issue should be resolved. If you are still experiencing issues, please send us an email at acasupport at microsoft dot com with you subscription id, container app name and timeframe of missing logs

@JPeer264
Copy link

JPeer264 commented May 4, 2024

Not sure if this is related since mine is within ACIs.

The Azure dashboard itself always shows "Loading..." when the init containers are still running (State = Running). Over the CLI, however, I get an error stating the container group is not yet ready:

$ az container logs --name <container_group> --container-name <container_name> --resource-group <resource_group>
# (ContainerGroupDeploymentNotReady) The container group '<container_group>' is not ready for the operation.
# Code: ContainerGroupDeploymentNotReady
# Message: The container group '<container_group>' is not ready for the operation.

In the Azure dashboard the logs are shown when State = Terminated.
However, as soon as the init container has finished (State = Terminated) then the logs can be seen both in the dashboard and the CLI (assuming the same command is executed as before).

@bandwiches
Copy link

Issue persists as of Dec 2024. It's confusing at best whether this is resolved or being worked on.

Confirmed Workaround: export to log analytics


This issue was opened May 2023. Latest update was Dec 2023:

@kobeyy - this issue should be resolved. If you are still experiencing issues, please send us an email at acasupport at microsoft dot com with you subscription id, container app name and timeframe of missing logs

This issue remains open to this day (Dec 2024) despite being "resolved".


#1336 was opened Nov 2024 and appears to be the same issue. Last update on Nov 7 was:

Thank you guys for raising this, we've added this to the backlog of our public roadmap. If you find yourself in the same situation and need this issue fixed please upvote here for us to get prioritization signal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
investigating currently looking into the issue observability related to observability Provisioning Related to deployment issues, revision provisioning, etc.
Projects
None yet
Development

No branches or pull requests

8 participants