tail 800 lines journald: incomplete output

[edsantiago]

Two flakes in one week:

# podman-remote [options] logs --tail 800 <CID>
...
...makes it only to 400-something, or perhaps even just zero lines
? Failure [3.295 seconds]
         Podman logs
         /var/tmp/go/src/github.com[/containers/podman/test/e2e/logs_test.go:26](https://github.com/containers/podman/blob/e874fc105c0fcfa291e256071be8b48dc4f347f8/test/e2e/logs_test.go#L26)
           tail 800 lines: journald [It]
           /var/tmp/go/src/github.com[/containers/podman/test/e2e/logs_test.go:116](https://github.com/containers/podman/blob/e874fc105c0fcfa291e256071be8b48dc4f347f8/test/e2e/logs_test.go#L116)
         
           Expected [long string] to have length 800

Podman logs [It] tail 800 lines: journald

• fedora-35 : int remote fedora-35 root host [remote]
  • PR shell completion: use more constants in the code #14272
    • 05-19 09:16
    • 05-19 09:16
    • 05-19 09:16
• fedora-36 : int remote fedora-36 root host [remote]
  • PR DO NOT MERGE: buildah vendor treadmill #13808
    • 05-25 09:10
    • 05-25 09:10
    • 05-25 09:10

[Luap99]

I think I saw this on many PRs recently.
@edsantiago Can you post the recent failures to check if it is only on remote?

[edsantiago]

Two more on 05-25, none since then (but then again, long weekend with very very few PRs)

Podman logs [It] tail 800 lines: journald

• fedora-35 : int podman fedora-35 root host
  • PR Remove unused archive flag from diff commands #14364
    • 05-25 12:55
• fedora-35 : int remote fedora-35 root host [remote]
  • PR Support remote deadlock errors in rm #14366
    • 05-25 15:27
    • 05-25 15:27
    • 05-25 15:27

[Luap99]

Yeah I saw this fail on a couple of my unmerged PRs, https://storage.googleapis.com/cirrus-ci-6707778565701632-fcae48/artifacts/containers/podman/4912015679422464/html/int-remote-fedora-35-root-host.log.html

[edsantiago]

Ah yes, I should've mentioned my flake logger only captures merged PRs. Identifying flakes on in-progress PRs was an exercise I deferred "for later" (and has not happened yet).

[edsantiago]

Anyhow: the logs above show one instance on non-remote, and curiously it was a one-time-only: the remote failures are all three-time failures.

[Luap99]

Just any idea but the test never actually waits for the container to complete. It call podman run -d ... followed by podman logs.
I think it is likely that the container/conmon did not write all the output before we call logs.

Let me open a PR with an extra podman wait then we will see if that worked or not.

[vrothberg]

That sounds plausible. Nice catch, @Luap99.