Dirty metadata buffer

[andsens]

Somethings amiss on bootup. dmesg says:

JBD: Spotted dirty metadata buffer (dev = xvda1, blocknr = 1). There's a risk of filesystem corruption in case of system crash.

[yanfalies]

I had this problem. I created a plugin which added mount options for ext file systems. barrier=0 fixes it.

[andsens]

Thing is, I already do that.

[mioi]

Hi there, I just started encountering this same problem yesterday. Launching a new ec2 instance with a freshly-built AMI works fine (except for the "JBD: Spotted dirty metadata buffer" in dmesg) but upon reboot, it fails to come up. Before rebooting it though, executing this seems to fix it (ignoring the warnings):

fsck -v /dev/xvda1

But still, have you come up with what is causing this? Thanks.

[andsens]

That's good info! Hm, I wonder what needs repairing though. Does fsck output any details on this?
Maybe tune2fs is messing with something in 12-format-volume.

[andsens]

[...] but upon reboot, it fails to come up. Before rebooting it though, executing this seems to fix it

On second I am not exactly sure what you mean. Does it only appear on first boot or everytime?

[yanfalies]

I've seen it happen too, but after a reboot. Anecdotally I've heard this just happens on Amazon. It probably wouldn't hurt to run e2fsck -f on the newly formatted partition after unmounting.

[mioi]

So i commented out the case block at the bottom of tasks/50-ec2-scripts where it wants to install the change-root-uuid init.d script, and i don't see this dirty metadata buffer warning anymore.

is there some reason you want to change the uuid of the root partition to a random uuid?

[andsens]

Yes. Hm, so this might actually be tune2fs itself saying this? I guess changing the UUID of the root volume on boot is not a healthy thing to do, but I'm not sure how to go about that case otherwise.

[mioi]

maybe instead of changing the uuid of the root volume at first bootup, you can use the 'nouuid' flag with your mount command when attaching ebs volumes with duplicate uuid's during troubleshooting. i am not sure if/how this work as i have never done it.

[andsens]

Yes of course I can do that. I am thinking of all the poor folks out there who will have to google their way around a wonky error message. That's why I made this script in the first place, you really have to know unix before thinking of doing this.

[yanfalies]

How about you come at this problem side ways. If the JBD code is really the problem. Disable the journal on ext4 after you umount the final volume and before snapshotting using:

tune2fs -O ^has_journal

On first boot,

1. run the UUID randomizer task
2. renable journal option
3. force fsck on next boot.
4. then reboot.

[andsens]

Disable the journal on ext4 after you umount the final volume

This is certainly a solution, but it feels very hack-y and could mess with the stability of the image. If there is no other way, I would rather disable the script and add a hint in the yet-to-be-written documentation.