Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unhealthy Healthcheck after Pihole Update #1048

Closed
LacoDev opened this issue Apr 2, 2022 · 29 comments
Closed

Unhealthy Healthcheck after Pihole Update #1048

LacoDev opened this issue Apr 2, 2022 · 29 comments
Labels

Comments

@LacoDev
Copy link

LacoDev commented Apr 2, 2022

Versions

  • Pi-hole: v5.9.1
  • AdminLTE: v5.11
  • FTL: v5.14

Platform

  • OS and version: linux armv7l Raspbian GNU/Linux 10 (buster)
  • Platform: Raspberry Pi

Expected behavior

After updating pihole complete functionality as before with an healthy healthcheck

Actual behavior / bug

time.c:132: Operation not permitted timer.c:619: fatal error: RUNTIME_CHECK(isc_time_now((&now)) == 0) failed Aborted (core dumped)

Pihole with slowly recovering functionality, heat spikes of the cpu, Unknown Status in Menu bar + the above error of an unhealthy healthcheck

Steps to reproduce

Steps to reproduce the behavior:

  1. recreate docker container with latest pihole image
  2. See the health check
  3. Log in the dashboard

Debug Token

  • Debug was sucessful but failed to upload...
@rdwebdesign rdwebdesign transferred this issue from pi-hole/pi-hole Apr 2, 2022
@rdwebdesign
Copy link
Member

Apparently your pi-hole is not running correctly.

Please, could you try pihole/pihole:2022.04.2beta image (no need to add NET_ADMIN capability)?

@martinikolaev
Copy link

Apparently your pi-hole is not running correctly.

Please, could you try pihole/pihole:2022.04.2beta image (no need to add NET_ADMIN capability)?

I have same issue latest release, with beta version its all fixed now! Thank you.

@LacoDev
Copy link
Author

LacoDev commented Apr 2, 2022

Sadly still the same issue… I can’t tell why its happening… When I reverse the version to the docker image pihole/pihole:2022.02.1 everything works fine

@rdwebdesign
Copy link
Member

What docker version are you using?

@leachimus
Copy link

Same problem here. After the autoupdate to the current version, the web interface and the queries go crazy. Also the active status does not seem to work correctly. It appears after activate and reload, unknown.
When I went back to version pihole/pihole:2022.02.1, everything worked fine again.

I use OpenMediaVault 5 with all fixes. And further I have...

Raspbian GNU/Linux 10 (buster) armv7l
Raspberry Pi 4 Model B Rev 1.4
Kernel: 5.10.103-v7l+
Docker version 20.10.14, build a224086

@dschaper
Copy link
Member

dschaper commented Apr 3, 2022

Try 2022.04.2beta

@LacoDev
Copy link
Author

LacoDev commented Apr 3, 2022

What docker version are you using?

20.10.12 (API: 1.41)

@leachimus
Copy link

Try 2022.04.2beta

This is my probem. it don't work. Only pihole:2022.02.1 work fine.

I would have to upgrade my entire system to Bullseye, which I don't have time to do at this time. I hope you can find a working workaround for now.

@AcedExplorerABL
Copy link

If this is of any help to someone like me who does not have a lot of time to understand and fix the (seemingly) broken PiHole docker update.

I am on OMV 5.6.26-1
Docker 5:20.10.143-0debian-buster
Portainer 2.11.1
The only PiHole image that is working for me is 2022.02.1
I also had to change the DNSMASQ_USER to "root" in the container settings. Not 100% sure if this was required. If you get the sudo errors, this should help.

Interestingly, Portainer shows the image version as 2022.02.1 but the PiHole admin page identifies itself with the Docker tag 2022.04.01

@leachimus
Copy link

If this is of any help to someone like me who does not have a lot of time to understand and fix the (seemingly) broken PiHole docker update.

I am on OMV 5.6.26-1 Docker 5:20.10.143-0debian-buster Portainer 2.11.1 The only PiHole image that is working for me is 2022.02.1 I also had to change the DNSMASQ_USER to "root" in the container settings. Not 100% sure if this was required. If you get the sudo errors, this should help.

Interestingly, Portainer shows the image version as 2022.02.1 but the PiHole admin page identifies itself with the Docker tag 2022.04.01

I use the same configuration as you. The thing with the Docker tag is the label. When replacing the container in Portainer, delete all labels and the ENV that are not absolutely necessary. Then the current data will be generated automatically.

@91ajames
Copy link

91ajames commented Apr 3, 2022

hmmmm. no issue here. did run into a snag on my other pc's thou. had issues connecting to any site. Refresh ipconfig and all that. Had the ethernet disconnect and pc reboot twice. Everything here works fine. There was an issue but it has disappeared from messages. Might of freaked out when updating and resizing caused 90% of the sd card. or at least in the logs.

[2022-04-03 00:02:32.965 499M] Resizing "FTL-queries" from 2162688 to (53248 * 44) == 2342912 (/dev/shm: 3.2MB used, 67.1MB total, FTL uses ##MB)
non stop

But its at 61.9%.

Asus Tinkerboard - TinkerOS


Only thing I wish it kept in was the ipv6 connections just for obvious stats. Every update from pihole, I would need to re-enter the optional ipv6 every time for unbound. Its in the docker settings. Even the config script I made wouldn't be auto added in pihole for whatever reason but works for DNS1 & DNS2. Thought DNS3 & DNS4 would work. Other than why add ipv6 in there at all when its not really necessarily? Just saying. Again, stats sake.

@rdwebdesign
Copy link
Member

@AcedExplorerABL

I'm not sure this will work, but since you are using Portainer, my suggestion is to create a new container from zero.
Portainer usually retains old ENV VARs and Labels.

Please, try to remove you container completely, delete the local image and create the container again (pulling a fresh image).

@jbhorner
Copy link

I also had nothing but issues with this version. Gravity wouldn't update, the ability to manage requests ceased working after a period of time, and the web interface was odd. (Status would go to unknown, and the "Activate" option would appear. This was also true for the beta noted above. I reverted to 2022.02.1 and have had no issues.

@throwabird
Copy link

throwabird commented Apr 11, 2022

Can also confirm pretty much the exact behavior @jbhorner described with the latest version aswell as the beta. It also seemed to completely occupy a core of processing power.
I also tested it with a fresh install using https://github.com/pi-hole/docker-pi-hole/blob/master/docker-compose.yml.example resulting in the same behavior.
Reverting to pihole/pihole:2022.02.1 resolved the issue for now.

@PromoFaux
Copy link
Member

PromoFaux commented Apr 11, 2022

latest version aswell as the beta.

They're basically the same image.

Have you checked the upgrade notes on the readme? If your host OS is buster then you'll need to :)

https://github.com/pi-hole/docker-pi-hole#upgrade-notes

I have also added this as an important note to the release notes for 2022.04.2

@jbhorner
Copy link

Have you checked the upgrade notes on the readme? If your host OS is buster then you'll need to :)

Upgrading to Bullseye from Buster can be a pretty heavy lift, as the only "official" way to do so is to start fresh. Ultimately, this is what I had to do...imaged another SD with Bullseye, and then migrated several things from Buster. There are documented methods to do an in-place upgrade, but my attempt at that yielded more problems (and time) than simply doing it the official way.

I don't know what the guidelines are for including something in a readme versus release notes. For something like this, I personally feel it should be in the release notes. I review those between updates for most things. Readme files are, to a very high percentage for me, generally not very useful unless new to an initial install, etc. But, that could just be me.

@PromoFaux
Copy link
Member

I have also added this as an important note to the release notes for 2022.04.2

They have been (as of my previous post) retrospectively added to the release notes. Unfortunately it was not an issue were were 100% aware of at the time of release, still working on the finer details in these issues. You also do not have to do an upgrade from buster to bullseye, rather it's to do with the version libseccomp, which can be updated via backports - though I cannot officially endorse option this as I don't know what effects it may have on your system (should be fine, but I wouldn't want to guarantee it!)

@jbhorner
Copy link

You also do not have to do an upgrade from buster to bullseye, rather it's to do with the version libseccomp, which can be updated via backports

My history of making one-off changes like this without future glitches is poor. On other Linux systems, I've had dependencies break, updates "fix" manual changes, etc. (Warnings should reflect the one-off changes, but they don't always.) It might be a good option for others, though. I've just been bitten before and am trying to avoid more headaches.

It's good to see that the information is there for folks to heed the warning of what might happen with the latest updates. Thank you for the continued efforts!

@Vodalex
Copy link

Vodalex commented May 1, 2022

Hello everyone. I have deleted the volumes and created the container from scratch. Unfortunately with the latest pihole the container ist still unhealthy. Only using 2022.02.1 pihole fixes the problem. Even if I enter root as DNSMASQ_USER.
Will there be a fix for it?

@douglasshepherd
Copy link

Hi all, I have been having this issue for a couple of months now with no resolution. Am I to understand that the primary issue is that OMV's docker version is buster and needs to be upgraded to Bullseye?
I don't do a ton with OMV and working within it is unclear to me. For instance, it doesn't offer the chance to upgrade Docker (as far as I can tell) only to install it within the web GUI-interface.
I am running OMV 5.6.26-1 (Usul) on a Pi4b
system: Linux raspberrypi 5.10.103-v7l+ #1529 SMP Tue Mar 8 12:24:00 GMT 2022 armv7l GNU/Linux
omvextrasorg 5.6.6
Docker: 5:20.10.143-0debian-buster
Portainer 2.11.1
Currently working pihole image: pihole/pihole:2022.02.1

Installing newer pihole images returns unhealthy image and the time on the pihole (when i was able to get the newer versions running a month or so back) operates with incorrect time as it could not contact NTP.

Any thoughts would be appreciated. Thank you all for your work on these projects!

@rdwebdesign
Copy link
Member

What is the docker version output on your system?

@douglasshepherd
Copy link

Client: Docker Engine - Community
Version: 20.10.14
API version: 1.41
Go version: go1.16.15
Git commit: a224086
Built: Thu Mar 24 01:48:44 2022
OS/Arch: linux/arm
Context: default
Experimental: true

Server: Docker Engine - Community
Engine:
Version: 20.10.14
API version: 1.41 (minimum version 1.12)
Go version: go1.16.15
Git commit: 87a90dc
Built: Thu Mar 24 01:46:45 2022
OS/Arch: linux/arm
Experimental: false
containerd:
Version: 1.5.11
GitCommit: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc:
Version: 1.0.3
GitCommit: v1.0.3-0-gf46b6ba
docker-init:
Version: 0.19.0
GitCommit: de40ad0

@rdwebdesign
Copy link
Member

There is a known issue (caused by a recent docker update) when you run a new docker version (>= 20.10.14) on a buster system.

Please read our Upgrade Notes (second item).

Our main suggestion to fix this is to upgrade your system to bullseye.

If you can't upgrade, there is a known workaround (linked at Upgrade Notes - try it at your own risk).

Please, read these 2 comments to understand the workaround:

@edgd1er
Copy link
Contributor

edgd1er commented May 5, 2022

At 30.06.2022 openmediavault 5.x will become EOL. This means no security/bugfix updates will be released anymore. Please upgrade to 6.x to be up-to-date.

If upgrading is possible, the current version 6 has no problem with pihole as omv is based on the latest debian s version.

There are few warning to know before upgrading:
https://www.openmediavault.org/

@douglasshepherd
Copy link

Ok, this makes more sense to me. I didn’t know that OMV 5 was EOL so soon - I will start planning my upgrades. Thank you both!

@Vodalex
Copy link

Vodalex commented May 5, 2022

Thanks everyone. I have executed om-release-upgrade and successfully upgraded from 5 to 6 after this the pihole container ist healthy and starting without any problem. Thank you very much

@github-actions
Copy link

github-actions bot commented Jun 4, 2022

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.

@github-actions github-actions bot added the stale label Jun 4, 2022
@leachimus
Copy link

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.

I have updated to the latest Debian version. It wasn't easy, but it worked in the end and now everything is running again. I believe that sooner or later everyone has to take this step.

@github-actions github-actions bot removed the stale label Jun 4, 2022
@github-actions
Copy link

github-actions bot commented Jul 4, 2022

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests