Homeassistant docker image doesn't include build tools. DHT can't work

[ghost]

@Vanuan commented on Aug 2, 2018, 12:44 AM UTC:

Home Assistant release with the issue:

Docker hub image:
homeassistant/raspberrypi3-homeassistant
Looks like alpine based

Last working Home Assistant release (if known):

N/A

Operating environment (Hass.io/Docker/Windows/etc.):

Raspbian, hassio, docker

Component/platform:

https://www.home-assistant.io/components/sensor.dht/

Description of problem:
It requires a wheel module which can't be build without gcc.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

sensor:
  - platform: dht
    sensor: DHT11
    pin: 4
    monitored_conditions:
      - temperature
      - humidity

Traceback (if applicable):

Command "/usr/local/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-i52jguvt/Adafruit-Python-DHT/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-eivkfo33/install-record.txt --single-version-externally-managed --prefix  --compile --user --prefix=" failed with error code 1 in /tmp/pip-install-i52jguvt/Adafruit-Python-DHT/

Additional information:
Apart of that, Adafruit_Python_DHT==1.3.2 looks broken: adafruit/Adafruit_Python_DHT#102

So 2 issues to fix:

1. add gcc to homeassistant issues (to be able to install wheel addons)
2. change Adafruit_Python_DHT version to 1.3.4

This issue was moved by awarecan from home-assistant/home-assistant#15788.

[ghost]

@Vanuan commented on Aug 2, 2018, 12:47 AM UTC:

Looks like second issue mitigated by #15706

[ghost]

@technicalpickles commented on Aug 3, 2018, 5:15 PM UTC:

Is there any other output in the failure? I don't see anything mentioning gcc specifically.

I'm trying to figure out what tags and publishes these, but there isn't any info on dockerhub 🤔 https://hub.docker.com/r/homeassistant/raspberrypi3-homeassistant/

[ghost]

@technicalpickles commented on Aug 3, 2018, 11:52 PM UTC:

I think this might be a duplicate of #15712 ... no solution there though.

[ghost]

@Vanuan commented on Aug 4, 2018, 7:31 AM UTC:

Well, here's how I figured it was absense of gcc:

1. Run pip install Adafruit_Python_DHT==1.3.2
2. Failure message doesn't mention anything specific
3. Run apk add alpine-sdk
4. Run pip install again
5. See the failure now mentions missing *.c files
6. Run pip install Adafruit_DHT==1.3.4
7. See the package successfully installed

[ghost]

@Vanuan commented on Aug 4, 2018, 8:06 AM UTC:

I've tried to figure out the source Dockerfile for raspberry too. So I went to microbadger service which decompiles "closed source" images (apparently there's no such thing, unless you squash the layers):
https://microbadger.com/images/homeassistant/raspberrypi3-homeassistant
Now I see that gcc is there, so I'm totally confused.
Maybe it has been recently replaced.

[ghost]

@Vanuan commented on Aug 4, 2018, 10:08 AM UTC:

Ok, I think I found what puts images to Docker Hub:
https://github.com/home-assistant/hassio-build/tree/master/autodeploy

According to the source it does the following (looping forever each 10 minutes):

1. Fetches the GitHub releases page through API
2. For each release runs the builder
3. Builder builds docker images for each architecture and machine by cloning dockerfiles from git repo

Now the important part. Where are these dockerfiles for each architecture and machine are stored?

TADA 🎉

That was quite hard to find.

That's the most advanced build system for docker images I've ever seen. It builds images with container builders built in containers by cloning git repo with dockerfiles for each machine.

[ghost]

@Vanuan commented on Aug 4, 2018, 10:18 AM UTC:

So now I can track it down: all these apk add are not actually persisted:

apk add --no-cache --virtual .build-dependencies
...
apk del .build-dependencies

So the final image doesn't have gcc and all other build dependencies in place.
But I still see the DHT installed there:

https://github.com/home-assistant/hassio-homeassistant/blob/da1e02c6ea7a6a22ea1ac374fe6c0ad31497499c/machine/raspberrypi3#L39-L50

But why it does pip install at runtime? This pip package should already exist according to the dockerfile, right? Probably homeassistant plugin system is somehow not aware that pip already installed. I'm not proficient with pip, but maybe it has something to do with virtualenv?

[ghost]

@Vanuan commented on Aug 4, 2018, 11:24 AM UTC:

Maybe pvizeli can shed some light here?

[ghost]

@Vanuan commented on Aug 4, 2018, 11:34 AM UTC:

I think I have an idea. The new Dockerfile change now installs "Adafruit_Python_DHT==1.3.3" while homeassistant dht plugin still requires 1.3.2

So here's a much bigger problem: there's disconnect between hassio images and pip requirement in the source code of dht:
https://github.com/home-assistant/home-assistant/blob/master/homeassistant/components/sensor/dht.py

Well, I don't even know how that would work. This whole system is so complicated. I don't think it's gonna work without extracting all the plugins into their own containers with some kind of rpc (grpc?) to talk with homeassistant core.

[ghost]

@Vanuan commented on Aug 4, 2018, 11:44 AM UTC:

Yeah, it looks like I have latest image of raspberrypi3-homeassistant which has dht 0.3.2 hardcoded in dht.py, but 0.3.3 in the Dockerfile.

Even if we pull dev branch of hass and latest branch of hassio-hass there's mismatch:

Adafruit-DHT==1.3.3 vs Adafruit_Python_DHT=1.3.3

[ghost]

@awarecan commented on Aug 4, 2018, 2:55 PM UTC:

#15706 is not in 0.75.0, and just be included in 0.75.1

[ghost]

@Vanuan commented on Aug 4, 2018, 5:29 PM UTC:

awarecan Even with this change, are you sure it won't require changes on hassio-homeassistant repo?

Have you read anything I wrote about missing gcc?

[ghost]

@technicalpickles commented on Aug 4, 2018, 7:49 PM UTC:

Vanuan nice sleuthing 🎉

One trick I'm using over in #15698 to avoid mentioning a version number in a Dockerfile is to grep & sed it out of requirements.txt in a Dockerfile. I'm not sure if the context the machine builds, if they would have access to that file though 🤔

[ghost]

@awarecan commented on Aug 5, 2018, 12:58 AM UTC:

Vanuan please report hassio issue in its repo https://github.com/home-assistant/hassio/issues

[pvizeli]

With home-assistant 0.75.1 is the problem fix and it install the needed variant

[kaiomatico]

@pvizeli is that a question? Because in my test on hassOS/hassIO with 0.75.1 the dht sensor still does not work.

[rbmaster]

Can confirm the issue as well. I planned on trying to fix it by using the development SSH but I couldn't even get that to work on a Rpi1-ModelB and didn't get the time or will to debug that. I would just change to clean debian if performance was not that much worse on an Rpi1.

[TracKer]

Got this issue on home-assistant v0.80.0

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.