homeassistant: update to 0.87.1 and generic installer and service

[chickenandpork]

HomeAssistant is fairly outdated, and it's versioned as just a date whereas upstream uses version numbers. Additionally, I found that any build re-hit the upstream for a HEAD checkout, so any two builds are not reproducible, and the date and versions are unrelated.

I'm a real fan of caching upstream resources :)

I've used a cross/homeassistant as a way to pull out the upstream and store into the DISTRIB_DIR for re-use. It's an imperfect fit (see Makefile likes 12-23), it would be cleaner to create a mk/spksrc.gitpull.mk or such to make a reusable component for this and to make it a bit cleaner. Looking for feedback on that.

cross/homeassistant stores the upstream githash and version -- this duplication is a few other packages (ie python, python3) which I saw as a source of risk. Looking for feedback.

I've got an additional commit to push that improves HomeAssistant's HomeKit, but trying to keep this to a reusable diff to land and share before going further.

Resolves #3115 #3373 #3573 #3543 #3584
Requires #3396

Checklist

• Build rule all-supported completed successfully
• Package upgrade completed successfully (repeatedly while testing)
• New installation of package completed successfully

-- boilerplate -- (sorry)

I've been granted permission by my employer to contribute to spksrc without risk of IP confusion. I am required to explicitly write that my contribution to this project is solely in my personal capacity, and I am not conveying any rights to any intellectual property of any third parties.

[hogi1200]

Hi @chickenandpork,
first of all I'd like to say a big thank you for your efforts in updating Python to 3.5.5! 👍
I wanted to try home assistant on my nas for quite a while now, but always hesitated, because I would have been forced to stay on an older version of home assistant due to the old version of python provided by synology.

So I was following the issue #3396 closely, and finally managed to install home assistant 0.75.3 on my DS116.
I manually installed the spk from safihire's build (because final version is not yet published on synocommunity repo) and then installed home assistant in a virtual environment by "pip install homeassistant==0.75.3".

Unfortunatelly, I was unable to install latest version (>0.76.0) because of the required package cryptography==2.3.1. Pip was unable to install cffi-1.11.5 because there is apparently no compiler on my nas. There are further problems with home assistant components "cloud" and "discovery" (when activated in the configuration) which want to install additional python packages (e.g. netifaces, warrant==0.6.1).
Sadly, my knowledge of python ends here, so I was hoping you could help me out with the following questions:

1.) It is my understanding that some python packages include c-code which needs to be compiled for the processor architecture. When installing such a package with pip, it will try to download a pre-compiled version (wheel) or if none matches (e.g. for my ARM processor), then it will try to compile with a compiler on the system. Is this correct?

2.) Synology nas do not have a compiler by default, right? Is it possible to install a compiler on the nas? I tried to execute gcc (from https://sourceforge.net/projects/dsgpl/files/DSM%206.1%20Tool%20Chains/Marvell%20Armada%2038x%20Linux%203.10.102/) on the nas but got an "wrong exec format" error. Is this compiler just for cross-compilation?

3.) Is is theoretically possible to include a cross-compiled version of cffi with the python3 delivery (spk) itself?
Similar to the sqlite3 module you added.

4.) I see now, apart from python, you are also making a spk for homeassistant itself. Would such a spk already include a cross-compiled cffi? What about other packages that are dynamically installed by home assistant (depending on configuration)?

5.) What are the advantages of having a homeassistant spk compared to installing via pip? Since homeassistant is updated quite often (2 weeks interval), wouldn't it mean a lot of effort for keeping the spk version up-to-date?

Thanks a lot in advance! :)

[chickenandpork]

hogi1200, send me email at {your nickname}@{my nickname}.com (it'll forward to me). Let's swap some email and you may choose to share what works for you.

[hogi1200]

Thanks for your offer!
I can happily report that I got it working already :)

For anyone interested, this is what I did to get the cross-compiled python packages:
1.) installed development environment in a virtual machine with 64bit debian linux (https://github.com/SynoCommunity/spksrc#setup-development-environment)
2.) built the python3 spk for my architecture according to wiki to verify that the toolchain is working.
3.) modified the "requirements.txt" file of the python3 package and added the package I need to get cross-compiled (e.g. cffi==1.11.5)
4.) build the python3 spk again. the toolchain automatically cross-compiles the added package and stores it as pre-compiled wheel in the subfolder wheelhouse inside the spk file.
5.) extract the wheel file from the spk, copy to nas and install manually with pip. voila!

[ymartin59]

Package requires migration to generic service support: https://github.com/SynoCommunity/spksrc/wiki/Package-Migration
You can inspire from PR #3431 as an example.

[ymartin59]

I find it overkill compare to simply copy-paste of version string used in all packages...

[ymartin59]

Please remove that file

[ymartin59]

Is that really useful?

[ymartin59]

There is easier way to get git hash. See https://github.com/SynoCommunity/spksrc/blob/master/cross/gateone/Makefile as an example.
By the way, I recommend to stick to git tag 0.80.3

[ymartin59]

I have worked on it associated with #3529

[ymartin59]

Some fixes to provide:

• make virtual env writable for components autoinstallation
• set HOME to write pip cache into var/

[ymartin59]

This is pending - I expect to include "bluepy" and "openzwave" support before publishing homeassistant in latest version.

[ymartin59]

It now builds and runs over Python 3.6. Will now add openZwave...

[ymartin59]

@chickenandpork Hello. I need your help... x64 builds but other cross-compiled targets fail with:

     File "src/build_bcrypt.py", line 21, in <module>
        ffi = FFI()
      File "/tmp/pip-build-env-ot3drld4/overlay/lib/python3.6/site-packages/cffi/api.py", line 48, in __init__
        import _cffi_backend as backend
    ImportError: /tmp/pip-build-env-ot3drld4/overlay/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory
    
    ----------------------------------------
Command "/spksrc/native/python3/work-native/install/usr/local/bin/python3 /spksrc/native/python3/work-native/install/usr/local/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp4q9sryym" failed with error code 1 in /spksrc/spk/homeassistant/work-aarch64-6.1/wheelbuild/bcrypt
You are using pip version 19.0.2, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
../../mk/spksrc.wheel.mk:61: recipe for target 'build_wheel_target' failed

And I wonder why pip does not find/load library located at spk/homeassistant/work-aarch64-6.1/install/var/packages/homeassistant/target/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so? What/where is the site-packages location search path?

[chickenandpork]

I can look in around 9 hrs :-/

…

On Feb 26, 2019, at 09:50, Yves Martin ***@***.***> wrote: @chickenandpork Hello. I need your help... x64 builds but other cross-compiled targets fail with: File "src/build_bcrypt.py", line 21, in <module> ffi = FFI() File "/tmp/pip-build-env-ot3drld4/overlay/lib/python3.6/site-packages/cffi/api.py", line 48, in __init__ import _cffi_backend as backend ImportError: /tmp/pip-build-env-ot3drld4/overlay/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory ---------------------------------------- Command "/spksrc/native/python3/work-native/install/usr/local/bin/python3 /spksrc/native/python3/work-native/install/usr/local/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp4q9sryym" failed with error code 1 in /spksrc/spk/homeassistant/work-aarch64-6.1/wheelbuild/bcrypt You are using pip version 19.0.2, however version 19.0.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command. ../../mk/spksrc.wheel.mk:61: recipe for target 'build_wheel_target' failed And I wonder why pip does not find/load library located at spk/homeassistant/work-aarch64-6.1/install/var/packages/homeassistant/target/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so? What/where is the site-packages location search path? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[ymartin59]

@chickenandpork Hello Allan. Sorry I missed your reply. I have understood and fixed "requirements.txt" to publish. It is OK but only local at the moment... I am working on openzwave and will create PR then.

[chickenandpork]

It’s ok, my build server was randomly synflooding, so something went bad. Nuked it. Rebuilding it. Config-as-code anyone? :-/

…

On Feb 28, 2019, at 09:13, Yves Martin ***@***.***> wrote: @chickenandpork Hello Allan. Sorry I missed your reply. I have understood and fixed "requirements.txt" to publish. It is OK but only local at the moment... I am working on openzwave and will create PR then. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.