Addons autoinstaller and env-based filter

Dockerfile

@@ -53,40 +53,32 @@ ONBUILD USER odoo
 ARG PYTHONOPTIMIZE=2
 ARG WKHTMLTOPDF_VERSION=0.12.4
 ARG WKHTMLTOPDF_CHECKSUM='049b2cdec9a8254f0ef8ac273afaf54f7e25459a273e27189591edc7d7cf29db'
-ENV OPENERP_SERVER=/opt/odoo/auto/odoo.conf \
-    UNACCENT=true \
-    # Git and git-aggregator
-    GIT_AUTHOR_NAME=docker-odoo \
-    EMAIL=https://hub.docker.com/r/tecnativa/odoo \
-    DEPTH_DEFAULT=1 \
+ENV DEPTH_DEFAULT=1 \
     DEPTH_MERGE=100 \
-    # Postgres
-    WAIT_DB=true \
-    # PuDB debugger
+    EMAIL=https://hub.docker.com/r/tecnativa/odoo \
+    GIT_AUTHOR_NAME=docker-odoo \
+    LC_ALL=C.UTF-8 \
+    OPENERP_SERVER=/opt/odoo/auto/odoo.conf \
+    PATH="~/.local/bin:$PATH" \
     PUDB_RDB_HOST=0.0.0.0 \
     PUDB_RDB_PORT=6899 \
-    # WDB debugger
+    UNACCENT=true \
+    WAIT_DB=true \
     WDB_NO_BROWSER_AUTO_OPEN=True \
     WDB_SOCKET_SERVER=wdb \
     WDB_WEB_PORT=1984 \
-    WDB_WEB_SERVER=localhost \
-    # Other
-    LC_ALL=C.UTF-8 \
-    PATH="~/.local/bin:$PATH"
+    WDB_WEB_SERVER=localhost
 
 # Other requirements and recommendations to run Odoo
 # See https://github.com/$ODOO_SOURCE/blob/$ODOO_VERSION/debian/control
 RUN apt-get update \
     && apt-get -y upgrade \
     && apt-get install -y --no-install-recommends \
-        # Odoo direct dependencies
         python ruby-compass \
-        # Odoo indirect dependencies
         fontconfig libfreetype6 libxml2 libxslt1.1 libjpeg62-turbo zlib1g \
         libfreetype6 liblcms2-2 libopenjpeg5 libtiff5 tk tcl libpq5 \
         libldap-2.4-2 libsasl2-2 libx11-6 libxext6 libxrender1 \
         locales-all zlibc \
-        # This image's facilities
         bzip2 ca-certificates curl gettext-base git nano npm \
         openssh-client telnet xz-utils \
     && curl https://bootstrap.pypa.io/get-pip.py | python /dev/stdin --no-cache-dir \

README.md

@@ -196,58 +196,65 @@ web:
     - web_responsive
 ```
 
-You can bundle [several YAML documents][] if you want to logically group your
-addons and some repos are repeated among groups, by separating each document
-with `---`:
+Advanced features:
 
-```yaml
-# Spanish Localization
-l10n-spain:
-    - l10n_es
-server-tools:
-    - date_range
----
-# SEO tools
-website:
-    - website_blog_excertp_img
-server-tools: # Here we repeat server-tools, but no problem because it's a
-              # different document
-    - html_image_url_extractor
-    - html_text
-```
-
-You can add all modules in a repo by using a `*`:
-
-```yaml
-website:
-    - "*"
-```
+- You can bundle [several YAML documents][] if you want to logically group your
+  addons and some repos are repeated among groups, by separating each document
+  with `---`.
 
-Important notes:
+- Addons under `private` and `odoo/addons` are linked automatically unless you
+  specify them.
 
-- Do not add repos for the required [`odoo`][] and [`private`][] directories;
-  those are automatic.
+- You can use `ONLY` to supply a dictionary of environment variables and a
+  list of possible values to enable that document in the matching environments.
 
-- Only addons here are symlinked in [`/opt/odoo/auto/addons`][]. Addons from
-  the required directories above are added directly in the [`odoo.conf`][]
-  file.
-
-- This means that if you have an addon with the same name in any of [`odoo`][],
-  [`private`][] or [`/opt/odoo/auto/addons`][] directories, this will be the
-  importance order in which they will be loaded (from most to least important):
+- If an addon is found in several places at the same time, it will get linked
+  according to this priority table:
 
   1. Addons in [`private`][].
-  2. Custom addons listed in [`addons.yaml`][].
-  3. Core Odoo addons from [`./odoo/addons`][`odoo`].
+  2. Addons in other repositories (in case one is matched in several, it will
+     be random, BEWARE!). Better have no duplicated names if possible.
+  3. Core Odoo addons from [`odoo/addons`][`odoo`].
 
-  Although it is better to simply have no name conflicts if possible.
+- If an addon is specified but not available at runtime, it will fail silently.
 
-- Any other addon not listed here will not be usable in Odoo (and will be
-  removed by default, to keep the resulting image thin).
+- You can use any wildcards supported by [Python's glob module][glob].
 
-- If you list 2 addons with the same name, you'll get a build error.
+This example shows these advanced features:
 
-- If you use the wildcard (`*`), it must be encapsulated in quotes.
+```yaml
+# Spanish Localization
+l10n-spain:
+  - l10n_es # Overrides built-in l10n_es under odoo/addons
+server-tools:
+  - "*date*" # All modules that contain "date" in their name
+  - module_auto_update # Makes `autoupdate` script actually autoupdate addons
+web:
+  - "*" # All web addons
+---
+# Different YAML document to separate SEO Tools
+website:
+  - website_blog_excertp_img
+server-tools: # Here we repeat server-tools, but no problem because it's a
+              # different document
+  - html_image_url_extractor
+  - html_text
+---
+# Enable demo ribbon only for devel and test environments
+ONLY:
+  PGDATABASE: # This environment variable must exist and be in the list
+    - devel
+    - test
+server-tools:
+  - web_environment_ribbon
+---
+# Enable special authentication methods only in production environment
+ONLY:
+  PGDATABASE:
+    - prod
+server-tools:
+  - auth_*
+```
 
 ##### `/opt/odoo/custom/dependencies/*.txt`
 
@@ -310,6 +317,15 @@ now keep this in mind:
 
 ## Bundled tools
 
+### `addons-install`
+
+A handy CLI tool to automate addon management based on the current environment.
+It allows you to install, update, test and/or list private, extra and/or core
+addons available to current container, based on current [`addons.yaml`][]
+configuration.
+
+Call `addons-install --help` for usage instructions.
+
 ### [`nano`][]
 
 The CLI text editor we all know, just in case you need to inspect some bug in
@@ -915,6 +931,7 @@ scaffolding versions is preserved.
 [builds]: https://hub.docker.com/r/tecnativa/odoo-base/builds/
 [docker-socket-proxy]: https://hub.docker.com/r/tecnativa/docker-socket-proxy/
 [Fish]: http://fishshell.com/
+[glob]: https://docs.python.org/3/library/glob.html
 [Let's Encrypt]: https://letsencrypt.org/
 [OCA]: https://odoo-community.org/
 [OCB]: https://github.com/OCA/OCB

bin/addons-install

@@ -0,0 +1,62 @@
+#!/usr/bin/env python
+from __future__ import print_function
+import sys
+from argparse import ArgumentParser
+from subprocess import check_call
+from odoobaselib import addons_config, CORE, PRIVATE, logging
+
+# Define CLI options
+parser = ArgumentParser(description="Install addons in current environment")
+parser.add_argument(
+    "-c", "--core", action="store_true",
+    help="Install all Odoo core addons")
+parser.add_argument(
+    "-e", "--extra", action="store_true",
+    help="Install all extra addons")
+parser.add_argument(
+    "-l", "--list", action="store_true",
+    help="Only list addons instead of installing them")
+parser.add_argument(
+    "-p", "--private", action="store_true",
+    help="Install all private addons")
+parser.add_argument(
+    "-s", "--separator", type=str, default=",",
+    help="String that separates addons when using --list")
+parser.add_argument(
+    "-t", "--test", action="store_true",
+    help="Run unit tests for these addons, usually combined with --update")
+parser.add_argument(
+    "-u", "--update", action="store_true",
+    help="Update addons instead of installing them")
+
+# Check no CLI conflicts
+args = parser.parse_args()
+if not (args.private or args.core or args.extra):
+    parser.error("You have to choose an option at least")
+if args.list and args.update:
+    parser.error("Cannot --list and --update together")
+if args.separator != "," and not args.list:
+    parser.error("Cannot use --separator without --list")
+
+# Generate the matching addons set
+addons = set()
+for addon, repo in addons_config():
+    core_ok = args.core and repo == CORE
+    extra_ok = args.extra and repo not in {CORE, PRIVATE}
+    private_ok = args.private and repo == PRIVATE
+    if private_ok or core_ok or extra_ok:
+        addons.add(addon)
+
+# Do the required action
+if not addons:
+    sys.exit("No addons found")
+addons = args.separator.join(sorted(addons))
+if args.list:
+    print(addons)
+else:
+    command = ["odoo", "--stop-after-init"]
+    if args.test:
+        command += ["--test-enable", "--workers", "0"]
+    command += ["--update" if args.update else "--init", addons]
+    logging.info("Executing %s", " ".join(command))
+    check_call(command)

build.d/400-clean

@@ -5,40 +5,37 @@ import os
 import shutil
 import sys
 
-import odoobaselib
+from odoobaselib import addons_config, CORE, CLEAN, PRIVATE, SRC_DIR
 
-if not odoobaselib.CLEAN:
+if not CLEAN:
     logging.warning("Not cleaning garbage")
     sys.exit()
 
-config = odoobaselib.addons_config()
-for directory in os.listdir(odoobaselib.SRC_DIR):
+addons = set(addons_config())
+repos = {addon[1] for addon in addons} | {CORE, PRIVATE}
+for directory in os.listdir(SRC_DIR):
     # Special directories must be preserved
-    if directory in {"private", "odoo"}:
-        continue
+    if directory == "odoo":
+        directory = CORE
 
     # Skip regular files
-    full = os.path.join(odoobaselib.SRC_DIR, directory)
+    full = os.path.join(SRC_DIR, directory)
     if not os.path.isdir(full):
         continue
 
     # Remove directories not listed in addons.yaml
-    if directory not in config:
+    if directory not in repos:
         logging.info("Removing directory %s", full)
         shutil.rmtree(full)
         continue
 
-    # Skip wildcards
-    if '*' in config[directory]:
-        continue
-
     # Traverse addons
     for subdirectory in os.listdir(full):
         subfull = os.path.join(full, subdirectory)
         # Skip regular files
         if not os.path.isdir(subfull):
             continue
         # Remove addon if not used
-        if subdirectory not in config[directory]:
+        if (subdirectory, directory) not in addons:
             logging.info("Removing subdirectory %s", subfull)
             shutil.rmtree(subfull)

conf.d/10-addons.conf

@@ -1,3 +1,3 @@
 [options]
 ; Addons in priority order: private, then other repos, then base Odoo
-addons_path = /opt/odoo/custom/src/private,/opt/odoo/auto/addons,/opt/odoo/custom/src/odoo/addons
+addons_path = /opt/odoo/auto/addons

entrypoint.d/40-addons-link

@@ -2,39 +2,26 @@
 # -*- coding: utf-8 -*-
 import logging
 import os
+from glob import iglob
 
-import odoobaselib
+from odoobaselib import (
+    addons_config,
+    ADDONS_DIR,
+    ADDONS_YAML,
+    SRC_DIR,
+)
 
 logging.info(
     "Linking all addons from %s in %s",
-    odoobaselib.ADDONS_YAML,
-    odoobaselib.ADDONS_DIR)
+    ADDONS_YAML,
+    ADDONS_DIR)
 
-for repo, addons in odoobaselib.addons_config().items():
-    while addons:
-        addon = addons.pop()
-        if addon == '*':
-            addon_dir = os.path.join(odoobaselib.SRC_DIR, repo)
-            try:
-                modules = next(os.walk(addon_dir))[1]
-            except StopIteration:
-                continue
-            for module in modules:
-                if not module.startswith('.') and module != 'setup':
-                    addons.append(module)
-            continue
-        src = os.path.relpath(
-            os.path.join(odoobaselib.SRC_DIR, repo, addon),
-            odoobaselib.ADDONS_DIR)
-        dst = os.path.join(odoobaselib.ADDONS_DIR, addon)
-        try:
-            ok = os.readlink(dst) == src
-        except OSError:
-            ok = False
-        if not ok:
-            try:
-                os.remove(dst)
-            except OSError:
-                pass
-            os.symlink(src, dst)
-            logging.debug("Linked %s in %s", src, dst)
+# Remove all links in addons dir
+for link in iglob(os.path.join(ADDONS_DIR, "*")):
+    os.remove(link)
+# Add new links
+for addon, repo in addons_config():
+    src = os.path.relpath(os.path.join(SRC_DIR, repo, addon), ADDONS_DIR)
+    dst = os.path.join(ADDONS_DIR, addon)
+    os.symlink(src, dst)
+    logging.debug("Linked %s in %s", src, dst)