initial workflow tests

.github/workflows/integration-k3d.yaml → .github/workflows/integration.yaml

@@ -1,7 +1,7 @@
 # This workflow will install Python dependencies, run tests and lint with a single version of Python
 # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
 
-name: K3D Integration Tests
+name: Integration Tests
 
 on:
   push:
@@ -13,10 +13,13 @@ permissions:
   contents: read
 
 jobs:
-  build:
-    name: K3D Integration Tests
+  test:
+    name: Integration Tests
     runs-on: ubuntu-latest
-
+    strategy:
+      fail-fast: false
+      matrix:
+        deploymode: ["local", "k3d", "addon"]
     steps:
       - uses: actions/checkout@v4
       - name: Set up Python 3.11
@@ -30,18 +33,36 @@ jobs:
       - name: Lint with Ruff
         run: |
           ruff check .
+          ruff check --select I .
       - name: Create cluster
+        if: startsWith(matrix.deploymode, 'k3') == true
         shell: bash
         run: bash scripts/setup_k3d.sh
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           SKIP_CREATION: true
           SKIP_REGISTRY_CREATION: true
           SKIP_READINESS: true
+      - name: install playwright deps
+        run: |
+          playwright install --with-deps chromium
+      - name: Copy content to addon context
+        if: startsWith(matrix.deploymode, 'addon') == true
+        id: copy
+        run: bash scripts/copy_content_to_addon_context.sh
       - name: Install Dapr
         shell: bash
         run: |
           wget -q https://raw.githubusercontent.com/dapr/cli/master/install/install.sh -O - | /bin/bash
+      - name: initialize dapr
+        run: |
+          dapr init --slim
       - name: Test with pytest
         run: |
-          pytest -v -s -rA -c tests/pytest.ini --deploy-mode=k3d --replay-mode=replay
+          pytest -v -s -rA -c tests/pytest.ini --deploy-mode=${{ matrix.deploymode }} --replay-mode=replay
+      - name: Upload debug playwright screenshots
+        if: failure()
+        uses: actions/upload-artifact@v4
+        with:
+          name: playwright-screenshots-${{ matrix.deploymode }}
+          path: "**/fail-*.png"

README.md

@@ -1,6 +1,6 @@
-# Home ~~Automation~~ Intelligence
+# Mindctrl: Home ~~Automation~~ Intelligence :thought_balloon:
 
-Manage your home automation LLM prompts, available LLMs and evaluate changes with MLflow
+An Intelligence hosting platform for automating your life
 
 [![Addon Builder](https://github.com/akshaya-a/mindctrl/actions/workflows/addon-builder.yaml/badge.svg)](https://github.com/akshaya-a/mindctrl/actions/workflows/addon-builder.yaml)
 [![Addon Integration](https://github.com/akshaya-a/mindctrl/actions/workflows/integration-addon.yaml/badge.svg)](https://github.com/akshaya-a/mindctrl/actions/workflows/integration-addon.yaml)
@@ -9,7 +9,59 @@ Manage your home automation LLM prompts, available LLMs and evaluate changes wit
 
 ---
 
-![promptlab](assets/mlflowhass-promptlab.png)
+![promptlab](assets/mctrl_arch_v1.png "Mindctrl Architecture")
+_Mindctrl architecture, courtesy of the amazing [excalidraw](https://excalidraw.com/)_
+
+A more disciplined incorporation of AI into your home automation setup. This project aims to provide a platform for managing and evaluating conversational prompts for Large Language Models (LLMs) alongside traditional ML techniques in Home Assistant or other home automation platforms (bring your own events and triggers).
+
+## Features
+
+- [x] Manage deployed models or prompts with a versioned registry
+- [x] Evaluate prompts with a simple UI
+- [ ] Customize your agents
+
+## Platform
+
+- [ ] Scale from [a single device](#home-assistant-addon) :computer: ...
+  - [x] to [a selfhosted cluster](#kubernetes) :globe_with_meridians: ...
+  - [ ] to [the cloud](#cloud) :cloud:
+- [ ] Deploy onto
+  - [x] amd64 or ...
+  - [x] aarch64 architectures, and feel free to contribute support for ...
+  - [ ] the rest
+- [ ] Developed on
+  - [x] Linux, but it's mostly containerized so it should work on ...
+  - [ ] Windows or ...
+  - [ ] MacOS with little effort
+
+## Goals / OKRs
+
+1. [Increase Spousal Approval Rating](#reason-about-state-changes) (SAR) by 10% by the end of the quarter
+2. [Gain a better understanding](#why) of true AI/ML application deployment + tooling required for successful end-to-end scenarios by building from first principles (no -> light -> heavy AI-focused frameworks)
+3. Incorporate latest GenAI techniques to evaluate utility
+   - Memory, tool-calling and RAG have lots of room to grow
+4. Justify the purchase of a new GPU
+
+## Getting Started
+
+> [!WARNING]
+> This project is in early development :see_no_evil: and is not yet ready for production use. Or any use, really. But if you're feeling adventurous, read on!
+
+First, decide how you want to integrate Mindctrl into your home automation setup. You can choose from the following options:
+
+### Home Assistant Addon
+
+1. [Install the Mindctrl Home Assistant Addon](https://www.home-assistant.io/common-tasks/os#installing-third-party-add-ons)
+2. Configure the addon with your database and (optional) broker details
+
+### Kubernetes
+
+- [ ] TODO: [TuringPi + K3S instructions](https://docs.turingpi.com/docs/how-to-plan-kubernetes-installation) more scoped to mindctrl. In the interim, intrepid explorers can look at what [the tests do](tests/utils/cluster.py)
+- [ ] TODO: Convert the kubectl commands to helm charts
+
+### Cloud
+
+- [ ] TODO: Azure Container Apps instructions
 
 ## Why?
 
@@ -19,6 +71,8 @@ Manage your home automation LLM prompts, available LLMs and evaluate changes wit
 
 ### Better manage conversational prompts
 
+![promptlab](assets/mlflowhass-promptlab.png)
+
 Home Assistant has a convenient prompt template tool to generate prompts for LLMs. However, it's not easy to manage these prompts. I can't tell if my new prompt is better than the last one, change tracking is not easy, and live editing means switching between the developer tools and the prompt template tool. There's a better way! Enter MLflow with its new PromptLab UI and integrated evaluation tools + tracking.
 
 ### Reason about state changes
@@ -33,19 +87,7 @@ LLMs can be used to reason about state changes more naturally. Can we send the s
 
 - (motion, time of day, device usage) -> "is AK asleep?"
 
-### Wait what about Langchain?
-
-You heard about Langchain but not this whole MLflow business - what's up with that? [Read more about how all this fits together!](/docs/prompt-techniques.md)
-
-## How?
-
-### Getting Started
-
-1. Install the MLflow Gateway
-2. Install the MLflow Tracking Service
-3. Install the MLflow Home Integration
-
-## What?
+## What are you talking about?
 
 - LLM: Large Language Model, a model that can generate text based on a prompt
 - [MLflow](https://mlflow.org/): An open source platform for the machine learning lifecycle

addons/mindctrl/rootfs/usr/bin/run_dashboard.sh

@@ -12,5 +12,10 @@ if [[ -n "$ingress_entry" ]]; then
     bashio::log.info "running dashboard with prefix $SERVER_BASE_HREF"
 fi
 
+bashio::log.info $HOME
+ls -la $HOME/.dapr/bin
+bashio::log.info "Starting dapr placement server"
+$HOME/.dapr/bin/placement
 
+bashio::log.info "Starting dapr dashboard"
 dapr dashboard -a 0.0.0.0 -p 9999

addons/mindctrl/rootfs/usr/bin/run_traefik.sh

@@ -22,7 +22,7 @@ export HASS_INGRESS_ENTRY="${ingress_entry}"
 bashio::log.info "Starting traefik..."
 /traefik version
 # TODO: until this is unified, keep in sync with testcontainer
-/traefik --accesslog=true --accesslog.format=json --log.level=DEBUG --api=true --api.dashboard=true --api.insecure=true \
+/traefik --accesslog=true --accesslog.format=json --log.level=DEBUG --api=true \
     --entrypoints.http.address=':80' \
-    --ping=true \
+    --ping=true --entryPoints.ping.address=:8082 --ping.entryPoint=ping \
     --providers.file.filename /.context/services/ingress/traefik-config.yaml

custom_components/mindctrl/__init__.py

@@ -3,8 +3,10 @@
 
 from __future__ import annotations
 
+import asyncio
+
 from homeassistant.components import conversation as haconversation
-from homeassistant.components.hassio import AddonManager, AddonError, AddonState
+from homeassistant.components.hassio import AddonError, AddonManager, AddonState
 from homeassistant.config_entries import ConfigEntry
 from homeassistant.core import HomeAssistant, callback
 from homeassistant.exceptions import (
@@ -13,15 +15,10 @@
 from homeassistant.helpers import config_validation as cv
 from homeassistant.helpers.typing import ConfigType
 
-import asyncio
-
 from .addon import get_addon_manager
-
-from .const import ADDON_NAME, CONF_URL, CONF_USE_ADDON, DOMAIN, _LOGGER
-
-from .services import MindctrlClient, async_register_services
+from .const import _LOGGER, ADDON_NAME, CONF_URL, CONF_USE_ADDON, DOMAIN
 from .conversation import MLflowAgent
-
+from .services import MindctrlClient, async_register_services
 
 CONNECT_TIMEOUT = 10
 

custom_components/mindctrl/addon.py

@@ -8,7 +8,7 @@
 from homeassistant.core import HomeAssistant, callback
 from homeassistant.helpers.singleton import singleton
 
-from .const import ADDON_SLUG, DOMAIN, _LOGGER, ADDON_NAME
+from .const import _LOGGER, ADDON_NAME, ADDON_SLUG, DOMAIN
 
 DATA_ADDON_MANAGER = f"{DOMAIN}_addon_manager"
 

custom_components/mindctrl/config_flow.py

@@ -1,29 +1,29 @@
+import asyncio
+import uuid
 from typing import Any
+
+import aiohttp
+import voluptuous as vol
 from homeassistant import config_entries, exceptions
-from homeassistant.core import HomeAssistant, callback
 from homeassistant.const import CONF_URL
-from homeassistant.helpers.aiohttp_client import async_get_clientsession
+from homeassistant.core import HomeAssistant, callback
 
 # from homeassistant.components.zeroconf import ZeroconfServiceInfo
 from homeassistant.data_entry_flow import (
     FlowResult,
 )
-import voluptuous as vol
-import asyncio
-import aiohttp
-import uuid
+from homeassistant.helpers.aiohttp_client import async_get_clientsession
 
 from .const import (
+    _LOGGER,
     ADDON_HOST,
     ADDON_PORT,
     CONF_ADDON_LOG_LEVEL,
     CONF_INTEGRATION_CREATED_ADDON,
-    DOMAIN,
-    _LOGGER,
     CONF_USE_ADDON,
+    DOMAIN,
 )
 
-
 DEFAULT_URL = f"http://{ADDON_HOST}:{ADDON_PORT}"
 TITLE = "mindctrl"
 

custom_components/mindctrl/conversation.py

@@ -1,17 +1,17 @@
+from typing import Literal
+
+import mlflow
 from homeassistant.components import conversation
+from homeassistant.config_entries import ConfigEntry
 from homeassistant.const import MATCH_ALL
 from homeassistant.core import (
     HomeAssistant,
 )
-from homeassistant.config_entries import ConfigEntry
-from typing import Literal
-from homeassistant.util import ulid
-from homeassistant.helpers import intent, template
 from homeassistant.exceptions import (
     TemplateError,
 )
-import mlflow
-
+from homeassistant.helpers import intent, template
+from homeassistant.util import ulid
 
 from .const import _LOGGER
 

custom_components/mindctrl/entity.py

@@ -1,13 +1,14 @@
 """AdGuard Home base entity."""
 
 from __future__ import annotations
+
 from abc import ABC, abstractmethod
 
 from homeassistant.config_entries import SOURCE_HASSIO, ConfigEntry
 from homeassistant.helpers.device_registry import DeviceEntryType, DeviceInfo
 from homeassistant.helpers.entity import Entity
 
-from .const import ADDON_SLUG, DOMAIN, _LOGGER
+from .const import _LOGGER, ADDON_SLUG, DOMAIN
 from .services import MindctrlClient