IL-421 classification user journey with Docker, Kubernetes setup

* feat: WIP Classification runs as container on localhost - Enhance use case Jupyter Notebook to include guidance for

TASK: IL-421

* WIP: Add classificationservice FastAPI setup and Dockerfile
TASK: IL-421

* IL-421 expose docker port to localhost

* IL-421 kubernetes yaml

* IL-421 Kubernetes notes

---------

Co-authored-by: Florian Schepers <[email protected]>
Co-authored-by: FelixFehse <[email protected]>

src/examples/issue_classification_user_journey.ipynb

@@ -43,6 +43,8 @@
    "outputs": [],
    "source": [
     "### Helper methods ###\n",
+    "\n",
+    "\n",
     "def display_histograms(\n",
     "    expected_labels_histogram: dict[str, int],\n",
     "    predicted_labels_histogram: dict[str, int],\n",
@@ -727,6 +729,68 @@
     "\n",
     "Feel free to further play around and improve our classification example. "
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Classification Service\n",
+    "\n",
+    "Outline\n",
+    "- Setup FastAPI Service\n",
+    "- Setup Docker\n",
+    "- Setup Kubernetes\n",
+    "- Setup Pulumi\n",
+    "- Check that it is working\n",
+    "\n",
+    "### Environment variables\n",
+    "TODO: Describe how to set environment variables \n",
+    "\n",
+    "### FastAPI service\n",
+    "TODO: Describe FastAPI setup for `ClassificationService.py` \n",
+    "\n",
+    "In order to start the Classification service open a new terminal and execute the command\n",
+    "```shell\n",
+    "hypercorn .\\src\\examples\\issue_classification_user_journey\\ClassificationService:app\n",
+    "```\n",
+    "\n",
+    "### Docker\n",
+    "\n",
+    "__Important__: For some reason, git adds a null terminator to the `requirements.txt` file. This crashed the docker build step. You might have to manually delete the null terminator at the end of the file.\n",
+    "\n",
+    "Write dockerfile that will create a new docker container with the FastAPI program.\n",
+    "We assume, your github token for the Intelligence Layer is exported to the `GITHUB_TOKEN` environmental variable. \n",
+    "Build the docker container with\n",
+    "```shell\n",
+    "docker build . -t classification-service:local -f Dockerfile --secret id=GITHUB_TOKEN\n",
+    "```\n",
+    "Run the docker container and \n",
+    "```shell\n",
+    "docker run -p 8000:80 classification-service:local;2D                                                           \n",
+    "```\n",
+    "\n",
+    "poetry export --without-hashes --format=requirements.txt > requirements.txt\n",
+    "\n",
+    "### Kubernetes\n",
+    "\n",
+    "- Install `minikube`, see e.g. [here](https://minikube.sigs.k8s.io/docs/start/)\n",
+    "- `minikube start`\n",
+    "- Setup current shell to use minikube's docker daemon: `eval $(minikube -p minikube docker-env)`\n",
+    "- build docker container as above\n",
+    "    - check if docker image exists:\n",
+    "        - `minikube ssh`\n",
+    "        - `docker image ls`, you should see it listed.\n",
+    "- Start Kubernetes : `kubectl apply -f classification_deployment.yaml`\n",
+    "- list all Kubernetes pods: `kubectl get pods -A`\n",
+    "- forward port: `kubectl port-forward banana-7f84bb87d9-dk4ww  8000:80` Note: you will have to specify the exact pod name (here: `banana-7f84bb87d9-dk4ww`)\n",
+    "- Check, if you can access `localhost:8000`.\n",
+    "    - check `curl -X POST localhost:8000/classify --json '{\"chunk\": \"long text\", \"labels\": [\"abc\", \"def\"]}'`\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
   }
  ],
  "metadata": {

src/examples/issue_classification_user_journey/Dockerfile

@@ -0,0 +1,31 @@
+FROM ubuntu:latest as builder
+
+RUN apt-get update && apt-get upgrade -y \
+    && apt-get install -y python3 python3-pip python3-venv git
+
+RUN mkdir /app
+COPY requirements.txt /app/requirements.txt
+
+RUN python3 -m venv /app/venv
+ENV PATH="/app/venv/bin:$PATH"
+
+RUN pip install --upgrade pip
+RUN --mount=type=secret,id=GITHUB_TOKEN \
+    GITHUB_TOKEN=$(cat /run/secrets/GITHUB_TOKEN) \
+    pip install -r /app/requirements.txt
+
+
+FROM ubuntu:latest as runtime
+
+# Delete apt package lists immediately to save ~45MB. This has to be done in the same RUN command,
+# otherwise the data is already persisted in another layer.
+RUN apt-get update && apt-get upgrade -y \
+    && apt-get install -y python3 \
+    && rm -r /var/lib/apt/lists/*
+
+COPY --from=builder /app /app
+COPY main.py /app/main.py
+
+ENV PATH="/app/venv/bin:$PATH"
+WORKDIR /app
+ENTRYPOINT [ "hypercorn", "main:app", "--bind", "0.0.0.0:80", "--access-logfile", "-", "--access-logformat", "%(h)s %(l)s %(l)s %(t)s \"%(r)s\" %(s)s %(b)s %(L)s \"%(f)s\" \"%(a)s\""]

src/examples/issue_classification_user_journey/classification_deployment.yaml

@@ -0,0 +1,23 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: banana
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: banana
+  template:
+    metadata:
+      labels:
+        app: banana
+    spec:
+      containers:
+      - env:
+        - name: AA_TOKEN
+          valueFrom:
+            secretKeyRef:
+              key: AA_TOKEN
+              name: aa-token
+        image: classification-service:local
+        name: banana

src/examples/issue_classification_user_journey/main.py

@@ -0,0 +1,97 @@
+import http
+import os
+from http import HTTPStatus
+from typing import Annotated, Sequence
+
+from aleph_alpha_client import Client
+from dotenv import load_dotenv
+from fastapi import Depends, FastAPI, HTTPException, Request, Response
+from fastapi.datastructures import URL
+
+from intelligence_layer.connectors import AlephAlphaClientProtocol
+from intelligence_layer.core import LuminousControlModel, NoOpTracer, Task
+from intelligence_layer.use_cases import (
+    ClassifyInput,
+    PromptBasedClassify,
+    SingleLabelClassifyOutput,
+)
+
+# Minimal FastAPI app ##########################################################
+
+app = FastAPI()
+
+
+@app.get("/")
+def root() -> Response:
+    return Response(content="Classification Service", status_code=HTTPStatus.OK)
+
+
+# Authentication ###############################################################
+
+
+class AuthService:
+    def is_valid_token(self, token: str, permissions: Sequence[str], url: URL) -> bool:
+        # Add your authentication logic here
+        print(f"Checking permission for route: {url.path}")
+        return True
+
+
+class PermissionChecker:
+    def __init__(self, permissions: Sequence[str] = []):
+        self.permissions = permissions
+
+    def __call__(
+        self,
+        request: Request,
+        auth_service: Annotated[AuthService, Depends(AuthService)],
+    ) -> None:
+        token = request.headers.get("Authorization") or ""
+        try:
+            if not auth_service.is_valid_token(token, self.permissions, request.url):
+                raise HTTPException(HTTPStatus.UNAUTHORIZED)
+        except RuntimeError:
+            raise HTTPException(HTTPStatus.INTERNAL_SERVER_ERROR)
+
+
+permission_checker_for_user = PermissionChecker(["User"])
+
+
+# Intelligence Layer Task ######################################################
+
+PROMPT = """Identify the department that would be responsible for handling the given request.
+Reply with only the department name."""
+
+load_dotenv()
+
+
+def client() -> Client:
+    return Client(
+        token=os.environ["AA_TOKEN"],
+        host=os.getenv("AA_CLIENT_BASE_URL", "https://api.aleph-alpha.com"),
+    )
+
+
+def default_model(
+    app_client: Annotated[AlephAlphaClientProtocol, Depends(client)],
+) -> LuminousControlModel:
+    return LuminousControlModel("luminous-supreme-control", client=app_client)
+
+
+def classification_task(
+    model: Annotated[LuminousControlModel, Depends(default_model)],
+) -> PromptBasedClassify:
+    return PromptBasedClassify(instruction=PROMPT, model=model)
+
+
+@app.post(
+    "/classify",
+    # dependencies=[Depends(PermissionChecker(["User"]))],
+    status_code=http.HTTPStatus.OK,
+)
+def classification_task_route(
+    input: ClassifyInput,
+    task: Annotated[
+        Task[ClassifyInput, SingleLabelClassifyOutput], Depends(classification_task)
+    ],
+) -> SingleLabelClassifyOutput:
+    return task.run(input, NoOpTracer())