Slimmed-down version of Dispatch for PoCs and basic usage

[kars7e]

Feature Request

Detailed Description

Provide a single-binary Dispatch server that does not require Kubernetes to run. Ideally, the only requirement should be access to a Docker daemon.

Context

Dispatch currently consists of 8+ base services, many dependent services, but most importantly, it requires Kubernetes to be deployed. This is intentional as we aim to support large-scale, multi-tenant environments with many functions/APIs/events being processed at the same time.
There are situations where this scale is not needed though. When a user first tries Dispatch, he/she often deploys only a few resources of each kind. This is more than enough to get familiar with Dispatch and its model. However, the Dispatch requirements may be just too much for the user. We found that local Kubernetes installation, even with the help of Minikube, is often problematic and disappointing. When a cluster is finally available, the footprint of Dispatch control plane consumes more compute & storage resources than the functions created for testing. It will be beneficial for users to be able to test Dispatch in a much simpler fashion before they commit to it and make the hassle of installing all the dependencies.

Possible Implementation

The current Dispatch CLI and swagger API definitions should remain unchanged. Dispatch services have no overlapping API endpoints, so they could be combined into a single binary which handles requests for all Dispatch resources (single binary could be then used to produce single container/single OVA deliverables).

The goal is to maintain the same experience for the user, so as many features of "cluster" Dispatch as possible should be available in the "single-host" version of it. All services depend on a datastore. We already have a support for libkv "file" storage which could be used in the single-host version.

Here is a break down by service for per-service dependencies:

Function Manager

Function Manager expects a FaaS driver to create, delete and run functions. A new, Docker-based driver would be created. The expectation will be that function manager has network access to both docker daemon and containers deployed on it.

Image Manager

Image manager only requires Docker daemon. As the same daemon will be used by function manager, there is no need for image registry.

Secrets Store

Secret store would have a new libkv-based backend.

Events Manager

Event Drivers would have a new Docker-based backend. Subscriptions would have a complete in-memory implementation (no external message queue).

API Manager

API Manager could support API Gateway within the Dispatch itself, all paths not starting with /v1 could be routed to it. New in-memory backend would be added

Identity Manager

Single-host Dispatch instance could probably live without auth, and use single organization. However we could investigate if supporting auth with service accounts would be possible.

Service Manager

Service manager depends on k8s catalog which might be hard to replace. This service might not be supported in single-host version.

Complexity

• Low - Simple enhancement or bug fix, no architectural changes or refactoring
• Medium - Change requires some thought, but is relatively isolated
• High - Significant architectural change or large refactor

[berndtj]

This is great. Initially, we should just not worry about API gateway and Service manager.