Skip to content

Commit

Permalink
#322 Set config store to be included by default
Browse files Browse the repository at this point in the history
- Update documentation to live under supporting components
  • Loading branch information
carter-cundiff committed Sep 12, 2024
1 parent 1ac07a6 commit df97de2
Show file tree
Hide file tree
Showing 5 changed files with 176 additions and 258 deletions.
3 changes: 3 additions & 0 deletions DRAFT_RELEASE_NOTES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Major Additions

## Universal Configurations Store
The Configuration Store is a tool that enables the various configurations for a project to be centrally defined and managed. It then provides a standardized way of accessing them, allowing the environment specific configurations to be dynamically provided to the their respective resources within the project at runtime. See the [official documentation](https://boozallen.github.io/aissemble/aissemble/current-dev/configuration-store.html) for more details on leveraging the configuration store.

## aiSSEMBLE Infrastructure Helm Chart
Created a helm chart with the necessary infrastructure for deploying your aiSSEMBLE project. This chart includes support for [Argo CD](https://argo-cd.readthedocs.io/en/stable/), [Jenkins](https://www.jenkins.io/), and [Nginx Ingress](https://docs.nginx.com/nginx-ingress-controller/). See the [chart README](https://github.com/boozallen/aissemble/tree/dev/extensions/extensions-helm/aissemble-infrastructure-chart#readme) for more details.

Expand Down
1 change: 1 addition & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
* Supporting Components
** xref:alerting-details.adoc[]
** xref:bias-detection.adoc[]
** xref:configuration-store.adoc[]
** xref:data-access-details.adoc[]
** xref:data-encryption.adoc[]
** xref:data-profiling-details.adoc[]
Expand Down
171 changes: 171 additions & 0 deletions docs/modules/ROOT/pages/configuration-store.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
= Universal Configuration Store

== Overview
The Universal Configuration Store is a tool that enables the various configurations for a project to be centrally defined and managed.
It then provides a standardized way of accessing them, allowing the environment specific configurations to be dynamically provided to
their respective resources within the project at runtime.

=== Setup
To setup the configuration store, one must first create properties files for storing the configurations. Each individual configuration is
represented as a property key value pair within the file. These files must then be made accessible to the configuration store kubernetes
resource for ingestion. See the applicable sections below for more details on defining configurations as properties, creating environment
specific property overrides, mounting the properties, and specifying the configuration store backend storage type.

==== Defining Configuration Properties
Properties represent key values pairs for storing the project configurations. The key values pairs are stored as string types using
the standard `.properties` file format. Each property then has an associated group name determined by the name of the file. Properties
are then accessed using a combination of their group name and property name. Accessing the properties is covered in more detail in the
<<Usage>> section below.

Here is an example of a base properties file named `messaging.properties`:
[source,properties]
----
connector=smallrye-kafka
topic=baseTopic
----
This file defines two properties, `connector` and `topic`, each with the group name `messaging`.

===== Environment Specific Overrides
In addition to the base properties file introduced above, one can optionally specify a secondary properties file that houses environment
specific configurations. These configurations will then used to override their respective base configurations.

Continuing with the example above, suppose one defines an additional properties file named `messaging.properties` like the following:
[source,properties]
----
topic=ciTopic
auth-token=auth-secret
----

Then when these two properties files are reconciled within the store, it results in a configuration equivalent to the following:
[source,properties]
----
connector=smallrye-kafka
topic=ciTopic
auth-token=auth-secret
----

NOTE: The `topic` property is overwritten when the environment specific value `ciTopic`.

==== Mounting Properties
In order for the store to read the configurations, the properties files will need to be placed into a https://kubernetes.io/docs/concepts/storage/persistent-volumes/#lifecycle-of-a-volume-and-claimpersistent[Persistent Volume,role=external,window=_blank] (PV)
mounted to the configuration store kubernetes resource. It is assumed a PV will meet most projects needs and is the most widely accepted
structured solution. The configuration service will then consume these properties files and place them into the configuration store. Please
see the configuration-store https://github.com/boozallen/aissemble/blob/{git-tree}/extensions/extensions-helm/aissemble-configuration-store-chart/README.md#persistent-volume-properties[helm chart documentation,role=external,window=_blank] for more information on mounting the PV to the service.

===== Properties File Locations
Within the configuration store's helm chart, one can specify environment variables with the URIs that specify where the project's various
properties files are located within the PV. It is required to specify a base property URI that houses the base/default property
configurations. In addition to the base URI, one can optionally specify a secondary URI for properties files that house the environment
specific configurations.

NOTE: Multiple configuration properties files can be stored at either URI.

Since it is common practice to define separate Helm chart values for each environment of a project's development lifecycle, it is
encouraged to define one shared base URI and respective environment specific URIs, each housing the relevant overrides.

The following Helm chart configuration demonstrates the URI specifications for an example CI deployment:
[source,yaml]
----
env:
BASE_PROPERTY: <URI housing base property configurations>
ENVIRONMENT_PROPERTY: <URI housing CI-specific property overrides/augmentations>
----

==== Backend Storage Options
The universal configuration store offers robust production ready configuration storage using https://developer.hashicorp.com/vault[Vault,role=external,window=_blank]. This is an excellent solution for large
enterprise applications. For smaller applications looking for a more light weight but still robust storage option, the configuration store
can be set to use https://github.com/TechnologyBrewery/krausening[Krausening,role=external,window=_blank]. The configuration store can also be set up to use in memory storage, this is best
for quick local testing.

The default property storage is Krausening. To change it to use one of the other options, set the environment
variable `STORAGE_CLASS` to the desired value. The values currently available are `krausening`, `vault`, and `inMemory`.

=== Usage
After the configurations are ingested and reconciled within the store, the next step is to access them within a project using one of
the two available methods: the API endpoint or kubernetes resource injection. See the applicable sections below for more details on
utilizing these approaches.

==== API Endpoint
The configuration store includes a kubernetes service with the endpoints detailed below. The API endpoint provides a simple yet flexible way
to access any configurations from within a project.

.Endpoints
[cols="2,1,4"]
|===
| Endpoint Path | HTTP Method | Description

| `/aissemble-properties/{groupName}/{propertyName}`
| `GET`
| Returns a JSON object containing the value of a property with the given group name and property name.

| `/aissemble-properties/healthcheck`
| `GET`
| Returns a plain text string used for validating the service is reachable.

|===

Here is an example `curl` request to the configuration store service containing the properties files defined above:
[source,bash]
----
curl http://configuration-store.config-store-ns.svc:8083/aissemble-properties/messaging/topic
----

Resulting JSON object:
[source,json]
----
{
"groupName": "messaging",
"name": "topic",
"value": "ciTopic"
}
----

NOTE: The name of the configuration service may vary depending on the projects helm chart configuration, by default it is
`http://configuration-store.config-store-ns.svc:8083`

==== Resource Injection
The configuration store provisions a https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/[kubernetes admission webhook,role=external,window=_blank] as part of its deployment. This webhook is used for injecting property values
from the configuration store directly into kubernetes resources as they deployed to the cluster.

To insert a configuration value into a kubernetes resource, the resource must include the following label within its metadata:
[source,yaml]
----
metadata:
labels:
aissemble-configuration-store: enabled
----

Then anywhere within the resource definition, all instances of `$getConfigValue(groupName={groupName};propertyName={propertyName})` will
be injected with the respective value from the configuration store.

Here is an example ConfigMap making use of resource injection with the properties files defined above. This is the resource definition before
it is deployed to the cluster:
[source,yaml]
----
apiVersion: v1
kind: ConfigMap
metadata:
name: messaging-config-map
labels:
aissemble-configuration-store: enabled
data:
messaging.properties: |-
connector=$getConfigValue(groupName=messaging;propertyName=connector)
topic=$getConfigValue(groupName=messaging;propertyName=topic)
auth-token=$getConfigValue(groupName=messaging;propertyName=auth-token)
----

Resulting resource on the cluster:
----
apiVersion: v1
kind: ConfigMap
metadata:
name: messaging-config-map
labels:
aissemble-configuration-store: enabled
data:
messaging.properties: |-
connector=smallrye-kafka
topic=ciTopic
auth-token=auth-secret
----
Loading

0 comments on commit df97de2

Please sign in to comment.