Skip to content

Commit

Permalink
#332 Generate Config Store by default
Browse files Browse the repository at this point in the history
- Update documentation to live under supporting components
- #219 improve manual action message keys
  • Loading branch information
carter-cundiff committed Sep 16, 2024
1 parent 760c05f commit 602b286
Show file tree
Hide file tree
Showing 16 changed files with 314 additions and 392 deletions.
3 changes: 3 additions & 0 deletions DRAFT_RELEASE_NOTES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Major Additions

## Universal Configurations Store
The Configuration Store is a tool that enables the various configurations for a project to be centrally defined and managed. It then provides a standardized way of accessing them, allowing the environment specific configurations to be dynamically provided to the their respective resources within the project at runtime. See the [official documentation](https://boozallen.github.io/aissemble/aissemble/current-dev/configuration-store.html) for more details on leveraging the configuration store.

## aiSSEMBLE Infrastructure Helm Chart
Created a helm chart with the necessary infrastructure for deploying your aiSSEMBLE project. This chart includes support for [Argo CD](https://argo-cd.readthedocs.io/en/stable/), [Jenkins](https://www.jenkins.io/), and [Nginx Ingress](https://docs.nginx.com/nginx-ingress-controller/). See the [chart README](https://github.com/boozallen/aissemble/tree/dev/extensions/extensions-helm/aissemble-infrastructure-chart#readme) for more details.

Expand Down
1 change: 1 addition & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
* Supporting Components
** xref:alerting-details.adoc[]
** xref:bias-detection.adoc[]
** xref:configuration-store.adoc[]
** xref:data-access-details.adoc[]
** xref:data-encryption.adoc[]
** xref:data-profiling-details.adoc[]
Expand Down
179 changes: 179 additions & 0 deletions docs/modules/ROOT/pages/configuration-store.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
= Universal Configuration Store

== Overview
The Universal Configuration Store is a tool that enables the various configurations for a project to be centrally defined and managed.
It then provides a standardized way of accessing them, allowing the environment specific configurations to be dynamically provided to
their respective resources within the project at runtime.

== Setup
To setup the Configuration Store one must <<defining-configuration-properties,define the configurations>> through properties files, optionally creating <<environment-specific-overrides,environment
specific overrides>>. Then the Configuration Store is ready for <<deploying-the-configuration-store,deployment>> to a Kubernetes cluster, using the <<backend-storage-types,backend storage type>>
of choice.

=== Defining Configuration Properties
The Configuration Store utilizes the opensource project https://github.com/TechnologyBrewery/krausening[Krausening,role=external,window=_blank] for managing its configurations, where each configuration is
represented by a key value pair property. The key values pairs are stored as string types using the standard `.properties` file
format.

Here is an example properties file:
[source,properties]
----
connector=smallrye-kafka
topic=baseTopic
----
This file defines two configuration properties: `connector` and `topic`.

TIP: For sensitive configuration values, Krausening supports https://github.com/TechnologyBrewery/krausening/blob/dev/krausening/README.md#krausening-in-four-pints-leveraging-jasypt-for-encryptingdecrypting-properties[encrypting and decrypting properties,role=external,window=_blank]

==== Environment Specific Overrides
In addition to the properties file introduced above, one can optionally specify a secondary properties file that houses environment
specific configurations. These configurations will then override their respective base configurations.

Continuing with the example above, suppose one defines an additional properties file like the following:
[source,properties]
----
topic=ciTopic
auth-token=auth-secret
----

Then when these two properties files are reconciled within the store, it results in a configuration equivalent to the following:
[source,properties]
----
connector=smallrye-kafka
topic=ciTopic
auth-token=auth-secret
----

NOTE: The `topic` property is overwritten when the environment specific value `ciTopic`.

=== Deploying the Configuration Store
The https://github.com/boozallen/aissemble/blob/{git-tree}/extensions/extensions-helm/aissemble-configuration-store-chart/README.md[Configuration Store Helm chart,role=external,window=_blank] is used for easily deploying it to a Kubernetes cluster. As part of the deployment, one must
provide the store access to the custom configurations by <<mounting-properties-files,mounting the properties files>> to the Kubernetes resource. Additionally,
the <<backend-storage-types,backend storage type>> can be adjusted to fit a project's needs.

NOTE: When deploying a project locally, the Configuration Store must be manually deployed first before all other resources. Please
create a https://github.com/boozallen/aissemble/issues[Github Issue,role=external,window=_blank] if additional assistance is required.

==== Mounting Properties Files
Properties files are attached to the local file system of the Configuration Store within Kubernetes through a mounted volume. To mount
a volume to the Configuration Store, one must set the Helm chart's https://github.com/boozallen/aissemble/blob/{git-tree}/extensions/extensions-helm/aissemble-configuration-store-chart/README.md#persistent-volume-properties[persistent volume properties,role=external,window=_blank]. Then the Helm chart's https://github.com/boozallen/aissemble/blob/{git-tree}/extensions/extensions-helm/aissemble-configuration-store-chart/README.md#environment-variables[environment
variables,role=external,window=_blank] must be updated with the corresponding location(s) of the properties files within that volume. It is required to specify
a base property location that houses the base/default property configurations. In addition to the base location, one can optionally specify a
secondary location for properties files that house the environment specific configurations.

NOTE: Multiple configuration properties files can be stored at either location.

=== Backend Storage Types
By default, the Configuration Store uses Krausening for it backend property storage. This provides a more light weight but still robust
storage option perfect for smaller applications.

For enterprise applications that have more robust requirements, it is recommended to utilize https://developer.hashicorp.com/vault[Vault,role=external,window=_blank]. Additionally, the configuration
store can be set up to use in-memory storage &mdash; this is best for unit testing.

To change it to use one of the other options, set the environment variable `STORAGE_CLASS` to the desired value. The values currently
available are `krausening`, `vault`, and `inMemory`.

== Usage
After the configurations are ingested and reconciled within the store, the next step is to access them within a project using one of
the two available methods: the <<rest-endpoint,REST endpoint>> or <<kubernetes-resource-injection-recommended,Kubernetes resource injection>>.
Configuration properties are identified by using a group name and property name, where group name is determined by the name of the file
the property is defined in.

Given an example properties file named `messaging.properties`:
[source,properties]
----
connector=smallrye-kafka
----

The property name would be `connector` and group name would be `messaging`.

=== REST Endpoint
The Configuration Store includes a Kubernetes service with the endpoints detailed below. The API endpoint provides a simple yet flexible way
to access any configurations from within a project.


==== GET/aissemble-properties/{groupName}/{propertyName}
Returns a JSON object containing the value of a property with the given group name and property name.

*Parameters*

|===
|*Name* | *Description*
|`groupName`
|The name of the file the property is defined in (excluding `.properties`).

|`propertyName`
|The name of the property

|===

Example GET request using the properties files defined above:
[source,bash]
----
curl http://configuration-store.config-store-ns.svc:8083/aissemble-properties/messaging/topic
----

Example JSON output:
[source,json]
----
{
"groupName": "messaging",
"name": "topic",
"value": "ciTopic"
}
----

NOTE: The name of the configuration service may vary depending on the project's Helm chart configuration, by default it is
`http://configuration-store.config-store-ns.svc:8083`

==== GET/aissemble-properties/healthcheck
Returns http `200` and a plain text string used for validating the service is reachable.


=== Kubernetes Resource Injection (Recommended)
The Configuration Store provisions a https://Kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/[Kubernetes admission webhook,role=external,window=_blank] as part of its deployment. This webhook is used for injecting property values
from the Configuration Store directly into Kubernetes resources as they are deployed to the cluster.

To insert a configuration value into a Kubernetes resource, the resource must include the following label within its metadata:
[source,yaml]
----
metadata:
labels:
aissemble-configuration-store: enabled
----

Then anywhere within the resource definition, all instances of `$getConfigValue(groupName={groupName};propertyName={propertyName})` will
be injected with the respective value from the Configuration Store.

Here is an example ConfigMap making use of resource injection with the properties files defined above. This is the resource definition before
it is deployed to the cluster:
[source,yaml]
----
apiVersion: v1
kind: ConfigMap
metadata:
name: messaging-config-map
labels:
aissemble-configuration-store: enabled
data:
messaging.properties: |-
connector=$getConfigValue(groupName=messaging;propertyName=connector)
topic=$getConfigValue(groupName=messaging;propertyName=topic)
auth-token=$getConfigValue(groupName=messaging;propertyName=auth-token)
----

Resulting resource on the cluster:
[source,yaml]
----
apiVersion: v1
kind: ConfigMap
metadata:
name: messaging-config-map
labels:
aissemble-configuration-store: enabled
data:
messaging.properties: |-
connector=smallrye-kafka
topic=ciTopic
auth-token=auth-secret
----
Loading

0 comments on commit 602b286

Please sign in to comment.