Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#1 📝 adding notes about documentation that is being worked as pa… #7

Merged
merged 1 commit into from
Apr 29, 2024
Merged

#1 📝 adding notes about documentation that is being worked as pa… #7

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 33 additions & 29 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
# aiSSEMBLE
[![Documentation](https://img.shields.io/badge/documentation-GitHub_Pages-blue)](https://boozallen.github.io/aissemble/current-dev/index.html)

## aiSSEMBLE Overview

### Purpose of the aiSSEMBLE

aiSSEMBLE is Booz Allen's lean manufacturing approach for holistically designing, developing and fielding
AI solutions across the engineering lifecycle from data processing to model building, tuning, and training to secure
operational deployment. This repository consists of standardized components which make it easy for dev teams to quickly
reuse and apply to their project to drive consistency, reliability and low delivery risk. aiSSEMBLE offers projects the
rapid generation of necessary scaffolding, boilerplate libraries, and container images with the flexibility of custom
configuration. It consists of pre-fabricated components that can be used as is within your projects and generated
AI solutions across the engineering lifecycle from data processing to model building, tuning, and training to secure
operational deployment. This repository consists of standardized components which make it easy for dev teams to quickly
reuse and apply to their project to drive consistency, reliability and low delivery risk. aiSSEMBLE offers projects the
rapid generation of necessary scaffolding, boilerplate libraries, and container images with the flexibility of custom
configuration. It consists of pre-fabricated components that can be used as is within your projects and generated
capabilities that can be extended.

### Languages and frameworks used to implement aiSSEMBLE
Expand All @@ -25,19 +24,25 @@ Many languages can be useful across the full breadth of AI solutions. Currently,
In addition, the following build tools and container frameworks are an important part of aiSSEMBLE:
* Fermenter MDA
* Maven
* Habushu Maven Plugin (builds Python modules)
* Orphedomos Maven Plugin (build Docker modules)
* Helm Maven Plugin
* Habushu Maven Plugin (builds Python modules)
* Orphedomos Maven Plugin (build Docker modules)
* Helm Maven Plugin
* Kubernetes
* Helm

### Releasing aiSSEMBLE
### Detailed Documentation

For details on releasing aiSSEMBLE, please see the README located [within our devops folder](https://github.com/boozallen/aissemble/blob/dev/devops/README.md).
aiSSEMBLE documentation will be posted on GitHub pages along with this repository. We are [actively porting our existing
documentation](https://github.com/boozallen/aissemble/issues/5) and will update this section with a link as soon as it is up and available.

### aiSSEMBLE Releases

aiSSEMBLE is currently released about once a month, but we intend to increase to around twice a month as we get our
processes adjusted and honed into the public GitHub

## Configurations

For details on using the configuration tool, please consult our [Configuring Your Environment guidance](https://boozallen.github.io/aissemble/current-dev/configurations.html).
For details on using the configuration tool, please consult our [Configuring Your Environment guidance (LINK COMING SOON)](https://boozallen.github.io/aissemble/current-dev/configurations.html).

## Build

Expand All @@ -57,23 +62,23 @@ The following steps will build aiSSEMBLE. *You must follow the configuration gui

### Helpful Profiles
The aiSSEMBLE baseline project provides several build profiles that may be helpful for different development environments.
To activate each one, use the standard Maven syntax: `mvn clean install -P[profile_name]`, for
instance, `mvn clean install -PnoRdAdmin`. There are many profiles you can find in the root `pom.xml` file. The
To activate each one, use the standard Maven syntax: `mvn clean install -P[profile_name]`, for
instance, `mvn clean install -PnoRdAdmin`. There are many profiles you can find in the root `pom.xml` file. The
following profiles are often useful when first starting with aiSSEMBLE:

* *noRdAdmin*: For configurations that disallow granting administrator privileges to Rancher Desktop. Testing frameworks
leveraged by aiSSEMBLE may, at times, assume that the docker unix socket is located at `/var/run/docker.sock`, which is
not the case when presented with a non-elevated Rancher installation. Activating this profile will override the
`DOCKER_HOST` seen by these dependencies, pointing it instead at `unix://$HOME/.rd/docker.sock`.
* *integration-test*: Some integration tests require Docker and automatically start/stop Docker Compose services while
executing tests (i.e. see the test/test-mda-models/test-data-delivery-pyspark-patterns module). **Note that the Maven
build does not build the Docker images directly. The images are built within the Kubernetes cluster to speed up
development builds and save disk space.**
* *noRdAdmin*: For configurations that disallow granting administrator privileges to Rancher Desktop. Testing frameworks
leveraged by aiSSEMBLE may, at times, assume that the docker unix socket is located at `/var/run/docker.sock`, which is
not the case when presented with a non-elevated Rancher installation. Activating this profile will override the
`DOCKER_HOST` seen by these dependencies, pointing it instead at `unix://$HOME/.rd/docker.sock`.
* *integration-test*: Some integration tests require Docker and automatically start/stop Docker Compose services while
executing tests (i.e. see the test/test-mda-models/test-data-delivery-pyspark-patterns module). **Note that the Maven
build does not build the Docker images directly. The images are built within the Kubernetes cluster to speed up
development builds and save disk space.**

## Use a Maven Archetype to Create a New aiSSEMBLE-Based Project

The first step in creating a new project is to leverage Maven’s archetype functionality to incept a new Maven project
that will contain all of your aiSSEMBLE component implementations - Data Delivery and Machine Learning pipelines as
The first step in creating a new project is to leverage Maven’s archetype functionality to incept a new Maven project
that will contain all of your aiSSEMBLE component implementations - Data Delivery and Machine Learning pipelines as
well as Path to Production modules.

Open a terminal to the location in which you want your project to live and execute the following command:
Expand All @@ -91,9 +96,9 @@ This command will trigger an interactive questionnaire giving you the opportunit
5. projectGitUrl
6. projectName

* For details on these fields refer to https://boozallen.github.io/aissemble/current-dev/archetype.html#_use_a_maven_archetype_to_create_a_new_project
* For details on these fields refer to (LINK COMING SOON) https://boozallen.github.io/aissemble/current-dev/archetype.html#_use_a_maven_archetype_to_create_a_new_project

* For detailed instructions on adding a pipeline refer to https://boozallen.github.io/aissemble/current-dev/add-pipelines-to-build.html
* For detailed instructions on adding a pipeline refer to (LINK COMING SOON) https://boozallen.github.io/aissemble/current-dev/add-pipelines-to-build.html

## Troubleshooting

Expand All @@ -115,8 +120,7 @@ the `test-data-delivery-pyspark-patterns` module:
SERVER ERROR: Bad Gateway url=https://dl.bintray.com/spark-packages/maven/org/antlr/antlr4-master/4.7/antlr4-master-4.7.jar
SERVER ERROR: Bad Gateway url=https://dl.bintray.com/spark-packages/maven/org/antlr/antlr-master/3.5.2/antlr-master-3.5.2.jar
```
If this occurs, remove your local Ivy cache (`rm -rf ~/.ivy2`) and then manually download the dependency that failed to
If this occurs, remove your local Ivy cache (`rm -rf ~/.ivy2`) and then manually download the dependency that failed to
download. Taking the above error message as an example, the following Maven command would download the needed commons-math3 jar:

`mvn org.apache.maven.plugins:maven-dependency-plugin:get -Dartifact=org.apache.commons:commons-math3:3.2`

`mvn org.apache.maven.plugins:maven-dependency-plugin:get -Dartifact=org.apache.commons:commons-math3:3.2`