Dev -> master (#68)

* Increment Version * Fixed issues while making structure for neon api (#67) * Increment Version * Fixed issue when conversation id specified but unset (#69) * Increment Version * Project dependencies update (#70) * Migrated Python version to 3.10 * Cleaned up and restructured dependencies * Added pre commit hooks * Ran black formatter * Made setting up MySQLConnector optional * Updated project requirements * updated python version in references * Incremented subversion * Increment Version * Update for Helm deployment (#71) * Update containers to fully support ovos-config * WIP troubleshooting container init failures * Troubleshooting unit test failures * Troubleshooting unit test failures * Fix legacy config file checks * Replace 'update' with 'update_many' per PyMongo docs https://pymongo.readthedocs.io/en/stable/api/pymongo/collection.html#pymongo.collection.Collection.update_many Update command validation to reference known commands enum directly --------- Co-authored-by: Daniel McKnight <[email protected]> * Increment Version * Support for admin api (#72) * Added K8S and Rabbit MQ management utilities * added callback on missing k8s config * Fixed backward compatibility issue with pymongo * Added submind state processor * Moved out mq validation utility * incremented subversion * added retry-wait for klatchat observer * Fixed issue with caching * Simplified socket io logic * Added license notice --------- Co-authored-by: NeonKirill <[email protected]> * Increment Version * Fix configuration handling (#75) Co-authored-by: Daniel McKnight <[email protected]> * Increment Version * Added support for banning/unbanning subminds and admins endpoint for fetching chats data (#76) * Increment Version * Update PyPI action spec (#73) * Increment Version * Replaces `pull_master` with `propose_release` action (#78) Updates deployment automation to address #77 Co-authored-by: Daniel McKnight <[email protected]> * Increment Version * Reorganized Mongo DB API (#79) * Refactored Mongo DB API to be more granular, optimized and generified queries in place * Fixed unittests * Addressed comments * Fixed issue with prompts * removed redundant property from request_tts * Increment Version * Fixed issue with unauthorized user * Increment Version * Troubleshooting Proctored Conversations (#80) * Increment Version * Upgraded Socket IO to use any transport * Increment Version * Support for personas (#81) * Initial implementation of Personas API in klatchat * Added personas endpoints and optimized middleware * Support for generic configs management --------- Co-authored-by: NeonKirill <[email protected]> * Increment Version * Alpha -> Dev (#85) * added new deployment instructions to target alpha * Disabled pull request hook for CI flow * Fixed import issue * fixing "/ is not a connected namespace." * incremented controller expiration time and decreased ttl length * Fixing issues with updating user account (#83) * Reorganized Socket IO handlers * Fixing Chat Messages Overflow Issue (#84) * Removed dependency on chat_flow property to track messages in conversation * Increment Version --------- Co-authored-by: NeonDaniel <[email protected]> Co-authored-by: NeonKirill <[email protected]>
NeonGeckoCom · Apr 13, 2024 · 1a541b6 · 1a541b6
1 parent e71dc42
commit 1a541b6
Show file tree

Hide file tree

Showing 124 changed files with 6,386 additions and 3,152 deletions.
diff --git a/.github/workflows/deploy_containers.yml b/.github/workflows/deploy_containers.yml
@@ -5,17 +5,13 @@ on:
     branches:
       - dev
       - main
-  pull_request:
+      - alpha
   workflow_dispatch:
     inputs:
       version:
         description: 'Images tag'
         required: true
-        default: 'dev'
-#      images:
-#        description: 'Include Images (klatchat_observer|chat_client|chat_server) space-separated'
-#        required: false
-#        default: klatchat_observer chat_client chat_server
+        default: 'alpha'
 
 permissions:
   contents: write
@@ -57,17 +53,10 @@ jobs:
       if: github.event_name == 'workflow_dispatch'
       run: |
         echo "VERSION=${{ github.event.inputs.version }}" >> $GITHUB_ENV
-#        echo ${{ github.event.inputs.images }} >> $IMAGES
-    - name: Apply Pull Request Environments
-      if: github.event_name == 'pull_request'
-      run: |
-        echo "VERSION=${{ github.event.pull_request.base.ref }}" >> $GITHUB_ENV
-#        echo klatchat_observer chat_client chat_server >> $IMAGES
     - name: Apply Push Environments
       if: github.event_name == 'push'
       run: |
-        echo "VERSION=$(echo ${GITHUB_HEAD_REF} | tr / -)" >> $GITHUB_ENV
-#        echo klatchat_observer chat_client chat_server >> $IMAGES
+        echo "VERSION=$(echo ${{ github.ref_name }} | tr / -)" >> $GITHUB_ENV
     - name: Clean Up Version
       run: |
         echo "VERSION=${VERSION//['/']/-}" >> $GITHUB_ENV

diff --git a/.github/workflows/propose_release.yml b/.github/workflows/propose_release.yml
@@ -0,0 +1,27 @@
+name: Propose Stable Release
+on:
+  workflow_dispatch:
+    inputs:
+      release_type:
+        type: choice
+        description: Release Type
+        options:
+          - patch
+          - minor
+          - major
+jobs:
+  update_version:
+    uses: neongeckocom/.github/.github/workflows/propose_semver_release.yml@master
+    with:
+      branch: dev
+      release_type: ${{ inputs.release_type }}
+      update_changelog: True
+  pull_changes:
+    uses: neongeckocom/.github/.github/workflows/pull_master.yml@master
+    needs: update_version
+    with:
+      pr_reviewer: neonreviewers
+      pr_assignee: ${{ github.actor }}
+      pr_draft: false
+      pr_title: ${{ needs.update_version.outputs.version }}
+      pr_body: ${{ needs.update_version.outputs.changelog }}
diff --git a/.github/workflows/publish_release.yml b/.github/workflows/publish_release.yml
@@ -26,7 +26,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v1
         with:
-          python-version: 3.8
+          python-version: '3.10'
       - name: Install Build Tools
         run: |
           python -m pip install build wheel
@@ -35,6 +35,6 @@ jobs:
         run: |
           python setup.py bdist_wheel
       - name: Publish to Test PyPI
-        uses: pypa/gh-action-pypi-publish@master
+        uses: pypa/gh-action-pypi-publish@release/v1
         with:
-          password: ${{secrets.PYPI_TOKEN}}
+          password: ${{secrets.PYPI_TOKEN}}
diff --git a/.github/workflows/publish_test_build.yml b/.github/workflows/publish_test_build.yml
@@ -18,7 +18,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v1
         with:
-          python-version: 3.8
+          python-version: '3.10'
       - name: Install Build Tools
         run: |
           python -m pip install build wheel
@@ -34,6 +34,6 @@ jobs:
         run: |
           python setup.py bdist_wheel
       - name: Publish to Test PyPI
-        uses: pypa/gh-action-pypi-publish@master
+        uses: pypa/gh-action-pypi-publish@release/v1
         with:
           password: ${{secrets.PYPI_TOKEN}}
diff --git a/.github/workflows/pull_master.yml b/.github/workflows/pull_master.yml
diff --git a/.github/workflows/unit_tests.yml b/.github/workflows/unit_tests.yml
@@ -7,7 +7,7 @@ jobs:
   unit_tests:
     strategy:
       matrix:
-        python-version: [ '3.8', '3.9' ]
+        python-version: [ '3.10' ]
       max-parallel: 1
     runs-on: ubuntu-latest
     env:
@@ -24,6 +24,7 @@ jobs:
           python -m pip install --upgrade pip
           pip install -r requirements/requirements.txt
           pip install -r requirements/test_requirements.txt
+          pip install -r requirements/legacy_migration_requirements.txt
       - name: Get Credential
         run: |
           mkdir -p ~/.local/share/neon
@@ -51,7 +52,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v1
         with:
-          python-version: 3.8
+          python-version: '3.10'
       - name: Install Build Tools
         run: |
           python -m pip install build wheel

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -0,0 +1,16 @@
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v2.3.0
+    hooks:
+    -   id: check-yaml
+    -   id: check-json
+    -   id: check-symlinks
+    -   id: end-of-file-fixer
+    -   id: trailing-whitespace
+    -   id: pretty-format-json
+    -   id: requirements-txt-fixer
+    -   id: sort-simple-yaml
+-   repo: https://github.com/psf/black
+    rev: 22.10.0
+    hooks:
+    -   id: black
diff --git a/chat_client/__main__.py b/chat_client/__main__.py
@@ -31,7 +31,10 @@
 from chat_client.wsgi import app
 
 
-if __name__ == '__main__':
-    uvicorn.run(app=app, host=os.environ.get('HOST', '127.0.0.1'),
-                port=int(os.environ.get('PORT', 8001)),
-                log_level=os.environ.get('LOG_LEVEL', 'INFO').lower())
+if __name__ == "__main__":
+    uvicorn.run(
+        app=app,
+        host=os.environ.get("HOST", "127.0.0.1"),
+        port=int(os.environ.get("PORT", 8001)),
+        log_level=os.environ.get("LOG_LEVEL", "INFO").lower(),
+    )
diff --git a/chat_client/app.py b/chat_client/app.py
@@ -47,65 +47,84 @@
 sys.path.append(os.path.pardir)
 sys.path.append(os.path.dirname(os.path.abspath(__file__)))
 
-from .blueprints import base as base_blueprint, \
-                        chat as chat_blueprint, \
-                        components as components_blueprint
+from .blueprints import (
+    base as base_blueprint,
+    chat as chat_blueprint,
+    components as components_blueprint,
+)
 
 
 def create_app() -> FastAPI:
     """
-        Application factory for the Klatchat Client
+    Application factory for the Klatchat Client
     """
-    app_version = get_version('chat_client/version.py')
-    LOG.name = os.environ.get('LOG_NAME', 'client_err')
-    LOG.base_path = os.environ.get('LOG_BASE_PATH', '.')
-    LOG.init(config={'level': os.environ.get('LOG_LEVEL', 'INFO'), 'path': os.environ.get('LOG_PATH', os.getcwd())})
-    logger = LOG.create_logger('chat_client')
+    app_version = get_version("chat_client/version.py")
+    LOG.name = os.environ.get("LOG_NAME", "client_err")
+    LOG.base_path = os.environ.get("LOG_BASE_PATH", ".")
+    LOG.init(
+        config={
+            "level": os.environ.get("LOG_LEVEL", "INFO"),
+            "path": os.environ.get("LOG_PATH", os.getcwd()),
+        }
+    )
+    logger = LOG.create_logger("chat_client")
     logger.addHandler(logging.StreamHandler())
-    LOG.info(f'Starting Klatchat Client v{app_version}')
-    chat_app = FastAPI(title="Klatchat Client",
-                       version=app_version)
+    LOG.info(f"Starting Klatchat Client v{app_version}")
+    chat_app = FastAPI(title="Klatchat Client", version=app_version)
 
     @chat_app.middleware("http")
     async def log_requests(request: Request, call_next):
         """Logs requests and gracefully handles Internal Server Errors"""
-        idem = ''.join(random.choices(string.ascii_uppercase + string.digits, k=6))
+        idem = "".join(random.choices(string.ascii_uppercase + string.digits, k=6))
         LOG.info(f"rid={idem} start request path={request.url.path}")
         start_time = time.time()
         try:
             response = await call_next(request)
             process_time = (time.time() - start_time) * 1000
-            formatted_process_time = '{0:.2f}'.format(process_time)
-            LOG.info(f"rid={idem} completed_in={formatted_process_time}ms status_code={response.status_code}")
+            formatted_process_time = "{0:.2f}".format(process_time)
+            LOG.info(
+                f"rid={idem} completed_in={formatted_process_time}ms status_code={response.status_code}"
+            )
             return response
         except ConnectionError as ex:
             LOG.error(ex)
             from .client_config import app_config
-            return Response(f'Connection error : {app_config["SERVER_URL"]}', status_code=404)
+
+            return Response(
+                f'Connection error : {app_config["SERVER_URL"]}', status_code=404
+            )
         except Exception as ex:
             LOG.error(f"rid={idem} received an exception {ex}")
-        return Response(f'Chat server error occurred', status_code=500)
+        return Response(f"Chat server error occurred", status_code=500)
 
     # Redirects any not found pages to chats page
     @chat_app.exception_handler(StarletteHTTPException)
     async def custom_http_exception_handler(request, exc):
         if exc.status_code == status.HTTP_404_NOT_FOUND:
             return RedirectResponse("/chats")
 
-    __cors_allowed_origins = os.environ.get('COST_ALLOWED_ORIGINS', '') or '*'
+    __cors_allowed_origins = os.environ.get("CORS_ALLOWED_ORIGINS", "") or "*"
 
-    LOG.info(f'CORS_ALLOWED_ORIGINS={__cors_allowed_origins}')
+    LOG.info(f"CORS_ALLOWED_ORIGINS={__cors_allowed_origins}")
 
     chat_app.add_middleware(
         CORSMiddleware,
-        allow_origins=__cors_allowed_origins.split(','),
+        allow_origins=__cors_allowed_origins.split(","),
         allow_credentials=True,
         allow_methods=["*"],
         allow_headers=["*"],
     )
-    static_suffix = '/build' if os.environ.get('KLAT_ENV', 'dev').upper() == 'PROD' else ''
-    chat_app.mount("/css", StaticFiles(directory=f"chat_client/static/css{static_suffix}"), name="css")
-    chat_app.mount("/js", StaticFiles(directory=f"chat_client/static/js{static_suffix}"), name="js")
+    static_suffix = (
+        "/build" if os.environ.get("KLAT_ENV", "dev").upper() == "PROD" else ""
+    )
+    chat_app.mount(
+        "/css",
+        StaticFiles(directory=f"chat_client/static/css{static_suffix}"),
+        name="css",
+    )
+    chat_app.mount(
+        "/js", StaticFiles(directory=f"chat_client/static/js{static_suffix}"), name="js"
+    )
     chat_app.mount("/img", StaticFiles(directory=f"chat_client/static/img"), name="img")
 
     chat_app.include_router(base_blueprint.router)