Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Data connection saved-object type for external connections #7925

Merged
merged 2 commits into from
Sep 4, 2024

Conversation

goyamegh
Copy link
Contributor

@goyamegh goyamegh commented Aug 29, 2024

Description

Today MDS plugin only supports OpenSearch datasources. With the new saved-object type data-connection, the intention is to allow the external data connections like CloudWatch and Security Lake.

Issues Resolved

NA

Screenshot

NA

Testing the changes

Saved objects API testing sample inputs (unit tests added as well):

1. Create:
Input: 
POST http://localhost:5601/api/saved_objects/data-connection/12345
{
  "attributes": {
    "connectionId": "security-lake",
    "type": "AWS CloudWatch"
  }
}
Output: 
{
"type":  "data-connection",
"id":  "1234567",
"attributes":  {
"connectionId":  "security-lake",
"type":  "AWS CloudWatch"
},
"references":  [],
"updated_at":  "2024-08-29T21:38:20.344Z"
}

2. Update
Input:
PUT http://localhost:5601/api/saved_objects/data-connection/12345
{
  "attributes": {
    "connectionId": "security-lake",
    "type": "AWS CloudWatch"
  }
}

3. Bulk create - failure
Input:
POST http://localhost:5601/api/saved_objects/_bulk_create
[
    {
      "type": "data-connection",
      "id": "12345",
      "attributes": {
        "connectionId": "security-lake",
        "type": "AWS Security "
      }
    },
    {
      "type": "data-connection",
      "id": "67890",
      "attributes": {
        "connectionId": "cloudwatch",
        "type": "AWS CloudWatch1"
      }
    }
  ]
Output:
{
    "statusCode": 400,
    "error": "Bad Request",
    "message": "Invalid attribute type: 'AWS Security ' for connection id: 'security-lake': Bad Request"
}

4. Bulk update
PUT http://localhost:5601/api/saved_objects/_bulk_update
[
    {
      "type": "data-connection",
      "id": "12345",
      "attributes": {
        "connectionId": "security-lake",
        "type": "AWS Security "
      }
    },
    {
      "type": "data-connection",
      "id": "67890",
      "attributes": {
        "connectionId": "cloudwatch",
        "type": "AWS CloudWatch1"
      }
    }
  ]
Output: 
{
    "statusCode": 400,
    "error": "Bad Request",
    "message": "Updating a data-connection saved-object type is not supported: Bad Request"
}

3. Bulk create - success
Input:
POST http://localhost:5601/api/saved_objects/_bulk_create
[
    {
      "type": "data-connection",
      "id": "12345",
      "attributes": {
        "connectionId": "security-lake1",
        "type": "AWS Security Lake"
      }
    },
    {
      "type": "data-connection",
      "id": "67890",
      "attributes": {
        "connectionId": "cloudwatch1",
        "type": "AWS CloudWatch"
      }
    }
  ]
Output:
{
    "saved_objects": [
        {
            "type": "data-connection",
            "id": "123456",
            "attributes": {
                "connectionId": "security-lake1",
                "type": "AWS Security Lake"
            },
            "references": [],
            "updated_at": "2024-08-29T21:37:52.262Z",
            "version": "1"
        },
        {
            "type": "data-connection",
            "id": "678901",
            "attributes": {
                "connectionId": "cloudwatch1",
                "type": "AWS CloudWatch"
            },
            "references": [],
            "updated_at": "2024-08-29T21:37:52.262Z",
            "version": "1"
        }
    ]
}

Changelog

  • feat: Introduce a data-connection saved-object type for external data connections

Check List

  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

Copy link
Contributor

ℹ️ Manual Changeset Creation Reminder

Please ensure manual commit for changeset file 7925.yml under folder changelogs/fragments to complete this PR.

If you want to use the available OpenSearch Changeset Bot App to avoid manual creation of changeset file you can install it in your forked repository following this link.

For more information about formatting of changeset files, please visit OpenSearch Auto Changeset and Release Notes Tool.

Copy link
Contributor

❌ Changelog Entry Missing Hyphen

Changelog entries must begin with a hyphen (-).

Copy link

codecov bot commented Aug 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.11%. Comparing base (adcaba7) to head (8632bae).
Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7925   +/-   ##
=======================================
  Coverage   61.11%   61.11%           
=======================================
  Files        3691     3691           
  Lines       87340    87340           
  Branches    13435    13435           
=======================================
  Hits        53381    53381           
  Misses      30706    30706           
  Partials     3253     3253           
Flag Coverage Δ
Linux_1 29.31% <ø> (ø)
Linux_2 56.24% <ø> (ø)
Linux_3 37.98% <ø> (ø)
Linux_4 29.58% <ø> (ø)
Windows_1 29.33% <ø> (ø)
Windows_2 56.20% <ø> (ø)
Windows_3 37.98% <ø> (ø)
Windows_4 29.58% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

kavilla
kavilla previously approved these changes Sep 3, 2024
@goyamegh goyamegh force-pushed the main-data-connection branch 2 times, most recently from f5e633d to d57311b Compare September 3, 2024 22:44
amsiglan
amsiglan previously approved these changes Sep 3, 2024
kavilla
kavilla previously approved these changes Sep 4, 2024
ruanyl
ruanyl previously approved these changes Sep 4, 2024
@goyamegh goyamegh force-pushed the main-data-connection branch from 0d21468 to cd8e468 Compare September 4, 2024 19:41
Copy link
Contributor

github-actions bot commented Sep 4, 2024

❌ Invalid Prefix For Manual Changeset Creation

Invalid description prefix. Found "feat". Only "skip" entry option is permitted for manual commit of changeset files.

If you were trying to skip the changelog entry, please use the "skip" entry option in the ##Changelog section of your PR description.

@amsiglan amsiglan merged commit 3caaa9d into opensearch-project:main Sep 4, 2024
67 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 4, 2024
* Add Data connection saved-object type for external connections

Signed-off-by: Megha Goyal <[email protected]>

* Changeset file for PR #7925 created/updated

---------

Signed-off-by: Megha Goyal <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
(cherry picked from commit 3caaa9d)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 4, 2024
* Add Data connection saved-object type for external connections

Signed-off-by: Megha Goyal <[email protected]>

* Changeset file for PR #7925 created/updated

---------

Signed-off-by: Megha Goyal <[email protected]>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
(cherry picked from commit 3caaa9d)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
LDrago27 pushed a commit that referenced this pull request Sep 5, 2024
#8000)

* Add Data connection saved-object type for external connections

* Changeset file for PR #7925 created/updated

---------
(cherry picked from commit 3caaa9d)

Signed-off-by: Megha Goyal <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
ashwin-pc pushed a commit that referenced this pull request Sep 10, 2024
#8001)

* Add Data connection saved-object type for external connections



* Changeset file for PR #7925 created/updated

---------



(cherry picked from commit 3caaa9d)

Signed-off-by: Megha Goyal <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
Co-authored-by: Huy Nguyen <[email protected]>
Comment on lines +21 to +27
type: 'text',
},
type: {
type: 'text',
},
meta: {
type: 'text',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should these types be keyword? and what is meta?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants