Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gemini connector integration #183668

Merged
merged 129 commits into from
Jun 6, 2024
Merged
Show file tree
Hide file tree
Changes from 81 commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
47a3e94
feat: Gemini dashboard creation
honeyn303 Apr 10, 2024
fa2b7b9
feat: Added the API Key attribute
honeyn303 Apr 22, 2024
5893522
fix: Fixed the api error.
honeyn303 Apr 23, 2024
a2c919a
fix: Remove the access & secret keys input field and populated the de…
honeyn303 Apr 24, 2024
918699c
feat: Added the configuration parameters for generateContent api
honeyn303 Apr 24, 2024
7704bd5
feat: Modified the code to use Vertex AI endpoint instead of AI studio
honeyn303 May 7, 2024
31efa17
feat: Added Project ID & location configuration field.
honeyn303 May 7, 2024
0afa1fe
fix: modified variable apiKey to accessToken
honeyn303 May 7, 2024
2575b19
feat: changed the default gemini model version to gemini-1.5-pro
honeyn303 May 8, 2024
98a4d40
Rebased this branch to the gemini branch
honeyn303 May 8, 2024
289b644
removed unused dependencies
honeyn303 May 9, 2024
156264e
removing file
honeyn303 May 9, 2024
f6f3e32
rebasing with the main branch
honeyn303 May 9, 2024
d179be0
Merge pull request #5 from honeyn303/gemini-ai-assistant
honeyn303 May 9, 2024
49a9dc0
Merge branch 'elastic:main' into gemini-assistant
honeyn303 May 9, 2024
b8099c0
Merge branch 'elastic:main' into gemini-assistant
honeyn303 May 10, 2024
ba83a51
Merge branch 'elastic:main' into gemini-assistant
honeyn303 May 13, 2024
a9411c0
Merge branch 'elastic:main' into gemini-assistant
honeyn303 May 14, 2024
230c713
ai assistant issue
honeyn303 May 14, 2024
a30af87
fix
honeyn303 May 14, 2024
2cedc33
Merge branch 'elastic:main' into gemini-bug-fix
honeyn303 May 14, 2024
01fc5fe
bug fix
honeyn303 May 14, 2024
0a9a26f
Merge branch 'elastic:main' into gemini-bug-fix
honeyn303 May 15, 2024
3d8472f
fix: Fixed the issue with the test api throwing an error
honeyn303 May 15, 2024
e689126
corrected signal parameter missing issue in gemini connector
rohanxz May 15, 2024
1654aea
Merge branch 'elastic:main' into gemini-bug-fix
honeyn303 May 15, 2024
6426c8d
Adding unit test cases for Gemini
shivankawasthi11 May 15, 2024
c371347
Revert "Adding unit test cases for Gemini"
shivankawasthi11 May 16, 2024
23032b2
Merge branch 'elastic:main' into gemini-bug-fix
honeyn303 May 16, 2024
8b80d4a
feat: Removed the Observability featureID, code clean up and added do…
honeyn303 May 16, 2024
2b680d9
feat: uncommented code
honeyn303 May 16, 2024
d68ca63
Merge branch 'elastic:main' into gemini-connector-integration
honeyn303 May 16, 2024
5da317f
fxi
honeyn303 May 16, 2024
bd47bfc
Merge branch 'elastic:main' into gemini-connector-integration
honeyn303 May 16, 2024
1c0605a
fixed sceret parameters
rohanxz May 16, 2024
90c1714
Merge branch 'elastic:main' into gemini-connector-integration
honeyn303 May 16, 2024
332ef77
Merge branch 'elastic:main' into gemini-connector-integration
honeyn303 May 16, 2024
57fcfbb
Merge branch 'elastic:main' into gemini-connector-integration
honeyn303 May 19, 2024
53d0ba1
fix: addressing the PR#183668 comments
honeyn303 May 19, 2024
644edc4
fix: code tweaks
honeyn303 May 19, 2024
0b12b47
fix: code clean up
honeyn303 May 19, 2024
f4f632d
Adding unit tests for Gemini
shivankawasthi11 May 17, 2024
223ddce
Updating unit test to implement OAuth
shivankawasthi11 May 20, 2024
ace52eb
feat: add token tracking for non stream responses
honeyn303 May 20, 2024
41673a8
initial commit: Adding functional tests (might need further tweaks)
honeyn303 May 20, 2024
d28edad
functional test cases for gemini
honeyn303 May 20, 2024
f626e00
fix: tweaks to functional test
honeyn303 May 21, 2024
4d748dc
Merge branch 'elastic:main' into gemini-connector-integration
honeyn303 May 21, 2024
ef455e8
fix: tweaks to oauth implementation
honeyn303 May 21, 2024
0522284
feat: Adding client side test cases
honeyn303 May 21, 2024
26f6776
feat: added the required documentation
honeyn303 May 21, 2024
da7fd52
fixes to the documentation
honeyn303 May 21, 2024
f652084
fix: Fixed token tracking issue for run and test sub-action
honeyn303 May 21, 2024
73dfdbe
feat: Added server side test cases
honeyn303 May 21, 2024
c2f3ff3
Delete .yarnrc
honeyn303 May 21, 2024
df11787
Delete .yarn/releases/yarn-1.22.19.cjs
honeyn303 May 21, 2024
e965f89
restored .yarnrc
rohanxz May 21, 2024
42c4eaa
Delete yarn.lock.orig
honeyn303 May 21, 2024
b6132cc
Update .yarnrc
honeyn303 May 21, 2024
54e34d4
feat: added docs images
honeyn303 May 21, 2024
a8a4598
Update llm.ts
honeyn303 May 21, 2024
047f8e7
Update constants.ts
honeyn303 May 21, 2024
4491673
Update index.tsx
honeyn303 May 21, 2024
4755c45
Merge branch 'elastic:main' into gemini-connector-integration
honeyn303 May 21, 2024
c48b14f
Merge branch 'main' into gemini-connector-integration
kibanamachine May 22, 2024
25cf790
fix: pr review comments
honeyn303 May 22, 2024
dbc6f70
fix: removed invoke ai and invoke stream
honeyn303 May 22, 2024
6c471a5
adding gemini files to CODEOWNERS
honeyn303 May 22, 2024
33fa26a
fix: es lint errors
honeyn303 May 22, 2024
1c9419b
fix: eslint error
honeyn303 May 22, 2024
1e19e54
fix: unique translations id
honeyn303 May 22, 2024
97526c0
fix: linting errors and added checks to validate the credentials JSON…
honeyn303 May 22, 2024
3c2363b
Fixing PR comments on unit tests
shivankawasthi11 May 23, 2024
776d940
Updating test body for Gemini
shivankawasthi11 May 23, 2024
0da2eb9
fix: removed invokeai and invoke stream schema
honeyn303 May 23, 2024
5d26a2b
fix: add proper credentials json file to the test case and few code t…
honeyn303 May 23, 2024
a20af20
fix: changed the credentials json format
honeyn303 May 23, 2024
1e3f330
fix: updated the json file to include the updated config and secret p…
honeyn303 May 23, 2024
0b02118
fix: the CI error related to node scripts/yarn_deduplicate
honeyn303 May 23, 2024
2fe2fa4
Merge branch 'main' into gemini-connector-integration
kibanamachine May 23, 2024
47af483
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine May 23, 2024
ee2a23a
fix: pr review comment changes
honeyn303 May 23, 2024
ccf2647
Updating unit tests to fix PR comments
shivankawasthi11 May 24, 2024
eb478a9
Merge branch 'main' into gemini-connector-integration
kibanamachine May 24, 2024
40098b1
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine May 24, 2024
21c8479
Merge branch 'main' into gemini-connector-integration
honeyn303 May 26, 2024
9c51343
fix: liniting issues
honeyn303 May 28, 2024
f11e36d
fix: eslint issues
honeyn303 May 28, 2024
0888b57
fix: linitng errors
honeyn303 May 28, 2024
94939e2
fix: using ES storage for storing access tokens
honeyn303 May 26, 2024
f6b570b
fix: few tweaks
honeyn303 May 27, 2024
78422fb
fix: test failures
honeyn303 May 28, 2024
2dac8ff
reverting changes to connector_token_client.ts
honeyn303 May 28, 2024
21bd29d
fix: removed console logs
honeyn303 May 28, 2024
375a316
fix: linting errors
honeyn303 May 28, 2024
85fa2dc
fix: changed the sorted field
honeyn303 May 28, 2024
123f206
fix: linting errors
honeyn303 May 29, 2024
5cd663a
Merge branch 'main' into gemini-connector-integration
kibanamachine May 29, 2024
9e162e3
Test and lint errors fix
shivankawasthi11 May 29, 2024
2e5b95d
fix: test failures
honeyn303 May 29, 2024
5ff956b
fix: test failures
honeyn303 May 29, 2024
19cca4f
Fixing test case
shivankawasthi11 May 30, 2024
883be6b
resolved PR review comments and refactored get_oauth_token file
rohanxz May 30, 2024
ce59195
added Unit tests for get_gcp_oauth_access_token.test
rohanxz May 30, 2024
ac0fbe3
Merge remote-tracking branch 'origin' into gemini-connector-integration
rohanxz May 30, 2024
b04aaca
updated yarn.lock
rohanxz May 30, 2024
bf07435
Removing the test
honeyn303 May 31, 2024
6abbcd8
resolved linting errors
rohanxz May 31, 2024
a391b96
fix: replaced the update function
honeyn303 May 31, 2024
a80be35
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine May 31, 2024
906f028
resolving conflit in connector_token_client.ts
honeyn303 May 31, 2024
2dced08
Merge branch 'main' into gemini-connector-integration
honeyn303 May 31, 2024
c9d3afb
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine May 31, 2024
b493737
resolving linting issue
rohanxz May 31, 2024
67032a1
Merge remote-tracking branch 'origin' into gemini-connector-integration
honeyn303 May 31, 2024
671e7d2
Merge branch 'elastic:main' into gemini-connector-integration
honeyn303 May 31, 2024
fd66af2
resolved conflicts
honeyn303 May 31, 2024
04ba7df
Merge branch 'main' into gemini-connector-integration
kibanamachine May 31, 2024
ac71670
Merge branch 'main' into gemini-connector-integration
honeyn303 Jun 3, 2024
a5dd164
Merge branch 'main' into gemini-connector-integration
kibanamachine Jun 3, 2024
0dfce94
fix: addressing PR review comments
honeyn303 Jun 3, 2024
08b4deb
fix: build test failures
honeyn303 Jun 3, 2024
c738d03
Delete .yarnrc
honeyn303 Jun 3, 2024
3bde569
Revert "Delete .yarnrc"
honeyn303 Jun 5, 2024
f967c76
removing file as requested
honeyn303 Jun 5, 2024
5680339
reverting changes to .yarnrc
honeyn303 Jun 5, 2024
e7a07db
Merge branch 'main' into gemini-connector-integration
kibanamachine Jun 5, 2024
61666f3
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Jun 5, 2024
00298b4
Merge branch 'main' into gemini-connector-integration
honeyn303 Jun 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -1484,6 +1484,11 @@ x-pack/test/security_solution_cypress/cypress/tasks/expandable_flyout @elastic/
/x-pack/plugins/stack_connectors/server/connector_types/bedrock @elastic/security-generative-ai
/x-pack/plugins/stack_connectors/common/bedrock @elastic/security-generative-ai

# Gemini
/x-pack/plugins/stack_connectors/public/connector_types/gemini @elastic/security-generative-ai
/x-pack/plugins/stack_connectors/server/connector_types/gemini @elastic/security-generative-ai
/x-pack/plugins/stack_connectors/common/gemini @elastic/security-generative-ai

## Defend Workflows owner connectors
/x-pack/plugins/stack_connectors/public/connector_types/sentinelone @elastic/security-defend-workflows
/x-pack/plugins/stack_connectors/server/connector_types/sentinelone @elastic/security-defend-workflows
Expand Down
147,529 changes: 147,529 additions & 0 deletions .yarn/releases/yarn-1.22.19.cjs

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions docs/management/action-types.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ a| <<d3security-action-type,D3 Security>>

| Send a request to D3 Security.

a| <<gemini-action-type,{gemini}>>

| Send a request to {gemini}.

a| <<email-action-type,Email>>

| Send email from your server.
Expand Down
74 changes: 74 additions & 0 deletions docs/management/connectors/action-types/gemini.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
[[gemini-action-type]]
== {gemini} connector and action
++++
<titleabbrev>{gemini}</titleabbrev>
++++
:frontmatter-description: Add a connector that can send requests to {gemini}.
:frontmatter-tags-products: [kibana]
:frontmatter-tags-content-type: [how-to]
:frontmatter-tags-user-goals: [configure]


The {gemini} connector uses https://github.com/axios/axios[axios] to send a POST request to {gemini}. The connector uses the <<execute-connector-api,run connector API>> to send the request.

[float]
[[define-gemini-ui]]
=== Create connectors in {kib}

You can create connectors in *{stack-manage-app} > {connectors-ui}*. For example:

[role="screenshot"]
image::management/connectors/images/gemini-connector.png[{gemini} connector]
// NOTE: This is an autogenerated screenshot. Do not edit it directly.

[float]
[[gemini-connector-configuration]]
==== Connector configuration

{gemini} connectors have the following configuration properties:

Name:: The name of the connector.
API URL:: The {gemini} request URL.
PROJECT ID:: The project which has Vertex AI endpoint enabled.
Region:: The GCP region where the Vertex AI endpoint enabled.
Default model:: The GAI model for {gemini} to use. Current support is for the Google Gemini models, defaulting to gemini-1.5-pro-preview-0409. The model can be set on a per request basis by including a "model" parameter alongside the request body.
Credentials JSON:: The GCP service account JSON file for authentication.

[float]
[[gemini-action-configuration]]
=== Test connectors

You can test connectors with the <<execute-connector-api,run connector API>> or
as you're creating or editing the connector in {kib}. For example:

[role="screenshot"]
image::management/connectors/images/gemini-params.png[{gemini} params test]
// NOTE: This is an autogenerated screenshot. Do not edit it directly.

The {gemini} actions have the following configuration properties.

Body:: A stringified JSON payload sent to the {gemini} Invoke Model API URL. For example:
+
[source,text]
--

{
body: JSON.stringify({
contents: [{
role: user,
parts: [{ text: 'Write the first line of a story about a magic backpack.' }]
}],
generation_config: {
temperature: 0,
maxOutputTokens: 8192
}
})
}
--
Model:: An optional string that will overwrite the connector's default model. For

[float]
[[gemini-connector-networking-configuration]]
=== Connector networking configuration

Use the <<action-settings, Action configuration settings>> to customize connector networking configurations, such as proxies, certificates, or TLS settings. You can set configurations that apply to all your connectors or use `xpack.actions.customHostSettings` to set per-host configurations.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/management/connectors/index.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
include::action-types/bedrock.asciidoc[leveloffset=+1]
include::action-types/d3security.asciidoc[leveloffset=+1]
include::action-types/email.asciidoc[leveloffset=+1]
include::action-types/gemini.asciidoc[leveloffset=+1]
include::action-types/resilient.asciidoc[leveloffset=+1]
include::action-types/index.asciidoc[leveloffset=+1]
include::action-types/jira.asciidoc[leveloffset=+1]
Expand Down
7 changes: 6 additions & 1 deletion docs/settings/alert-action-settings.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ WARNING: This feature is available in {kib} 7.17.4 and 8.3.0 onwards but is not
A boolean value indicating that a footer with a relevant link should be added to emails sent as alerting actions. Default: true.

`xpack.actions.enabledActionTypes` {ess-icon}::
A list of action types that are enabled. It defaults to `["*"]`, enabling all types. The names for built-in {kib} action types are prefixed with a `.` and include: `.email`, `.index`, `.jira`, `.opsgenie`, `.pagerduty`, `.resilient`, `.server-log`, `.servicenow`, .`servicenow-itom`, `.servicenow-sir`, `.slack`, `.swimlane`, `.teams`, `.tines`, `.torq`, `.xmatters`, `.gen-ai`, `.bedrock`, `.d3security`, and `.webhook`. An empty list `[]` will disable all action types.
A list of action types that are enabled. It defaults to `["*"]`, enabling all types. The names for built-in {kib} action types are prefixed with a `.` and include: `.email`, `.index`, `.jira`, `.opsgenie`, `.pagerduty`, `.resilient`, `.server-log`, `.servicenow`, .`servicenow-itom`, `.servicenow-sir`, `.slack`, `.swimlane`, `.teams`, `.tines`, `.torq`, `.xmatters`, `.gen-ai`, `.bedrock`, `.gemini`, `.d3security`, and `.webhook`. An empty list `[]` will disable all action types.
+
Disabled action types will not appear as an option when creating new connectors, but existing connectors and actions of that type will remain in {kib} and will not function.

Expand Down Expand Up @@ -277,6 +277,7 @@ A configuration URL that varies by connector:
+
--
* For an <<bedrock-action-type,{bedrock} connector>>, specifies the {bedrock} request URL.
* For an <<gemini-action-type,{gemini} connector>>, specifies the {gemini} request URL.
* For a <<openai-action-type,OpenAI connector>>, specifies the OpenAI request URL.
* For a <<resilient-action-type,{ibm-r} connector>>, specifies the {ibm-r} instance URL.
* For a <<jira-action-type,Jira connector>>, specifies the Jira instance URL.
Expand Down Expand Up @@ -341,6 +342,7 @@ The default model to use for requests, which varies by connector:
+
--
* For an <<bedrock-action-type,{bedrock} connector>>, current support is for the Anthropic Claude models. Defaults to `anthropic.claude-3-sonnet-20240229-v1:0`.
* For a <<gemini-action-type,{gemini} connector>>, current support is for the Gemini models. Defaults to `gemini-1.5-pro-preview-0409`.
* For a <<openai-action-type,OpenAI connector>>, it is optional and applicable only when `xpack.actions.preconfigured.<connector-id>.config.apiProvider` is `OpenAI`.
--

Expand Down Expand Up @@ -483,6 +485,9 @@ For an <<bedrock-action-type,{bedrock} connector>>, specifies the AWS access key

`xpack.actions.preconfigured.<connector-id>.secrets.apikey`::
An API key secret that varies by connector:

`xpack.actions.preconfigured.<connector-id>.secrets.credentialsJSON`::
For an <<gemini-action-type,{gemini} connector>>, specifies the GCP service account credentials JSON file for authentication.
+
--
* For a <<openai-action-type,OpenAI connector>>, specifies the OpenAI or Azure OpenAI API key for authentication.
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -1018,6 +1018,7 @@
"getopts": "^2.2.5",
"getos": "^3.1.0",
"globby": "^11.1.0",
"google-auth-library": "^9.10.0",
"gpt-tokenizer": "^2.1.2",
"handlebars": "4.7.8",
"he": "^1.2.0",
Expand Down
10 changes: 10 additions & 0 deletions x-pack/packages/kbn-elastic-assistant/impl/mock/connectors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,16 @@ export const mockActionTypes = [
isSystemActionType: true,
supportedFeatureIds: ['generativeAI'],
} as ActionType,
{
id: '.gemini',
name: 'Gemini',
enabled: true,
enabledInConfig: true,
enabledInLicense: true,
minimumLicenseRequired: 'basic',
isSystemActionType: true,
supportedFeatureIds: ['generativeAI'],
} as ActionType,
];

export const mockConnectors: AIConnector[] = [
Expand Down
Loading