Skip to content

Commit

Permalink
Update UI for integrations setup (opensearch-project#1052)
Browse files Browse the repository at this point in the history 
* Stub catalog reader interface

Signed-off-by: Simeon Widdis <[email protected]>

* Add basic catalog functionality to new catalog reader

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor validation logic with a deeper interface

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor validation logic with a deeper interface

Signed-off-by: Simeon Widdis <[email protected]>

* Remove redundant test.

This test is unneeded after 12c4bcf

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for new validators

Signed-off-by: Simeon Widdis <[email protected]>

* Make better failure mode for invalid objects

Signed-off-by: Simeon Widdis <[email protected]>

* Generalize Result type

Signed-off-by: Simeon Widdis <[email protected]>

* Convert backend to use catalog reader (unstable)

Signed-off-by: Simeon Widdis <[email protected]>

* Repair tests for integrations class (unstable)

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor repository for new integration interface

Signed-off-by: Simeon Widdis <[email protected]>

* Fix outer repository and backend tests

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for sample data

Signed-off-by: Simeon Widdis <[email protected]>

* Add CatalogReader JavaDocs

Signed-off-by: Simeon Widdis <[email protected]>

* Repair integrations builder

Signed-off-by: Simeon Widdis <[email protected]>

* Remove extra commented test

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unnecessary log statement

Signed-off-by: Simeon Widdis <[email protected]>

* Repair getSchemas behavior to return correct type

Let it be known at on this day, with this commit,
I have truly grokked why we don't use `any` in typescript.

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for getSchemas

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for asset and sample data backend methods

Signed-off-by: Simeon Widdis <[email protected]>

* Break flyout validation methods out of constructing method

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for extracted flyout methods

Signed-off-by: Simeon Widdis <[email protected]>

* Switch validation method to use ValidationResult

Signed-off-by: Simeon Widdis <[email protected]>

* Swap out flyout for hello-world setup page

Signed-off-by: Simeon Widdis <[email protected]>

* Add basic step incrementing

Signed-off-by: Simeon Widdis <[email protected]>

* Add basic field skeleton for each step

Signed-off-by: Simeon Widdis <[email protected]>

* Add a cancel button

Signed-off-by: Simeon Widdis <[email protected]>

* Add config type to developing form

Signed-off-by: Simeon Widdis <[email protected]>

* Flatten integration config

Signed-off-by: Simeon Widdis <[email protected]>

* Add sample data table modal

Signed-off-by: Simeon Widdis <[email protected]>

* Add toggle for standard and advanced asset config

Signed-off-by: Simeon Widdis <[email protected]>

* Simplify imports

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor major class names

Signed-off-by: Simeon Widdis <[email protected]>

* (WIP) begin refactoring functionality into adaptor

Signed-off-by: Simeon Widdis <[email protected]>

* Finish migrating functionality to data adaptor

Signed-off-by: Simeon Widdis <[email protected]>

* Rename integration types for more clarity

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor component usage

Signed-off-by: Simeon Widdis <[email protected]>

* Connect forms to config state

Signed-off-by: Simeon Widdis <[email protected]>

* Fix filetype selector

Signed-off-by: Simeon Widdis <[email protected]>

* Remove hardcoded name in path

Signed-off-by: Simeon Widdis <[email protected]>

* Write one snapshot test

Signed-off-by: Simeon Widdis <[email protected]>

* Add more tests

Signed-off-by: Simeon Widdis <[email protected]>

* Fix test naming

Signed-off-by: Simeon Widdis <[email protected]>

* Update obsolete snapshots

Signed-off-by: Simeon Widdis <[email protected]>

* Move integration creation helpers to own file

Signed-off-by: Simeon Widdis <[email protected]>

* Break out integration creation methods

Signed-off-by: Simeon Widdis <[email protected]>

* Isolate more create_integration helpers

Signed-off-by: Simeon Widdis <[email protected]>

* Simplify setup form

Signed-off-by: Simeon Widdis <[email protected]>

* Add data source picker items

Signed-off-by: Simeon Widdis <[email protected]>

* Add better selector logic

Signed-off-by: Simeon Widdis <[email protected]>

* Add queries for data sources

Signed-off-by: Simeon Widdis <[email protected]>

* Switch from selector to combobox

Signed-off-by: Simeon Widdis <[email protected]>

* Update snapshots

Signed-off-by: Simeon Widdis <[email protected]>

* Connect validation button to data source validation method

Signed-off-by: Simeon Widdis <[email protected]>

* Reimplement add integration button

Signed-off-by: Simeon Widdis <[email protected]>

* Temporarily remove validate button

Signed-off-by: Simeon Widdis <[email protected]>

* Simplify dynamic table term selection

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused validate code

Signed-off-by: Simeon Widdis <[email protected]>

* Undo wildcard import

Signed-off-by: Simeon Widdis <[email protected]>

* Switch from proxy to dataconnections endpoint

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused table fields

Signed-off-by: Simeon Widdis <[email protected]>

* Switch dataconnections base to const

Signed-off-by: Simeon Widdis <[email protected]>

* Add console proxy to route constants

Signed-off-by: Simeon Widdis <[email protected]>

* Update snapshots

Signed-off-by: Simeon Widdis <[email protected]>

* Move color to constants

Signed-off-by: Simeon Widdis <[email protected]>

* Move index name validation to constants and improve matching

Signed-off-by: Simeon Widdis <[email protected]>

* Move test constants to test constants

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>
  • Loading branch information
@Swiddis @mengweieric
Swiddis authored and mengweieric committed Oct 13, 2023
1 parent 340057a commit 2b1e2d9
Show file tree
Hide file tree
Showing 13 changed files with 2,194 additions and 2,552 deletions.
4 changes: 4 additions & 0 deletions common/constants/integrations.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,7 @@

export const OPENSEARCH_DOCUMENTATION_URL = 'https://opensearch.org/docs/latest/integrations/index';
export const ASSET_FILTER_OPTIONS = ['index-pattern', 'search', 'visualization', 'dashboard'];
export const VALID_INDEX_NAME = /^[a-z\d\.][a-z\d\._\-\*]*$/;

// Upstream doesn't export this, so we need to redeclare it for our use.
export type Color = 'success' | 'primary' | 'warning' | 'danger' | undefined;
1 change: 1 addition & 0 deletions common/constants/shared.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ export const EVENT_ANALYTICS = '/event_analytics';
export const SAVED_OBJECTS = '/saved_objects';
export const SAVED_QUERY = '/query';
export const SAVED_VISUALIZATION = '/vis';
export const CONSOLE_PROXY = '/api/console/proxy';

// Server route
export const PPL_ENDPOINT = '/_plugins/_ppl';
Expand Down
2,681 changes: 1,199 additions & 1,482 deletions ...omponents/integrations/components/__tests__/__snapshots__/setup_integration.test.tsx.snap
View file

Large diffs are not rendered by default.

329 changes: 1 addition & 328 deletions public/components/integrations/components/__tests__/added_integration_flyout.test.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,7 @@
import { configure, mount } from 'enzyme';
import Adapter from 'enzyme-adapter-react-16';
import { waitFor } from '@testing-library/react';
import {
AddIntegrationFlyout,
checkDataSourceName,
doTypeValidation,
doNestedPropertyValidation,
doPropertyValidation,
fetchDataSourceMappings,
fetchIntegrationMappings,
doExistingDataSourceValidation,
} from '../add_integration_flyout';
import * as add_integration_flyout from '../add_integration_flyout';
import { AddIntegrationFlyout } from '../add_integration_flyout';
import React from 'react';
import { HttpSetup } from '../../../../../../../src/core/public';

Expand Down Expand Up @@ -44,320 +34,3 @@ describe('Add Integration Flyout Test', () => {
});
});
});

describe('doTypeValidation', () => {
it('should return true if required type is not specified', () => {
const toCheck = { type: 'string' };
const required = {};

const result = doTypeValidation(toCheck, required);

expect(result.ok).toBe(true);
});

it('should return true if types match', () => {
const toCheck = { type: 'string' };
const required = { type: 'string' };

const result = doTypeValidation(toCheck, required);

expect(result.ok).toBe(true);
});

it('should return true if object has properties', () => {
const toCheck = { properties: { prop1: { type: 'string' } } };
const required = { type: 'object' };

const result = doTypeValidation(toCheck, required);

expect(result.ok).toBe(true);
});

it('should return false if types do not match', () => {
const toCheck = { type: 'string' };
const required = { type: 'number' };

const result = doTypeValidation(toCheck, required);

expect(result.ok).toBe(false);
});
});

describe('doNestedPropertyValidation', () => {
it('should return true if type validation passes and no properties are required', () => {
const toCheck = { type: 'string' };
const required = { type: 'string' };

const result = doNestedPropertyValidation(toCheck, required);

expect(result.ok).toBe(true);
});

it('should return false if type validation fails', () => {
const toCheck = { type: 'string' };
const required = { type: 'number' };

const result = doNestedPropertyValidation(toCheck, required);

expect(result.ok).toBe(false);
});

it('should return false if a required property is missing', () => {
const toCheck = { type: 'object', properties: { prop1: { type: 'string' } } };
const required = {
type: 'object',
properties: { prop1: { type: 'string' }, prop2: { type: 'number' } },
};

const result = doNestedPropertyValidation(toCheck, required);

expect(result.ok).toBe(false);
});

it('should return true if all required properties pass validation', () => {
const toCheck = {
type: 'object',
properties: {
prop1: { type: 'string' },
prop2: { type: 'number' },
},
};
const required = {
type: 'object',
properties: {
prop1: { type: 'string' },
prop2: { type: 'number' },
},
};

const result = doNestedPropertyValidation(toCheck, required);

expect(result.ok).toBe(true);
});
});

describe('doPropertyValidation', () => {
it('should return true if all properties pass validation', () => {
const rootType = 'root';
const dataSourceProps = {
prop1: { type: 'string' },
prop2: { type: 'number' },
};
const requiredMappings = {
root: {
template: {
mappings: {
properties: {
prop1: { type: 'string' },
prop2: { type: 'number' },
},
},
},
},
};

const result = doPropertyValidation(rootType, dataSourceProps as any, requiredMappings);

expect(result.ok).toBe(true);
});

it('should return false if a property fails validation', () => {
const rootType = 'root';
const dataSourceProps = {
prop1: { type: 'string' },
prop2: { type: 'number' },
};
const requiredMappings = {
root: {
template: {
mappings: {
properties: {
prop1: { type: 'string' },
prop2: { type: 'boolean' },
},
},
},
},
};

const result = doPropertyValidation(rootType, dataSourceProps as any, requiredMappings);

expect(result.ok).toBe(false);
});

it('should return false if a required nested property is missing', () => {
const rootType = 'root';
const dataSourceProps = {
prop1: { type: 'string' },
};
const requiredMappings = {
root: {
template: {
mappings: {
properties: {
prop1: { type: 'string' },
prop2: { type: 'number' },
},
},
},
},
};

const result = doPropertyValidation(rootType, dataSourceProps as any, requiredMappings);

expect(result.ok).toBe(false);
});
});

describe('checkDataSourceName', () => {
it('Filters out invalid index names', () => {
const result = checkDataSourceName('ss4o_logs-no-exclams!', 'logs');

expect(result.ok).toBe(false);
});

it('Filters out incorrectly typed indices', () => {
const result = checkDataSourceName('ss4o_metrics-test-test', 'logs');

expect(result.ok).toBe(false);
});

it('Accepts correct indices', () => {
const result = checkDataSourceName('ss4o_logs-test-test', 'logs');

expect(result.ok).toBe(true);
});
});

describe('fetchDataSourceMappings', () => {
it('Retrieves mappings', async () => {
const mockHttp = {
post: jest.fn().mockResolvedValue({
source1: { mappings: { properties: { test: true } } },
source2: { mappings: { properties: { test: true } } },
}),
} as Partial<HttpSetup>;

const result = fetchDataSourceMappings('sample', mockHttp as HttpSetup);

await expect(result).resolves.toMatchObject({
source1: { properties: { test: true } },
source2: { properties: { test: true } },
});
});

it('Catches errors', async () => {
const mockHttp = {
post: jest.fn().mockRejectedValue(new Error('Mock error')),
} as Partial<HttpSetup>;

const result = fetchDataSourceMappings('sample', mockHttp as HttpSetup);

await expect(result).resolves.toBeNull();
});
});

describe('fetchIntegrationMappings', () => {
it('Returns schema mappings', async () => {
const mockHttp = {
get: jest.fn().mockResolvedValue({ data: { mappings: { test: true } }, statusCode: 200 }),
} as Partial<HttpSetup>;

const result = fetchIntegrationMappings('target', mockHttp as HttpSetup);

await expect(result).resolves.toStrictEqual({ test: true });
});

it('Returns null if response fails', async () => {
const mockHttp = {
get: jest.fn().mockResolvedValue({ statusCode: 404 }),
} as Partial<HttpSetup>;

const result = fetchIntegrationMappings('target', mockHttp as HttpSetup);

await expect(result).resolves.toBeNull();
});

it('Catches request error', async () => {
const mockHttp = {
get: jest.fn().mockRejectedValue(new Error('mock error')),
} as Partial<HttpSetup>;

const result = fetchIntegrationMappings('target', mockHttp as HttpSetup);

await expect(result).resolves.toBeNull();
});
});

describe('doExistingDataSourceValidation', () => {
it('Catches and returns checkDataSourceName errors', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest
.spyOn(add_integration_flyout, 'checkDataSourceName')
.mockReturnValue({ ok: false, errors: ['mock'] });

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', false);
});

it('Catches data stream fetch errors', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest.spyOn(add_integration_flyout, 'checkDataSourceName').mockReturnValue({ ok: true });
jest.spyOn(add_integration_flyout, 'fetchDataSourceMappings').mockResolvedValue(null);
jest
.spyOn(add_integration_flyout, 'fetchIntegrationMappings')
.mockResolvedValue({ test: { template: { mappings: {} } } });

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', false);
});

it('Catches integration fetch errors', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest.spyOn(add_integration_flyout, 'checkDataSourceName').mockReturnValue({ ok: true });
jest
.spyOn(add_integration_flyout, 'fetchDataSourceMappings')
.mockResolvedValue({ test: { properties: {} } });
jest.spyOn(add_integration_flyout, 'fetchIntegrationMappings').mockResolvedValue(null);

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', false);
});

it('Catches type validation issues', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest.spyOn(add_integration_flyout, 'checkDataSourceName').mockReturnValue({ ok: true });
jest
.spyOn(add_integration_flyout, 'fetchDataSourceMappings')
.mockResolvedValue({ test: { properties: {} } });
jest
.spyOn(add_integration_flyout, 'fetchIntegrationMappings')
.mockResolvedValue({ test: { template: { mappings: {} } } });
jest
.spyOn(add_integration_flyout, 'doPropertyValidation')
.mockReturnValue({ ok: false, errors: ['mock'] });

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', false);
});

it('Returns no errors if everything passes', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest.spyOn(add_integration_flyout, 'checkDataSourceName').mockReturnValue({ ok: true });
jest
.spyOn(add_integration_flyout, 'fetchDataSourceMappings')
.mockResolvedValue({ test: { properties: {} } });
jest
.spyOn(add_integration_flyout, 'fetchIntegrationMappings')
.mockResolvedValue({ test: { template: { mappings: {} } } });
jest.spyOn(add_integration_flyout, 'doPropertyValidation').mockReturnValue({ ok: true });

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', true);
});
});
Loading

0 comments on commit 2b1e2d9

Please sign in to comment.