Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HTTP] Versioned router implementation #153543

Merged
merged 43 commits into from
Mar 28, 2023
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
30c31e1
move types to "core-versioned-http-server"
jloleysens Mar 22, 2023
52aff31
rename "version" to "versioned" in all files
jloleysens Mar 22, 2023
e5a80bc
export types
jloleysens Mar 22, 2023
1e88fac
remove the concept of versioned http toolkit
jloleysens Mar 22, 2023
1d28474
remove concept of versioned toolkit from code
jloleysens Mar 22, 2023
b191455
added getRoutes function and moved required "access" property to top-…
jloleysens Mar 22, 2023
3b74e08
Added VersionedRouterRoute type
jloleysens Mar 22, 2023
d720fdf
one small step for versioned http
jloleysens Mar 22, 2023
2124cae
added some check for "not found" 406 responses
jloleysens Mar 23, 2023
5a76452
move to "from" pattern for internal versioned router
jloleysens Mar 23, 2023
dc6b41b
update index exports
jloleysens Mar 23, 2023
1890a96
update index exports
jloleysens Mar 23, 2023
51f63db
updated types a bit more
jloleysens Mar 23, 2023
256a47f
got first iteration of the versioned router working, need a lot more …
jloleysens Mar 23, 2023
78a9e59
ran yarn kbn bootstrap
jloleysens Mar 23, 2023
02f7eef
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Mar 23, 2023
a2601b2
[CI] Auto-commit changed files from 'node scripts/generate codeowners'
kibanamachine Mar 23, 2023
929b3ef
remove export of non-existent types
jloleysens Mar 23, 2023
7422334
moved types to core-http-server, bye bye boilerplate
jloleysens Mar 23, 2023
9a751f2
move implementation to core/http
jloleysens Mar 23, 2023
f9241f6
run yarn kbn bootstrap and moved some internal types to internal package
jloleysens Mar 23, 2023
879575b
remove unhelpful comment
jloleysens Mar 23, 2023
f76be86
remove unnecessary type cast
jloleysens Mar 23, 2023
c83fdea
fix jest config and rename Internal... to Core...
jloleysens Mar 23, 2023
83d1d48
register route in the contstructor
jloleysens Mar 23, 2023
fd84e23
improve readbility of nullish checking
jloleysens Mar 23, 2023
3a7dfb7
[CI] Auto-commit changed files from 'node scripts/generate codeowners'
kibanamachine Mar 23, 2023
2bbc777
added some more tests
jloleysens Mar 23, 2023
91d54bf
updated test message
jloleysens Mar 23, 2023
82a2147
clean making the core request mutable
jloleysens Mar 24, 2023
3e79036
Merge branch 'main' into versioned-router-impl
jloleysens Mar 24, 2023
ae1a918
add "type" to type only import
jloleysens Mar 24, 2023
991984d
added status code to body result
jloleysens Mar 24, 2023
e7f3b6d
Pass in the mutated request
jloleysens Mar 24, 2023
d0363a3
improve if statement to run validation if there is at least some inpu…
jloleysens Mar 24, 2023
4ebba29
use mutable kibana request
jloleysens Mar 24, 2023
ff4c755
Merge branch 'main' into versioned-router-impl
jloleysens Mar 27, 2023
2970540
slight update to types
jloleysens Mar 27, 2023
0cf0319
added internal comment and updated test
jloleysens Mar 27, 2023
9e2e542
Merge branch 'main' into versioned-router-impl
jloleysens Mar 28, 2023
9ae403e
remove example.ts file, move to doc comment
jloleysens Mar 28, 2023
abb110d
refactor names of some versioning specific types
jloleysens Mar 28, 2023
7abb5da
fix ts issue
jloleysens Mar 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,8 @@ packages/core/usage-data/core-usage-data-base-server-internal @elastic/kibana-co
packages/core/usage-data/core-usage-data-server @elastic/kibana-core
packages/core/usage-data/core-usage-data-server-internal @elastic/kibana-core
packages/core/usage-data/core-usage-data-server-mocks @elastic/kibana-core
packages/core/versioning/core-version-http-server @elastic/kibana-core
packages/core/versioning/core-versioned-http-server @elastic/kibana-core
packages/core/versioning/core-versioned-http-server-internal @elastic/kibana-core
jloleysens marked this conversation as resolved.
Show resolved Hide resolved
x-pack/plugins/cross_cluster_replication @elastic/platform-deployment-management
packages/kbn-crypto @elastic/kibana-security
packages/kbn-crypto-browser @elastic/kibana-core
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,8 @@
"@kbn/core-usage-data-base-server-internal": "link:packages/core/usage-data/core-usage-data-base-server-internal",
"@kbn/core-usage-data-server": "link:packages/core/usage-data/core-usage-data-server",
"@kbn/core-usage-data-server-internal": "link:packages/core/usage-data/core-usage-data-server-internal",
"@kbn/core-version-http-server": "link:packages/core/versioning/core-version-http-server",
"@kbn/core-versioned-http-server": "link:packages/core/versioning/core-versioned-http-server",
"@kbn/core-versioned-http-server-internal": "link:packages/core/versioning/core-versioned-http-server-internal",
"@kbn/cross-cluster-replication-plugin": "link:x-pack/plugins/cross_cluster_replication",
"@kbn/crypto": "link:packages/kbn-crypto",
"@kbn/crypto-browser": "link:packages/kbn-crypto-browser",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ export {
isKibanaResponse,
KibanaResponse,
} from './src/response';
export { RouteValidator } from './src/validator';
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,9 @@ export class CoreKibanaRequest<

constructor(
request: RawRequest,
public readonly params: Params,
public readonly query: Query,
public readonly body: Body,
public params: Params,
public query: Query,
public body: Body,
jloleysens marked this conversation as resolved.
Show resolved Hide resolved
// @ts-expect-error we will use this flag as soon as http request proxy is supported in the core
// until that time we have to expose all the headers
private readonly withoutSecretHeaders: boolean
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# @kbn/core-versioned-http-server-internal

This package contains the implementation for sever-side HTTP versioning.

## Experimental

See notes in `@kbn/core-versioned-http-server`
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,4 @@
* Side Public License, v 1.
*/

// TODO: export once types are ready
export {};
export { InternalVersionedRouter } from './src';
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

module.exports = {
preset: '@kbn/test/jest_node',
rootDir: '../../../..',
roots: ['<rootDir>/packages/core/versioning/core-versioned-http-server-internal'],
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"type": "shared-common",
"id": "@kbn/core-versioned-http-server-internal",
"owner": "@elastic/kibana-core"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"name": "@kbn/core-versioned-http-server-internal",
"private": true,
"version": "1.0.0",
"author": "Kibana Core",
"license": "SSPL-1.0 OR Elastic License 2.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

export { InternalVersionedRouter } from './internal_versioned_router';
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { schema } from '@kbn/config-schema';
import type { IRouter, RequestHandler } from '@kbn/core-http-server';
import { httpServiceMock } from '@kbn/core-http-server-mocks';
import { VERSION_HEADER } from './internal_versioned_route';
import { InternalVersionedRouter } from '.';
import { kibanaResponseFactory } from '@kbn/core-http-router-server-internal';

describe('Versioned route', () => {
jloleysens marked this conversation as resolved.
Show resolved Hide resolved
let router: IRouter;
const handlerFn: RequestHandler = async (ctx, req, res) => res.ok({ body: { foo: 1 } });
beforeEach(() => {
router = httpServiceMock.createRouter();
});

it('can register multiple handlers', () => {
const versionedRouter = InternalVersionedRouter.from({ router });
versionedRouter
.get({ path: '/test/{id}', access: 'internal' })
.addVersion({ version: '1', validate: false }, handlerFn)
.addVersion({ version: '2', validate: false }, handlerFn)
.addVersion({ version: '3', validate: false }, handlerFn);
const routes = versionedRouter.getRoutes();
expect(routes).toHaveLength(1);
const [route] = routes;
expect(route.handlers).toHaveLength(3);
// We only register one route with the underlying router
expect(router.get).toHaveBeenCalledTimes(1);
});

it('does not allow specifying a handler for the same version more than once', () => {
const versionedRouter = InternalVersionedRouter.from({ router });
expect(() =>
versionedRouter
.get({ path: '/test/{id}', access: 'internal' })
.addVersion({ version: '1', validate: false }, handlerFn)
.addVersion({ version: '1', validate: false }, handlerFn)
.addVersion({ version: '3', validate: false }, handlerFn)
).toThrowError(`Version 1 handler has already been registered for the route "get /test/{id}"`);
});

it('runs input validations', async () => {
let handler: RequestHandler;

let validatedBody = false;
let validatedParams = false;
let validatedQuery = false;
let validatedOutputBody = false;

(router.post as jest.Mock).mockImplementation((opts: unknown, fn) => (handler = fn));
const versionedRouter = InternalVersionedRouter.from({ router });
versionedRouter.post({ path: '/test/{id}', access: 'internal' }).addVersion(
{
version: '1',
validate: {
request: {
body: schema.object({
foo: schema.number({
validate: () => {
validatedBody = true;
},
}),
}),
params: schema.object({
foo: schema.number({
validate: () => {
validatedParams = true;
},
}),
}),
query: schema.object({
foo: schema.number({
validate: () => {
validatedQuery = true;
},
}),
}),
},
response: {
body: schema.object({
foo: schema.number({
validate: () => {
validatedOutputBody = true;
},
}),
}),
},
},
},
handlerFn
);

const kibanaResponse = await handler!(
{} as any,
{
headers: { [VERSION_HEADER]: '1' },
body: { foo: 1 },
params: { foo: 1 },
query: { foo: 1 },
} as any,
kibanaResponseFactory
);

expect(kibanaResponse.status).toBe(200);
expect(validatedBody).toBe(true);
expect(validatedParams).toBe(true);
expect(validatedQuery).toBe(true);
expect(validatedOutputBody).toBe(true);
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { schema } from '@kbn/config-schema';
import type {
RequestHandler,
IRouter,
RequestHandlerContextBase,
KibanaRequest,
KibanaResponseFactory,
} from '@kbn/core-http-server';
import type {
Version,
AddVersionOpts,
VersionedRoute,
VersionedRouteConfig,
} from '@kbn/core-versioned-http-server';
import type { CoreKibanaRequest } from '@kbn/core-http-router-server-internal';
import type { Method } from './types';

import { validate } from './validate';

type Options = AddVersionOpts<unknown, unknown, unknown, unknown>;

export const VERSION_HEADER = 'TBD';

// This validation is a pass-through so that we can apply our version-specific validation later
const passThroughValidation = { body: schema.any(), params: schema.any(), query: schema.any() };

export class InternalVersionedRoute implements VersionedRoute {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect most of the code in here will be implementing the actual API specification and making sure versioned routes adhere to it.

private readonly handlers = new Map<
Version,
{
fn: RequestHandler;
options: Options;
}
>();

public static from({
router,
method,
path,
options,
}: {
router: IRouter;
method: Method;
path: string;
options: VersionedRouteConfig<Method>;
}) {
return new InternalVersionedRoute(router, method, path, options);
}

private constructor(
private readonly router: IRouter,
public readonly method: Method,
public readonly path: string,
public readonly options: VersionedRouteConfig<Method>,
// TODO: Make "true" dev-only
private readonly validateResponses: boolean = true
) {}

/** This is where we must implement the versioned spec once it is available */
private requestHandler = async (
ctx: RequestHandlerContextBase,
req: KibanaRequest,
res: KibanaResponseFactory
) => {
const version = req.headers[VERSION_HEADER] as undefined | Version;
if (!version) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking out loud: we may want to just default to the latest version instead.

return res.custom({
statusCode: 406,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using 406 seems smart. I'd just want to confirm that we're totally fine using it here, as we're not per-say performing content negociation with like standard headers. But tbh I actually like it.

body: `Version expected at [${this.method}] [${this.path}]. Please specify a version in the ${VERSION_HEADER} header.`,
});
}

const handler = this.handlers.get(version);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking out loud: depending on whether we use semver and have some concept of fallback, we may want more logic than just direct get from the map. But we can't do better than that right now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

if (!handler) {
return res.custom({
statusCode: 406,
body: `No ${version} available for ${this.method} ${this.path}. Available versions are: ${[
this.handlers.keys(),
].join(',')}`,
});
}

const coreKibanaRequest = req as CoreKibanaRequest;
if (handler.options.validate && handler.options.validate.request) {
jloleysens marked this conversation as resolved.
Show resolved Hide resolved
try {
const { body, params, query } = validate(
req,
handler.options.validate.request,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, my opinion regarding your thoughts:

First, I have to admit I missed that too when we were discussing about this. Kibana requests are instantiated directly with their validation schemas, which is a problem for us here...

Now,

I think the correct approach would be to have our own implementation of KibanaRequest that would wrap this CoreKibanaRequest. It would delegates (or 'proxy') most of the prop accesses to the underlying CoreKibanaRequest, except for params, query and body. That would be the correct way to address the problem here.

That being said, and even if what you did is more of a workaround/hack, I'd say it would be fine-ish if we want to keep it that way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was thinking about using a JS Proxy for this purpose. But I know it introduces a performance overhead for access. Manually wrapping felt like overhead.

I'm happy to go the Proxy route if we feel performance does not matter (I think it is about 1/3 the speed of normal access). I concluded that it is better to leave our performance budget as intact as possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't use a proxy, just a boring, manually maintained wrapper with get-based accessors.

But, again, just allowing to override the properties as you did is perfectly fine unless we find a reason to want more.

handler.options.version
);
coreKibanaRequest.body = body;
coreKibanaRequest.params = params;
coreKibanaRequest.query = query;
} catch (e) {
return res.custom({
statusCode: 400,
body: e.message,
});
jloleysens marked this conversation as resolved.
Show resolved Hide resolved
}
} else {
// Preserve behavior of not passing through unvalidated data
coreKibanaRequest.body = {};
coreKibanaRequest.params = {};
coreKibanaRequest.query = {};
}

const result = await handler.fn(ctx, req, res);

if (this.validateResponses && handler.options.validate && handler.options.validate.response) {
const { response } = handler.options.validate;
try {
validate(
req,
{ body: response.body, unsafe: { body: response.unsafe } },
handler.options.version
);
} catch (e) {
return res.custom({
statusCode: 500,
body: `Failed output validation: ${e.message}`,
});
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:oh_no_you_didnt:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄 gotta love structural typing


return result;
};

public addVersion(options: Options, handler: RequestHandler<any, any, any, any>): VersionedRoute {
if (this.handlers.has(options.version)) {
throw new Error(
`Version ${
options.version
} handler has already been registered for the route "${this.method.toLowerCase()} ${
this.path
}"`
);
}

this.handlers.set(options.version, { fn: handler, options });
if (this.handlers.size === 1) {
this.router[this.method](
{
path: this.path,
validate: passThroughValidation,
options: this.options,
},
this.requestHandler
);
}
jloleysens marked this conversation as resolved.
Show resolved Hide resolved
return this;
}

public getHandlers(): Array<{ fn: RequestHandler; options: Options }> {
return [...this.handlers.values()];
}
}
Loading