Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Add kibana setting for file data visualizer max file size #64427

Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions x-pack/plugins/ml/common/constants/settings.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export const FILE_DATA_VISUALIZER_MAX_FILE_SIZE = 'ml:fileDataVisualizerMaxFileSize';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not mandatory, but if you follow my other suggestions, it might be a good idea to synch here too:

Suggested change
export const FILE_DATA_VISUALIZER_MAX_FILE_SIZE = 'ml:fileDataVisualizerMaxFileSize';
export const FILE_DATA_VISUALIZER_MAX_FILE_SIZE = 'ml:DataVisualizerMaxFileSize';

16 changes: 0 additions & 16 deletions x-pack/plugins/ml/common/types/ml_config.ts

This file was deleted.

7 changes: 1 addition & 6 deletions x-pack/plugins/ml/public/application/app.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,10 @@ import { KibanaContextProvider } from '../../../../../src/plugins/kibana_react/p
import { setDependencyCache, clearCache } from './util/dependency_cache';
import { setLicenseCache } from './license';
import { MlSetupDependencies, MlStartDependencies } from '../plugin';
import { MlConfigType } from '../../common/types/ml_config';

import { MlRouter } from './routing';

type MlDependencies = MlSetupDependencies &
MlStartDependencies & {
mlConfig: MlConfigType;
};
type MlDependencies = MlSetupDependencies & MlStartDependencies;

interface AppProps {
coreStart: CoreStart;
Expand Down Expand Up @@ -78,7 +74,6 @@ export const renderApp = (
http: coreStart.http,
security: deps.security,
urlGenerators: deps.share.urlGenerators,
mlConfig: deps.mlConfig,
});

const mlLicense = setLicenseCache(deps.licensing);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@ import numeral from '@elastic/numeral';
import { ml } from '../../../../services/ml_api_service';
import { AnalysisResult, InputOverrides } from '../../../../../../common/types/file_datavisualizer';
import {
MAX_FILE_SIZE,
MAX_FILE_SIZE_BYTES,
ABSOLUTE_MAX_FILE_SIZE_BYTES,
FILE_SIZE_DISPLAY_FORMAT,
} from '../../../../../../common/constants/file_datavisualizer';
import { getMlConfig } from '../../../../util/dependency_cache';
import { getUiSettings } from '../../../../util/dependency_cache';
import { FILE_DATA_VISUALIZER_MAX_FILE_SIZE } from '../../../../../../common/constants/settings';

const DEFAULT_LINES_TO_SAMPLE = 1000;
const UPLOAD_SIZE_MB = 5;
Expand Down Expand Up @@ -62,13 +64,13 @@ export function readFile(file: File) {
}

export function getMaxBytes() {
const maxFileSize = getMlConfig().file_data_visualizer.max_file_size;
const maxFileSize = getUiSettings().get(FILE_DATA_VISUALIZER_MAX_FILE_SIZE, MAX_FILE_SIZE);
// @ts-ignore
const maxBytes = numeral(maxFileSize.toUpperCase()).value();
if (maxBytes < MAX_FILE_SIZE_BYTES) {
return MAX_FILE_SIZE_BYTES;
}
return maxBytes < ABSOLUTE_MAX_FILE_SIZE_BYTES ? maxBytes : ABSOLUTE_MAX_FILE_SIZE_BYTES;
return maxBytes <= ABSOLUTE_MAX_FILE_SIZE_BYTES ? maxBytes : ABSOLUTE_MAX_FILE_SIZE_BYTES;
}

export function getMaxBytesFormatted() {
Expand Down
11 changes: 0 additions & 11 deletions x-pack/plugins/ml/public/application/util/dependency_cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import {
} from 'kibana/public';
import { SharePluginStart } from 'src/plugins/share/public';
import { SecurityPluginSetup } from '../../../../security/public';
import { MlConfigType } from '../../../common/types/ml_config';

export interface DependencyCache {
timefilter: DataPublicPluginSetup['query']['timefilter'] | null;
Expand All @@ -43,7 +42,6 @@ export interface DependencyCache {
security: SecurityPluginSetup | null;
i18n: I18nStart | null;
urlGenerators: SharePluginStart['urlGenerators'] | null;
mlConfig: MlConfigType | null;
}

const cache: DependencyCache = {
Expand All @@ -64,7 +62,6 @@ const cache: DependencyCache = {
security: null,
i18n: null,
urlGenerators: null,
mlConfig: null,
};

export function setDependencyCache(deps: Partial<DependencyCache>) {
Expand All @@ -85,7 +82,6 @@ export function setDependencyCache(deps: Partial<DependencyCache>) {
cache.security = deps.security || null;
cache.i18n = deps.i18n || null;
cache.urlGenerators = deps.urlGenerators || null;
cache.mlConfig = deps.mlConfig || null;
}

export function getTimefilter() {
Expand Down Expand Up @@ -206,13 +202,6 @@ export function getGetUrlGenerator() {
return cache.urlGenerators.getUrlGenerator;
}

export function getMlConfig() {
if (cache.mlConfig === null) {
throw new Error("mlConfig hasn't been initialized");
}
return cache.mlConfig;
}

export function clearCache() {
console.log('clearing dependency cache'); // eslint-disable-line no-console
Object.keys(cache).forEach(k => {
Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/ml/public/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { PluginInitializer, PluginInitializerContext } from 'kibana/public';
import { PluginInitializer } from 'kibana/public';
import './index.scss';
import {
MlPlugin,
Expand All @@ -19,6 +19,6 @@ export const plugin: PluginInitializer<
MlPluginStart,
MlSetupDependencies,
MlStartDependencies
> = (context: PluginInitializerContext) => new MlPlugin(context);
> = () => new MlPlugin();

export { MlPluginSetup, MlPluginStart };
13 changes: 1 addition & 12 deletions x-pack/plugins/ml/public/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,7 @@
*/

import { i18n } from '@kbn/i18n';
import {
Plugin,
CoreStart,
CoreSetup,
AppMountParameters,
PluginInitializerContext,
} from 'kibana/public';
import { Plugin, CoreStart, CoreSetup, AppMountParameters } from 'kibana/public';
import { ManagementSetup } from 'src/plugins/management/public';
import { SharePluginStart } from 'src/plugins/share/public';
import { UsageCollectionSetup } from 'src/plugins/usage_collection/server';
Expand All @@ -25,7 +19,6 @@ import { LicenseManagementUIPluginSetup } from '../../license_management/public'
import { setDependencyCache } from './application/util/dependency_cache';
import { PLUGIN_ID, PLUGIN_ICON } from '../common/constants/app';
import { registerFeature } from './register_feature';
import { MlConfigType } from '../common/types/ml_config';

export interface MlStartDependencies {
data: DataPublicPluginStart;
Expand All @@ -41,10 +34,7 @@ export interface MlSetupDependencies {
}

export class MlPlugin implements Plugin<MlPluginSetup, MlPluginStart> {
constructor(private readonly initializerContext: PluginInitializerContext) {}

setup(core: CoreSetup<MlStartDependencies, MlPluginStart>, pluginsSetup: MlSetupDependencies) {
const mlConfig = this.initializerContext.config.get<MlConfigType>();
core.application.register({
id: PLUGIN_ID,
title: i18n.translate('xpack.ml.plugin.title', {
Expand All @@ -67,7 +57,6 @@ export class MlPlugin implements Plugin<MlPluginSetup, MlPluginStart> {
usageCollection: pluginsSetup.usageCollection,
licenseManagement: pluginsSetup.licenseManagement,
home: pluginsSetup.home,
mlConfig,
},
{
element: params.element,
Expand Down
15 changes: 0 additions & 15 deletions x-pack/plugins/ml/server/config.ts

This file was deleted.

2 changes: 0 additions & 2 deletions x-pack/plugins/ml/server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,3 @@ import { MlServerPlugin } from './plugin';
export { MlPluginSetup, MlPluginStart } from './plugin';

export const plugin = (ctx: PluginInitializerContext) => new MlServerPlugin(ctx);

export { config } from './config';
34 changes: 34 additions & 0 deletions x-pack/plugins/ml/server/lib/register_settings.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { CoreSetup } from 'kibana/server';
import { i18n } from '@kbn/i18n';
import { schema } from '@kbn/config-schema';
import { FILE_DATA_VISUALIZER_MAX_FILE_SIZE } from '../../common/constants/settings';
import { MAX_FILE_SIZE } from '../../common/constants/file_datavisualizer';

export function registerKibanaSettings(coreSetup: CoreSetup) {
coreSetup.uiSettings.register({
[FILE_DATA_VISUALIZER_MAX_FILE_SIZE]: {
name: i18n.translate('xpack.ml.maxFileSizeSettingsName', {
defaultMessage: 'File Data Visualizer maximum file upload size',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the docs we don't differentiate between the "Data Visualizer" and the "File Data Visualizer" (I think it's because they're both on the same "Data Visualizer" tab). IMO it's not necessary to have the "File" qualifier here either:

Suggested change
defaultMessage: 'File Data Visualizer maximum file upload size',
defaultMessage: 'Data Visualizer maximum file upload size',

}),
value: MAX_FILE_SIZE,
description: i18n.translate('xpack.ml.maxFileSizeSettingsDescription', {
defaultMessage:
'Sets the file size limit when importing data in the File Data Visualizer. The highest supported value for this setting is 1GB.',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto re "File Data Visualizer":

Suggested change
'Sets the file size limit when importing data in the File Data Visualizer. The highest supported value for this setting is 1GB.',
'Sets the file size limit when importing data in the Data Visualizer. The highest supported value for this setting is 1GB.',

}),
category: ['Machine Learning'],
schema: schema.string(),
validation: {
regexString: '\\d+[mMgG][bB]',
message: i18n.translate('xpack.ml.maxFileSizeSettingsError', {
defaultMessage: 'Should be a valid data size. e.g. 200MB, 1GB',
}),
},
},
});
}
3 changes: 3 additions & 0 deletions x-pack/plugins/ml/server/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ import { MlServerLicense } from './lib/license';
import { createSharedServices, SharedServices } from './shared_services';
import { userMlCapabilities, adminMlCapabilities } from '../common/types/capabilities';
import { setupCapabilitiesSwitcher } from './lib/capabilities';
import { registerKibanaSettings } from './lib/register_settings';

declare module 'kibana/server' {
interface RequestHandlerContext {
Expand Down Expand Up @@ -122,6 +123,8 @@ export class MlServerPlugin implements Plugin<MlPluginSetup, MlPluginStart, Plug
},
});

registerKibanaSettings(coreSetup);

this.mlLicense.setup(plugins.licensing.license$, [
(mlLicense: MlLicense) => initSampleDataSets(mlLicense, plugins),
]);
Expand Down