Skip to content

Commit

Permalink
Move metrics to setup and add cgroup metrics (elastic#76730)
Browse files Browse the repository at this point in the history
# Conflicts:
#	src/core/server/config/deprecation/core_deprecations.ts
  • Loading branch information
joshdover committed Sep 9, 2020
1 parent 63db25b commit 6f86c5f
Show file tree
Hide file tree
Showing 31 changed files with 628 additions and 44 deletions.
2 changes: 1 addition & 1 deletion docs/development/core/server/kibana-plugin-core-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
| [LoggerFactory](./kibana-plugin-core-server.loggerfactory.md) | The single purpose of <code>LoggerFactory</code> interface is to define a way to retrieve a context-based logger instance. |
| [LoggingServiceSetup](./kibana-plugin-core-server.loggingservicesetup.md) | Provides APIs to plugins for customizing the plugin's logger. |
| [LogMeta](./kibana-plugin-core-server.logmeta.md) | Contextual metadata |
| [MetricsServiceSetup](./kibana-plugin-core-server.metricsservicesetup.md) | |
| [MetricsServiceSetup](./kibana-plugin-core-server.metricsservicesetup.md) | APIs to retrieves metrics gathered and exposed by the core platform. |
| [NodesVersionCompatibility](./kibana-plugin-core-server.nodesversioncompatibility.md) | |
| [OnPostAuthToolkit](./kibana-plugin-core-server.onpostauthtoolkit.md) | A tool set defining an outcome of OnPostAuth interceptor for incoming request. |
| [OnPreAuthToolkit](./kibana-plugin-core-server.onpreauthtoolkit.md) | A tool set defining an outcome of OnPreAuth interceptor for incoming request. |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [MetricsServiceSetup](./kibana-plugin-core-server.metricsservicesetup.md) &gt; [collectionInterval](./kibana-plugin-core-server.metricsservicesetup.collectioninterval.md)

## MetricsServiceSetup.collectionInterval property

Interval metrics are collected in milliseconds

<b>Signature:</b>

```typescript
readonly collectionInterval: number;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [MetricsServiceSetup](./kibana-plugin-core-server.metricsservicesetup.md) &gt; [getOpsMetrics$](./kibana-plugin-core-server.metricsservicesetup.getopsmetrics_.md)

## MetricsServiceSetup.getOpsMetrics$ property

Retrieve an observable emitting the [OpsMetrics](./kibana-plugin-core-server.opsmetrics.md) gathered. The observable will emit an initial value during core's `start` phase, and a new value every fixed interval of time, based on the `opts.interval` configuration property.

<b>Signature:</b>

```typescript
getOpsMetrics$: () => Observable<OpsMetrics>;
```

## Example


```ts
core.metrics.getOpsMetrics$().subscribe(metrics => {
// do something with the metrics
})

```

Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,18 @@

## MetricsServiceSetup interface

APIs to retrieves metrics gathered and exposed by the core platform.

<b>Signature:</b>

```typescript
export interface MetricsServiceSetup
```

## Properties

| Property | Type | Description |
| --- | --- | --- |
| [collectionInterval](./kibana-plugin-core-server.metricsservicesetup.collectioninterval.md) | <code>number</code> | Interval metrics are collected in milliseconds |
| [getOpsMetrics$](./kibana-plugin-core-server.metricsservicesetup.getopsmetrics_.md) | <code>() =&gt; Observable&lt;OpsMetrics&gt;</code> | Retrieve an observable emitting the [OpsMetrics](./kibana-plugin-core-server.opsmetrics.md) gathered. The observable will emit an initial value during core's <code>start</code> phase, and a new value every fixed interval of time, based on the <code>opts.interval</code> configuration property. |

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [OpsMetrics](./kibana-plugin-core-server.opsmetrics.md) &gt; [collected\_at](./kibana-plugin-core-server.opsmetrics.collected_at.md)

## OpsMetrics.collected\_at property

Time metrics were recorded at.

<b>Signature:</b>

```typescript
collected_at: Date;
```
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export interface OpsMetrics

| Property | Type | Description |
| --- | --- | --- |
| [collected\_at](./kibana-plugin-core-server.opsmetrics.collected_at.md) | <code>Date</code> | Time metrics were recorded at. |
| [concurrent\_connections](./kibana-plugin-core-server.opsmetrics.concurrent_connections.md) | <code>OpsServerMetrics['concurrent_connections']</code> | number of current concurrent connections to the server |
| [os](./kibana-plugin-core-server.opsmetrics.os.md) | <code>OpsOsMetrics</code> | OS related metrics |
| [process](./kibana-plugin-core-server.opsmetrics.process.md) | <code>OpsProcessMetrics</code> | Process related metrics |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [OpsOsMetrics](./kibana-plugin-core-server.opsosmetrics.md) &gt; [cpu](./kibana-plugin-core-server.opsosmetrics.cpu.md)

## OpsOsMetrics.cpu property

cpu cgroup metrics, undefined when not running in a cgroup

<b>Signature:</b>

```typescript
cpu?: {
control_group: string;
cfs_period_micros: number;
cfs_quota_micros: number;
stat: {
number_of_elapsed_periods: number;
number_of_times_throttled: number;
time_throttled_nanos: number;
};
};
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [OpsOsMetrics](./kibana-plugin-core-server.opsosmetrics.md) &gt; [cpuacct](./kibana-plugin-core-server.opsosmetrics.cpuacct.md)

## OpsOsMetrics.cpuacct property

cpu accounting metrics, undefined when not running in a cgroup

<b>Signature:</b>

```typescript
cpuacct?: {
control_group: string;
usage_nanos: number;
};
```
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ export interface OpsOsMetrics

| Property | Type | Description |
| --- | --- | --- |
| [cpu](./kibana-plugin-core-server.opsosmetrics.cpu.md) | <code>{</code><br/><code> control_group: string;</code><br/><code> cfs_period_micros: number;</code><br/><code> cfs_quota_micros: number;</code><br/><code> stat: {</code><br/><code> number_of_elapsed_periods: number;</code><br/><code> number_of_times_throttled: number;</code><br/><code> time_throttled_nanos: number;</code><br/><code> };</code><br/><code> }</code> | cpu cgroup metrics, undefined when not running in a cgroup |
| [cpuacct](./kibana-plugin-core-server.opsosmetrics.cpuacct.md) | <code>{</code><br/><code> control_group: string;</code><br/><code> usage_nanos: number;</code><br/><code> }</code> | cpu accounting metrics, undefined when not running in a cgroup |
| [distro](./kibana-plugin-core-server.opsosmetrics.distro.md) | <code>string</code> | The os distrib. Only present for linux platforms |
| [distroRelease](./kibana-plugin-core-server.opsosmetrics.distrorelease.md) | <code>string</code> | The os distrib release, prefixed by the os distrib. Only present for linux platforms |
| [load](./kibana-plugin-core-server.opsosmetrics.load.md) | <code>{</code><br/><code> '1m': number;</code><br/><code> '5m': number;</code><br/><code> '15m': number;</code><br/><code> }</code> | cpu load metrics |
Expand Down
16 changes: 12 additions & 4 deletions docs/setup/settings.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@ which may cause a delay before pages start being served.
Set to `false` to disable Console. *Default: `true`*

| `cpu.cgroup.path.override:`
| Override for cgroup cpu path when mounted in a
manner that is inconsistent with `/proc/self/cgroup`.
| *deprecated* This setting has been renamed to `ops.cGroupOverrides.cpuPath`
and the old name will no longer be supported as of 8.0.

| `cpuacct.cgroup.path.override:`
| Override for cgroup cpuacct path when mounted
in a manner that is inconsistent with `/proc/self/cgroup`.
| *deprecated* This setting has been renamed to `ops.cGroupOverrides.cpuAcctPath`
and the old name will no longer be supported as of 8.0.

| `csp.rules:`
| A https://w3c.github.io/webappsec-csp/[content-security-policy] template
Expand Down Expand Up @@ -438,6 +438,14 @@ not saved in {es}. *Default: `data`*
| Set the interval in milliseconds to sample
system and process performance metrics. The minimum value is 100. *Default: `5000`*

| `ops.cGroupOverrides.cpuPath:`
| Override for cgroup cpu path when mounted in a
manner that is inconsistent with `/proc/self/cgroup`.

| `ops.cGroupOverrides.cpuAcctPath:`
| Override for cgroup cpuacct path when mounted
in a manner that is inconsistent with `/proc/self/cgroup`.

| `server.basePath:`
| Enables you to specify a path to mount {kib} at if you are
running behind a proxy. Use the `server.rewriteBasePath` setting to tell {kib}
Expand Down
1 change: 1 addition & 0 deletions src/core/public/core_app/status/lib/load_status.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ const mockedResponse: StatusResponse = {
],
},
metrics: {
collected_at: new Date('2020-01-01 01:00:00'),
collection_interval_in_millis: 1000,
os: {
platform: 'darwin' as const,
Expand Down
2 changes: 2 additions & 0 deletions src/core/server/config/deprecation/core_deprecations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ export const coreDeprecationProvider: ConfigDeprecationProvider = ({
renameFromRoot('xpack.telemetry.config', 'telemetry.config'),
renameFromRoot('xpack.telemetry.banner', 'telemetry.banner'),
renameFromRoot('xpack.telemetry.url', 'telemetry.url'),
renameFromRoot('cpu.cgroup.path.override', 'ops.cGroupOverrides.cpuPath'),
renameFromRoot('cpuacct.cgroup.path.override', 'ops.cGroupOverrides.cpuAcctPath'),
configPathDeprecation,
dataPathDeprecation,
rewriteBasePathDeprecation,
Expand Down
1 change: 1 addition & 0 deletions src/core/server/legacy/legacy_service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,7 @@ export class LegacyService implements CoreService {
getTypeRegistry: startDeps.core.savedObjects.getTypeRegistry,
},
metrics: {
collectionInterval: startDeps.core.metrics.collectionInterval,
getOpsMetrics$: startDeps.core.metrics.getOpsMetrics$,
},
uiSettings: { asScopedToClient: startDeps.core.uiSettings.asScopedToClient },
Expand Down
115 changes: 115 additions & 0 deletions src/core/server/metrics/collectors/cgroup.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import mockFs from 'mock-fs';
import { OsCgroupMetricsCollector } from './cgroup';

describe('OsCgroupMetricsCollector', () => {
afterEach(() => mockFs.restore());

it('returns empty object when no cgroup file present', async () => {
mockFs({
'/proc/self': {
/** empty directory */
},
});

const collector = new OsCgroupMetricsCollector({});
expect(await collector.collect()).toEqual({});
});

it('collects default cgroup data', async () => {
mockFs({
'/proc/self/cgroup': `
123:memory:/groupname
123:cpu:/groupname
123:cpuacct:/groupname
`,
'/sys/fs/cgroup/cpuacct/groupname/cpuacct.usage': '111',
'/sys/fs/cgroup/cpu/groupname/cpu.cfs_period_us': '222',
'/sys/fs/cgroup/cpu/groupname/cpu.cfs_quota_us': '333',
'/sys/fs/cgroup/cpu/groupname/cpu.stat': `
nr_periods 444
nr_throttled 555
throttled_time 666
`,
});

const collector = new OsCgroupMetricsCollector({});
expect(await collector.collect()).toMatchInlineSnapshot(`
Object {
"cpu": Object {
"cfs_period_micros": 222,
"cfs_quota_micros": 333,
"control_group": "/groupname",
"stat": Object {
"number_of_elapsed_periods": 444,
"number_of_times_throttled": 555,
"time_throttled_nanos": 666,
},
},
"cpuacct": Object {
"control_group": "/groupname",
"usage_nanos": 111,
},
}
`);
});

it('collects override cgroup data', async () => {
mockFs({
'/proc/self/cgroup': `
123:memory:/groupname
123:cpu:/groupname
123:cpuacct:/groupname
`,
'/sys/fs/cgroup/cpuacct/xxcustomcpuacctxx/cpuacct.usage': '111',
'/sys/fs/cgroup/cpu/xxcustomcpuxx/cpu.cfs_period_us': '222',
'/sys/fs/cgroup/cpu/xxcustomcpuxx/cpu.cfs_quota_us': '333',
'/sys/fs/cgroup/cpu/xxcustomcpuxx/cpu.stat': `
nr_periods 444
nr_throttled 555
throttled_time 666
`,
});

const collector = new OsCgroupMetricsCollector({
cpuAcctPath: 'xxcustomcpuacctxx',
cpuPath: 'xxcustomcpuxx',
});
expect(await collector.collect()).toMatchInlineSnapshot(`
Object {
"cpu": Object {
"cfs_period_micros": 222,
"cfs_quota_micros": 333,
"control_group": "xxcustomcpuxx",
"stat": Object {
"number_of_elapsed_periods": 444,
"number_of_times_throttled": 555,
"time_throttled_nanos": 666,
},
},
"cpuacct": Object {
"control_group": "xxcustomcpuacctxx",
"usage_nanos": 111,
},
}
`);
});
});
Loading

0 comments on commit 6f86c5f

Please sign in to comment.