Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use scaled_floats for percentages in ES mapping #2156

Merged
merged 1 commit into from
Aug 2, 2016
Merged

Use scaled_floats for percentages in ES mapping #2156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ https://github.com/elastic/beats/compare/v5.0.0-alpha5...master[Check the HEAD d

*Metricbeat*

- Use the new scaled_float Elasticsearch type for the percentage values. {pull}2156[2156]

*Packetbeat*

*Topbeat*
Expand Down
11 changes: 8 additions & 3 deletions libbeat/scripts/generate_template.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,19 +178,24 @@ def fill_field_properties(args, field, defaults, path):
}

elif field["type"] in ["geo_point", "date", "long", "integer",
"double", "float", "half_float", "boolean"]:
"double", "float", "half_float", "scaled_float",
"boolean"]:
# Convert all integer fields to long
if field["type"] == "integer":
field["type"] = "long"

if args.es2x and field["type"] == "half_float":
# ES 2.x doesn't support half floats, so convert to floats
if args.es2x and field["type"] in ["half_float", "scaled_float"]:
# ES 2.x doesn't support half or scaled floats, so convert to float
field["type"] = "float"

properties[field["name"]] = {
"type": field.get("type")
}

if field["type"] == "scaled_float":
properties[field["name"]]["scaling_factor"] = \
field.get("scaling_factor", 1000)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice


elif field["type"] in ["dict", "list"]:
if field.get("dict-type") == "keyword":
# add a dynamic template to set all members of
Expand Down
66 changes: 33 additions & 33 deletions metricbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,39 +141,39 @@ CPU stats.
[float]
=== apache.status.cpu.load

type: half_float
type: scaled_float

CPU Load.


[float]
=== apache.status.cpu.user

type: half_float
type: scaled_float

CPU user load.


[float]
=== apache.status.cpu.system

type: half_float
type: scaled_float

System cpu.


[float]
=== apache.status.cpu.children_user

type: half_float
type: scaled_float

CPU of children user.


[float]
=== apache.status.cpu.children_system

type: half_float
type: scaled_float

CPU of children system.

Expand Down Expand Up @@ -1899,7 +1899,7 @@ CPU Core number.
[float]
=== system.core.user.pct

type: half_float
type: scaled_float

The percentage of CPU time spent in user space. On multi-core systems, you can have percentages that are greater than 100%. For example, if 3 cores are at 60% use, then the `cpu.user_p` will be 180%.

Expand All @@ -1915,7 +1915,7 @@ The amount of CPU time spent in user space.
[float]
=== system.core.system.pct

type: half_float
type: scaled_float

The percentage of CPU time spent in kernel space.

Expand All @@ -1931,7 +1931,7 @@ The amount of CPU time spent in kernel space.
[float]
=== system.core.nice.pct

type: half_float
type: scaled_float

The percentage of CPU time spent on low-priority processes.

Expand All @@ -1947,7 +1947,7 @@ The amount of CPU time spent on low-priority processes.
[float]
=== system.core.idle.pct

type: half_float
type: scaled_float

The percentage of CPU time spent idle.

Expand All @@ -1963,7 +1963,7 @@ The amount of CPU time spent idle.
[float]
=== system.core.iowait.pct

type: half_float
type: scaled_float

The percentage of CPU time spent in wait (on disk).

Expand All @@ -1979,7 +1979,7 @@ The amount of CPU time spent in wait (on disk).
[float]
=== system.core.irq.pct

type: half_float
type: scaled_float

The percentage of CPU time spent servicing and handling hardware interrupts.

Expand All @@ -1995,7 +1995,7 @@ The amount of CPU time spent servicing and handling hardware interrupts.
[float]
=== system.core.softirq.pct

type: half_float
type: scaled_float

The percentage of CPU time spent servicing and handling software interrupts.

Expand All @@ -2011,7 +2011,7 @@ The amount of CPU time spent servicing and handling software interrupts.
[float]
=== system.core.steal.pct

type: half_float
type: scaled_float

The percentage of CPU time spent in involuntary wait by the virtual CPU while the hypervisor was servicing another processor. Available only on Unix.

Expand All @@ -2034,63 +2034,63 @@ The amount of CPU time spent in involuntary wait by the virtual CPU while the hy
[float]
=== system.cpu.user.pct

type: half_float
type: scaled_float

The percentage of CPU time spent in user space. On multi-core systems, you can have percentages that are greater than 100%. For example, if 3 cores are at 60% use, then the `cpu.user_p` will be 180%.


[float]
=== system.cpu.system.pct

type: half_float
type: scaled_float

The percentage of CPU time spent in kernel space.


[float]
=== system.cpu.nice.pct

type: half_float
type: scaled_float

The percentage of CPU time spent on low-priority processes.


[float]
=== system.cpu.idle.pct

type: half_float
type: scaled_float

The percentage of CPU time spent idle.


[float]
=== system.cpu.iowait.pct

type: half_float
type: scaled_float

The percentage of CPU time spent in wait (on disk).


[float]
=== system.cpu.irq.pct

type: half_float
type: scaled_float

The percentage of CPU time spent servicing and handling hardware interrupts.


[float]
=== system.cpu.softirq.pct

type: half_float
type: scaled_float

The percentage of CPU time spent servicing and handling software interrupts.


[float]
=== system.cpu.steal.pct

type: half_float
type: scaled_float

The percentage of CPU time spent in involuntary wait by the virtual CPU while the hypervisor was servicing another processor. Available only on Unix.

Expand Down Expand Up @@ -2314,7 +2314,7 @@ The used disk space in bytes.
[float]
=== system.filesystem.used.pct

type: half_float
type: scaled_float

The percentage of used disk space.

Expand Down Expand Up @@ -2380,47 +2380,47 @@ Load averages.
[float]
=== system.load.1

type: half_float
type: scaled_float

Load average for the last minute.


[float]
=== system.load.5

type: half_float
type: scaled_float

Load average for the last 5 minutes.


[float]
=== system.load.15

type: half_float
type: scaled_float

Load average for the last 15 minutes.


[float]
=== system.load.norm.1

type: half_float
type: scaled_float

Load divided by the number of cores for the last minute.


[float]
=== system.load.norm.5

type: half_float
type: scaled_float

Load divided by the number of cores for the last 5 minutes.


[float]
=== system.load.norm.15

type: half_float
type: scaled_float

Load divided by the number of cores for the last 15 minutes.

Expand Down Expand Up @@ -2459,7 +2459,7 @@ The total amount of free memory in bytes. This value does not include memory con
[float]
=== system.memory.used.pct

type: half_float
type: scaled_float

The percentage of used memory.

Expand Down Expand Up @@ -2490,7 +2490,7 @@ Actual free memory in bytes. It is calculated based on the OS. On Linux it consi
[float]
=== system.memory.actual.used.pct

type: half_float
type: scaled_float

The percentage of actual used memory.

Expand Down Expand Up @@ -2528,7 +2528,7 @@ Available swap memory.
[float]
=== system.memory.swap.used.pct

type: half_float
type: scaled_float

The percentage of used swap memory.

Expand Down Expand Up @@ -2694,7 +2694,7 @@ The amount of CPU time the process spent in user space.
[float]
=== system.process.cpu.total.pct

type: half_float
type: scaled_float

The percentage of CPU time spent by the process since the last update. Its value is similar to the %CPU value of the process displayed by the top command on Unix systems.

Expand Down Expand Up @@ -2748,7 +2748,7 @@ The Resident Set Size. The amount of memory the process occupied in main memory
[float]
=== system.process.memory.rss.pct

type: half_float
type: scaled_float

The percentage of memory the process occupied in main memory (RAM).

Expand Down
Loading