Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add normalize aggregation #4886

Merged
merged 2 commits into from
Jul 31, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/aggregations.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,8 @@ There are many different types of pipeline aggregation, each computing different

* <<moving-function-aggregation-usage,Moving Function Aggregation Usage>>

* <<normalize-aggregation-usage,Normalize Aggregation Usage>>

* <<percentiles-bucket-aggregation-usage,Percentiles Bucket Aggregation Usage>>

* <<serial-differencing-aggregation-usage,Serial Differencing Aggregation Usage>>
Expand Down Expand Up @@ -321,6 +323,8 @@ include::aggregations/pipeline/moving-average/moving-average-simple-aggregation-

include::aggregations/pipeline/moving-function/moving-function-aggregation-usage.asciidoc[]

include::aggregations/pipeline/normalize/normalize-aggregation-usage.asciidoc[]

include::aggregations/pipeline/percentiles-bucket/percentiles-bucket-aggregation-usage.asciidoc[]

include::aggregations/pipeline/serial-differencing/serial-differencing-aggregation-usage.asciidoc[]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
:ref_current: https://www.elastic.co/guide/en/elasticsearch/reference/master

:github: https://github.com/elastic/elasticsearch-net

:nuget: https://www.nuget.org/packages

////
IMPORTANT NOTE
==============
This file has been generated from https://github.com/elastic/elasticsearch-net/tree/master/src/Tests/Tests/Aggregations/Pipeline/Normalize/NormalizeAggregationUsageTests.cs.
If you wish to submit a PR for any spelling mistakes, typos or grammatical errors for this file,
please modify the original csharp file found at the link and submit the PR with that change. Thanks!
////

[[normalize-aggregation-usage]]
=== Normalize Aggregation Usage

A parent pipeline aggregation which calculates the specific normalized/rescaled value for a specific bucket value.
Values that cannot be normalized, will be skipped using the skip gap policy.

NOTE: Valid for Elasticsearch 7.9.0+ with at least basic license level

==== Fluent DSL example

[source,csharp]
----
a => a
.DateHistogram("projects_started_per_month", dh => dh
.Field(p => p.StartedOn)
.CalendarInterval(DateInterval.Month)
.Aggregations(aa => aa
.Sum("commits", sm => sm
.Field(p => p.NumberOfCommits)
)
.Normalize("percent_of_commits", aaa => aaa
.BucketsPath("commits")
.Method(NormalizeMethod.PercentOfSum)
.Format("00.00%")
)
)
)
----

==== Object Initializer syntax example

[source,csharp]
----
new DateHistogramAggregation("projects_started_per_month")
{
Field = "startedOn",
CalendarInterval = DateInterval.Month,
Aggregations = new SumAggregation("commits", "numberOfCommits") &&
new NormalizeAggregation("percent_of_commits", "commits")
{
Method = NormalizeMethod.PercentOfSum,
Format = "00.00%"
}
}
----

[source,javascript]
.Example json output
----
{
"projects_started_per_month": {
"date_histogram": {
"field": "startedOn",
"calendar_interval": "month"
},
"aggs": {
"commits": {
"sum": {
"field": "numberOfCommits"
}
},
"percent_of_commits": {
"normalize": {
"buckets_path": "commits",
"method": "percent_of_sum",
"format": "00.00%"
}
}
}
}
}
----

==== Handling Responses

[source,csharp]
----
response.ShouldBeValid();

var projectsPerMonth = response.Aggregations.DateHistogram("projects_started_per_month");
projectsPerMonth.Should().NotBeNull();
projectsPerMonth.Buckets.Should().NotBeNull();
projectsPerMonth.Buckets.Count.Should().BeGreaterThan(0);

foreach (var bucket in projectsPerMonth.Buckets)
{
var normalize = bucket.Normalize("percent_of_commits");
normalize.Value.Should().BeGreaterOrEqualTo(0);
normalize.ValueAsString.Should().NotBeNullOrEmpty();
}
----

1 change: 1 addition & 0 deletions docs/code-standards/descriptors.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,7 @@ var methods = from d in YieldAllDescriptors()
where !(m.Name == nameof(RankFeatureSigmoidFunctionDescriptor.Pivot) && dt == typeof(RankFeatureSigmoidFunctionDescriptor))
where !(m.Name == nameof(DateHistogramGroupSourceDescriptor<object>.CalendarInterval) && dt == typeof(DateHistogramGroupSourceDescriptor<>))
where !(m.Name == nameof(DateHistogramGroupSourceDescriptor<object>.FixedInterval) && dt == typeof(DateHistogramGroupSourceDescriptor<>))
where !(m.Name == nameof(NormalizeAggregationDescriptor.Method) && dt == typeof(NormalizeAggregationDescriptor))

select new {m, d, p};

Expand Down
2 changes: 2 additions & 0 deletions src/Nest/Aggregations/AggregateDictionary.cs
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,8 @@ public FiltersAggregate Filters(string key)

public SingleBucketAggregate Nested(string key) => TryGet<SingleBucketAggregate>(key);

public ValueAggregate Normalize(string key) => TryGet<ValueAggregate>(key);

public SingleBucketAggregate ReverseNested(string key) => TryGet<SingleBucketAggregate>(key);

public SingleBucketAggregate Children(string key) => TryGet<SingleBucketAggregate>(key);
Expand Down
15 changes: 15 additions & 0 deletions src/Nest/Aggregations/AggregationContainer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,10 @@ public interface IAggregationContainer
[DataMember(Name = "nested")]
INestedAggregation Nested { get; set; }

/// <inheritdoc cref="INormalizeAggregation"/>
[DataMember(Name = "normalize")]
INormalizeAggregation Normalize { get; set; }

/// <inheritdoc cref="IParentAggregation"/>
[DataMember(Name = "parent")]
IParentAggregation Parent { get; set; }
Expand Down Expand Up @@ -356,6 +360,9 @@ public class AggregationContainer : IAggregationContainer

public INestedAggregation Nested { get; set; }

/// <inheritdoc cref="INormalizeAggregation"/>
public INormalizeAggregation Normalize { get; set; }

/// <inheritdoc cref="IParentAggregation"/>
public IParentAggregation Parent { get; set; }

Expand Down Expand Up @@ -513,6 +520,8 @@ public class AggregationContainerDescriptor<T> : DescriptorBase<AggregationConta

INestedAggregation IAggregationContainer.Nested { get; set; }

INormalizeAggregation IAggregationContainer.Normalize { get; set; }

IParentAggregation IAggregationContainer.Parent { get; set; }

IPercentileRanksAggregation IAggregationContainer.PercentileRanks { get; set; }
Expand Down Expand Up @@ -672,6 +681,12 @@ Func<NestedAggregationDescriptor<T>, INestedAggregation> selector
) =>
_SetInnerAggregation(name, selector, (a, d) => a.Nested = d);

/// <inheritdoc cref="INormalizeAggregation"/>
public AggregationContainerDescriptor<T> Normalize(string name,
Func<NormalizeAggregationDescriptor, INormalizeAggregation> selector
) =>
_SetInnerAggregation(name, selector, (a, d) => a.Normalize = d);

/// <inheritdoc cref="IParentAggregation"/>
public AggregationContainerDescriptor<T> Parent<TParent>(string name,
Func<ParentAggregationDescriptor<T, TParent>, IParentAggregation> selector
Expand Down
91 changes: 91 additions & 0 deletions src/Nest/Aggregations/Pipeline/Normalize/NormalizeAggregation.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
// Licensed to Elasticsearch B.V under one or more agreements.
// Elasticsearch B.V licenses this file to you under the Apache 2.0 License.
// See the LICENSE file in the project root for more information

using System.Runtime.Serialization;
using Elasticsearch.Net;
using Elasticsearch.Net.Utf8Json;

namespace Nest
{
/// <summary>
/// A parent pipeline aggregation which calculates the specific normalized/rescaled value for a specific bucket value.
/// Values that cannot be normalized, will be skipped using the skip gap policy.
/// <para />
/// Valid in Elasticsearch 7.9.0+ with at least basic license level.
/// </summary>
[InterfaceDataContract]
[ReadAs(typeof(NormalizeAggregation))]
public interface INormalizeAggregation : IPipelineAggregation
{
[DataMember(Name = "method")]
NormalizeMethod Method { get; set; }
}

/// <inheritdoc cref="INormalizeAggregation"/>
public class NormalizeAggregation
: PipelineAggregationBase, INormalizeAggregation
{
internal NormalizeAggregation() { }

public NormalizeAggregation(string name, SingleBucketsPath bucketsPath)
: base(name, bucketsPath) { }

internal override void WrapInContainer(AggregationContainer c) => c.Normalize = this;

/// <inheritdoc cref ="INormalizeAggregation.Method"/>
public NormalizeMethod Method { get; set; }
}

/// <inheritdoc cref="INormalizeAggregation"/>
public class NormalizeAggregationDescriptor
: PipelineAggregationDescriptorBase<NormalizeAggregationDescriptor, INormalizeAggregation, SingleBucketsPath>
, INormalizeAggregation
{
NormalizeMethod INormalizeAggregation.Method { get; set; }

/// <inheritdoc cref ="INormalizeAggregation.Method"/>
public NormalizeAggregationDescriptor Method(NormalizeMethod method) =>
Assign(method, (a, v) => a.Method = v);
}

[StringEnum]
public enum NormalizeMethod
{
/// <summary>
/// rescales the data such that the minimum number is zero, and the maximum number is 1, with the rest normalized linearly in-between.
/// </summary>
[EnumMember(Value = "rescale_0_1")]
RescaleZeroToOne,

/// <summary>
/// rescales the data such that the minimum number is zero, and the maximum number is 1, with the rest normalized linearly in-between.
/// </summary>
[EnumMember(Value = "rescale_0_100")]
RescaleZeroToOneHundred,

/// <summary>
/// normalizes each value so that it represents a percentage of the total sum it attributes to.
/// </summary>
[EnumMember(Value = "percent_of_sum")]
PercentOfSum,

/// <summary>
/// normalizes such that each value is normalized by how much it differs from the average.
/// </summary>
[EnumMember(Value = "mean")]
Mean,

/// <summary>
/// normalizes such that each value represents how far it is from the mean relative to the standard deviation
/// </summary>
[EnumMember(Value = "zscore")]
Zscore,

/// <summary>
/// normalizes such that each value is exponentiated and relative to the sum of the exponents of the original values.
/// </summary>
[EnumMember(Value = "softmax")]
Softmax
}
}
4 changes: 4 additions & 0 deletions src/Nest/Aggregations/Visitor/AggregationVisitor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ public interface IAggregationVisitor

void Visit(INestedAggregation aggregation);

void Visit(INormalizeAggregation aggregation);

void Visit(IParentAggregation aggregation);

void Visit(IReverseNestedAggregation aggregation);
Expand Down Expand Up @@ -233,6 +235,8 @@ public virtual void Visit(IRareTermsAggregation aggregation) { }

public virtual void Visit(INestedAggregation aggregation) { }

public virtual void Visit(INormalizeAggregation aggregation) { }

public virtual void Visit(IParentAggregation aggregation) { }

public virtual void Visit(ICardinalityAggregation aggregation) { }
Expand Down
4 changes: 4 additions & 0 deletions src/Nest/Aggregations/Visitor/AggregationWalker.cs
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,10 @@ public void Walk(IAggregationContainer aggregation, IAggregationVisitor visitor)
v.Visit(d);
Accept(v, d.Aggregations);
});
AcceptAggregation(aggregation.Normalize, visitor, (v, d) =>
{
v.Visit(d);
});
AcceptAggregation(aggregation.Parent, visitor, (v, d) =>
{
v.Visit(d);
Expand Down
Loading