Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Akka.Cluster.Metrics extension implementation #4126

Merged
merged 87 commits into from
Jan 27, 2020
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
2b7f153
Created project
IgorFedchenko Dec 20, 2019
d323eff
Updated proto to syntax3
IgorFedchenko Dec 20, 2019
29a3264
Generated messages file
IgorFedchenko Dec 24, 2019
f6a2ae6
Implemented collector builder
IgorFedchenko Dec 24, 2019
48682e1
Implemented new classes and interfaces
IgorFedchenko Dec 25, 2019
4ce8099
Minor fixes
IgorFedchenko Dec 25, 2019
71f6381
Implemented routing, metric selectors, fixed metric values storage
IgorFedchenko Dec 26, 2019
19ff99a
Fixed routing, added messages serialization (not complete)
IgorFedchenko Dec 26, 2019
9a52276
Merge branch 'dev' into cluster-metrics
IgorFedchenko Dec 26, 2019
afa8365
Kick off .net452 target
IgorFedchenko Dec 27, 2019
33624fc
Implemented messages serialization and other fixes and refactorings
IgorFedchenko Dec 27, 2019
0f806c6
Added dummy implementation for compression with TODO label
IgorFedchenko Dec 27, 2019
fe324fb
Added and fixed configuration spec
IgorFedchenko Dec 27, 2019
7f4e983
Added and fixed messages serialization
IgorFedchenko Dec 28, 2019
b42644a
Refactoring and fixes
IgorFedchenko Dec 28, 2019
0ef34ed
DynamicAccess fix
IgorFedchenko Dec 28, 2019
d6ac8dd
Added generation script for proto message types
IgorFedchenko Dec 28, 2019
5f078e5
Reusing AddressData protobuf message
IgorFedchenko Dec 28, 2019
39e5171
Fixed serializer configuration
IgorFedchenko Dec 28, 2019
a43e76b
Renamed Option.ForEach to Option.OnSuccess
IgorFedchenko Dec 28, 2019
a9ec08e
Merge branch 'dev' into cluster-metrics
IgorFedchenko Dec 28, 2019
e45c73c
Added collection spec + fixed implementation
IgorFedchenko Dec 30, 2019
dd84077
Added specs and applied fixes for MetricsSelectors
IgorFedchenko Dec 30, 2019
e5dcf83
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 1, 2020
5476934
Added new specs and fixes
IgorFedchenko Jan 3, 2020
3ca5bf6
More fixes
IgorFedchenko Jan 3, 2020
5e24ecf
Added spec and fixed weighted routees
IgorFedchenko Jan 3, 2020
af5cb25
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 3, 2020
93b88b1
Implemented DefaultCollector and updated metric names closer to .NET
IgorFedchenko Jan 5, 2020
6606492
Added specs and minor fixes for metrics collector
IgorFedchenko Jan 5, 2020
cab575e
Added MNTR tests (not all passing)
IgorFedchenko Jan 6, 2020
3087efb
MNTR tests configuration fix
IgorFedchenko Jan 6, 2020
52f04d4
Made DynamicAccess marked as InternalApi
IgorFedchenko Jan 6, 2020
295d699
Marked DynamicAccess methods with InternalApi
IgorFedchenko Jan 6, 2020
6825b2d
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 6, 2020
6da830a
Reflection and configuration fixes
IgorFedchenko Jan 6, 2020
da13922
MNTR specs fix
IgorFedchenko Jan 7, 2020
727220b
Merge remote-tracking branch 'origin/cluster-metrics' into cluster-me…
IgorFedchenko Jan 7, 2020
b84f86a
MNTR StatsSampleSpec fix (now passing)
IgorFedchenko Jan 7, 2020
f1a044d
MNTR ClusterMetricsEnabledSpec fix
IgorFedchenko Jan 7, 2020
f1c5386
MNTR fix from #4139
IgorFedchenko Jan 9, 2020
a390887
Fixed memory use metric
IgorFedchenko Jan 9, 2020
ac4d12e
AwaitAssertAsync fix for async assertion
IgorFedchenko Jan 10, 2020
0fe2955
MNTR option to clear output folder
IgorFedchenko Jan 10, 2020
df32600
Some fixes
IgorFedchenko Jan 10, 2020
079bcac
Fix props creation in routing
IgorFedchenko Jan 10, 2020
eb05abc
Failed specs fixes
IgorFedchenko Jan 10, 2020
0572cf4
Added API approval
IgorFedchenko Jan 11, 2020
29950c0
Dispose process objects after loading metrics
IgorFedchenko Jan 11, 2020
4f7a51e
Merge branch 'dev' into cluster-metrics
Aaronontheweb Jan 13, 2020
8ddb1a5
Minor fixes and refactoring
IgorFedchenko Jan 14, 2020
4ef2e8c
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 14, 2020
27b68e4
Updated API approval
IgorFedchenko Jan 14, 2020
454ca47
Fixed typo in reference.conf
IgorFedchenko Jan 15, 2020
e6bee49
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 15, 2020
8fa0fd5
Akka.Remove protobuf AddressData payload reuse
IgorFedchenko Jan 15, 2020
b4f6996
Merge branch 'cluster-metrics-address-reuse' into cluster-metrics
IgorFedchenko Jan 15, 2020
878fe84
API approvement
IgorFedchenko Jan 16, 2020
18be9ee
Refactored test
IgorFedchenko Jan 16, 2020
57f371e
Using GC.GetTotalMemory instead of Process.PrivateMemorySize64
IgorFedchenko Jan 16, 2020
0223f66
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 16, 2020
ff70a40
Fix CPU collection timings tracking
IgorFedchenko Jan 17, 2020
dfea281
DefaultCollector sample time tracking start/stop fix
IgorFedchenko Jan 20, 2020
22018bb
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 20, 2020
188e025
Increased metric sampling timeouts
IgorFedchenko Jan 20, 2020
ecda6ea
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 22, 2020
78b0109
Refactored test to be less racy
IgorFedchenko Jan 22, 2020
98b86da
Merge remote-tracking branch 'origin/cluster-metrics' into cluster-me…
IgorFedchenko Jan 22, 2020
9a9e1d2
Added CPU load value limitation
IgorFedchenko Jan 22, 2020
0e1495a
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 23, 2020
c45c7ec
Disabled flaky performance test
IgorFedchenko Jan 23, 2020
8318488
CPU truncate fix
IgorFedchenko Jan 23, 2020
55f2ed7
Merge branch 'dev' into cluster-metrics
Aaronontheweb Jan 23, 2020
d7f4ae3
Merge branch 'dev' into cluster-metrics
Aaronontheweb Jan 23, 2020
7ec40f3
Merge branch 'dev' into cluster-metrics
Aaronontheweb Jan 23, 2020
6bfb91d
Merge branch 'dev' into cluster-metrics
Aaronontheweb Jan 24, 2020
85c0d28
Fixed Protobuf payload class visability
IgorFedchenko Jan 25, 2020
2c4006c
Disabled invalid racy MNTR test
IgorFedchenko Jan 25, 2020
b78dc52
Increased timeout for cluster to be up
IgorFedchenko Jan 25, 2020
9e50bef
Merge branch 'dev' into cluster-metrics
IgorFedchenko Jan 25, 2020
990d19e
One more Protobuf payload visability fix
IgorFedchenko Jan 25, 2020
fdd27e1
Increased timeout even more
IgorFedchenko Jan 25, 2020
6afac7e
Refactored test to be more rebust to timings
IgorFedchenko Jan 25, 2020
f7e4432
Using local public AddressData structure
IgorFedchenko Jan 27, 2020
62f7684
MNTR clear-output option fix
IgorFedchenko Jan 27, 2020
61ef434
Fixed issue with MNTR spec
Aaronontheweb Jan 27, 2020
36ac347
Updated API approvement
IgorFedchenko Jan 27, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions src/Akka.sln
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Akka.Docs.Tests", "core\Akk
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Akka.Docs.Tutorials", "core\Akka.Docs.Tutorials\Akka.Docs.Tutorials.csproj", "{9AE636F2-988C-42E4-9B40-FF1F7177DF91}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Akka.Cluster.Metrics", "contrib\cluster\Akka.Cluster.Metrics\Akka.Cluster.Metrics.csproj", "{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}"
EndProject
Global
GlobalSection(Performance) = preSolution
HasPerformanceSessions = true
Expand Down Expand Up @@ -909,6 +911,18 @@ Global
{9AE636F2-988C-42E4-9B40-FF1F7177DF91}.Release|x64.Build.0 = Release|Any CPU
{9AE636F2-988C-42E4-9B40-FF1F7177DF91}.Release|x86.ActiveCfg = Release|Any CPU
{9AE636F2-988C-42E4-9B40-FF1F7177DF91}.Release|x86.Build.0 = Release|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Debug|Any CPU.Build.0 = Debug|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Debug|x64.ActiveCfg = Debug|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Debug|x64.Build.0 = Debug|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Debug|x86.ActiveCfg = Debug|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Debug|x86.Build.0 = Debug|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Release|Any CPU.ActiveCfg = Release|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Release|Any CPU.Build.0 = Release|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Release|x64.ActiveCfg = Release|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Release|x64.Build.0 = Release|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Release|x86.ActiveCfg = Release|Any CPU
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -998,6 +1012,7 @@ Global
{6F8FECD6-6E39-473E-9B9A-9EE22CBF479F} = {01167D3C-49C4-4CDE-9787-C176D139ACDD}
{159F6312-3F12-4A84-9E7C-0A4B4FD72D1E} = {01167D3C-49C4-4CDE-9787-C176D139ACDD}
{9AE636F2-988C-42E4-9B40-FF1F7177DF91} = {01167D3C-49C4-4CDE-9787-C176D139ACDD}
{1E26E439-C9E1-4DBE-A783-D5FE31FCDD6F} = {76F58DC4-19F1-43EF-A6E2-EC1CC8395AC5}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {03AD8E21-7507-4E68-A4E9-F4A7E7273164}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
<Project Sdk="Microsoft.NET.Sdk">
<Import Project="..\..\..\common.props" />

<PropertyGroup>
<AssemblyTitle>Akka.Cluster.Metrics</AssemblyTitle>
<Description>
The member nodes of the cluster can collect system health metrics and publish that to other cluster nodes
and to the registered subscribers on the system event bus with the help of Cluster Metrics Extension.
</Description>
<VersionSuffix>beta</VersionSuffix>
<TargetFramework>$(NetStandardLibVersion)</TargetFramework>
<PackageTags>$(AkkaPackageTags);network;cluster;sharding</PackageTags>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Google.Protobuf" Version="3.11.2" />
IgorFedchenko marked this conversation as resolved.
Show resolved Hide resolved
</ItemGroup>

<ItemGroup Condition=" '$(TargetFramework)' == '$(NetFrameworkLibVersion)' ">
<Reference Include="System.Core" />
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\..\..\core\Akka.Cluster\Akka.Cluster.csproj" />
<ProjectReference Include="..\..\..\core\Akka\Akka.csproj" />
IgorFedchenko marked this conversation as resolved.
Show resolved Hide resolved
</ItemGroup>

<PropertyGroup Condition=" '$(TargetFramework)' == '$(NetFrameworkLibVersion)' ">
IgorFedchenko marked this conversation as resolved.
Show resolved Hide resolved
<DefineConstants>$(DefineConstants);SERIALIZATION</DefineConstants>
</PropertyGroup>

<PropertyGroup Condition=" '$(TargetFramework)' == '$(NetStandardLibVersion)' ">
<DefineConstants>$(DefineConstants);CORECLR</DefineConstants>
</PropertyGroup>

<PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
<DefineConstants>$(DefineConstants);RELEASE</DefineConstants>
</PropertyGroup>

</Project>
124 changes: 124 additions & 0 deletions src/contrib/cluster/Akka.Cluster.Metrics/ClusterMetrics.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
// //-----------------------------------------------------------------------
// // <copyright file="ClusterMetricsExtension.cs" company="Akka.NET Project">
// // Copyright (C) 2009-2019 Lightbend Inc. <http://www.lightbend.com>
// // Copyright (C) 2013-2019 .NET Foundation <https://github.com/akkadotnet/akka.net>
// // </copyright>
// //-----------------------------------------------------------------------

using System;
using Akka.Actor;
using Akka.Annotations;
using Akka.Cluster.Metrics.Configuration;
using Akka.Cluster.Metrics.Events;
using Akka.Cluster.Metrics.Helpers;
using Akka.Cluster.Metrics.Serialization;

namespace Akka.Cluster.Metrics
{
/// <summary>
/// Cluster metrics extension.
///
/// Cluster metrics is primarily for load-balancing of nodes. It controls metrics sampling
/// at a regular frequency, prepares highly variable data for further analysis by other entities,
/// and publishes the latest cluster metrics data around the node ring and local eventStream
/// to assist in determining the need to redirect traffic to the least-loaded nodes.
///
/// Metrics sampling is delegated to the <see cref="IMetricsCollector"/>.
///
/// Smoothing of the data for each monitored process is delegated to the
/// <see cref="NodeMetrics.Types.EWMA"/> for exponential weighted moving average.
/// </summary>
public class ClusterMetrics : IExtension
{
private readonly ExtendedActorSystem _system;

/// <summary>
/// Cluster metrics settings
/// </summary>
public ClusterMetricsSettings Settings { get; }

/// <summary>
/// Creates new <see cref="ClusterMetrics"/> for given actor system
/// </summary>
/// <param name="system"></param>
public ClusterMetrics(ExtendedActorSystem system)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this private - force users to call the ClusterMetrics.Get(ActorSystem) method so we can guarantee a maximum of one ClusterMetrics instance per-ActorSystem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made this internal since it is used in ClusterMetricsExtensionProvider.

{
_system = system;
Settings = ClusterMetricsSettings.Create(system.Settings.Config);

Supervisor = _system.SystemActorOf(
Props.Create<ClusterMetricsSupervisor>().WithDispatcher(Settings.MetricsDispatcher).WithDeploy(Deploy.Local),
Settings.SupervisorName);
}

/// <summary>
/// Creates new <see cref="ClusterMetrics"/> for given actor system
/// </summary>
public static ClusterMetrics Get(ActorSystem system)
{
return system.WithExtension<ClusterMetrics, ClusterMetricsExtensionProvider>();
}

/// <summary>
/// INTERNAL API
///
/// Supervision strategy.
/// </summary>
[InternalApi]
public ClusterMetricsStrategy Strategy
{
get
{
return DynamicAccess.CreateInstanceFor<ClusterMetricsStrategy>(Settings.SupervisorStrategyProvider)
.GetOrElse(() =>
{
_system.Log.Error(
$"Configured strategy provider {Settings.SupervisorStrategyProvider} failed to load, " +
$"using default {typeof(ClusterMetricsStrategy).Name}.");
return new ClusterMetricsStrategy(Settings.SupervisorStrategyConfiguration);
})
.Get();
}
}

/// <summary>
/// Supervisor actor.
/// Accepts subtypes of <see cref="ClusterMetricsSupervisorMetadata.ICollectionControlMessage"/> to manage metrics collection at runtime
/// </summary>
public IActorRef Supervisor { get; }

/// <summary>
/// Subscribe user metrics listener actor unto <see cref="IClusterMetricsEvent"/>
/// events published by extension on the system event bus.
/// </summary>
public void Subscribe(IActorRef metricsListener)
{
_system.EventStream.Subscribe(metricsListener, typeof(IClusterMetricsEvent));
}

/// <summary>
/// Unubscribe user metrics listener actor from <see cref="IClusterMetricsEvent"/>
/// events published by extension on the system event bus.
/// </summary>
/// <param name="metricsListener"></param>
public void Unsubscribe(IActorRef metricsListener)
{
_system.EventStream.Unsubscribe(metricsListener, typeof(IClusterMetricsEvent));
}
}

/// <summary>
/// ClusterMetricsExtensionProvider
/// </summary>
public class ClusterMetricsExtensionProvider: ExtensionIdProvider<ClusterMetrics>
{
/// <summary>
/// Creates new <see cref="ClusterMetrics"/> for given actor system
/// </summary>
public override ClusterMetrics CreateExtension(ExtendedActorSystem system)
{
var extension = new ClusterMetrics(system);
return extension;
}
}
}
Loading