Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add generalized crossplatform support for Hyperion serializer. #4878

Merged
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions docs/articles/networking/serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,3 +303,80 @@ akka {
}
}
```

## Cross platform serialization compatibility in Hyperion
There are problems that can arise when migrating from old .NET Framework to the new .NET Core standard, mainly because of breaking namespace and assembly name changes between these platforms.
Hyperion implements a generic way of addressing this issue by transforming the names of these incompatible names during deserialization.

There are two ways to set this up, one through the HOCON configuration file, and the other by using the `HyperionSerializerSetup` class.

> [!NOTE]
> Only the first successful name transformation is applied, the rest are ignored.
> If you are matching several similar names, make sure that you order them from the most specific match to the least specific one.

### HOCON
HOCON example:
```
akka.actor.serialization-settings.hyperion.cross-platform-package-name-overrides = {
netfx = [
{
fingerprint = "System.Private.CoreLib,%core%",
rename-from = "System.Private.CoreLib,%core%",
rename-to = "mscorlib,%core%"
}]
netcore = [
{
fingerprint = "mscorlib,%core%",
rename-from = "mscorlib,%core%",
rename-to = "System.Private.CoreLib,%core%"
}]
net = [
{
fingerprint = "mscorlib,%core%",
rename-from = "mscorlib,%core%",
rename-to = "System.Private.CoreLib,%core%"
}]
}
```

In the example above, we're addressing the classic case where the core library name was changed between `mscorlib` in .NET Framework to `System.Private.CoreLib` in .NET Core.
This transform is already included inside Hyperion as the default cross platform support, and used here as an illustration only.

The HOCON configuration section is composed of three object arrays named `netfx`, `netcore`, and `net`, each corresponds, respectively, to .NET Framework, .NET Core, and the new .NET 5.0 and beyond.
The Hyperion serializer will automatically detects the platform it is running on currently and uses the correct array to use inside its deserializer. For example, if Hyperion detects
that it is running under .NET framework, then it will use the `netfx` array to do its deserialization transformation.

The way it works that when the serializer detects that the type name contains the `fingerprint` string, it will replace the string declared in the `rename-from`
property into the string declared in the `rename-to`.

In code, we can write this behaviour as:
```csharp
if(packageName.Contains(fingerprint)) packageName = packageName.Replace(rename-from, rename-to);
```

### HyperionSerializerSetup

This behaviour can also be implemented programatically by providing a `HyperionSerializerSetup` instance during `ActorSystem` creation.

```csharp
#if NETFRAMEWORK
var hyperionSetup = HyperionSerializerSetup.Empty
.WithPackageNameOverrides(new Func<string, string>[]
{
str => str.Contains("System.Private.CoreLib,%core%")
? str.Replace("System.Private.CoreLib,%core%", "mscorlib,%core%") : str
}
#elif NETCOREAPP
var hyperionSetup = HyperionSerializerSetup.Empty
.WithPackageNameOverrides(new Func<string, string>[]
{
str => str.Contains("mscorlib,%core%")
? str.Replace("mscorlib,%core%", "System.Private.CoreLib,%core%") : str
}
#endif

var bootstrap = BootstrapSetup.Create().And(hyperionSetup);
var system = ActorSystem.Create("actorSystem", bootstrap);
```

In the example above, we're using compiler directives to make sure that the correct name transform are used during compilation.
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,61 @@ public void Hyperion_serializer_should_allow_to_setup_custom_types_provider_with
Assert.Equal(typeof(DummyTypesProvider), serializer.Settings.KnownTypesProvider);
}
}

[Fact]
public void Hyperion_serializer_should_read_cross_platform_package_name_override_settings()
{
var config = ConfigurationFactory.ParseString(@"
akka.actor {
serializers.hyperion = ""Akka.Serialization.HyperionSerializer, Akka.Serialization.Hyperion""
serialization-bindings {
""System.Object"" = hyperion
}
serialization-settings.hyperion {
cross-platform-package-name-overrides = {
netfx = [
{
fingerprint = ""a"",
rename-from = ""b"",
rename-to = ""c""
}]
netcore = [
{
fingerprint = ""d"",
rename-from = ""e"",
rename-to = ""f""
}]
net = [
{
fingerprint = ""g"",
rename-from = ""h"",
rename-to = ""i""
}]
}
}
}
");
using (var system = ActorSystem.Create(nameof(HyperionConfigTests), config))
{
var serializer = (HyperionSerializer)system.Serialization.FindSerializerForType(typeof(object));
var overrides = serializer.Settings.PackageNameOverrides.ToList();
Assert.NotEmpty(overrides);
var @override = overrides[0];

#if NET471
Assert.Equal("acc", @override("abc"));
Assert.Equal("bcd", @override("bcd"));
#elif NETCOREAPP3_1
Assert.Equal("dff", @override("def"));
Assert.Equal("efg", @override("efg"));
#elif NET5_0
Assert.Equal("gii", @override("ghi"));
Assert.Equal("hij", @override("hij"));
#else
throw new Exception("Test can not be completed because no proper compiler directive is set for this test build");
#endif
}
}
}

class DummyTypesProvider : IKnownTypesProvider
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Akka.Actor;
using Akka.Configuration;
using Akka.TestKit;
using Xunit;
using Xunit.Abstractions;
using FluentAssertions;

namespace Akka.Serialization.Hyperion.Tests
{
public class HyperionSerializerSetupSpec : AkkaSpec
{
private static Config Config
=> ConfigurationFactory.ParseString(@"
akka.actor {
serializers {
hyperion = ""Akka.Serialization.Hyperion, Akka.Serialization.Hyperion""
}

serialization-bindings {
""System.Object"" = hyperion
}
}
");

public HyperionSerializerSetupSpec(ITestOutputHelper output) : base (Config, output)
{ }

[Fact]
public void Setup_should_be_converted_to_settings_correctly()
{
var setup = HyperionSerializerSetup.Empty
.WithPreserveObjectReference(true)
.WithKnownTypeProvider<NoKnownTypes>();
var settings =
new HyperionSerializerSettings(false, false, typeof(DummyTypesProvider), new Func<string, string>[] { s => $"{s}.." });
var appliedSettings = setup.ApplySettings(settings);

appliedSettings.PreserveObjectReferences.Should().BeTrue(); // overriden
appliedSettings.VersionTolerance.Should().BeFalse(); // default
appliedSettings.KnownTypesProvider.Should().Be(typeof(NoKnownTypes)); // overriden
appliedSettings.PackageNameOverrides.Count().Should().Be(1); // from settings
appliedSettings.PackageNameOverrides.First()("a").Should().Be("a..");
}

[Fact]
public void Setup_package_override_should_work()
{
var setup = HyperionSerializerSetup.Empty
.WithPackageNameOverrides(new Func<string, string>[]
{
s => s.Contains("Hyperion.Override")
? s.Replace(".Override", "")
: s
});

var settings = HyperionSerializerSettings.Default;
var appliedSettings = setup.ApplySettings(settings);

var adapter = appliedSettings.PackageNameOverrides.First();
adapter("My.Hyperion.Override").Should().Be("My.Hyperion");
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,18 @@
//-----------------------------------------------------------------------

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Runtime.Serialization;
using Akka.Actor;
using Akka.Configuration;
using Akka.Serialization.Hyperion;
using Akka.Util;
using Hyperion;
using HySerializer = Hyperion.Serializer;

// ReSharper disable once CheckNamespace
namespace Akka.Serialization
Expand All @@ -28,7 +32,7 @@ public class HyperionSerializer : Serializer
/// </summary>
public readonly HyperionSerializerSettings Settings;

private readonly Hyperion.Serializer _serializer;
private readonly HySerializer _serializer;

/// <summary>
/// Initializes a new instance of the <see cref="HyperionSerializer"/> class.
Expand Down Expand Up @@ -57,7 +61,7 @@ public HyperionSerializer(ExtendedActorSystem system, Config config)
public HyperionSerializer(ExtendedActorSystem system, HyperionSerializerSettings settings)
: base(system)
{
this.Settings = settings;
Settings = settings;
var akkaSurrogate =
Surrogate
.Create<ISurrogated, ISurrogate>(
Expand All @@ -66,13 +70,22 @@ public HyperionSerializer(ExtendedActorSystem system, HyperionSerializerSettings

var provider = CreateKnownTypesProvider(system, settings.KnownTypesProvider);

if (system != null)
{
var settingsSetup = system.Settings.Setup.Get<HyperionSerializerSetup>()
.GetOrElse(HyperionSerializerSetup.Empty);

settingsSetup.ApplySettings(Settings);
}

_serializer =
new Hyperion.Serializer(new SerializerOptions(
new HySerializer(new SerializerOptions(
preserveObjectReferences: settings.PreserveObjectReferences,
versionTolerance: settings.VersionTolerance,
surrogates: new[] { akkaSurrogate },
knownTypes: provider.GetKnownTypes(),
ignoreISerializable:true));
ignoreISerializable:true,
packageNameOverrides: settings.PackageNameOverrides));
}

/// <summary>
Expand Down Expand Up @@ -156,7 +169,8 @@ public sealed class HyperionSerializerSettings
public static readonly HyperionSerializerSettings Default = new HyperionSerializerSettings(
preserveObjectReferences: true,
versionTolerance: true,
knownTypesProvider: typeof(NoKnownTypes));
knownTypesProvider: typeof(NoKnownTypes),
packageNameOverrides: new List<Func<string, string>>());

/// <summary>
/// Creates a new instance of <see cref="HyperionSerializerSettings"/> using provided HOCON config.
Expand All @@ -174,15 +188,42 @@ public sealed class HyperionSerializerSettings
public static HyperionSerializerSettings Create(Config config)
{
if (config.IsNullOrEmpty())
throw ConfigurationException.NullOrEmptyConfig<HyperionSerializerSettings>("akka.serializers.hyperion");
throw ConfigurationException.NullOrEmptyConfig<HyperionSerializerSettings>("akka.actor.serialization-settings.hyperion");

var typeName = config.GetString("known-types-provider", null);
var type = !string.IsNullOrEmpty(typeName) ? Type.GetType(typeName, true) : null;

var framework = RuntimeInformation.FrameworkDescription;
string frameworkKey;
if (framework.Contains(".NET Framework"))
frameworkKey = "netfx";
else if (framework.Contains(".NET Core"))
frameworkKey = "netcore";
else
frameworkKey = "net";

var packageNameOverrides = new List<Func<string, string>>();
var overrideConfigs = config.GetValue($"cross-platform-package-name-overrides.{frameworkKey}");
if (overrideConfigs != null)
{
var configs = overrideConfigs.GetArray().Select(value => value.GetObject());
foreach (var obj in configs)
{
var fingerprint = obj.GetKey("fingerprint").GetString();
var renameFrom = obj.GetKey("rename-from").GetString();
var renameTo = obj.GetKey("rename-to").GetString();
packageNameOverrides.Add(packageName =>
packageName.Contains(fingerprint)
? packageName.Replace(renameFrom, renameTo)
: packageName);
}
}

return new HyperionSerializerSettings(
preserveObjectReferences: config.GetBoolean("preserve-object-references", true),
versionTolerance: config.GetBoolean("version-tolerance", true),
knownTypesProvider: type);
knownTypesProvider: type,
packageNameOverrides: packageNameOverrides);
}

/// <summary>
Expand All @@ -207,14 +248,37 @@ public static HyperionSerializerSettings Create(Config config)
/// </summary>
public readonly Type KnownTypesProvider;

/// <summary>
/// A list of lambda functions, used to transform incoming deserialized
/// package names before they are instantiated
/// </summary>
public readonly IEnumerable<Func<string, string>> PackageNameOverrides;

/// <summary>
/// Creates a new instance of a <see cref="HyperionSerializerSettings"/>.
/// </summary>
/// <param name="preserveObjectReferences">Flag which determines if serializer should keep track of references in serialized object graph.</param>
/// <param name="versionTolerance">Flag which determines if field data should be serialized as part of type manifest.</param>
/// <param name="knownTypesProvider">Type implementing <see cref="IKnownTypesProvider"/> to be used to determine a list of types implicitly known by all cooperating serializer.</param>
/// <exception cref="ArgumentException">Raised when `known-types-provider` type doesn't implement <see cref="IKnownTypesProvider"/> interface.</exception>
[Obsolete]
public HyperionSerializerSettings(bool preserveObjectReferences, bool versionTolerance, Type knownTypesProvider)
: this(preserveObjectReferences, versionTolerance, knownTypesProvider, new List<Func<string, string>>())
{ }

/// <summary>
/// Creates a new instance of a <see cref="HyperionSerializerSettings"/>.
/// </summary>
/// <param name="preserveObjectReferences">Flag which determines if serializer should keep track of references in serialized object graph.</param>
/// <param name="versionTolerance">Flag which determines if field data should be serialized as part of type manifest.</param>
/// <param name="knownTypesProvider">Type implementing <see cref="IKnownTypesProvider"/> to be used to determine a list of types implicitly known by all cooperating serializer.</param>
/// <param name="packageNameOverrides">TBD</param>
/// <exception cref="ArgumentException">Raised when `known-types-provider` type doesn't implement <see cref="IKnownTypesProvider"/> interface.</exception>
public HyperionSerializerSettings(
bool preserveObjectReferences,
bool versionTolerance,
Type knownTypesProvider,
IEnumerable<Func<string, string>> packageNameOverrides)
{
knownTypesProvider = knownTypesProvider ?? typeof(NoKnownTypes);
if (!typeof(IKnownTypesProvider).IsAssignableFrom(knownTypesProvider))
Expand All @@ -223,6 +287,7 @@ public HyperionSerializerSettings(bool preserveObjectReferences, bool versionTol
PreserveObjectReferences = preserveObjectReferences;
VersionTolerance = versionTolerance;
KnownTypesProvider = knownTypesProvider;
PackageNameOverrides = packageNameOverrides;
}
}
}
Loading