Add test execution at the end of build process

[masesdevelopers]

Is your feature request related to a problem? Please describe.
Currently no tests are executed.

Describe the solution you'd like
The build workflow can execute some unit tests to verify execution.
Meanwhile the test projects can be enhanced to use the Microsoft.NET.Test.Sdk and/or Microsoft.TestPlatform.TestHost

Describe alternatives you've considered
N/A

Additional context
N/A

[masesdevelopers]

For MacOS and Windows try to start local ZooKeeper and Apache Kafka broker then execute the tests

[masesdevelopers]

The test execution reports:

00::44::23:8223 - System.InvalidOperationException: BridgeInstance is null.
   at MASES.JCOBridge.C2JBridge.JVMBridgeException.IExecute[TReturn](String methodName, Object[] args)
   at Java.Lang.Throwable.get_Message()
   at MASES.EntityFrameworkCore.KNet.Storage.Internal.KafkaCluster.CreateTable(IEntityType entityType, Int32 cycle) in /_/src/net/KEFCore/Storage/Internal/KafkaCluster.cs:line 230

[masesdevelopers]

The test execution reports:

00::44::23:8223 - System.InvalidOperationException: BridgeInstance is null.
   at MASES.JCOBridge.C2JBridge.JVMBridgeException.IExecute[TReturn](String methodName, Object[] args)
   at Java.Lang.Throwable.get_Message()
   at MASES.EntityFrameworkCore.KNet.Storage.Internal.KafkaCluster.CreateTable(IEntityType entityType, Int32 cycle) in /_/src/net/KEFCore/Storage/Internal/KafkaCluster.cs:line 230

Add a check in

KEFCore/src/net/KEFCore/Storage/Internal/KafkaCluster.cs

Line 230 in 9312fea

if (ex.Message.Contains("deletion"))

to verify if ex.BridgeInstance is not null, otherwise cast to JVMBridgeException and retrieve Message. The problem comes from masesgroup/JNet#429 and how Java.Lang.Throwable manages the information.

[masesdevelopers]

Within https://github.com/masesgroup/KEFCore/actions/runs/9655461268 many tests fails. Here the summary:

• .NET: net8.0
• OS
  • Windows
    • JDK: Corretto 21
  • Linux
    • Oracle 21
    • Zulu 21
    • Temurin 21
    • Microsoft 21
  • MacOS 13 and MacOS 14 (macos-latest)
    • Corretto 21
    • Microsoft 21
    • Temurin 21
    • Oracle 21
    • Zulu 21

The errors occurs always in net8.0, and sparsely for many JDK 21, when the array of bytes of key related to the Kafka record is retrieved from the JVM.
However the problem is raised in many way within KNet code, so the issue shall be solved there.

[masesdevelopers]

Within https://github.com/masesgroup/KEFCore/actions/runs/9655461268 many tests fails. Here the summary:

* .NET: net8.0

* OS
  
  * Windows
    
    * JDK: Corretto 21
  * Linux
    
    * Oracle 21
    * Zulu 21
    * Temurin 21
    * Microsoft 21
  * MacOS 13 and MacOS 14 (macos-latest)
    
    * Corretto 21
    * Microsoft 21
    * Temurin 21
    * Oracle 21
    * Zulu 21

The errors occurs always in net8.0, and sparsely for many JDK 21, when the array of bytes of key related to the Kafka record is retrieved from the JVM. However the problem is raised in many way within KNet code, so the issue shall be solved there.

Executing other tests, some exceptions are raised within KEFCore itself:

Unhandled exception. System.Text.Json.JsonException: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
 ---> System.Text.Json.JsonReaderException: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true. LineNumber: 0 | BytePositionInLine: 0.
   at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
   at System.Text.Json.Utf8JsonReader.Read()
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](ReadOnlySpan`1 utf8Json, JsonSerializerOptions options)
   at MASES.EntityFrameworkCore.KNet.Serialization.Json.DefaultKEFCoreSerDes.ValueContainer`1.JsonRaw`1.DeserializeWithHeaders(String topic, Headers headers, Byte[] data) in /_/src/net/KEFCore.SerDes/DefaultKEFCoreSerDes.cs:line 318
   at MASES.EntityFrameworkCore.KNet.Serialization.Json.DefaultKEFCoreSerDes.ValueContainer`1.JsonRaw`1.Deserialize(String topic, Byte[] data) in /_/src/net/KEFCore.SerDes/DefaultKEFCoreSerDes.cs:line 313
   at MASES.KNet.Streams.KeyValue`4..ctor(IGenericSerDesFactory factory, KeyValue`2 value, ISerDes`2 keySerDes, ISerDes`2 valueSerDes, Boolean fromPrefetched)
   at MASES.KNet.Streams.State.KeyValueIterator`4.PrefetchableLocalEnumerator.ConvertObject(Object input)
   at MASES.JCOBridge.C2JBridge.JVMBridgeBasePrefetchableEnumerator.Advance()
   at MASES.JCOBridge.C2JBridge.JVMBridgeBasePrefetchableEnumerator.Execute(Object o)
   at System.Threading.Thread.StartHelper.Callback(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)

and the process fails due to unhandled exception in the thread used to prefetch the information. The specific exception is System.Text.Json.JsonReaderException which reports LineNumber: 0 | BytePositionInLine: 0: seems the data received does not contains information, i.e. the array of bytes is empty.

The previous condition pairs with the following:

Unhandled exception. System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'startIndex')
   at System.BitConverter.ToInt32(Byte[] value, Int32 startIndex)
   at MASES.KNet.Serialization.KNetSerialization.DeserializeInt(Boolean fallbackToKafka, String topic, Byte[] data)
   at MASES.KNet.Serialization.SerDes`2.Deserialize(String topic, TJVMT data)
   at MASES.KNet.Streams.KeyValue`4..ctor(IGenericSerDesFactory factory, KeyValue`2 value, ISerDes`2 keySerDes, ISerDes`2 valueSerDes, Boolean fromPrefetched)
   at MASES.KNet.Streams.State.KeyValueIterator`4.PrefetchableLocalEnumerator.ConvertObject(Object input)
   at MASES.JCOBridge.C2JBridge.JVMBridgeBasePrefetchableEnumerator.Advance()
   at MASES.JCOBridge.C2JBridge.JVMBridgeBasePrefetchableEnumerator.Execute(Object o)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)

where System.ArgumentOutOfRangeException most probably reports that the value input of System.BitConverter.ToInt32 has a length of zero and MASES.KNet.Serialization.KNetSerialization.DeserializeInt try to start from zero: see explanation in https://learn.microsoft.com/it-it/dotnet/api/system.bitconverter.toint32?view=net-8.0#system-bitconverter-toint32(system-byte()-system-int32)

The first kind of exception shall be managed in KEFCore, while the second one is related to KNet.

[masesdevelopers]

Windows is not a supported OS to run Apache Kafka and many tests fails on Windows because the broker, or ZooKeeper, closes.

Try use Docker, only on Windows, since it seems to be installed in Windows 2022 image, the command shall be something like:

docker run -e KNET_DOCKER_RUNNING_MODE=server -p 9092:9092 ghcr.io/masesgroup/knet:latest 

maybe the access to ghcr.io will be forbidden, so it is needed an authentication.

[masesdevelopers]

Windows is not a supported OS to run Apache Kafka and many tests fails on Windows because the broker, or ZooKeeper, closes.

Try use Docker, only on Windows, since it seems to be installed in Windows 2022 image, the command shall be something like:

docker run -e KNET_DOCKER_RUNNING_MODE=server -p 9092:9092 ghcr.io/masesgroup/knet:latest 

maybe the access to ghcr.io will be forbidden, so it is needed an authentication.

The docker version available in Windows runner cannot activate linux images due to virtualization limitation, so there is no chance to use this method.

To avoid problems on OSes not supporting Apache Kafka maybe it is mandatory to activate an external container/VM, but the network can be a bottleneck.

[masesdevelopers]

Some tests fails for other reason than the previous: specifically under windows and with JDK 17. Looking at the available dumps, sometime there is an access violation during preparation of the KNetSerialization class. However, trying to reproduce the issue locally, the exception is not raised; between GitHub and local test the differences are:

• OS: Windows Server for GitHub, Windows Desktop in local
• JDK: locally tested only with a Temurin JDK 17
• .NET 6/8: both at latest version

It is possible to highlight that KNetReplicator never raises such exception and the exception is not raised from the same test even if the order of execution is always the same.