Skip to content

Commit

Permalink
Removed string from definition of KNet generic types (#64)
Browse files Browse the repository at this point in the history
* Fix documentation

* #59: removed ProducerByEntity because the new EntityTypeProducer must be used with each Entity

* Code cleanup

* Version upgrade
  • Loading branch information
masesdevelopers authored Oct 1, 2023
1 parent cc4f749 commit 295a9dd
Show file tree
Hide file tree
Showing 25 changed files with 475 additions and 278 deletions.
40 changes: 28 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,42 @@
# KEFCore: the EntityFrameworkCore provider for Apache Kafka
# KEFCore: Entity Framework Core provider for Apache Kafka

[![CI_BUILD](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml) [![CI_RELEASE](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml)
KEFCore is the Entity Framework Core provider for Apache Kafka.
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use Apache Kafka as a distributed database.

### Libraries and Tools

[![latest version](https://img.shields.io/nuget/v/MASES.EntityFrameworkCore.KNet)](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet) [![downloads](https://img.shields.io/nuget/dt/MASES.EntityFrameworkCore.KNet)](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet)

KEFCore is the EntityFrameworkCore provider for Apache Kafka.
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use Apache Kafka as a distributed database.
### Pipelines

This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
[![CI_BUILD](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml)
[![CI_RELEASE](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml)

---

## Scope of the project

This project aims to create a provider to access the information stored within an Apache Kafka cluster using the paradigm behind Entity Framework.
The project is based on available information within the official [EntityFrameworkCore repository](https://github.com/dotnet/efcore), many classes was copied from there as reported in the official documentation within the Microsoft website at https://docs.microsoft.com/en-us/ef/core/providers/writing-a-provider.

### Community and Contribution

Do you like the project?
- Request your free [community subscription](https://www.jcobridge.com/pricing-25/).

Do you want to help us?
- put a :star: on this project
- open [issues](https://github.com/masesgroup/KEFCore/issues) to request features or report bugs :bug:
- improves the project with Pull Requests

This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

## Summary

* [Roadmap](src/documentation/articles/roadmap.md)
* [Actual state](src/documentation/articles/actualstate.md)
* [KEFCore usage](src/documentation/articles/usage.md)

## Runtime engine

KEFCore uses [KNet](https://github.com/masesgroup/KNet), and indeed [JCOBridge](https://www.jcobridge.com) with its [features](https://www.jcobridge.com/features/), to obtain many benefits:
Expand All @@ -36,13 +59,6 @@ Have a look at the following JCOBridge resources:
- [Commercial Edition](https://www.jcobridge.com/pricing-25/)
- Latest release: [![JCOBridge nuget](https://img.shields.io/nuget/v/MASES.JCOBridge)](https://www.nuget.org/packages/MASES.JCOBridge)

---
## Summary

* [Roadmap](src/documentation/articles/roadmap.md)
* [Actual state](src/documentation/articles/actualstate.md)
* [KEFCore usage](src/documentation/articles/usage.md)

---

KAFKA is a registered trademark of The Apache Software Foundation. KEFCore has no affiliation with and is not endorsed by The Apache Software Foundation.
Expand Down
2 changes: 1 addition & 1 deletion src/documentation/articles/intro.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Welcome to KEFCore

KEFCore is the EntityFrameworkCore provider for Apache Kafka.
KEFCore is the Entity Framework Core provider for Apache Kafka.
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use Apache Kafka as a distributed database.

This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
1 change: 1 addition & 0 deletions src/documentation/articles/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ The roadmap can be synthetized in the following points:

* Create a first working provider based on InMemory provider
* Extends the first provider with new features able to create Apache Kafka Streams topology to retrieve information
* Use KNetCompactedReplicator beside Apache Kafka Streams
4 changes: 2 additions & 2 deletions src/documentation/articles/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@
### Installation

EF Core for Apache Kafka is available on [NuGet](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet):
Entity Framework Core provider for Apache Kafka is available on [NuGet](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet):

```sh
dotnet add package MASES.EntityFrameworkCore.KNet
```

### Basic usage

The following code demonstrates basic usage of EF Core for Apache Kafka.
The following code demonstrates basic usage of Entity Framework Core provider for Apache Kafka.
For a full tutorial configuring the `KafkaDbContext`, defining the model, and creating the database, see [KafkaDbContext](kafkadbcontext.md) in the docs.

```cs
Expand Down
4 changes: 2 additions & 2 deletions src/documentation/index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# KEFCore: the EntityFrameworkCore provider for Apache Kafka
# KEFCore: Entity Framework Core provider for Apache Kafka

[![CI_BUILD](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml) [![CI_RELEASE](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml)

KEFCore is the EntityFrameworkCore provider for Apache Kafka.
KEFCore is the Entity Framework Core provider for Apache Kafka.
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use Apache Kafka as a distributed database.

This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
Expand Down
2 changes: 1 addition & 1 deletion src/net/Common/Common.props
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<Owners>MASES s.r.l.</Owners>
<Authors>MASES s.r.l.</Authors>
<Company>MASES s.r.l.</Company>
<Version>0.7.2.0</Version>
<Version>0.8.0.0</Version>
<TargetFrameworks>net6.0</TargetFrameworks>
<LangVersion>latest</LangVersion>
<GenerateAssemblyInfo>true</GenerateAssemblyInfo>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ public interface IKafkaSingletonOptions : ISingletonOptions

string? BootstrapServers { get; }

bool ProducerByEntity { get; }
//bool ProducerByEntity { get; }

bool UseCompactedReplicator { get; }

Expand Down
40 changes: 20 additions & 20 deletions src/net/KEFCore/Infrastructure/Internal/KafkaOptionsExtension.cs
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ public class KafkaOptionsExtension : IDbContextOptionsExtension
private string? _databaseName;
private string? _applicationId;
private string? _bootstrapServers;
private bool _producerByEntity = false;
//private bool _producerByEntity = false;
private bool _useCompactedReplicator = false;
private bool _usePersistentStorage = false;
private int _defaultNumPartitions = 1;
Expand All @@ -60,7 +60,7 @@ protected KafkaOptionsExtension(KafkaOptionsExtension copyFrom)
_databaseName = copyFrom._databaseName;
_applicationId = copyFrom._applicationId;
_bootstrapServers = copyFrom._bootstrapServers;
_producerByEntity = copyFrom._producerByEntity;
//_producerByEntity = copyFrom._producerByEntity;
_useCompactedReplicator = copyFrom._useCompactedReplicator;
_usePersistentStorage = copyFrom._usePersistentStorage;
_defaultNumPartitions = copyFrom._defaultNumPartitions;
Expand All @@ -85,7 +85,7 @@ protected KafkaOptionsExtension(KafkaOptionsExtension copyFrom)

public virtual string BootstrapServers => _bootstrapServers!;

public virtual bool ProducerByEntity => _producerByEntity;
//public virtual bool ProducerByEntity => _producerByEntity;

public virtual bool UseCompactedReplicator => _useCompactedReplicator;

Expand Down Expand Up @@ -139,14 +139,14 @@ public virtual KafkaOptionsExtension WithBootstrapServers(string bootstrapServer
return clone;
}

public virtual KafkaOptionsExtension WithProducerByEntity(bool producerByEntity = false)
{
var clone = Clone();
//public virtual KafkaOptionsExtension WithProducerByEntity(bool producerByEntity = false)
//{
// var clone = Clone();

clone._producerByEntity = producerByEntity;
// clone._producerByEntity = producerByEntity;

return clone;
}
// return clone;
//}

public virtual KafkaOptionsExtension WithCompactedReplicator(bool useCompactedReplicator = false)
{
Expand Down Expand Up @@ -242,7 +242,7 @@ public virtual Properties StreamsOptions(string applicationId)
{
props.Remove(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG);
}
props.Put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.Serdes$StringSerde", true, SystemClassLoader));
props.Put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.Serdes$ByteArraySerde", true, SystemClassLoader));
if (props.ContainsKey(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG))
{
props.Remove(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG);
Expand Down Expand Up @@ -277,16 +277,16 @@ public virtual Properties ProducerOptions()
{
props.Put(ProducerConfig.LINGER_MS_CONFIG, 1);
}
if (props.ContainsKey(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG))
{
props.Remove(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG);
}
props.Put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.StringSerializer", true, SystemClassLoader));
if (props.ContainsKey(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG))
{
props.Remove(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG);
}
props.Put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.StringSerializer", true, SystemClassLoader));
//if (props.ContainsKey(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG))
//{
// props.Remove(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG);
//}
//props.Put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.StringSerializer", true, SystemClassLoader));
//if (props.ContainsKey(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG))
//{
// props.Remove(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG);
//}
//props.Put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.StringSerializer", true, SystemClassLoader));

return props;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ public virtual void Initialize(IDbContextOptions options)
DatabaseName = kafkaOptions.DatabaseName;
ApplicationId = kafkaOptions.ApplicationId;
BootstrapServers = kafkaOptions.BootstrapServers;
ProducerByEntity = kafkaOptions.ProducerByEntity;
//ProducerByEntity = kafkaOptions.ProducerByEntity;
UseCompactedReplicator = kafkaOptions.UseCompactedReplicator;
UsePersistentStorage = kafkaOptions.UsePersistentStorage;
DefaultNumPartitions = kafkaOptions.DefaultNumPartitions;
Expand Down
10 changes: 5 additions & 5 deletions src/net/KEFCore/Infrastructure/KafkaDbContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@ public KafkaDbContext(DbContextOptions options) : base(options)
/// Use persistent storage
/// </summary>
public virtual bool UsePersistentStorage { get; set; } = false;
/// <summary>
/// Use a producer for each Entity
/// </summary>
public bool UseProducerByEntity { get; set; } = false;
///// <summary>
///// Use a producer for each Entity
///// </summary>
//public bool UseProducerByEntity { get; set; } = false;
/// <summary>
/// Use <see cref="MASES.KNet.Replicator.KNetCompactedReplicator{TKey, TValue}"/> instead of Apache Kafka Streams
/// </summary>
Expand Down Expand Up @@ -104,7 +104,7 @@ protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
o.StreamsConfig(StreamsConfigBuilder ?? o.EmptyStreamsConfigBuilder).WithDefaultNumPartitions(DefaultNumPartitions);
o.WithUsePersistentStorage(UsePersistentStorage);
o.WithProducerByEntity(UseProducerByEntity);
//o.WithProducerByEntity(UseProducerByEntity);
o.WithCompactedReplicator(UseCompactedReplicator);
o.WithDefaultReplicationFactor(DefaultReplicationFactor);
});
Expand Down
40 changes: 20 additions & 20 deletions src/net/KEFCore/Infrastructure/KafkaDbContextOptionsBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -93,26 +93,26 @@ public virtual KafkaDbContextOptionsBuilder WithUseNameMatching(bool useNameMatc
return this;
}

/// <summary>
/// Enables creation of producer for each <see cref="IEntity"/>
/// </summary>
/// <remarks>
/// See <see href="https://aka.ms/efcore-docs-dbcontext-options">Using DbContextOptions</see>, and
/// <see href="https://github.com/masesgroup/KEFCore">The EF Core Kafka database provider</see> for more information and examples.
/// </remarks>
/// <param name="producerByEntity">If <see langword="true" />, then each entity will have its own <see cref="KafkaProducer"/>.</param>
/// <returns>The same builder instance so that multiple calls can be chained.</returns>
public virtual KafkaDbContextOptionsBuilder WithProducerByEntity(bool producerByEntity = false)
{
var extension = OptionsBuilder.Options.FindExtension<KafkaOptionsExtension>()
?? new KafkaOptionsExtension();

extension = extension.WithProducerByEntity(producerByEntity);

((IDbContextOptionsBuilderInfrastructure)OptionsBuilder).AddOrUpdateExtension(extension);

return this;
}
///// <summary>
///// Enables creation of producer for each <see cref="IEntity"/>
///// </summary>
///// <remarks>
///// See <see href="https://aka.ms/efcore-docs-dbcontext-options">Using DbContextOptions</see>, and
///// <see href="https://github.com/masesgroup/KEFCore">The EF Core Kafka database provider</see> for more information and examples.
///// </remarks>
///// <param name="producerByEntity">If <see langword="true" />, then each entity will have its own <see cref="KafkaProducer"/>.</param>
///// <returns>The same builder instance so that multiple calls can be chained.</returns>
//public virtual KafkaDbContextOptionsBuilder WithProducerByEntity(bool producerByEntity = false)
//{
// var extension = OptionsBuilder.Options.FindExtension<KafkaOptionsExtension>()
// ?? new KafkaOptionsExtension();

// extension = extension.WithProducerByEntity(producerByEntity);

// ((IDbContextOptionsBuilderInfrastructure)OptionsBuilder).AddOrUpdateExtension(extension);

// return this;
//}

/// <summary>
/// Enables use of <see cref="MASES.KNet.Replicator.KNetCompactedReplicator{TKey, TValue}"/>
Expand Down
1 change: 1 addition & 0 deletions src/net/KEFCore/KEFCore.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
<IncludeAssets>All</IncludeAssets>
<PrivateAssets>None</PrivateAssets>
</PackageReference>
<PackageReference Include="MASES.KNet.Serialization.Json" Version="2.1.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="7.0.11" PrivateAssets="none" />
</ItemGroup>
</Project>
Loading

0 comments on commit 295a9dd

Please sign in to comment.