Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removed string from definition of KNet generic types #64

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 28 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,42 @@
# KEFCore: the EntityFrameworkCore provider for Apache Kafka
# KEFCore: Entity Framework Core provider for Apache Kafka

[![CI_BUILD](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml) [![CI_RELEASE](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml)
KEFCore is the Entity Framework Core provider for Apache Kafka.
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use Apache Kafka as a distributed database.

### Libraries and Tools

[![latest version](https://img.shields.io/nuget/v/MASES.EntityFrameworkCore.KNet)](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet) [![downloads](https://img.shields.io/nuget/dt/MASES.EntityFrameworkCore.KNet)](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet)

KEFCore is the EntityFrameworkCore provider for Apache Kafka.
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use Apache Kafka as a distributed database.
### Pipelines

This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
[![CI_BUILD](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml)
[![CI_RELEASE](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml)

---

## Scope of the project

This project aims to create a provider to access the information stored within an Apache Kafka cluster using the paradigm behind Entity Framework.
The project is based on available information within the official [EntityFrameworkCore repository](https://github.com/dotnet/efcore), many classes was copied from there as reported in the official documentation within the Microsoft website at https://docs.microsoft.com/en-us/ef/core/providers/writing-a-provider.

### Community and Contribution

Do you like the project?
- Request your free [community subscription](https://www.jcobridge.com/pricing-25/).

Do you want to help us?
- put a :star: on this project
- open [issues](https://github.com/masesgroup/KEFCore/issues) to request features or report bugs :bug:
- improves the project with Pull Requests

This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

## Summary

* [Roadmap](src/documentation/articles/roadmap.md)
* [Actual state](src/documentation/articles/actualstate.md)
* [KEFCore usage](src/documentation/articles/usage.md)

## Runtime engine

KEFCore uses [KNet](https://github.com/masesgroup/KNet), and indeed [JCOBridge](https://www.jcobridge.com) with its [features](https://www.jcobridge.com/features/), to obtain many benefits:
Expand All @@ -36,13 +59,6 @@ Have a look at the following JCOBridge resources:
- [Commercial Edition](https://www.jcobridge.com/pricing-25/)
- Latest release: [![JCOBridge nuget](https://img.shields.io/nuget/v/MASES.JCOBridge)](https://www.nuget.org/packages/MASES.JCOBridge)

---
## Summary

* [Roadmap](src/documentation/articles/roadmap.md)
* [Actual state](src/documentation/articles/actualstate.md)
* [KEFCore usage](src/documentation/articles/usage.md)

---

KAFKA is a registered trademark of The Apache Software Foundation. KEFCore has no affiliation with and is not endorsed by The Apache Software Foundation.
Expand Down
2 changes: 1 addition & 1 deletion src/documentation/articles/intro.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Welcome to KEFCore

KEFCore is the EntityFrameworkCore provider for Apache Kafka.
KEFCore is the Entity Framework Core provider for Apache Kafka.
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use Apache Kafka as a distributed database.

This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
1 change: 1 addition & 0 deletions src/documentation/articles/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ The roadmap can be synthetized in the following points:

* Create a first working provider based on InMemory provider
* Extends the first provider with new features able to create Apache Kafka Streams topology to retrieve information
* Use KNetCompactedReplicator beside Apache Kafka Streams
4 changes: 2 additions & 2 deletions src/documentation/articles/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@

### Installation

EF Core for Apache Kafka is available on [NuGet](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet):
Entity Framework Core provider for Apache Kafka is available on [NuGet](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet):

```sh
dotnet add package MASES.EntityFrameworkCore.KNet
```

### Basic usage

The following code demonstrates basic usage of EF Core for Apache Kafka.
The following code demonstrates basic usage of Entity Framework Core provider for Apache Kafka.
For a full tutorial configuring the `KafkaDbContext`, defining the model, and creating the database, see [KafkaDbContext](kafkadbcontext.md) in the docs.

```cs
Expand Down
4 changes: 2 additions & 2 deletions src/documentation/index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# KEFCore: the EntityFrameworkCore provider for Apache Kafka
# KEFCore: Entity Framework Core provider for Apache Kafka

[![CI_BUILD](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml) [![CI_RELEASE](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml/badge.svg)](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml)

KEFCore is the EntityFrameworkCore provider for Apache Kafka.
KEFCore is the Entity Framework Core provider for Apache Kafka.
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use Apache Kafka as a distributed database.

This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
Expand Down
2 changes: 1 addition & 1 deletion src/net/Common/Common.props
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<Owners>MASES s.r.l.</Owners>
<Authors>MASES s.r.l.</Authors>
<Company>MASES s.r.l.</Company>
<Version>0.7.2.0</Version>
<Version>0.8.0.0</Version>
<TargetFrameworks>net6.0</TargetFrameworks>
<LangVersion>latest</LangVersion>
<GenerateAssemblyInfo>true</GenerateAssemblyInfo>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ public interface IKafkaSingletonOptions : ISingletonOptions

string? BootstrapServers { get; }

bool ProducerByEntity { get; }
//bool ProducerByEntity { get; }

bool UseCompactedReplicator { get; }

Expand Down
40 changes: 20 additions & 20 deletions src/net/KEFCore/Infrastructure/Internal/KafkaOptionsExtension.cs
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ public class KafkaOptionsExtension : IDbContextOptionsExtension
private string? _databaseName;
private string? _applicationId;
private string? _bootstrapServers;
private bool _producerByEntity = false;
//private bool _producerByEntity = false;
private bool _useCompactedReplicator = false;
private bool _usePersistentStorage = false;
private int _defaultNumPartitions = 1;
Expand All @@ -60,7 +60,7 @@ protected KafkaOptionsExtension(KafkaOptionsExtension copyFrom)
_databaseName = copyFrom._databaseName;
_applicationId = copyFrom._applicationId;
_bootstrapServers = copyFrom._bootstrapServers;
_producerByEntity = copyFrom._producerByEntity;
//_producerByEntity = copyFrom._producerByEntity;
_useCompactedReplicator = copyFrom._useCompactedReplicator;
_usePersistentStorage = copyFrom._usePersistentStorage;
_defaultNumPartitions = copyFrom._defaultNumPartitions;
Expand All @@ -85,7 +85,7 @@ protected KafkaOptionsExtension(KafkaOptionsExtension copyFrom)

public virtual string BootstrapServers => _bootstrapServers!;

public virtual bool ProducerByEntity => _producerByEntity;
//public virtual bool ProducerByEntity => _producerByEntity;

public virtual bool UseCompactedReplicator => _useCompactedReplicator;

Expand Down Expand Up @@ -139,14 +139,14 @@ public virtual KafkaOptionsExtension WithBootstrapServers(string bootstrapServer
return clone;
}

public virtual KafkaOptionsExtension WithProducerByEntity(bool producerByEntity = false)
{
var clone = Clone();
//public virtual KafkaOptionsExtension WithProducerByEntity(bool producerByEntity = false)
//{
// var clone = Clone();

clone._producerByEntity = producerByEntity;
// clone._producerByEntity = producerByEntity;

return clone;
}
// return clone;
//}

public virtual KafkaOptionsExtension WithCompactedReplicator(bool useCompactedReplicator = false)
{
Expand Down Expand Up @@ -242,7 +242,7 @@ public virtual Properties StreamsOptions(string applicationId)
{
props.Remove(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG);
}
props.Put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.Serdes$StringSerde", true, SystemClassLoader));
props.Put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.Serdes$ByteArraySerde", true, SystemClassLoader));
if (props.ContainsKey(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG))
{
props.Remove(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG);
Expand Down Expand Up @@ -277,16 +277,16 @@ public virtual Properties ProducerOptions()
{
props.Put(ProducerConfig.LINGER_MS_CONFIG, 1);
}
if (props.ContainsKey(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG))
{
props.Remove(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG);
}
props.Put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.StringSerializer", true, SystemClassLoader));
if (props.ContainsKey(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG))
{
props.Remove(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG);
}
props.Put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.StringSerializer", true, SystemClassLoader));
//if (props.ContainsKey(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG))
//{
// props.Remove(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG);
//}
//props.Put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.StringSerializer", true, SystemClassLoader));
//if (props.ContainsKey(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG))
//{
// props.Remove(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG);
//}
//props.Put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Class.ForName("org.apache.kafka.common.serialization.StringSerializer", true, SystemClassLoader));

return props;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ public virtual void Initialize(IDbContextOptions options)
DatabaseName = kafkaOptions.DatabaseName;
ApplicationId = kafkaOptions.ApplicationId;
BootstrapServers = kafkaOptions.BootstrapServers;
ProducerByEntity = kafkaOptions.ProducerByEntity;
//ProducerByEntity = kafkaOptions.ProducerByEntity;
UseCompactedReplicator = kafkaOptions.UseCompactedReplicator;
UsePersistentStorage = kafkaOptions.UsePersistentStorage;
DefaultNumPartitions = kafkaOptions.DefaultNumPartitions;
Expand Down
10 changes: 5 additions & 5 deletions src/net/KEFCore/Infrastructure/KafkaDbContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@ public KafkaDbContext(DbContextOptions options) : base(options)
/// Use persistent storage
/// </summary>
public virtual bool UsePersistentStorage { get; set; } = false;
/// <summary>
/// Use a producer for each Entity
/// </summary>
public bool UseProducerByEntity { get; set; } = false;
///// <summary>
///// Use a producer for each Entity
///// </summary>
//public bool UseProducerByEntity { get; set; } = false;
/// <summary>
/// Use <see cref="MASES.KNet.Replicator.KNetCompactedReplicator{TKey, TValue}"/> instead of Apache Kafka Streams
/// </summary>
Expand Down Expand Up @@ -104,7 +104,7 @@ protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
o.StreamsConfig(StreamsConfigBuilder ?? o.EmptyStreamsConfigBuilder).WithDefaultNumPartitions(DefaultNumPartitions);
o.WithUsePersistentStorage(UsePersistentStorage);
o.WithProducerByEntity(UseProducerByEntity);
//o.WithProducerByEntity(UseProducerByEntity);
o.WithCompactedReplicator(UseCompactedReplicator);
o.WithDefaultReplicationFactor(DefaultReplicationFactor);
});
Expand Down
40 changes: 20 additions & 20 deletions src/net/KEFCore/Infrastructure/KafkaDbContextOptionsBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -93,26 +93,26 @@ public virtual KafkaDbContextOptionsBuilder WithUseNameMatching(bool useNameMatc
return this;
}

/// <summary>
/// Enables creation of producer for each <see cref="IEntity"/>
/// </summary>
/// <remarks>
/// See <see href="https://aka.ms/efcore-docs-dbcontext-options">Using DbContextOptions</see>, and
/// <see href="https://github.com/masesgroup/KEFCore">The EF Core Kafka database provider</see> for more information and examples.
/// </remarks>
/// <param name="producerByEntity">If <see langword="true" />, then each entity will have its own <see cref="KafkaProducer"/>.</param>
/// <returns>The same builder instance so that multiple calls can be chained.</returns>
public virtual KafkaDbContextOptionsBuilder WithProducerByEntity(bool producerByEntity = false)
{
var extension = OptionsBuilder.Options.FindExtension<KafkaOptionsExtension>()
?? new KafkaOptionsExtension();

extension = extension.WithProducerByEntity(producerByEntity);

((IDbContextOptionsBuilderInfrastructure)OptionsBuilder).AddOrUpdateExtension(extension);

return this;
}
///// <summary>
///// Enables creation of producer for each <see cref="IEntity"/>
///// </summary>
///// <remarks>
///// See <see href="https://aka.ms/efcore-docs-dbcontext-options">Using DbContextOptions</see>, and
///// <see href="https://github.com/masesgroup/KEFCore">The EF Core Kafka database provider</see> for more information and examples.
///// </remarks>
///// <param name="producerByEntity">If <see langword="true" />, then each entity will have its own <see cref="KafkaProducer"/>.</param>
///// <returns>The same builder instance so that multiple calls can be chained.</returns>
//public virtual KafkaDbContextOptionsBuilder WithProducerByEntity(bool producerByEntity = false)
//{
// var extension = OptionsBuilder.Options.FindExtension<KafkaOptionsExtension>()
// ?? new KafkaOptionsExtension();

// extension = extension.WithProducerByEntity(producerByEntity);

// ((IDbContextOptionsBuilderInfrastructure)OptionsBuilder).AddOrUpdateExtension(extension);

// return this;
//}

/// <summary>
/// Enables use of <see cref="MASES.KNet.Replicator.KNetCompactedReplicator{TKey, TValue}"/>
Expand Down
1 change: 1 addition & 0 deletions src/net/KEFCore/KEFCore.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
<IncludeAssets>All</IncludeAssets>
<PrivateAssets>None</PrivateAssets>
</PackageReference>
<PackageReference Include="MASES.KNet.Serialization.Json" Version="2.1.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="7.0.11" PrivateAssets="none" />
</ItemGroup>
</Project>
Loading
Loading