JSON Fliox

News

Moved Friflo.Engine.ECS to its own GitHub Repository
Published Friflo.Engine.ECS v3.0.0 preview.6
New features: Entity Relationships, Relations, full-text Search and Range queries.
Created a new GitHub repository ECS.CSharp.Benchmark - Common use-cases
comparing the performance of multiple C# ECS projects with simple benchmarks.

JSON Fliox

A client / server ORM for .NET for SQL & NoSQL databases with focus on Web apps and performance.
Database access is realized by creating a single class to query, create, update or delete table / container records.

This class - a FlioxClient - provide type safety and support validation, batching, transactions and messaging.
A FlioxHub can be used to serve hosted databases via HTTP using ASP.NET Core or an HttpListener.

As Fliox is an ORM it has similarities to projects like Entity Framework Core , Ruby on Rails , Django or Hibernate

Unique ORM Features

As of writing this the list below show unique features not supported by other ORMs like:
EFCore, Dapper, NHibernate, LINQ to DB, PetaPoco or ServiceStack.

Fliox provide two schema models for all SQL databases:
- Relational - the default - storing each entity class property in its own column.
- JSON columns storing entities in a single column called: json_data.
  This enables schema changes without database migrations and avoid OR impedance mismatch .
Create automatically common REST route handlers GET, PUT, PATCH, DELETE to execute LINQ query filters
and CRUD operations (bulk or single resource) for each table using ASP.NET Core.
FlioxClient's can be used client-side on user devices by using a HTTP or WebSocket connection.
Using the listed ORMs above require to disclose the database credentials to the public.
FlioxClient's can also be used in web applications (JS/WASM) using HTTP or WebSocket's.
Not supported by the mentioned ORMs as they require proprietary - non HTTP based - protocols.
Support Pub-Sub to receive push events for subscribed database changes like: create, update, delete & merge.
Provide a Web UI - the Hub Explorer - to explore databases using a web browser.
Supports batching. Multiple database operations are combined into a single request send to the server.
This enables execution of database transactions in a single request and enable non-chatty data communication.

TL;DR

Try the example Hub online running on AWS - DemoHub (EC2 instance: t2-micro, us-east-1)
The DemoHub .NET project is available at 🚀 friflo/Fliox.Examples.

Note: JSON Fliox is not a UI library. It is designed for simple integration in .NET and Web UI frameworks.

🚩 Content

Features
Quickstart
Providers
Examples ❯ 🚀 friflo/Fliox.Examples
Hub
- Client ❯ README.md
- Host ❯ README.md
- Explorer ❯ README.md
- DB ❯ README.md
- Protocol ❯ README.md
Fliox
- Schema ❯ README.md
- Mapper ❯ README.md
Project
- API ❯ friflo/fliox-docs
- Properties
- Principles
- Build ❯ README.md
Motivation
Credits

🎨 Features

Compact list of features supported by Clients and Hubs

ASP.NET Core & HttpListener integration
- REST API - JSON Schema / OpenAPI
- GraphQL API
- Batch API - HTTP, WebSocket & UDP
Database
- CRUD operations - bulk and single resource
- Transactions - Begin, Commit & Rollback - for SQL databases
- Schema
- Queries - LINQ expressions
- Container relations (associations)
- Entity validation
- Send messages (event) & commands (RPC) with validation
- Pub-Sub - entity changes, messages and commands
Hub Explorer - the admin page
Monitoring
Authentication / Authorization
Code generation
- C#, Typescript & Kotlin
- JSON Schema, OpenAPI Schema & GraphQL Schema
- Schema documentation & class diagram
Compatibility
- Frameworks: .NET, .NET Core, .NET Framework and .NET Standard
- tested on Windows, macOS, Linux & Unity 2021 or higher

Features are explained in more detail in the sections: Hub and Fliox

Performance characteristics
RTT request / response roundtrip	0.3 ms
Pub-Sub delay send message ➞ subscriber event	sub millisecond latency
Pub-Sub throughput FIFO 3 subscribers each using a WebSocket	50k messages / sec
Query request 1M rows, each row 124 byte => response 125MB	1.3 sec
Throughput request / response WebSocket, 4 concurrent clients	27k requests / sec
ASP.NET Core Hub integration	1 LOC Startup.cs
Minimal Client & Server with: REST, CRUD, Queries, Pub-Sub & Explorer	60 LOC Client & Server
run on Intel(R) Core(TM) i7-4790K CPU 4.00GHz

⏩ Quickstart

Direct database access

Create a Console Application and add the following dependencies:

dotnet add package Friflo.Json.Fliox.Hub
dotnet add package Friflo.Json.Fliox.Hub.SQLite

Create a TodoClient client to specify the database schema.

📄 TodoClient.cs

public class TodoClient : FlioxClient
{
    // --- containers
    public  readonly    EntitySet <long, Job>   jobs;
    
    public TodoClient(FlioxHub hub, string dbName = null) : base (hub, dbName) { }
}

// ---------------------------------- entity models ----------------------------------
public class Job
{
    [Key]       public  long    id;
    [Required]  public  string  title;
                public  bool?   completed;
}

The following code create / open a SQLite database by using TodoClient as the database schema.
It also perform some database operations like: UpsertRange() & Query()

📄 Program.cs

    var schema      = DatabaseSchema.Create<TodoClient>();
    var database    = new SQLiteDatabase("todo_db", "Data Source=todo.sqlite3", schema);
    await database.SetupDatabaseAsync(); // for development: create database or update ist schema
    var hub         = new FlioxHub(database);

    var client      = new TodoClient(hub);
    client.jobs.UpsertRange(new[] {
        new Job { id = 1, title = "Buy milk", completed = true },
        new Job { id = 2, title = "Buy cheese", completed = false }
    });
    var jobs = client.jobs.Query(job => job.completed == true);
    await client.SyncTasks(); // execute UpsertRange & Query task

    foreach (var job in jobs.Result) {
        Console.WriteLine($"{job.id}: {job.title}");
    }
    // output:  1: Buy milk

Run the application

dotnet run

Remote database access

Remote database access - client / servers setup - require two console applications:

HTTP server to host a single - or multiple - databases
HTTP client to access a hosted database

1. HTTP Server

Add dependency to Hub Explorer to host a Web UI to browse databases.
The Hub Explorer is optional but speedup development. It contains the static files for the Web UI.

dotnet add package Friflo.Json.Fliox.Hub.Explorer

Replace the code in 📄 Program.cs above to host a database by an HTTP server.

📄 Program.cs (server)

    var schema      = DatabaseSchema.Create<TodoClient>();
    var database    = new SQLiteDatabase("todo_db", "Data Source=todo.sqlite3", schema);
    await database.SetupDatabaseAsync(); // for development: create database or update ist schema
    var hub         = new FlioxHub(database);
    hub.Info.Set ("TodoHub", "dev", "https://github.com/friflo/Fliox.Examples/tree/main/Todo", "rgb(0 171 145)"); // optional
    hub.UseClusterDB(); // required by HubExplorer
    hub.UsePubSub();    // optional - enables Pub-Sub
    // --- create HttpHost
    var httpHost    = new HttpHost(hub, "/fliox/");
    httpHost.UseStaticFiles(HubExplorer.Path); // nuget: https://www.nuget.org/packages/Friflo.Json.Fliox.Hub.Explorer
    
    HttpServer.RunHost("http://localhost:5000/", httpHost); // http://localhost:5000/fliox/

Start the server and check the Hub Explorer is available at http://localhost:5000/fliox/

dotnet run

ASP.NET Core integration

ASP.NET Core integration requires the nuget package.

dotnet add package Friflo.Json.Fliox.Hub.AspNetCore

Integration into an existing WebApplication app is enabled adding

    app.MapHost("/fliox/{*path}", httpHost);`

Or create an WebApplication from scratch by replacing HttpServer.RunHost() in the snippet above by

    var app = WebApplication.Create();
    app.MapRedirect("/", httpHost);
    app.MapHost("/fliox/{*path}", httpHost);
    app.Run();

C# documentation in Hub Explorer (optional)

The C# documentation of TodoClient and other model classes can be utilized in the Hub Explorer.
Therefor add the following xml snippet to the *.csproj. It will copy the *.xml files next to the *.dll files.
The server read and add the documentation to schema definition.

  <PropertyGroup>
	<GenerateDocumentationFile>True</GenerateDocumentationFile>
  </PropertyGroup>
  <!-- Copy XML files from all PackageReferences to output dir -->
  <Target Name="_ResolveCopyLocalNuGetPkgXmls" AfterTargets="ResolveReferences">
    <ItemGroup>
      <ReferenceCopyLocalPaths Include="@(ReferenceCopyLocalPaths->'%(RootDir)%(Directory)%(Filename).xml')" Condition="'%(ReferenceCopyLocalPaths.NuGetPackageId)'!='' and Exists('%(RootDir)%(Directory)%(Filename).xml')" />
    </ItemGroup>
  </Target>

2. HTTP Client

Create a second Console application to access the hosted database via HTTP.

Add required nuget dependencies

dotnet add package Friflo.Json.Fliox.Hub

Copy 📄 TodoClient.cs from above to Console project.

Note: The intention is to implement the TodoClient and its model classes in a separate library project.
Client & server applications reference this library as their dependency.
But for simplicity create a copy in the client application for now.

📄 Program.cs (client)

    var hub     = new WebSocketClientHub("todo_db", "ws://localhost:5000/fliox/");
    var client  = new TodoClient(hub);
    var jobs    = client.jobs.Query(job => job.completed == true);
    client.jobs.SubscribeChanges(Change.All, (changes, context) => {
        Console.WriteLine(changes);
    });
    await client.SyncTasks(); // execute Query & SubscribeChanges task

    foreach (var job in jobs.Result) {
        Console.WriteLine($"{job.id}: {job.title}");
    }
    // output:  1: Buy milk
    Console.WriteLine("\n wait for events ... (exit with: CTRL + C)\n note: generate events by clicking 'Save' on a record in the Hub Explorer\n");
    await Task.Delay(3_600_000); // wait 1 hour

Ensure the server is running and start the client application

dotnet run

⛁ Database providers

Database	class / nuget	connection string examples
in-memory	`MemoryDatabase`	none
	build-in
file-system	`FileDatabase`	path of root folder
	build-in
SQLite	`SQLiteDatabase`	`"Data Source=test_db.sqlite3"`
		`dotnet add package Friflo.Json.Fliox.Hub.SQLite`
MySQL	`MySQLDatabase`	`"Server=localhost;User ID=root;Password=;Database=test_db;"`
		`dotnet add package Friflo.Json.Fliox.Hub.MySQL`
MariaDB	`MariaDBDatabase`	`"Server=localhost;User ID=root;Password=;Database=test_db;"`
		`dotnet add package Friflo.Json.Fliox.Hub.MySQL`
PostgreSQL	`PostgreSQLDatabase`	`"Host=localhost;Username=postgres;Password=postgres;Database=test_db;"`
		`dotnet add package Friflo.Json.Fliox.Hub.PostgreSQL`
SQL Server	`SQLServerDatabase`	`"Data Source=.;Integrated Security=True;Database=test_db"`
		`dotnet add package Friflo.Json.Fliox.Hub.SQLServer`

The connection string of each provider is documented in its nuget README

Example snippet to create a database using SQLite looks like:

    var connection  = "Data Source=todo_db.sqlite3";
    var schema      = DatabaseSchema.Create<TodoClient>();
    var database    = new SQLiteDatabase("todo_db", connection, schema);

🚀 Examples

📄 friflo/Fliox.Examples

A separate git repository with two small ready-to-run examples (70 LOC & 550 LOC) using Fliox Clients and Servers.
Build and run a server with Gitpod using VSCode in the browser without installing anything.

screenshot: DemoHub server logs

📦 Hub

Namespace Friflo.Json.Fliox.Hub.*
Assembly Friflo.Json.Fliox.Hub.dll

Client

📄 README.md

Fliox clients are strongly typed C# classes used to access SQL or NoSQL databases.
They are implemented by creating a class e.g. MyClient extending FlioxClient.
The database containers are represented as properties in the derived class MyClient.

These classes also acts as a database schemas. They can be assigned to databases hosted on the Hub.
Doing this enables features like:

JSON validation of entities aka records
generate a JSON Schema, an OpenAPI Schema and a GraphQL Schema
generate a HTML Schema documentation and a UML class diagram
generate classes for various programming languages: C#, Typescript & Kotlin

The MyClient can be used to declare custom database commands using DTO's as input and result types.

Host

📄 README.md

A HttpHost instance is used to host multiple databases.
It is designed to be integrated into HTTP servers like ASP.NET Core.
This enables access to hosted databases via HTTP, WebSocket or UDP supporting the following Web API's:

REST
GraphQL
Batch API

A FlioxHub instance is used to configure the hosted databases, authentication / authorization and Pub-Sub.
This FlioxHub instance need to be passed to the constructor of the HttpHost

Explorer

📄 README.md
Assembly Friflo.Json.Fliox.Hub.Explorer.dll

The Hub Explorer is an admin page used to access databases, containers and entities hosted by a Fliox Hub.
The Explorer also enables to execute application specific database commands.

screenshot: Hub Explorer

DB

📄 README.md

Provide a set of support databases used to:

serve the Hub configuration - used by the Hub Explorer. Schema: ClusterStore
serve monitoring data. Schema: MonitorStore
perform user authentication, authorization and management. Schema: UserStore

Protocol

📄 README.md

The Protocol is the communication interface between a FlioxClient and a FlioxHub.
Web clients can use this Protocol to access a Hub using the Batch API via HTTP & JSON.
A language specific API - e.g. written in Typescript, Kotlin, ... - is not a requirement.

The Protocol is not intended to be used by C# .NET clients directly.
Instead they are using a FlioxClient that is optimized to transform API calls into the Protocol.

📦 Fliox

Namespace Friflo.Json.Fliox.*
Assembly Friflo.Json.Fliox.dll

Schema

📄 README.md

This module enables transforming schemas expressed by a set of C# classes into other programming languages and schema formats like:

C#, Typescript, Kotlin
HTML documentation, Schema Class Diagram
JSON Schema, OpenAPI Schema, GraphQL Schema

Its main purpose is to generate schemas and types for various languages of classes extending FlioxClient.
The screenshots below show Hub pages utilizing the schemas mentioned above.

screenshot: MonitorStore schema as class diagram

screenshots: Schema documentation, Swagger UI & GraphiQL

Mapper

📄 README.md

This module enables serialization / deserialization of C# .NET objects to / from JSON.
Its feature set and API is similar to the .NET packages:

The module is utilized by the assembly Friflo.Json.Fliox.Hub to serialize entities and DTO's.
Its also used for serialization of the supported protocols: REST, GraphQL and Batch API.

🔧 Project

API

The Fliox C# .NET API is CLS-compliant
The API is available at fliox-docs API Reference

Properties

The goal of the library, its components and API is to be easy digestible for software developers.
The properties describe the characteristics of this project - at least what it aims for.
These properties are targeted to the needs of users using the library.
They fit mostly the aspects described in CUPID-for joyful coding .

Topics of the CUPID properties focused by this project are

Composable
- No 3rd party dependencies
- Compatibility .NET Core 3.1 or higher, .NET Framework 4.6.2 or higher and Unity 2020.1 or higher
- Seamless integration into existing ASP.NET Core applications with a handful lines of code
- Independence from other parts of existing applications
- Fliox Clients and Hubs are unit testable without mocking
- Replace the underlying database without changing application code
- Add custom code / schema generators without changing the application code
Predictable
- API surface is as small as possible
- API: class, method and property names are short, simple, and easy to pronounce
- Observable
  - Monitoring is integral part of the Hub
  - The ToString() methods of classes show only relevant state to avoid noise in debugging sessions
  - Error and runtime assertion messages are short and expressive
Domain based
- Enable implementing compact applications which are easy to read and to maintain

Principles

A set of rules followed by this project to aim for simplicity and performance. See Principles

Build

📄 README.md

The project Json.Tests contains a console application and unit tests.
Build and run instructions for .NET and Unity are in the README file.

unit tests
Code coverage: 86% measured with JetBrains • docCover

Passed! - Failed: 0, Passed: 842, Skipped: 0, Total: 842, Duration: 7 s - Friflo.Json.Tests.dll (net6.0)

summarized logs of unit test execution - they are executed in

🔥 Motivation

The main driver of this project is the development of an competitive online multiplayer game - a still unresolved task in my todo list.
The foundation to achieve this is commonly a module called Netcode in online multiplayer games.
The key aspects of Netcode are: Synchronizing game state, messaging, low latency, high throughput, minimal use of system resources, reliability & easy to use API.
As Unity is selected as the Game engine C# .NET is the way to go.

Another objective is to create an open source software project which may have the potential to be popular.
As I have 15+ years experience as a software developer in enterprise environment - Shout-Out to HERE Technologies - I decided to avoid a Vendor Lock-In to Unity and target for a solution which fits also the needs of common .NET projects.
So development is entirely done with .NET Core while checking Unity compatibility on a regular basis.

The result is a project with a feature set useful in common & gaming projects and targeting for optimal performance.
The common ground of both areas is the need of databases.
In context of game development the game state (Players, NPC, objects, ...) is represented as an in-memory database to enable low latency, high throughput and minimal use of system resources.
In common projects databases are used to store any kind of data persistent by using a popular DBMS.
Specific for online gaming is the ability to send messages from one client to another in real time. This is enabled by supporting Pub-Sub with sub millisecond latency on localhost.

🙏 Credits


NUnit	C#	unit testing of the library in the CLR and Unity
FluentAssertions	C#	unit testing of the library
SQLitePCL.raw	C#	used by DB Provider for SQLite
SqlClient	C#	used by DB Provider for Microsoft SQL Server
MySqlConnector	C#	used by DB Provider for MySQL, MariaDB and MySQL compatible DBMS
Npgsql	C#	used by DB Provider for PostgreSQL
Microsoft.Azure.Cosmos	C#	used by DB Provider for CosmosDB
SIPSorcery	C#	WebRTC - Real-time communication between Web clients without server
GraphQL.NET Parser	C#	used by package: Friflo.Json.Fliox.Hub.GraphQL
MdDocs	C#	for fliox-docs API Reference
.NET platform	C# .NET	the platform providing compiler, runtime, IDE's & ASP.NET Core
Swagger	static JS	a REST / OpenAPI UI linked by the Hub Explorer
GraphiQL	static JS	a GraphQL UI linked by the Hub Explorer
Mermaid	static JS	class diagram for database schema linked by the Hub Explorer
Monaco Editor	static JS	used as JSON editor integrated in the Hub Explorer
WinMerge	Application	heavily used in this project
Inscape	Application	to create SVG's for this project

💖 Like this project?
Leave a ⭐ at friflo/Friflo.Json.Fliox

Happy coding!

License

This project is licensed under LGPLv3.

Name		Name	Last commit message	Last commit date
Latest commit History 17,031 Commits
.github		.github
.idea		.idea
.nupkgs		.nupkgs
.run		.run
.vscode		.vscode
Demos~		Demos~
Engine		Engine
Json.Tests		Json.Tests
Json		Json
Playground		Playground
docs		docs
scripts		scripts
.gitattributes		.gitattributes
.gitignore		.gitignore
.gitpod.yml		.gitpod.yml
BannedSymbols.txt		BannedSymbols.txt
CONTRIBUTING.md		CONTRIBUTING.md
Directory.Build.props		Directory.Build.props
Dockerfile		Dockerfile
Friflo.Json.Fliox.asmdef		Friflo.Json.Fliox.asmdef
Friflo.Json.Fliox.sln		Friflo.Json.Fliox.sln
Friflo.Json.Fliox.sln.DotSettings		Friflo.Json.Fliox.sln.DotSettings
LICENSE		LICENSE
README.md		README.md
csc.rsp		csc.rsp
global.json		global.json
package.json		package.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

JSON Fliox

News

JSON Fliox

Unique ORM Features

🚩 Content

🎨 Features

⏩ Quickstart

Direct database access

Remote database access

1. HTTP Server

2. HTTP Client

⛁ Database providers

🚀 Examples

📦 Hub

Client

Host

Explorer

DB

Protocol

📦 Fliox

Schema

Mapper

🔧 Project

API

Properties

Principles

Build

🔥 Motivation

🙏 Credits

License

About

Releases 34

Languages

License

friflo/Friflo.Json.Fliox

Folders and files

Latest commit

History

Repository files navigation

JSON Fliox

News

JSON Fliox

Unique ORM Features

🚩 Content

🎨 Features

⏩ Quickstart

Direct database access

Remote database access

1. HTTP Server

2. HTTP Client

⛁ Database providers

🚀 Examples

📦 Hub

Client

Host

Explorer

DB

Protocol

📦 Fliox

Schema

Mapper

🔧 Project

API

Properties

Principles

Build

🔥 Motivation

🙏 Credits

License

About

Topics

Resources

License

Stars

Watchers

Forks

Releases 34

Languages