Project structure review

[ndr-brt]

Feature Request

Review the overall project structure

Which Areas Would Be Affected?

all

Why Is the Feature Desired?

to have a more clear division from a component perspective.

Solution Proposal

The current EDC project structure reached a state where some things cannot be obtained (like permitting "extenders" to implement a custom api layer using the core services) and in general a not-so-clear state.

My proposal would be to have a more clear division from a component perspective.

A component is a particular EDC combination of dependencies that respond to a particular need, e.g.: control-plane, data-plane, data-plane-selector.
This would help developers to extricate themselves in the codebase more easily and separate responsibilities in a better way.

Spi

The spi module should be refactored as:

spi/catalog-spi
spi/control-plane
spi/control-plane/asset-spi
spi/control-plane/contract-spi
spi/control-plane/policy-spi
spi/control-plane/transfer-spi
spi/core-spi
spi/data-plane-spi
spi/data-plane-selector-spi
spi/transport-spi
spi/web-spi

Every component should have its own spi module, then other "common" spi could exist as well (like core, catalog, transport and web)

Some details:

• the asset-spi would be extracted from the core-spi
• the *Service interfaces should be moved to the control-plane/*-spi

Core

Responsibility of the core modules should be to provide a working base version of every component, so it should provide a default implementation for every interface present in the relative spimodule, so these three module will be introduced:

• control-plane-core: would contain all the default implementations of the stores and the services and an extension that register them as default
• data-plane-core: is the current data-plane-framework extension, will register all the default implementations
• data-plane-selector-core: is the current selector-core module

All of them should extend the base-core module. The defaults module would probably disappear (replaced by the <component>-core ones)

Extensions

This separation could be raised also in the extensions folder, as some extensions are needed only by one of the components, for example:

• control-plane: data management api, provisioners (s3, blobstorage, http...), "stores" sql/cosmos implementations (for assets, contracts, transfer process, policies...), data-plane-selector client, ...
• data-plane: implementations of the source/sink interfaces like data-plane-http, data-plane-s3, and so on.., data plane specific api (data-plane-api module)
• data-plane-selector: implementations of the DataPlaneInstanceStore
• common (needed by more than one component): observability api, vault, configuration, http, loggers, iam, ...

Type of Issue

cleanup

Checklist

• Project Structure: create decision record #1807
• Project structure: spi modules #1808
• Project structure: core modules #1809
• Project structure: extensions #1810
• Introduce service layer #1887
• Project strucure: move @Provider(isDefault = true) service providers to the respective core extension #1812
• refactor: clean up split spi modules #1468
• Spike: investigate the use of BOMs #1438
• Project structure: inline dataloading module #1833
• Project structure: introduce token-spi module #1834
• Tests: replace fake implementations with mocks #1063
• Project structure: update documentation #1844
• Project structure: remove nimbus-jose dependency from jwt-spi #1879
• ContractDefinitionService interface exists twice #1248
• Project structure: Move common/* modules to core/common/ folder #1884
• Refactor: join AssetLoader/AssetIndex and ContractDefinitionLoader/ContractDefinitionStore #1886
• Project structure: Policy classes and interfaces refactor #1967
• spi modules should not rely on core ones #2033