Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update architecture docs #305

Merged
merged 1 commit into from
May 15, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 99 additions & 5 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,18 @@ flowchart TD
subgraph BUILTIN_CMDS ["Built-in commands"]
direction LR
BLD[scarb build]
CLN[scarb clean]
FMT[scarb fmt]
RUN[scarb run]
NEW[scarb init/new]
UPD[scarb add/rm/update]
MET[scarb metadata]
CMD[scarb commands]
MAN[scarb manifest-path]
end
subgraph EXT_CMDS ["External subcommands system"]
direction LR
CAIRO_LS["Cairo Language Server"]
PROTOSTAR["scarb test\n(will be built from Protostar)"]
STARKNET["scarb starknet\n(will be built from Protostar)"]
end
Expand Down Expand Up @@ -58,10 +64,22 @@ classDiagram
}
class Config~'c~ {
- manifest path
- app exe path
- app dirs, like config, cache, PATH, etc.
- target directory handle
- ui
- logging configuration
- cairo plugins
- current profile
- global locks
- tokio runtime
}
class CodegenRepository {
- compilers
}
class StarknetContractCompiler {
}
class LibCompiler {
}
class Workspace~'c~ {
+ members() Iter~Package~
Expand All @@ -73,6 +91,9 @@ classDiagram
class Manifest {
+ Summary
+ Vec~Target~
+ compiler config
+ profile definitions
+ scripts
+ not important metadata
}
class Summary {
Expand All @@ -88,7 +109,7 @@ classDiagram
class Target {
+ TargetKind
+ String name
+ BTreeMap~String, *~ params
+ BTreeMap<String, *> params
}
class CompilationUnit {
Contains all information
Expand All @@ -105,6 +126,7 @@ classDiagram

Workspace~'c~ *-- Config~'c~
Workspace~'c~ o-- * Package
Config~'c~ *-- CodegenRepository
Package -- Manifest
Manifest -- Summary
Manifest o-- * Target
Expand All @@ -117,12 +139,12 @@ classDiagram
WorkspaceResolve *-- Package
CompilationUnit ..> WorkspaceResolve : is created from
CompilationUnit ..> Target : is created from
CodegenRepository *-- StarknetContractCompiler
CodegenRepository *-- LibCompiler
```

### Sources and the internal registry

**THIS IS NOT FULLY IMPLEMENTED YET.**

A _source_ is an object that finds and downloads remote packages based on names and versions.
The interface of sources is contained within the `Source` trait.
There are various `Source` implementation for different methods of downloading packages:
Expand All @@ -132,6 +154,8 @@ There are various `Source` implementation for different methods of downloading p
3. `RegistrySource` downloads packages from package registries.
4. And more...

**CURRENTLY ONLY PATH AND GIT SOURCES ARE IMPLEMENTED.**

The `Registry` object gathers all `Source` objects in a single mapping, and provides a unified interface for querying
_any_ package, no matter of its source.

Expand Down Expand Up @@ -182,9 +206,79 @@ overwritten.

### Targets

**THIS IS NOT IMPLEMENTED YET.**
Compilation targets are defined in the `Package` manifest.
Each target corresponds to source files which can be compiled into a package, and the way how the package is compiled.
Packages can have a built-in library target, and/or more externally defined targets.
When building a package, each target of this package will use exactly the same set of dependencies during compilation.
By default, if the manifest does not list any targets, Scarb will assume the library target with its default parameters.

TODO(mkaput): Write this section.
Each compilation target consists of target kind, source path and parameters table.
When building a project, compilation targets are used to construct compilation units.


### CodegenRepository

**Currently referred to as CompilerRepository within the implementation.**

To compile Cairo code while building a project into appropriate output targets, Scarb relies on Cairo compiler crate.
Each compilation target requires the Cairo compiler to be appropriately configured before building the project.
Scarb defines preconfigured Cairo compiler instances as structs implementing the `Compiler` trait.
You can see `StarknetContractCompiler` as an example of such implementation.

The CompilerRepository is used to manage the variety of preconfigured Cairo compilers available to Scarb.
Each compiler is identified by the target kind that uses it for compilation.
At any time, there can be only one compiler registered for a specific compilation target kind.
When building a project, Scarb retrieves appropriate compiler implementation from CodegenRepository by target kind.
The compiler will later be used to compile the project into the target output.

### Compilation units
Please see [scarb documentation](https://docs.swmansion.com/docs/reference/compilation-model) for more information.

### CairoPluginRepository

Thanks to the Cairo compiler architecture, the compilation process can be altered in certain ways by use of
the Semantic Plugins interface.
Within Scarb, this functionality is exposed to users as Cairo plugins.
Cairo plugin is a special Scarb package that defines a cairo-plugin target and provides additional interface for
instantiating compiler Semantic plugins.
The Cairo plugins can be used in Scarb to expand the capabilities of the Cairo compiler, empowering users to implement
extensions that would only be possible in the compiler codebase before.

Within Scarb, Cairo plugins are managed through `CairoPluginRepository`, residing in Scarb configuration object.
The plugin repository stores references to all Cairo plugins that are available during the compilation process.
Plugins appropriate to be used for building a specific package are applied to the compiler database before the compilation.
When using Scarb as a library, Cairo plugins can be defined with configuration builder.
If not specified otherwise, Scarb comes with predefined StarkNet Cairo plugin, that can be used for StarkNet contracts
compilation.
In the future, Scarb will also provide a way to define Cairo plugins as package dependencies.

**The mechanism for requiring Cairo plugins as package dependencies or compiling them in Scarb runtime is not implemented yet.**

Please see [the RFC document](https://github.com/software-mansion/scarb/discussions/227) for more information.

### RootDatabase management within the compiler

Cairo compiler relies on the `RootDatabase` for management of the compilation state.
When compiling a single compilation unit, Scarb creates and configures the database, and passes it to the Cairo compiler.
The compilation of each unit uses a separate database instance, which is then dropped after the compilation is finished.
The database is initialized with Cairo project configuration, obtained from the Scarb compilation unit, conditional
compilation parameters (see [scarb documentation](https://docs.swmansion.com/docs/reference/conditional-compilation)
for more information) and active Cairo plugins.

### Profiles implementation

Profiles provide a way to alter the compiler settings from the Scarb manifest file.
Please see the [scarb documentation](https://docs.swmansion.com/scarb/docs/reference/profiles) for general information
about profile use cases and syntax.

Overrides imposed by profile definition are applied towards the appropriate settings while converting `TomlManifest`
(serialization of Scarb manifest file) into the `Manifest` object.
First, profile definition is read from manifest file, including appropriate profile inheritance attributes.
The full definition of current profile is obtained by merging the parent profile definition with overrides from manifest.
Then, the definition is used to override appropriate setting, e.g. Cairo compiler configuration by merging appropriate
profile definition sections onto the default configuration.

The merging mentioned above is implemented as a merger of two plain `toml::Value` objects.

[pubgrub-algo-docs]: https://nex3.medium.com/pubgrub-2fb6470504f

Expand Down