How To

Design systems

Systems must be the only authoring point for any logic executed on entities. It's strictly forbidden to initiate any manipulation with entities outside of systems. Though, they can call into their dependencies and pieces of logic isolated in their own files and classes.

Systems can't contain any collections of components or entities that persist through multiple frames: everything should be stored in ECS directly.

Systems can contain temporary collections used for data aggregation: take a look at DeferredLoadingSystem.cs.

Systems can't contain a state. All states should be written and stored in ECS Worlds.

How to construct

Systems should have an internal constructor, it clearly indicates that we can't instantiate the system directly but are obliged to use ArchSystemsWorldBuilder.

Systems may accept shared dependencies in the .ctor such as:

Settings that apriori exist in a single instance: Quality, Partitioning, etc.
Pool Providers
Utility functionality (that for some reason is not static)
Configuration dedicated to the given system only, strategies and factories that are injected from the upper level: e.g. IConcurrentBudgetProvider

Every system should be inherited from BaseUnityLoopSystem: it provides common functionality for profiling and error reporting.

Follow Single Resposibility

Every system should execute a limited scope of responsibilities. It should be reflected in its name. There is no strict rule of how many queries it should have but if it grows beyond 200 lines of code consider splitting it into static counterparts.

Normally, every feature is represented by multiple systems that are bound by a certain execution order.

Distribute in groups

Decide in which Game Loop moment (SystemGroup) it will be executed. It purely depends on the system's designation, e.g.:

Physics manipulation should happen in PhysicsSystemGroup
Actions based on Transform.position or Transform.rotation - in PresentationSystemGroup as it is executed after transformation is applied in Unity

Consider creating your own group for a given feature: it will simplify defining dependencies between other groups and systems

How to write queries

There are four ways of writing queries:

Automatic generation is the most preferred one. It can be used in the systems only. But if you have a generic system it's impossible to use it as generic attributes are not supported in the version of C# used in Unity.
Iterating over chunks manually: GetChunkIterator(). The same code is generated by the source generator. You can consider this option in a generic class in some special cases.
World.InlineQuery can be used outside of the system itself and in generic cases. Its performance is very close to generated queries. See ReleasePoolableComponentSystem<T, TProvider> for a reference.
World.Query is the least preferred way of doing things as it uses delegates and can lead to closures unintentionally.

Good practices

Filter out by DeleteEntityIntention: it's undesirable to execute logic over entities marked for destruction

Performance Implication

System's Update should be allocation-free. In order to ensure this consider profiling before sending a feature for review.

When you define a system that operates in a scene context (not a global world) there will be as many instances of this system as worlds are loaded. Thus its Update may be executed many times in the same frame. You should keep the logic as simple as possible so every step of the system takes negligible time.

Every query produces an overhead. Try avoiding introducing multiple queries in the same system with the same filter. Invoke several different methods from one handler instead:

e.g. take a look at CalculateCharacterVelocitySystem: it uses a single entry point ResolveVelocity to calculate every kind of velocity in isolated pieces of logic: ApplyCharacterMovementVelocity, ApplyJump, ApplyGravity, ApplyAirDrag, etc.
another example is FinalizeGltfContainerLoadingSystem: in FinalizeLoading the static method ConfigureGltfContainerColliders.SetupColliders is called to execute logic encapsulated in its own class

Throttling

In order to optimize it further there is a concept of throttling: Systems registered in a Scene World do not execute unless there is a CRDT change from the JavaScript scene. This behavior is implemented in SystemGroupsUpdateGate.cs.

Throttling must be enabled manually by annotating with ThrottlingEnabled attribute. Not every system is suitable for throttling: for example Promises resolution should happen as soon as data is ready.

Enabling Throttling will significantly relieve CPU pressure.

How to manipulate components

World.Get API and Queries provide a ref access to the component. It makes it possible to modify a value type directly without the necessity of setting it back.

⚠️ You must use ref var, otherwise the value will be copied and changes won't be reflected, e.g.: ref var meshRendererComponent = ref world.Get<PrimitiveMeshRendererComponent>(entity);

⚠️ A severe ECS pitfall you may fall into: E.g. you have a query

        protected void TestQuery(in Entity entity, ref StreamableLoadingState state)
        {
            World.Add(entity, new StreamableLoadingResult<TAsset>());
            state.Value = StreamableLoadingState.Status.Finished;
        }

state.Value = StreamableLoadingState.Status.Finished; will not apply the change to the value you expect because you make a structural change (World.Add) before that line and moving between archetypes invalidates ref StreamableLoadingState state

You should be very cautious and apply all structural changes last!

It's super hard to detect as ref StreamableLoadingState state will not throw any exception but will silently point to another cell (in fact the same cell but affected by memcpy) in the reserved array in the archetype's chunk.

So the change will apply eventually to an indefinite component: lucky you if it is just an empty reserved cell but it can be also another valid entity that will be modified accidentally!

⚠️ Every Protobuf component needs to be registered at the ComponentsContainer in order to be correctly detected by systems

How to use "IsDirty" property on Protobuf Components

The partial definition of the specific Protobuf Component should be defined at IDirtyMarker.

ResetDirtyFlagSystem<T>.InjectToWorld(ref builder); should be called before the pertinent System InjectToWorld() call

How to clean up components

Cleaning up can result in (but not limited to) the following actions:

Returning to the pool

E.g.:

if (poolsRegistry.TryGetPool(component.ColliderType, out IComponentPool componentPool))
              componentPool.Release(component.Collider);

in ReleaseOutdatedColliderSystem.cs

Invalidating previously created Promises. Promises represent asynchronous loading operations. On component cleaning-up, you must ensure they don't leak and are properly interrupted.

E.g.:
```
component.Promise.ForgetLoading(world);
```
in CleanUpGltfContainerSystem

Dereferencing assets in a corresponding cache. Check Resources Unloading for more details.

      private void TryReleaseAsset(ref GltfContainerComponent component)
      {
          if (component.Promise.TryGetResult(World, out StreamableLoadingResult<GltfContainerAsset> result) && result.Succeeded)
          {
              cache.Dereference(component.Source, result.Asset);
              entityCollidersSceneCache.Remove(result.Asset);
          }
      }

in ResetGltfContainerSystem.cs

Any custom logic

entityCollidersSceneCache.Remove(result.Asset); to sync available colliders in the scene.

In AvatarInstantiatorSystem.cs complex logic connected to the Custom Skinning:

      private void InternalDestroyAvatar(ref AvatarShapeComponent avatarShapeComponent,
          ref AvatarCustomSkinningComponent skinningComponent, ref AvatarTransformMatrixComponent avatarTransformMatrixComponent,
          AvatarBase avatarBase)
      {
          CommonAvatarRelease(avatarShapeComponent, skinningComponent);
          avatarTransformMatrixComponent.Dispose();
          avatarPoolRegistry.Release(avatarBase);
      }

When to clean up components

When the component is removed

Component removal is initiated by the JavaScript scene logic. The common pattern to detect it is to apply an ECS query in the following scheme:
- The original SDK (Protobug) Component no longer exists. It will only happen when the scene deletes the component. When the whole scene is unloaded it's not the case
- A purely client-side component complementing an SDK one does
- DeleteEntityIntention does not as it should be handled by another query
E.g:
```
   [Query]
   [None(typeof(PBMeshRenderer), typeof(DeleteEntityIntention))]
   private void HandleComponentRemoval(ref PrimitiveMeshRendererComponent rendererComponent)
   {
       ReleaseMaterial.TryReleaseDefault(ref rendererComponent);

       if (poolsRegistry.TryGetPool(rendererComponent.PrimitiveMesh.GetType(), out IComponentPool componentPool))
           componentPool.Release(rendererComponent.PrimitiveMesh);
   }
```

When the entity is destroyed

Entity destruction is initiated by the JavaScript scene logic.

It can be detected by the presence of DeleteEntityIntention component. DeleteEntityIntention is not placed on any entities when the whole scene goes out of scope. DeleteEntityIntention survives only 1 frame. The automatic destruction of marked entities is performed in DestroyEntitiesSystem

E.g:

      [Query]
      [All(typeof(DeleteEntityIntention))]
      private void TryRelease(ref MaterialComponent materialComponent)
      {
          ReleaseMaterial.Execute(World, ref materialComponent, destroyMaterial);
      }

In some scenarios, clean-up logic may be heavy and can produce hiccups. In this case it's advisable to rely on frame-time IConcurrentBudgetProvider. If the budget is not available (and thus the clean-up logic is not executed) it's necessary to set the flag DeferDeletion on the DeleteEntityIntention component so the entity will survive.

E.g:

      [Query]
      private void DestroyAvatar(ref AvatarShapeComponent avatarShapeComponent, ref AvatarTransformMatrixComponent avatarTransformMatrixComponent,
          AvatarBase avatarBase, AvatarCustomSkinningComponent skinningComponent, ref DeleteEntityIntention deleteEntityIntention)
      {
          // Use frame budget for destruction as well
          if (!instantiationFrameTimeBudgetProvider.TrySpendBudget())
          {
              avatarBase.gameObject.SetActive(false);
              deleteEntityIntention.DeferDeletion = true;
              return;
          }

          InternalDestroyAvatar(ref avatarShapeComponent, ref skinningComponent, ref avatarTransformMatrixComponent, avatarBase);
          deleteEntityIntention.DeferDeletion = false;
      }

When the whole world is disposed of

Disposal of the world may happen when the player gets far enough away from the scene.

The only way to detect is to implement IFinalizeWorldSystem and add into the finalizeWorldSystems list on the world creation in a plugin.

E.g in GltfContainerPlugin:
```
var cleanUpGltfContainerSystem =
              CleanUpGltfContainerSystem.InjectToWorld(ref builder, assetsCache, sharedDependencies.EntityCollidersSceneCache);

finalizeWorldSystems.Add(cleanUpGltfContainerSystem);
```
void FinalizeComponents(in Query query); of IFinalizeWorldSystem:
- Currently, this method is ensured to be called on the main thread. However in the future for performance reason, we may revise it. So try to design the implementation in a thread-safe manner
- query corresponds to all entities with CRDTEntity component. Namely all SDK entities. You can safely ignore it
- You can provide your own query for the component of your interest
E.g
```
private static readonly QueryDescription ENTITY_DESTROY_QUERY = new QueryDescription()
         .WithAll<DeleteEntityIntention, GltfContainerComponent>();
```
from the previous example

Generalized scenarios

Some clean-up behaviour is common enough so it's generalized and can be reused across different components.

Returning reference-type components to the pool

All SDK components and some custom ones can be registered in ComponentPoolsRegistry. Then all components that correspond to SDK entities are grabbed by ReleaseReferenceComponentsSystem:

This system automatically returns them to the pool when the entity gets destroyed and the scene dies (by implementing IFinalizeWorldSystem)
This system does not return an SDK component to the pool when the component is removed by the scene (as it does not have knowledge about the client-side counterpart based on which it can infer if the component was ever processed). It should be done by custom code.

For all SDK (Protobuf) components Get and Release behaviour is pretty general and provided by the following extensions:

public static SDKComponentBuilder<T> WithPool<T>(this SDKComponentBuilder<T> sdkComponentBuilder, Action<T> onGet = null, Action<T> onRelease = null) where T: class, new()
      {
          sdkComponentBuilder.pool = new ComponentPool<T>(onGet: onGet, onRelease: onRelease);
          return sdkComponentBuilder;
      }

      /// <summary>
      ///     Provide a custom pool behavior for SDK components, it is a must
      /// </summary>
      public static SDKComponentBuilder<T> WithPool<T>(this SDKComponentBuilder<T> sdkComponentBuilder, IComponentPool<T> componentPool) where T: class, new()
      {
          sdkComponentBuilder.pool = componentPool;
          return sdkComponentBuilder;
      }

      /// <summary>
      ///     A shortcut to create a standard suite for Protobuf components
      /// </summary>
      /// <returns></returns>
      public static SDKComponentBridge AsProtobufComponent<T>(this SDKComponentBuilder<T> sdkComponentBuilder)
          where T: class, IMessage<T>, IDirtyMarker, new() =>
          sdkComponentBuilder.WithProtobufSerializer()
                             .WithPool(SetAsDirty)
                             .Build();

But it's useful to keep in mind that you can provide custom Get and Release behaviour for components being pooled without introducing a whole new system or query. Though the behavior should be simple and "static", and can rely on the data of that component only. For that the following methods exist

public static class ComponentPoolsRegistryExtensions
  {
      public static void AddComponentPool<T>(this IComponentPoolsRegistry componentPoolsRegistry, Action<T> onGet = null, Action<T> onRelease = null) where T: class, new()
      {
          componentPoolsRegistry.AddComponentPool(new ComponentPool<T>(onGet, onRelease));
      }
  }
...
public interface IComponentPoolsRegistry
  { 
      void AddGameObjectPool<T>(Func<T> creationHandler = null, Action<T> onRelease = null, int maxSize = 1024) where T: Component;

      void AddGameObjectPoolDCL<T>(Func<T> creationHandler = null, Action<T> onRelease = null, int maxSize = 1024) where T: Component;

      void AddComponentPool<T>(IComponentPool<T> componentPool) where T: class;
  }

Returning reference-type components to the pool indirectly

You may store poolable references in another component which is not pooled on its own.

E.g:
```
public struct PrimitiveColliderComponent : IPoolableComponentProvider<Collider>
  {
      public Collider Collider;
      public Type ColliderType;
      public PBMeshCollider.MeshOneofCase SDKType;

      Collider IPoolableComponentProvider<Collider>.PoolableComponent => Collider;

      Type IPoolableComponentProvider<Collider>.PoolableComponentType => ColliderType;

      public void Dispose() { }
  }
```
In order to provide basic clean-up behaviour for such scenarios do the following:
- Make your component implement IPoolableComponentProvider<out T>:
  - It can implement as many different T as needed
  - If your component is a structure you should implement Type PoolableComponentType => typeof(T); explicitly. Otherwise it will be boxed to take the default implementation from the interface
- Inject class ReleasePoolableComponentSystem<T, TProvider> with final arguments' types and add it to the list
E.g
```
finalizeWorldSystems.Add(ReleasePoolableComponentSystem<IPrimitiveMesh, PrimitiveMeshRendererComponent>.InjectToWorld(ref builder, componentPoolsRegistry));
```
- This system automatically returns components to the pool when the entity gets destroyed and the scene dies (by implementing IFinalizeWorldSystem)
- This system does not return an SDK component to the pool when the component is removed by the scene (as it does not have knowledge about the client-side counterpart based on which it can infer if the component was ever processed). It should be done by custom code.

If you require more complex clean up logic you can implement the system on your own and don't rely on this shortcut.

How to test systems

UnitySystemTestBase<TSystem> provides basic functionality for world creation and disposal.

In Tests you can create systems directly by calling a constructor. Consider exposing them by [InternalsVisibleTo] to tests.

Design components

In terms of ECS there is no difference between SDK components (from Proto) and written by us
If you need to enrich an entity (created with an SDK component) with additional data create a separate component: by filtering you will be able to recognize which components are not processed yet
Keep the balance between separate components and state:
- structural changes are expensive operations, if the logic supposes frequent/uncontrolled Adding or Removing components, it's preferred to have a single component and change its state instead.
- otherwise, it's advised to maintain a reasonable segregation and responsibilities distribution between different components
If you need to wait for data that is retrieved asynchronously create an AssetPromise<TAsset, TLoadingIntention>, e.g.:
- Asset Bundles
- GLTF
- Textures
- Any other data from web requests
You may have as many AssetPromises as needed and store them in a component or add them to an entity directly. Keep in mind it's a value type as well so whatever you do, ensure you operate with it by ref, otherwise the state won't be reflected.

Singletons

Some components can be natural singletons (e.g. PhysicsTickComponent): in our case, it means they exist in a single instance per World
They are created by systems in ctor or in Initialize
Then they can be used in Update by other systems
Instead of making a query every time such a component is needed consider caching it and save into a SingleInstanceEntity field

Test local scenes

In order walk around all the existing SDK test scenes that Decentraland provides for testing purposes we already have the way to do it in our project simply choosing, while we're playing the DynamicSceneLoading scene, the option https://sdk-test-scenes.decentraland.zone in the realm dropdown, then these scenes will load (see how to run specific tests scenes in the section below).

But there are sometimes, when we're modifying the implementation of an already existing SDK component or implementing a new one, it's very useful to have a custom scene where to experiment with that SDK component in a isolated way. For this purpose we can create a basic scene with the components that we're interested on test, run it in a local server and connect it to our project. To do this, let's follow the next steps:

1. Download the SDK7 Scene Template: Go to this repo and download it.

2. Run it in a local server: Open a console from the sdk7-scene-template folder and execute npm i. Once it finishes, execute npm run start. This will create the compiled bin file in sdk7-scene-template/bin/ and open the scene in your browser.

3. Modify the scene's code to your liking: Modify the code files inside sdk7-scene-template/src/ to have the scene that you need and observe how it automatically changes in your browser right after introduce the changes. Remember that you have the SDK Documentation available to learn how to implement each component from the SDK side.

4. Remove unnecessary code: Before attempting to test the scene from our Unity project, we will have to remove some unnecessary lines in the index.ts file. Specifically the related to initAssetPacks:

5. Link our project to our local scene: From this point on we can already close the scene in the browser (don't stop the local server in the console). We will go to the DynamicSceneLoading scene and add the new realm http://127.0.0.1:8000 in the EntryPoint game object.

6. Test our local scene from our project: Click on Play in Unity, select the new realm in the dropdown and that's it!

How to run specific Decentraland Test Scenes locally

You could be interested in test some scenes from the Decentraland repository but you only want to execute some of them in a isolated way without having to load all the rest of the scenes.

To do so, you can follow the steps below:

Clone the sdk7-goerli-plaza repository from here.
Modify the dcl-workspace.json file in order to have ONLY the list of scene paths that you want to test. For example:

{
  "folders": [
    {
      "path": "advanced-avatar-swap"
    },
    {
      "path": "avatar-swap"
    },
    {
      "path": "Cube"
    }
  ]
}

In the console, run npm i and then npm run start to run the scenes locally.
Click on Play in Unity and select the realm http://127.0.0.1:8000 in the dropdown.

And that's it! Only the specific scenes that you annotated in dcl-workspace.json will be compiled and executed, and will run together from Unity editor.

Test media streaming

In the current project, media streaming is implemented via external package, namely AVPro. This package is imported during the build process on CI. In order to test media streaming in editor

Import AVPro package. Trial/preview version of it can be downloaded from the official github repository. At the moment of writing this documentation AVPro version used in this project was 2.8.0.
Add csc.rsp file to the Assets folder with only one line -define:AV_PRO_PRESENT in this file.
Close and re-open Unity Editor, so the define symbols are updated.
Verify that symbol is recognized by checking DCL.MediaPlayer.asmdef in inspector. You should not see red sign in front of Define Constraints AV_PRO_PRESENT there.

After importing trial package you can

run MediaStreaming sdk-scene from StaticSceneLoader.unity scene to verify that media streaming is working. This scene includes both audio- and video streams.
run Main.unity scene on https://sdk-team-cdn.decentraland.org/ipfs/streaming-world-main realm and run to coordinates (-12,-3) to observe video played on the big screen (in the open cinema scene)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly