Deprecate/Private/Fork/Rename model changes Proposal

[davidfirst]

Edit: See the final implementation details here: #5111

Goals:

1. Enable deprecating a component locally, showing the component as "staged" in bit-status.
2. Allow export the component with this change and update the remote without having to run "bit deprecate --remote".
3. Same goes to the new "private" prop we plan to add. (Scope could have private components that other scopes, even with the same org can't use).
4. Allow to set deprecated/private fields on a lane, which will be merged to the component when the lane is merged.

Proposal:
Add a new model object "ComponentMetadata". It'll have the following fields:

type ComponentMetadata = {
  deprecated: boolean;
  private: boolean;
  log: Array<user, email, message, date, change>; // the "change" field is an auto-generated-message, such as "deprecate: true", "private: false" and so on.
  forkedFrom?: BitId; // for the newly created component, we want a reference to the old/renamed/forked one.
  newId?: BitId; // for the old/deprecated component, we want to hold a reference to the new one to suggest you on import to upgrade.
}

Change the components prop in the Lane object.
From: { id: BitId; head: Ref }
To: { id: BitId; head: Ref, metadata: ComponentMetadata }

• The hash of this object will be auto-generated and not content-addressable.
• The Component model will have a reference to this metadata by a new prop metadata.
• Upon bit deprecate this metadata object will be written to the objects dir and be referenced from the Component model.
• bit status will show components that have references to metadata as staged.
• bit export will merge the metadata object into the component and once the export succeeded, will delete the metadata object locally and remove the reference from the Component model.
• when checked out to a lane, bit export will export the metadata objects as well.

TBD: how do we know when a lane is staged due to changes in the metadata? maybe add exported: boolean field.

[davidfirst]

I discussed this with @ranm8 , and we'll be going another route.

• the metadata will be saved inside the Version object. As such, once a component is deprecated, bit status will show it as modified, and only the next tag/snap will change it to "staged". This way, no need a dedicated "log" field. It's part of the Version.
• to retrieve this metadata, go to the head prop of the Component object. Load the Version, and find it in the metadata prop.
• during a normal (non metadata) change, the metadata will be copied from the last version. Same as any other change we carry over from version to version (e.g. Env used). This way, the head always have metadata info.
• The scope aspect will expose slots for other aspects, which they could register and add data to this new metadata field. For example, the deprecation aspect could register and add deprecate field. A new aspect encapsulation will register and add private field. Ultimately, the metadata prop will look something like: metadata: { "@teambit/deprecation": { "deprecate": true } }.
• a new field metadata will be added to the .bitmap file. Once a user run a command such as bit deprecate, the changes will be written to the .bitmap file. so then, upon bit status the bit will be shown as modified. Once tagged/snapped, this metadata property will be copied as is into the Version object and then deleted from the .bitmap file.
• during the onComponentLoad hook, the workspace will load this metadata from the .bitmap and populate it into the component object, so then bit tag and bit status could get this info.
• the deprecated property on the Component object, will be deprecated :)

@GiladShoham , FYI. Any comments before I implement this, would be much appreciated.

[GiladShoham]

How do we merge metadata on lanes merge?
Not sure i like this approach. It's duplicating the same data again and again for each version.
Also it binds the metadata to the wrong entity (version instead of component)
I'll think about it more.

[davidfirst]

On lanes, it's going to be exactly the same. You snap the changes on a lane, so the head on the lane points to your changes in the snap.

About the duplication, I discussed this with Ran, his point is (and it makes sense to me) that we basically keep every unchanged metadata from version to version. For example, this piece of data that your component is using React env, it's a metadata of your component, and it is being saved in every new version you create.
In this case, it's even better, because the metadata object is going to be a file (contains JSON object, similar to what was saved in the .bitmap file), and inside the Version object, only the hash will be saved.
This file will be content addressable, so it'll be the same hash if nothing has changed, meaning, almost no duplication (exception the hash itself).

[GiladShoham]

@davidfirst The difference between the env example is that
you might have a component at v1.0.0 using react env and in v2.0.0 using node env (or a different version of the same env). and it's completely fine.
However what happens if you have a component deprecated in v1.0.0 but not in v2.0.0?
When I'll see the component it will appear as not deprecated (2.0.0 is the last version) but what will I see when selecting version 1.0.0 in the combo box in the UI? would I see it deprecated or not?
The duplication is not only about storage, it's about where is the source of truth.
(Anyway of course if we go that approach it's much better to have a model for the component meta to store it, not only because of storage as well)

[davidfirst]

After another discussion with Ran, we want to align all "metadata" types we have on the component.
Instead of having componentMetadata property on the Version object, the data will be saved as any other aspect. E.g.

extensions: [
 { name: "teambit.component/deprecation": data: { "deprecate": true } }
]

In terms of implementation, the Workspace provides an API to write metadata to the .bitmap file. When the component is loaded, the workspace during onComponentLoad, adds the data to the component object (inside extensions/aspects).
Besides the obvious benefit of not having to deal with two places to save data, it's also easier to communicate this to the users. No need to discuss about two types of metadata. In reality, they're all metadata of the component and they're all saved in the Version object.

This also helps with another feature we want. When running bit create --scope some-scope, we don't know to save the default-scope for that component, because in the .bitmap file, the scope is always empty for new components.
Same with env, when creating a new component, we want the ability to get the env from the template and save it for the new component.
With this change, bit create could write into the metadata property of the .bitmap file, something like this:

metadata: { 
  "teambit.workspace/workspace": { scope: "some-scope" },
  "teambit.envs/env": { "custom-react-env": {} }
}

Once the component is loaded, the workspace will override the defaultScope with this scope, and will set the envs according to this config. This will override any other configuration the workspace.jsonc has.

@GiladShoham , any insight here is welcome.

[davidfirst]

Discussed with Gilad.
To remove a property, we should use - sign. For example:

metadata: { 
  "teambit.component/deprecation": { deprecate: "-" },
}

We should not allow setting the minus sign to the aspect-id ("teambit.component/deprecation": "-"``), because the data saved in the model is not only from this metadataprop of .bitmap file, but also the data the aspects adds viaonComponentLoad` slot. Because it's a combination of the two, users could by mistake remove data that shouldn't have been removed.

About changing the defaultScope, a good place to do it is in componentDefaultScopeFromComponentDirAndNameWithoutConfigFile for getting it in the resolveComponentId method.
During the component load, a good place to have it is mergeWithWorkspaceRootConfigs method, here:

mergedObject.extensions = mergedObject.extensions.remove(componentId);
 mergedObject.defaultScope = componentConfigFromWorkspaceToMerge?.defaultScope || workspaceConfig?.defaultScope;

The first line, gets the data from the extensions, and the next one can set it accordingly.

[davidfirst]

Another discussion with Ran, a few more points:

• at first, we thought to remove the data from .bitmap during bit tag. However, problem is that if a user untag, we don't know how to put the data back to the .bitmap file easily. The reason is that in the Version object, it is saved inside the aspect config, but a config can come not only from .bitmap but also from different places, such as workspace.jsonc. In Bit core, we don't know which part of the config to put back in the .bitmap file, and we'll have the aspects to tell us that, which makes it harder for them to implement such a config. Also, it's not obvious on bit-untag when we need to put the config in .bitmap. If for example the user tagged twice, deprecated on the first tag, and untag the second, no need to write to the .bitmap file. For these reasons, we want the data to be removed upon export.
• the default-scope should not be part of this new metadata prop, for a few reasons: 1) it behaves differently, it doesn't get saved inside an aspect config. 2) it's impossible to add it to an existing component because it's practically renaming it, which can't be done this way, it needs to be done via deprecating the other component and creating a new component.

[davidfirst]

About removing the data from .bitmap.
As mentioned in the previous comment, it's problematic to remove it during tag. However, there is also an issue removing it during export. If a user deprecated on lane and snapped and then switch to main, the component will be deprecated unexpectedly.

One option (suggested by Ran) is to remove it during tag, and in order to not loose the data when untagging, this metadata will be saved in a separate file inside the scope.
It needs some more discussion though. Besides the complexity of managing another file, it can be an issue with the following scenario:
Say, I deprecated a component, tagged it and then undeprecated it.
If I untag now, I expect the component to remain undeprecated.
However, because I copy the metadata from this file into the .bitmap, it will be deprecated.

[davidfirst]

• Decided (with Ran and Gilad) to forget about this new file for now. The changes will be only in .bitmap and if a user untagged after deprecating, it won't re-populate the .bitmap with the previous data.
• the name metadata in .bitmap will be changed to config.
• About the merge issue described above, when the same aspect has config in the variants and in bitmap. It's fine to override whatever was in the variants by the bitmap entry as it is more specific. Similar to how component.json works.

[davidfirst]

Eventually ended up implementing this new file to support getting back the config on "bit reset".
See #6358

[davidfirst]

To summarize our options in regard to remove the config data from the .bitmap file:

1. remove it upon tag - two issues. 1) if there is a rule in workspace#variants that behaves differently than this config, after the tag, the component will be modified because now that the config is gone from the .bitmap, the workspace is the strongest source and the previous config, which is now in the model, is ignored. 2) when untagging, we don't know what config to return.
2. remove it upon export - problem is with lanes. if a user is on a lane, made a config change, snapped and switched to main, we don't want this config change to apply on main. Also, the first issue in # 1, still apply but instead of after tagging, it happens after exporting.
3. don't remove it at all - same issue as # 2.

[davidfirst]

Another conclusion - (discussed with Ran, Gilad & Itay).

Add a new prop __specific to the config prop of the extension in the model in case the config is coming from the .bitmap file. This way, we know that this config needs to be stronger than the variants in the workspace.jsonc.

Another option we were considering is to have this __specific prop in the variants instead of the models. It has a few advantages, e.g. for the end user importing a component, it won't be a confusion as to why this config is overridden by the variants but that config does not. (with the previous suggestion, it depends whether the original user used bit-env-set or variants). Also, with this approach, when importing component, by default, it won't be affected by variants, unless the user specifically added "__specific" to the variant.
The drawback of this approach is that it's a braking change for users, they won't understand why the variants stops working. With the original approach it's more of a gradual process.
One possibility to mitigate this breaking change is introducing "bit variants apply", which could apply the variants from workspace and put them in the bitmap by demand. However, as mentioned, we decided to go with the first approach.