feat: registryCustomizations

.sfdevrc.json

@@ -3,14 +3,14 @@
     "compile": {
       "clean": "if-file-deleted",
       "command": "tspc -p . --pretty --incremental",
-      "files": ["src/**/*.ts", "src/registry/*.json", "**/tsconfig.json", "messages/**"],
+      "files": ["src/**/*.ts", "src/registry/**/*.json", "**/tsconfig.json", "messages/**"],
       "output": ["lib/**", "*.tsbuildinfo"]
     },
     "test": {
       "dependencies": ["test:only", "test:compile", "test:registry-validation", "test:snapshot"]
     },
     "test:only": {
-      "command": "nyc mocha \"test/**/*.test.ts\" --exclude \"test/registry/registryValidation.test.ts\" --exclude \"test/snapshot\"",
+      "command": "nyc mocha \"test/**/*.test.ts\" --exclude \"test/registry/registryValidation.test.ts\" --exclude \"test/snapshot/**\"",
       "env": {
         "FORCE_COLOR": "2"
       },

HANDBOOK.md

@@ -102,8 +102,6 @@ The config file consists of a handful of different indexes.
 
 This file is large and luckily, not entirely crafted by hand. And because new metadata types are being added to the platform each release, we’ll need to update the [Metadata registry file]. The update-registry module in the scripts folder automatically updates the registry as best it can using a describeMetadata() call against a provided Salesforce org, without overwriting manual changes. It also attempts to update the indexes listed in the previous section. When generating a new version of the registry, it’s important to manually review the changes to ensure they make sense and aren’t destructive. When in doubt, test functionality with the new version. See [Contributing Metadata Types to the Registry](./contributing/metadata.md) in the development README on how to invoke the script with Yarn.
 
-Unfortunately. we sometimes need to manually change a type definition, albeit rarely. The `typeOverride.json` file allows us to overwrite any updates the script attempts to make that we don’t want to happen.
-
 🛠 _Another issue is we are limited by the permissions and licenses of the org that we are running the update script on, which may return incomplete describe information. We need to address this as soon as possible to not run into type gaps between releases. This is being worked on as of 7/16/2021._
 
 ### Breaking Down a Metadata Type Entry
@@ -148,6 +146,32 @@ registry.types.auradefinitionbundle.directoryName; // => 'aura'
 
 📝 _The registry object is “deeply frozen”, meaning none of its properties, even the nested ones, are mutable. This is to ensure that a consumer cannot change registry information in a process and potentially affect functionality._
 
+### Registry Variants
+
+It's possible that the registry isn't what a user wants
+
+- you're building a new type and want an easy way to test your registry changes
+- you disagree with the choice made in the registry (ex: I wish PermissionSet was decomposed instead of a giant file)
+
+There are two options available if you're in an sfdx project.
+
+- `registryCustomizations`: add your own partial of the the registry that'll be merged into the registry. It's a json object just like the registry itself.
+- `registryPresets`: SDR defines "preset" registryCustomizations and you say which ones you want applied. Each one is a partial, and you can have any or all of them by listing them. Names correspond to something in src/registry/presets, so your project file could use
+
+```json
+  "registryPresets": ["decomposePermissionSet", "decomposeSharingRules"]
+```
+
+if you want only those 2 presets.
+
+The naming convention is `decomposeFoo` where `Foo` is the top-level metadata type, and refers to `/presets/Foo.json`.
+
+This requires that any constructedRegistry know about your project directory. You'll see lots of code passing that around, and passing around constructed RegistryAccess to avoid getting the default if a custom one was already constructed.
+
+Be careful when instantiating classes (ex: ComponentSet) that will default a Registry or RegistryAccess--make sure to pass in a registry to preserve customizations.
+
+**Updating presets** If you do need to update a preset to make a breaking change, it's better to copy it to a new preset and give it a unique name (ex: `decomposeFooV2`). This preserves the existing behavior for existing projects with the old preset.
+
 ### Querying registry data
 
 While it’s perfectly fine to reference the registry export directly, the `RegistryAccess` class was created to make accessing the object a bit more streamlined. Querying types and searching the registry is oftentimes easier and cleaner this way and contains built-in checking for whether or not a metadata type exists. Here’s a comparison of using each:
@@ -683,7 +707,7 @@ Oftentimes we need to work with a unique collection of components. A `ComponentS
 
 Component sets key components using their fullName and type id, i.e. only one pair will ever be present in the set. This implies that anything that conforms to the `MetadataComponent` interface can be added to a set. `SourceComponent`s in fact have particular logic on how they are stored. While only one fullName and type pair can be present in a set, multiple source-backed components can map to the same pair under-the-hood. This is how splitting a CustomObject across multiple package directories is achieved - by treating each version of the component as a separate source-backed component that points to the same fullName and type pair in a set. When component merging happens, the conversion logic is able to combine the source components belonging to the same pair.
 
-The easiest way to build `ComponentSets` is by using the `ComponentSetBuilder` class's static `build()` function.  See <https://github.com/forcedotcom/source-deploy-retrieve/tree/main/src/collections/componentSetBuilder.ts> for details.  Simply pass a configuration object to `build()` and it takes care of the rest, including handling destructive changes!
+The easiest way to build `ComponentSets` is by using the `ComponentSetBuilder` class's static `build()` function. See <https://github.com/forcedotcom/source-deploy-retrieve/tree/main/src/collections/componentSetBuilder.ts> for details. Simply pass a configuration object to `build()` and it takes care of the rest, including handling destructive changes!
 
 Let’s look at some examples of adding components and testing membership:
 
@@ -827,7 +851,7 @@ The order in which we’ve examined the concepts up to this point has been inten
 
 Metadata API deploys and retrieves are asynchronous operations. Once a request has been made, the status of the operation needs to be polled to determine whether or not the request has finished processing. This lifecycle is encapsulated with the `MetadataApiDeploy` and `MetadataApiRetrieve` classes — one instance is meant to map to one operation. They expose useful methods to process the lifecycle and make other requests, which will be illustrated later. These both extend `MetadataTransfer` in order to share common functionality such as polling.
 
-The most common and simplest way to deploy and retrieve metadata is by using a [Component Set](#component-sets).  The easiest way to build `ComponentSets` is by using the `ComponentSetBuilder` class's static `build()` function.  See <https://github.com/forcedotcom/source-deploy-retrieve/tree/main/src/collections/componentSetBuilder.ts> for details.
+The most common and simplest way to deploy and retrieve metadata is by using a [Component Set](#component-sets). The easiest way to build `ComponentSets` is by using the `ComponentSetBuilder` class's static `build()` function. See <https://github.com/forcedotcom/source-deploy-retrieve/tree/main/src/collections/componentSetBuilder.ts> for details.
 
 ### Establishing an org connection
 

METADATA_SUPPORT.md

@@ -8,10 +8,8 @@ Currently, there are 0/0 supported metadata types.
 For status on any existing gaps, please search or file an issue in the [Salesforce CLI issues only repo](https://github.com/forcedotcom/cli/issues).
 To contribute a new metadata type, please see the [Contributing Metadata Types to the Registry](./contributing/metadata.md)
 
-|Metadata Type|Support|Notes|
-|:---|:---|:---|
-
-
+| Metadata Type | Support | Notes |
+| :------------ | :------ | :---- |
 
 ## Next Release (v61)
 
@@ -20,7 +18,7 @@ To contribute a new metadata type, please see the [Contributing Metadata Types t
 
 ## Additional Types
 
-> The following types are supported by this library but not in the coverage reports for either version.  These are typically
+> The following types are supported by this library but not in the coverage reports for either version. These are typically
 >
 > 1. types that have been removed from the metadata API but were supported in previous versions
 > 1. types that are available for pilots but not officially part of the metadata API (use with caution)

Registry Variants TODO.md

@@ -0,0 +1,29 @@
+there's a bug with converter.convert when converting mdapi to source with a long output directory--it'll use the entire directory for customLabels names (see snapshots inside the customLabels test)
+
+Other types seem to get a relative-to-proj path.
+
+Probably doesn't affect the CLI since we normally run the converts from inside a project rather than providing a long proj path.
+
+---
+
+## Open bugs
+
+## QA
+
+1. partial delete: deleting a member locally redeploys without that file
+1. partial delete: "removing" a member remotely deletes it locally if present
+1. conflicts get assigned to the correct place (both directions)
+
+---
+
+## doc work
+
+1. Contributing Metadata (how to test new types using variants before adding to the real registry)
+1. Migration process (convert, delete, convert?)
+
+---
+
+## Features
+
+1. how does it work with packaging
+1. how does it work with STL

package.json

@@ -47,15 +47,12 @@
     "@types/graceful-fs": "^4.1.9",
     "@types/mime": "2.0.3",
     "@types/minimatch": "^5.1.2",
-    "@types/proxy-from-env": "^1.0.4",
-    "@types/shelljs": "^0.8.15",
     "deep-equal-in-any-order": "^1.1.19",
     "deepmerge": "^4.3.1",
     "eslint-plugin-sf-plugin": "^1.17.4",
     "jsforce": "^2.0.0-beta.29",
     "mocha-junit-reporter": "^1.23.3",
     "mocha-snap": "^5.0.0",
-    "shelljs": "0.8.5",
     "ts-node": "^10.9.2",
     "ts-patch": "^3.1.1",
     "typescript": "^5.4.2"
@@ -78,7 +75,7 @@
     "prepare": "sf-install",
     "repl": "node --inspect ./scripts/repl.js",
     "test": "wireit",
-    "test:nuts": "mocha \"test/nuts/local/**/*.nut.ts\" --timeout 500000",
+    "test:nuts": "mocha \"test/nuts/local/**/*.nut.ts\" --timeout 500000 --parallel --job 20",
     "test:nuts:suggest": "mocha \"test/nuts/suggestType/suggestType.nut.ts\" --timeout 10000",
     "test:only": "wireit",
     "test:registry": "mocha ./test/registry/registryCompleteness.test.ts --timeout 50000",
@@ -108,7 +105,7 @@
       "command": "tspc -p . --pretty --incremental",
       "files": [
         "src/**/*.ts",
-        "src/registry/*.json",
+        "src/registry/**/*.json",
         "**/tsconfig.json",
         "messages/**"
       ],
@@ -165,7 +162,7 @@
       "output": []
     },
     "test:only": {
-      "command": "nyc mocha \"test/**/*.test.ts\" --exclude \"test/registry/registryValidation.test.ts\" --exclude \"test/snapshot\"",
+      "command": "nyc mocha \"test/**/*.test.ts\" --exclude \"test/registry/registryValidation.test.ts\" --exclude \"test/snapshot/**\"",
       "env": {
         "FORCE_COLOR": "2"
       },