Merging resources

ml-app-deployer 3.14.0 includes support for merging resource files from multiple configuration paths that reference the same resource. This support is initially intended to address the following two use cases:

1. A Data Hub [https://github.com/marklogic/marklogic-data-hub] application has multiple configuration files for the staging and final databases. Currently, multiple calls are made, one for each file. This can introduce issues if the files have different values for arrays - i.e. if two files for the same database have different sets of range indexes. Making a single call for each database avoids these issues and results in faster deployment times.
2. To allow for mlBundle dependencies in ml-gradle to contain configuration files that may need to be merged into an application's configuration files - i.e. a bundle may wish to provide some range indexes.

For version 3.14.0, merging is supported for the following resource types:

• Databases
• Privileges
• Roles
• Servers
• Users

How resources are merged together

Consider an example with two configuration paths, each having a database file with a database-name of "my-database". The first path has a database file with contents of:

{
	"database-name": "sample-app-one-database",
	"triple-index": false,
	"schema-database": "sample-app-schema-database",
	"range-element-index": [
		{
			"collation": "http://marklogic.com/collation/",
			"invalid-values": "reject",
			"localname": "id",
			"namespace-uri": "",
			"range-value-positions": false,
			"scalar-type": "string"
		}
	]
}

And the second configuration path has a database file with contents of:

{
	"database-name": "sample-app-one-database",
	"triple-index": true,
	"triggers-database": "sample-app-triggers-database",
	"range-element-index": [
		{
			"collation": "http://marklogic.com/collation/",
			"invalid-values": "reject",
			"localname": "otherId",
			"namespace-uri": "",
			"range-value-positions": false,
			"scalar-type": "string"
		}
	]
}

The contents of these files are first read in as JSON objects - Jackson ObjectNode objects, to be exact. The second ObjectNode is then merged into the first ObjectNode via the following rules:

• For any common properties with non-complex values, the value from the second object will overwrite that of the first
• For properties with arrays containing non-complex values, the values from the array in the second object will be added to the end of the array in the first object. Duplicates will be detected and discarded.
• For properties with arrays containing complex values (either objects or arrays), the arrays will be "shallow" merged, meaning that the contents of the array in the second object are added to the end of the array in the first object. The merge operation does not try to detect duplicates.
• Properties that exist in one object but not the other will be retained.

Based on the above rules, the output of merging the above two database objects is:

{
	"database-name": "sample-app-one-database",
	"triple-index": true,
	"schema-database": "sample-app-schema-database",
	"triggers-database": "sample-app-triggers-database",
	"range-element-index": [
		{
			"collation": "http://marklogic.com/collation/",
			"invalid-values": "reject",
			"localname": "id",
			"namespace-uri": "",
			"range-value-positions": false,
			"scalar-type": "string"
		},
		{
			"collation": "http://marklogic.com/collation/",
			"invalid-values": "reject",
			"localname": "otherId",
			"namespace-uri": "",
			"range-value-positions": false,
			"scalar-type": "string"
		}
	]
}

Merging JSON and XML resource files

If you are merging resource files, it is recommended to use JSON files. XML files are supported, but these must first be converted into JSON so that the file contents can be merged together as Jackson ObjectNode objects. This process involves using Jackson to unmarshal the XML content into Java objects, and then serializing those Java objects into ObjectNode objects.

This translation from XML to JSON depends on a set of annotations on Java objects in this project. Unfortunately, these annotations are not guaranteed to be up-to-date with the latest release of MarkLogic - there's typically a gap between the time MarkLogic is released (which may or may not add new elements to the Manage API schemas) and when these changes are incorporated into this project. Due to this, merging XML files is risky because the files may contain elements that are not yet mapped by the annotated Java objects, which would cause the elements to be ignored. Thus, if you need to merge resources files, JSON is the recommended format for configuration files as this annotated-based translation approach is not needed for JSON.