Merging resources

ml-app-deployer 3.14.0 includes support for merging resource files from multiple configuration paths that reference the same resource. This support is initially intended to address the following two use cases:

1. A Data Hub [https://github.com/marklogic/marklogic-data-hub] application has multiple configuration files for the staging and final databases. Currently, multiple calls are made, one for each file. This can introduce issues if the files have different values for arrays - i.e. if two files for the same database have different sets of range indexes. Making a single call for each database avoids these issues and results in faster deployment times.
2. To allow for mlBundle dependencies in ml-gradle to contain configuration files that may need to be merged into an application's configuration files - i.e. a bundle may wish to provide some range indexes.

For version 3.14.0, merging is supported for the following resource types:

• Databases
• Privileges
• Roles
• Servers
• Users

How resources are merged together

Consider an example with two configuration paths, each having a database file with a database-name of "my-database". The first path has a database file with contents of:

{
	"database-name": "sample-app-one-database",
	"triple-index": false,
	"schema-database": "sample-app-schema-database",
	"range-element-index": [
		{
			"collation": "http://marklogic.com/collation/",
			"invalid-values": "reject",
			"localname": "id",
			"namespace-uri": "",
			"range-value-positions": false,
			"scalar-type": "string"
		}
	]
}

And the second configuration path has a database file with contents of:

{
	"database-name": "sample-app-one-database",
	"triple-index": true,
	"triggers-database": "sample-app-triggers-database",
	"range-element-index": [
		{
			"collation": "http://marklogic.com/collation/",
			"invalid-values": "reject",
			"localname": "otherId",
			"namespace-uri": "",
			"range-value-positions": false,
			"scalar-type": "string"
		}
	]
}

The contents of these files are first read in as JSON objects - Jackson ObjectNode objects, to be exact. The second ObjectNode is then merged into the first ObjectNode via the following rules:

• For any common properties with non-complex values, the value from the second object will overwrite that of the first
• For properties with arrays containing non-complex values, the values from the array in the second object will be added to the end of the array in the first object. Duplicates will be detected and discarded.
• For properties with arrays containing complex values (either objects or arrays), the arrays will be "shallow" merged, meaning that the contents of the array in the second object are added to the end of the array in the first object. The merge operation does not try to detect duplicates.
• Properties that exist in one object but not the other will be retained.

Based on the above rules, the output of merging the above two database objects is:

{
	"database-name": "sample-app-one-database",
	"triple-index": true,
	"schema-database": "sample-app-schema-database",
	"triggers-database": "sample-app-triggers-database",
	"range-element-index": [
		{
			"collation": "http://marklogic.com/collation/",
			"invalid-values": "reject",
			"localname": "id",
			"namespace-uri": "",
			"range-value-positions": false,
			"scalar-type": "string"
		},
		{
			"collation": "http://marklogic.com/collation/",
			"invalid-values": "reject",
			"localname": "otherId",
			"namespace-uri": "",
			"range-value-positions": false,
			"scalar-type": "string"
		}
	]
}

Merging JSON and XML resource files

If you are merging resource files, it is recommended to use JSON files. XML files are supported, but these must first be converted into JSON so that the file contents can be merged together as Jackson ObjectNode objects. This process involves using Jackson to unmarshal the XML content into Java objects, and then serializing those Java objects into ObjectNode objects. An issue may arise if the Java objects, as defined in this project are not up-to-date with the latest Manage API schemas - the likely issue being that one or more properties are not unmarshalled because they are not yet recognized by the Java objects. Only using JSON files avoids any such issue from occurring.