How to add user-defined artifact metadata?

[ajborley]

There may be metadata that users would like to have associated with an artifact stored in Apicurio that does not fit into any of the currently available metadata fields, which are:

• name (string)
• description (string)
• labels (array of strings)
• createdBy (string)
• createdOn (long)
• modifiedBy (string)
• modifiedOn (long)
• id (string)
• version (int)
• type (ArtifactType)
• globalId (long)
• state (ArtifactState)

It would be useful to have a metadata field that can contain more free-form user-definable information. The labels metadata field can hold a list of Strings, although these seem to be more akin to tags to be used for organization/searching and are displayed as such in the UI.

In the Kubernetes world, resources have name-value pairs in the metadata.labels field, which is often used to hold additional information about the resource.

Apicurio could add an extra metadata field for name-value pairs, perhaps named properties (or maybe additionalProperties? Naming is hard :) ) which would be defined as a Java Map<String, String> object. This would cover simple name-value pairs, although it wouldn't allow more complex structures.

In addition to user-defined metadata, the ibmcompat API has fields in it's metadata objects that don't easily map to the Apicurio artifact metadata fields (such as the schema name (1) and state.comment fields (2)). Rather than adding those fields directly to the artifact metadata, a properties field could hold the information internally, and the ibmcompat API implementation could populate the additional fields using that data.

I'd be interested to hear what people think about this, and happy to provide a PR if it's though to be a good idea.

(1)

apicurio-registry/app/src/main/java/io/apicurio/registry/ibmcompat/model/SchemaSummary.java

Lines 35 to 40 in b7f2f41

	@JsonProperty("name")
	@NotNull
	@Size(min = 1, max = 100)
	public String getName() {
	return name;
	}

(2)

apicurio-registry/app/src/main/java/io/apicurio/registry/ibmcompat/model/SchemaState.java

Lines 72 to 77 in 64a78a7

	@JsonProperty("comment")
	@Size(max = 300)
	@JsonInclude(JsonInclude.Include.NON_NULL)
	public String getComment() {
	return comment;
	}

[ajborley]

Accidentally opened this half-way through creation. Reopening now :)

[antonmry]

This is something we were testing and it would help to have this feature. In our tests, only the 'description' was editable using the API and we would like to store information about the user who created the artifact. The field createdBy seems a great option but we didn't find how to populate it using the standard API (we didn't try with ccompat or ibmcompat).

This part could be also better documented at the API level to understand what fields aren't editable. There are examples but not a complete list.

https://github.com/Apicurio/apicurio-registry/blob/6cf2db6064581ac15e60770c5d75f1f4226f75a9/common/src/main/resources/META-INF/openapi.json

[EricWittmann]

Related issue: #361

[EricWittmann]

I think this is a must-have and we just need to work out the details. In particular, it would be very good to start with a proposal regarding how to modify the core Registry API to support this feature. I'm very open to suggestions.

I would add that there is some documentation about what fields can be modified here: https://studio-ws.apicur.io/sharing/76f236d3-1312-4548-a4d1-400487188c66#operation/updateArtifactMetaData

There is definitely room for documentation improvement there. I'm open to suggested changes to make things clearer!

Another question we'll need to address is whether users can search for artifacts based on these "additional properties". I would imagine the answer to that is "yes" since I can imagine that being a very powerful feature.

In any case, this is a very important enhancement and there's no reason not to get started on it right away! :)

[EricWittmann]

This has been fixed here: #777