Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged all remaining work from the 1.3.x to master #1127

Merged
merged 35 commits into from
Jan 15, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
da05a64
updated the release workflow to hopefully work on the 1.3.x branch
EricWittmann Oct 23, 2020
4c8bd00
Cherry-pick tests fixes to 1.3.x (#952)
famarting Oct 23, 2020
e8409ca
Added some documentation about configuring the registry UI (#946)
EricWittmann Oct 27, 2020
c54c605
added some documentation about artifact meta-data and custom properti…
EricWittmann Oct 27, 2020
f6e0f79
Added avro and json schema serde documentation (#957)
EricWittmann Oct 28, 2020
fb80f34
Update antora.yml
EricWittmann Oct 28, 2020
8bae786
Add registry client documentation (#955)
carlesarnal Oct 29, 2020
35b2dd1
Various maven related documentation changes (#964)
EricWittmann Oct 29, 2020
3df067b
documented how to override the topic names for kafka storage (#968)
EricWittmann Oct 30, 2020
1f397dc
Added docs for configuring default global rules (#963)
EricWittmann Oct 30, 2020
7928960
Document artifact states 965 (#967)
carlesarnal Oct 30, 2020
4dae0bd
Add Datum and Protobuf docs. (#971)
alesj Nov 2, 2020
c92f07b
Update antora.yml
EricWittmann Nov 3, 2020
a49ac3b
Automated update to Release Version:: 1.3.2.Final
apicurio-ci Nov 3, 2020
86c22bf
Automated update to next Snapshot Version: 1.3.3-SNAPSHOT
apicurio-ci Nov 3, 2020
e4d1588
Update antora.yml
EricWittmann Nov 3, 2020
baf90b6
clean up registry user docs and restructure (#1004)
smccarthy-ie Nov 11, 2020
c87d026
add revew feedback and tidy up (#1013)
smccarthy-ie Nov 17, 2020
0436784
more doc clean up, fomatting, modularization- no technical changes (#…
smccarthy-ie Nov 17, 2020
2ecedeb
Automated update to Release Version:: 1.3.2.Final
EricWittmann Nov 19, 2020
68da2af
Automated update to next Snapshot Version
EricWittmann Nov 19, 2020
ec76f49
Fix serializer to use the right (Registry's) schema. (#1021)
alesj Nov 23, 2020
ab9c219
Add check-period to globalId strategy. (#1025)
alesj Nov 25, 2020
90d9167
Doc release clean up (#1050)
smccarthy-ie Dec 2, 2020
8933e1f
clean up install steps based on verification testing with openshift 4…
smccarthy-ie Dec 16, 2020
7286153
Push docker images using "1.3.x-snapshot" tag
EricWittmann Dec 17, 2020
1e07840
Update verify.yaml
EricWittmann Dec 17, 2020
0d14148
Update release.yaml
EricWittmann Dec 17, 2020
f13d35c
Merged changes from 1.3.x branch
EricWittmann Jan 8, 2021
9d62ab3
Fixed some merge issues (bad choices when resolving conflicts or miss…
EricWittmann Jan 14, 2021
9e149ad
Merge branch 'master' into merges/from-1.3.x
EricWittmann Jan 15, 2021
849b2ba
Retry the schema fetch in the Kafka serializer to account for async s…
EricWittmann Jan 15, 2021
cb605a8
Merge branch 'merges/from-1.3.x' of github.com:EricWittmann/apicurio-…
EricWittmann Jan 15, 2021
a64c674
Merge branch 'master' into merges/from-1.3.x
EricWittmann Jan 15, 2021
d5f8a92
Fix for the check period test to support async storages
EricWittmann Jan 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,8 @@ jobs:
git config --global user.email "[email protected]"
git remote add origin "https://apicurio-ci:${{ secrets.ACCESS_TOKEN }}@github.com/Apicurio/apicurio-registry-examples.git"
git fetch
git checkout master
git branch --set-upstream-to=origin/master
git checkout ${{steps.metadata.outputs.branch}}
git branch --set-upstream-to=origin/${{steps.metadata.outputs.branch}}
git pull

- name: Apicurio Website Checkout
Expand All @@ -82,8 +82,8 @@ jobs:
git config --global user.email "[email protected]"
git remote add origin "https://apicurio-ci:${{ secrets.ACCESS_TOKEN }}@github.com/Apicurio/apicurio.github.io.git"
git fetch
git checkout master
git branch --set-upstream-to=origin/master
git checkout ${{steps.metadata.outputs.branch}}
git branch --set-upstream-to=origin/${{steps.metadata.outputs.branch}}
git pull

- name: Apicurio Playbook Checkout
Expand Down
29 changes: 29 additions & 0 deletions app/src/test/java/io/apicurio/registry/RegistrySerdeTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
import io.apicurio.registry.support.TestCmmn;
import io.apicurio.registry.support.Tester;
import io.apicurio.registry.types.ArtifactType;
import io.apicurio.registry.utils.IoUtil;
import io.apicurio.registry.utils.serde.AvroEncoding;
import io.apicurio.registry.utils.serde.AvroKafkaDeserializer;
import io.apicurio.registry.utils.serde.AvroKafkaSerializer;
Expand Down Expand Up @@ -126,6 +127,34 @@ public void testCachedSchema() throws Exception {
Assertions.assertEquals(id, idStrategy.findId(restClient, artifactId, ArtifactType.AVRO, schema));
}

@Test
public void testCheckPeriod() throws Exception {
Schema schema = new Schema.Parser().parse("{\"type\":\"record\",\"name\":\"myrecord5x\",\"fields\":[{\"name\":\"bar\",\"type\":\"string\"}]}");
String artifactId = generateArtifactId();
byte[] schemaContent = IoUtil.toBytes(schema.toString());
ArtifactMetaData amd = restClient.createArtifact(artifactId, ArtifactType.AVRO, new ByteArrayInputStream(schemaContent));
this.waitForGlobalId(amd.getGlobalId());

long pc = 5000L; // 5seconds check period ...

Map<String, Object> config = new HashMap<>();
config.put(SerdeConfig.CHECK_PERIOD_MS, String.valueOf(pc));
GlobalIdStrategy<Schema> idStrategy = new FindLatestIdStrategy<>();
idStrategy.configure(config, false);

long id1 = idStrategy.findId(restClient, artifactId, ArtifactType.AVRO, schema);
long id2 = idStrategy.findId(restClient, artifactId, ArtifactType.AVRO, schema);
Assertions.assertEquals(id1, id2); // should be less than 5seconds ...
retry(() -> restClient.getArtifactMetaDataByGlobalId(id2));

ArtifactMetaData amd_v2 = restClient.updateArtifact(artifactId, ArtifactType.AVRO, new ByteArrayInputStream(schemaContent));
this.waitForGlobalId(amd_v2.getGlobalId());
Thread.sleep(pc + 1);
retry(() -> Assertions.assertNotEquals(id2, restClient.getArtifactMetaData(artifactId).getGlobalId()));

Assertions.assertNotEquals(id2, idStrategy.findId(restClient, artifactId, ArtifactType.AVRO, schema));
}

@SuppressWarnings("unchecked")
@Test
public void testConfiguration() throws Exception {
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started/master.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ include::attributes.adoc[]
include::assemblies/getting-started/assembly-intro-to-the-registry.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-intro-to-registry-rules.adoc[leveloffset=+1]
ifdef::apicurio-registry[]
include::assemblies/getting-started/assembly-installing-the-registry-docker.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-installing-registry-docker.adoc[leveloffset=+1]
endif::[]
include::assemblies/getting-started/assembly-installing-the-registry-openshift.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-installing-registry-openshift.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-configuring-the-registry.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-managing-registry-artifacts-ui.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-managing-registry-artifacts-api.adoc[leveloffset=+1]
Expand Down
17 changes: 12 additions & 5 deletions docs/local-test-playbook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,23 @@ ui:
url: https://raw.githubusercontent.com/Apicurio/apicurio-docs-ui/master/dist/ui-bundle.zip
snapshot: true

runtime:
cache_dir: ./target/antora-cache

output:
dir: ./target/dist

asciidoc:
attributes:
plantuml-server-url: 'http://www.plantuml.com/plantuml'
plantuml-fetch-diagram: true
mod-loc: partial$
registry-overview: link:assembly-intro-to-the-registry.html[Introduction to Apicurio Registry]
registry-rules: link:assembly-intro-to-registry-rules.html[Introduction to Apicurio Registry rules]
registry-artifact-types: link:assembly-registry-reference.html[Apicurio Registry reference]
registry-rule-types: link:assembly-registry-reference.html[Apicurio Registry reference]
registry-rule-maturity-matrix: link:assembly-registry-reference.html[Apicurio Registry reference]
registry-reference: link:assembly-registry-reference.html[Apicurio Registry artifact reference]
installing-the-registry-docker: link:assembly-installing-registry-docker.html[Installing Apicurio Registry using Docker]
installing-the-registry-openshift: link:assembly-installing-registry-openshift.html[Installing Apicurio Registry on OpenShift]
installing-the-registry-storage-openshift: link:assembly-installing-registry-storage-openshift.html[Installing Apicurio Registry storage on OpenShift]
managing-registry-artifacts-ui: link:assembly-managing-registry-artifacts-ui.html[Managing Apicurio Registry content using the web console]
installing-the-registry-docker: link:assembly-installing-the-registry-docker.html[Installing Apicurio Registry using Docker]
installing-the-registry-openshift: link:assembly-installing-the-registry-openshift.html[Installing Apicurio Registry on OpenShift]
managing-registry-artifacts-api: link:assembly-managing-registry-artifacts-api.html[Managing Apicurio Registry content using the REST API]
kafka-client-serdes: link:assembly-using-kafka-client-serdes.html[Validating schemas using Kafka client serializers/deserializers]
13 changes: 8 additions & 5 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
* xref:getting-started/assembly-intro-to-the-registry.adoc[]
* xref:getting-started/assembly-intro-to-registry-rules.adoc[]
ifdef:getting-started/:apicurio-registry[]
* xref:getting-started/assembly-installing-the-registry-docker.adoc[]
endif:getting-started/:[]
* xref:getting-started/assembly-installing-the-registry-openshift.adoc[]
ifdef::apicurio-registry[]
* xref:getting-started/assembly-installing-registry-docker.adoc[]
endif::[]
* xref:getting-started/assembly-installing-registry-openshift.adoc[]
* xref:getting-started/assembly-installing-registry-storage-openshift.adoc[]
* xref:getting-started/assembly-configuring-the-registry.adoc[]
* xref:getting-started/assembly-managing-registry-artifacts-ui.adoc[]
* xref:getting-started/assembly-using-kafka-client-serdes.adoc[]
* xref:getting-started/assembly-managing-registry-artifacts-api.adoc[]
* xref:getting-started/assembly-managing-registry-artifacts-maven.adoc[]
* xref:getting-started/assembly-using-the-registry-client.adoc[]
* xref:getting-started/assembly-using-kafka-client-serdes.adoc[]
* xref:getting-started/assembly-registry-reference.adoc[]
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
include::{mod-loc}shared/all-attributes.adoc[]

[id="configuring-the-registry"]
= Configuring {registry} on OpenShift
= Configuring {registry} deployment on OpenShift

This chapter explains how to configure optional settings for {registry} health checks on OpenShift:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="installing-the-registry-docker"]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
// Metadata created by nebel
include::{mod-loc}shared/all-attributes.adoc[]

[id="installing-registry-ocp"]
= Installing {registry} on OpenShift

This chapter explains how to install {registry}:

* xref:installing-registry-operatorhub[]
//* xref:installing-registry-kafka-streams-template-storage[]

.Prerequisites
* {registry-overview}

NOTE: You can install more than one instance of {registry} depending on your environment. The number of instances depends on the number and type of artifacts stored in {registry} and on your chosen storage option.

ifdef::apicurio-registry[]
.Additional resources
* For details on building from source, see https://github.com/Apicurio/apicurio-registry.
endif::[]

//INCLUDES
//include::{mod-loc}getting-started/proc_installing-registry-kafka-streams-template-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-installing-registry-operatorhub.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,60 +1,49 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="installing-the-registry"]
= Installing {registry} on OpenShift

This chapter explains how to first install {registry} and then how to set up your chosen registry storage option: {kafka-streams}, embedded Infinispan, or PostgreSQL database.

.Prerequisites
* {registry-overview}
[id="installing-registry-storage"]
= Installing {registry} storage on OpenShift

.{registry} installation
* xref:installing-registry-operatorhub[]
//* xref:installing-registry-kafka-streams-template-storage[]
This chapter explains how to install and configure your chosen registry storage option: {kafka-streams}, embedded Infinispan, or PostgreSQL database.

.{kafka-streams} storage
* xref:installing-kafka-streams-operatorhub[]
* xref:installing-kafka-streams-operatorhub[]
* xref:setting-up-kafka-streams-storage[]
* xref:registry-kafka-topic-names[]

ifdef::apicurio-registry[]
.Embedded Infinispan storage
* xref:setting-up-infinispan-storage[]
.Embedded Infinispan storage
* xref:setting-up-infinispan-storage[]

.PostgreSQL database storage
* xref:installing-postgresql-operatorhub[]
* xref:setting-up-postgresql-storage[]
* xref:setting-up-postgresql-storage[]
endif::[]

ifdef::rh-service-registry[]
.Embedded Infinispan storage (Technology Preview)
* xref:setting-up-infinispan-storage[]
* xref:setting-up-infinispan-storage[]

.PostgreSQL database storage (Technology Preview)
* xref:installing-postgresql-operatorhub[]
* xref:setting-up-postgresql-storage[]
* xref:setting-up-postgresql-storage[]

[IMPORTANT]
====
{registry} storage in Infinispan or PostgreSQL is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production.
{registry} storage in Infinispan or PostgreSQL is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production.

These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview.
====
endif::[]

NOTE: You can install more than one instance of {registry} depending on your environment. The number of instances depends on your storage option, for example, your Kafka, Infinispan, or database cluster configuration, and on the number and type of artifacts stored in {registry}.

ifdef::apicurio-registry[]
.Additional resources
* For details on building from source, see https://github.com/Apicurio/apicurio-registry.
endif::[]
.Prerequisites
* {installing-the-registry-openshift}

//INCLUDES
//include::{mod-loc}getting-started/proc_installing-registry-kafka-streams-template-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-installing-registry-operatorhub.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-installing-kafka-streams-operatorhub.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-setting-up-kafka-streams-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-kafka-topic-names.adoc[leveloffset=+2]
include::{mod-loc}getting-started/proc-setting-up-infinispan-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-installing-postgresql-operatorhub.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-setting-up-postgresql-storage.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="intro-to-registry-rules"]
= {registry} content rules
//If the assembly covers a task, start the title with a verb in the gerund form, such as Creating or Configuring.

This chapter introduces the optional rules used to govern registry content and provides details on the available rule types:
This chapter introduces the optional rules used to govern registry content and provides details on the available rule configuration:

* xref:registry-rules[]
* xref:registry-rules-apply[]
* xref:registry-rules-work[]

* xref:registry-rules-work[]
* xref:registry-rules-config[]

//INCLUDES
include::{mod-loc}getting-started/con-registry-rules.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="intro-to-the-registry"]
Expand All @@ -14,6 +13,7 @@ This chapter introduces {registry} concepts and features and provides details on
* xref:registry-rest-api[]
* xref:registry-security[]
* xref:registry-storage[]
* xref:registry-web-console[]
* xref:client-serde[]
* xref:kafka-connect[]
* xref:registry-demo[]
Expand All @@ -26,6 +26,7 @@ include::{mod-loc}getting-started/con-registry-web-console.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-security.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-rest-api.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-web-console.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-serde.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-kafka-connect-converters.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-demo.adoc[leveloffset=+1]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="managing-registry-artifacts-api"]
= Managing {registry} content using the REST API
= Managing {registry} content using the REST API
//If the assembly covers a task, start the title with a verb in the gerund form, such as Creating or Configuring.

This chapter explains how to manage artifacts stored in the registry using the Registry REST API. This includes using Registry REST API commands, a Maven plug-in, or a Java client application:
This chapter describes the Registry REST API and shows how to use it manage artifacts stored in the registry:

* xref:registry-rest-api[]
* xref:managing-artifacts-using-rest-api[]
* xref:managing-artifacts-using-maven-plugin[]
* xref:managing-artifacts-using-client-code[]

.Additional resources
* link:{attachmentsdir}/registry-rest-api.htm[Apicurio Registry REST API documentation]

//INCLUDES
include::{mod-loc}getting-started/con-registry-rest-api.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-managing-artifacts-using-rest-api.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-managing-artifacts-using-maven-plugin.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-managing-artifacts-using-client-code.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Metadata created by nebel
include::{mod-loc}shared/all-attributes.adoc[]

[id="managing-registry-artifacts-maven"]
= Managing {registry} content using the Maven plug-in

This chapter explains how to manage artifacts stored in the registry using the {registry} Maven plug-in:

* xref:managing-artifacts-using-maven-plugin[]

.Prerequisites
* See {registry-overview}
* {registry} must be installed and running in your environment
* Maven must be installed and configured in your environment

//INCLUDES
include::{mod-loc}getting-started/proc-managing-artifacts-using-maven-plugin.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="managing-registry-artifacts-ui"]
Expand All @@ -8,11 +7,13 @@ include::{mod-loc}shared/all-attributes.adoc[]

This chapter explains how to manage artifacts stored in the registry using the {registry} web console. This includes uploading and browsing registry content, and configuring optional rules:

* xref:configuring-registry-ui[]
* xref:adding-artifacts-using-console[]
* xref:browsing-artifacts-using-console[]
* xref:configuring-rules-using-console[]

//INCLUDES
include::{mod-loc}getting-started/proc-configuring-registry-ui.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-adding-artifacts-using-console.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-browsing-artifacts-using-console.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-configuring-rules-using-console.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,21 +1,25 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="artifact-and-rule-types"]
= {registry} reference
[id="registry-artifact-reference"]
= {registry} artifact reference
//If the assembly covers a task, start the title with a verb in the gerund form, such as Creating or Configuring.

This chapter lists the supported artifact types and content rule types that are stored in {registry}.
This chapter provides details on the supported artifact types, states, metadata, and content rules that are stored in {registry}.

* xref:registry-artifact-types[]
* xref:registry-artifact-states[]
* xref:registry-artifact-metadata[]
* xref:registry-rule-types[]
* xref:registry-rule-maturity-matrix[]


.Additional resources
* For more detailed information on artifact and rule types, see the link:{attachmentsdir}/registry-rest-api.htm[Apicurio Registry REST API documentation]
* For more detailed information, see the link:{attachmentsdir}/registry-rest-api.htm[Apicurio Registry REST API documentation]

//INCLUDES
include::{mod-loc}getting-started/ref-registry-artifact-types.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-artifact-states.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-artifact-metadata.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-rule-types.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-rule-maturity-matrix.adoc[leveloffset=+1]
Loading