Add support for service registry service binding #21618

secondsun · 2021-11-22T14:56:12Z

This pull request adds to the apicurio registry avro extension the ability to consume service bindings injected by the service binding operator. This work is being done as part of adding service registry binding to the rhoas operator and cli.

The SBO provides the following bindings

oauthTokenUrl binds to channels' apicurio.auth.token.endpoint property +
clientId binds to channels' apicurio.auth.client.id property+
oauthRealm binds to channels' apicurio.auth.realm property+
registry binds to channels' apicurio.registry.url and mp.messaging.connector.smallrye-kafka.apicurio.registry.url properties+

== How to Test

=== Prerequisites

OpenShift 4.9 cluster with rhoas-operator and service binding operator installed
rhoas Cli
oc cli
console.redhat.com account

=== Cluster and Services Setup

Ensure that you have logged into your cluster with oc, and have logged into your Application Services account with rhoas login. Create a kafka instance with rhoas kafka create, and a service registry instance with rhoas service-registry create. Once your kafka and service registry entries are ready, create a kafka topic called movies. You can use hroas kafka topic create. You will then execute rhoas cluster connect twice - connecting the kafka instance and your service registry instance separately with each run. When you connect the kafka instance, you will be prompted to update your kafka ACLs and provided a command to do so using a service account that was created. It is important you do this.

We have provided a sample application(https://github.com/secondsun/kafka-avro-schema-quickstart/). This application is deployed to OpenShift as a container. One you have deployed the container, use rhoas cluster bind to first bind the kafka connection, then execute the command again to bind the service registry. Once the pod has restartted, use the route provided by openshift to navigate to /movies.html. You should see a screen showing a random movie title.

There is a version of the application already containerized with the extension here : quuay.io/secondsun/kafka-avro-schema-quickstart:latest

A video of the process is here :

https://www.youtube.com/watch?v=ul9JDjXLquw

quarkus-bot · 2021-11-22T14:56:14Z

Thanks for your pull request!

The title of your pull request does not follow our editorial rules. Could you have a look?

title should preferably start with an uppercase character (if it makes sense!)
title should not start with chore/docs/feat/fix/refactor but be a proper sentence

This message is automatically generated by a bot.

secondsun · 2021-11-22T14:56:47Z

@wtrocki @cescoffier @carlesarnal Here is the Binding extension for service registry. Proceed to comment.

...time/src/main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingConverter.java

quarkus-bot · 2021-11-22T15:11:34Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `0937c8f`

Status	Name	Step	Failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	Failures	Logs	Raw logs

Failures

⚙️ Initial JDK 11 Build #

- Failing: extensions/apicurio-registry-avro/runtime 
! Skipped: devtools/bom-descriptor-json docs extensions/apicurio-registry-avro/deployment and 8 more

📦 extensions/apicurio-registry-avro/runtime

✖ Failed to execute goal net.revelc.code.formatter:formatter-maven-plugin:2.17.0:validate (default) on project quarkus-apicurio-registry-avro: File '/home/runner/work/quarkus/quarkus/extensions/apicurio-registry-avro/runtime/src/main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingConverter.java' has not been previously formatted. Please format file and commit before running validation!

quarkus-bot · 2021-11-22T15:33:35Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `edc2559`

Status	Name	Step	Failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	Failures	Logs	Raw logs

Failures

⚙️ Initial JDK 11 Build #

- Failing: extensions/apicurio-registry-avro/runtime 
! Skipped: devtools/bom-descriptor-json docs extensions/apicurio-registry-avro/deployment and 8 more

📦 extensions/apicurio-registry-avro/runtime

✖ Failed to execute goal io.quarkus:quarkus-bootstrap-maven-plugin:999-SNAPSHOT:extension-descriptor (generate-extension-descriptor) on project quarkus-apicurio-registry-avro: Deployment artifact io.quarkus:quarkus-apicurio-registry-avro-deployment::jar:999-SNAPSHOT is missing the following dependencies from its configuration: io.quarkus:quarkus-kubernetes-service-binding-deployment::jar

...time/src/main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingConverter.java

carlesarnal

LGTM. I just think that maybe we should replace the combination of realm + auth URL by the token endpoint.

quarkus-bot · 2021-11-22T18:57:20Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `1e1e5ee`

Status	Name	Step	Failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	Failures	Logs	Raw logs

Failures

⚙️ Initial JDK 11 Build #

- Failing: extensions/apicurio-registry-avro/runtime 
! Skipped: devtools/bom-descriptor-json docs extensions/apicurio-registry-avro/deployment and 8 more

📦 extensions/apicurio-registry-avro/runtime

✖ Failed to execute goal io.quarkus:quarkus-bootstrap-maven-plugin:999-SNAPSHOT:extension-descriptor (generate-extension-descriptor) on project quarkus-apicurio-registry-avro: Deployment artifact io.quarkus:quarkus-apicurio-registry-avro-deployment::jar:999-SNAPSHOT is missing the following dependencies from its configuration: io.quarkus:quarkus-kubernetes-service-binding-deployment::jar

extensions/apicurio-registry-avro/deployment/pom.xml

cescoffier · 2021-11-23T07:03:56Z

...main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingExtensionProcessor.java

+            BuildProducer<ServiceProviderBuildItem> serviceProvider) {
+        if (capabilities.isPresent(Capability.KUBERNETES_SERVICE_BINDING)) {
+            serviceProvider.produce(
+                    new ServiceProviderBuildItem("io.quarkus.apicurio.registry.binding.ServiceRegistryBindingConverter",


Not related to that PR, but maybe we should have an SPI in the service binding extension with a dedicated build item (which will also about the capability check)

I am not sure what you mean here. Can you elaborate a bit please?

Imagine that instead to do:

if (capabilities.isPresent(Capability.KUBERNETES_SERVICE_BINDING)) { serviceProvider.produce( new ServiceProviderBuildItem("io.quarkus.apicurio.registry.binding.ServiceRegistryBindingConverter", ServiceRegistryBindingConverter.class.getName()));

You do:

producer.produce(new ServiceBindingConverter(ServiceRegistryBindingConverter.class));

Then the SBO processor would consumer this new build item and register the SPI provider.

Of course to avoid a dependency on the SBO extension deployment model, we would need to create an SPI module.

extensions/apicurio-registry-avro/runtime/pom.xml

cescoffier · 2021-11-23T07:15:30Z

...time/src/main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingConverter.java

+            registryUrl = binding.getProperties().get("registryurl");
+        }
+        if (registryUrl != null) {
+            properties.put("mp.messaging.connector.smallrye-kafka.apicurio.registry.url", registryUrl);


I would use "kafka." instead of "mp.messaging.connector.smallrye-kafka."

So, bare clients using Kafka default config will have the apicurio URL configured too.

...time/src/main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingConverter.java

geoand · 2021-11-24T08:30:44Z

Can you please squash the commits?

secondsun · 2021-11-24T22:42:08Z

I've implemented some feedback you all gave and have rerequested reviews. As an additional note : this works with native clients now, but it didn't before.

The image is at quay.io/secondsun/kafka-avro-schema-quickstart:native

secondsun · 2021-11-25T02:29:11Z

Ok, it seems like the binding works, but whatever I did to inject the http client causes vertx's connections to drop and not reconnect. I'll roll that back on Monday, but in the meantime if anyone sees anything that might cause that please comment.

...eployment/src/main/java/io/quarkus/apicurio/registry/avro/ApicurioRegistryAvroProcessor.java

geoand · 2021-11-26T08:49:02Z

Looks like all that's left for this to be ready is to use the Vert.x client, right?

cescoffier

It needs to use the vertx client.

secondsun · 2021-12-02T04:13:23Z

@geoand Sadly no, when I've updated to use the vertx client I get different errors.

For some reason the spi won't use the vertx client and is insisting on the jdk client.

2021-12-02 04:12:03,489 ERROR [io.qua.run.Application] (main) Failed to start application (with profile prod): java.util.ServiceConfigurationError: io.apicurio.rest.client.spi.ApicurioHttpClientProvider: Provider io.apicurio.rest.client.JdkHttpClientProvider not found
	at java.util.ServiceLoader.fail(ServiceLoader.java:589)
	at java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1212)
	at java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:106)
	at java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1265)
	at java.util.ServiceLoader$2.next(ServiceLoader.java:1306)
	at java.util.ServiceLoader$2.next(ServiceLoader.java:1297)
	at java.util.ServiceLoader$3.next(ServiceLoader.java:1395)
	at io.apicurio.rest.client.auth.OidcAuth.resolveProviderInstance(OidcAuth.java:81)
	at io.apicurio.rest.client.auth.OidcAuth.resolveApicurioHttpClient(OidcAuth.java:85)
	at io.apicurio.rest.client.auth.OidcAuth.<init>(OidcAuth.java:65)
	at io.apicurio.registry.serde.AbstractSchemaResolver.configureAuthWithUrl(AbstractSchemaResolver.java:232)
	at io.apicurio.registry.serde.AbstractSchemaResolver.configureClientWithBearerAuthentication(AbstractSchemaResolver.java:203)
	at io.apicurio.registry.serde.AbstractSchemaResolver.configure(AbstractSchemaResolver.java:75)
	at io.apicurio.registry.serde.DefaultSchemaResolver.configure(DefaultSchemaResolver.java:56)
	at io.apicurio.registry.serde.SchemaResolverConfigurer.configure(SchemaResolverConfigurer.java:75)
	at io.apicurio.registry.serde.AbstractKafkaSerDe.configure(AbstractKafkaSerDe.java:68)
	at io.apicurio.registry.serde.AbstractKafkaDeserializer.configure(AbstractKafkaDeserializer.java:62)
	at io.apicurio.registry.serde.avro.AvroKafkaDeserializer.configure(AvroKafkaDeserializer.java:68)
	at io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper.configure(DeserializerWrapper.java:82)
	at io.smallrye.reactive.messaging.kafka.impl.ReactiveKafkaConsumer.<init>(ReactiveKafkaConsumer.java:79)
	at io.smallrye.reactive.messaging.kafka.impl.KafkaSource.<init>(KafkaSource.java:93)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector.getPublisherBuilder(KafkaConnector.java:173)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector_ClientProxy.getPublisherBuilder(KafkaConnector_ClientProxy.zig:159)
	at io.smallrye.reactive.messaging.impl.ConnectorFactories.lambda$wrap$4(ConnectorFactories.java:80)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.createPublisher(ConfiguredChannelFactory.java:174)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.register(ConfiguredChannelFactory.java:136)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.initialize(ConfiguredChannelFactory.java:108)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory_ClientProxy.initialize(ConfiguredChannelFactory_ClientProxy.zig:189)
	at java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
	at io.smallrye.reactive.messaging.extension.MediatorManager.start(MediatorManager.java:189)
	at io.smallrye.reactive.messaging.extension.MediatorManager_ClientProxy.start(MediatorManager_ClientProxy.zig:220)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:41)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.notify(SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.zig:111)
	at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:320)
	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:302)
	at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:73)
	at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:128)
	at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:97)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(LifecycleEventsBuildStep$startupEvent1144526294.zig:80)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(LifecycleEventsBuildStep$startupEvent1144526294.zig:40)
	at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:677)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:104)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:67)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:41)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:120)
	at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)

geoand · 2021-12-02T06:17:45Z

That doesn't seem to be a Quarkus specific issue

cescoffier · 2021-12-02T08:15:40Z

@secondsun check if you don't have a META-INF/services file declaring the wrong client.

secondsun · 2021-12-02T14:53:53Z

@cescoffier I don't have a META-INF/services file; I think it is coming in from a dependency. I will double check though.

cescoffier · 2021-12-02T14:55:33Z

Even in the artifact providing OidcAuth? Look how the lookup is done and why it uses the wrong client.

carlesarnal · 2021-12-02T14:58:57Z

@secondsun this seems like a bug in the library providing OidcAuth, I'll take a look as soon as I can.

secondsun · 2021-12-02T15:00:34Z

@cescoffier I'll dig into that. @carlesarnal Do you have some time to take a look with me? You've helped with issues in the past and I think you're more familiar with this dependency.

cescoffier · 2021-12-02T15:01:56Z

I can help, but I don't know Apicurio code at all. @carlesarnal has a lot more knowledge on that code.

carlesarnal · 2021-12-02T15:02:24Z

@cescoffier I'll dig into that. @carlesarnal Do you have some time to take a look with me? You've helped with issues in the past and I think you're more familiar with this dependency.

I think there's a bug in the underlying library but haven't had the time to go through it yet. I'll try to take a look tomorrow.

secondsun · 2021-12-02T15:06:16Z

Thanks guys, I'm taking a look today and will let you guys know what I find / send prs where I need to if I get lucky

cescoffier · 2021-12-02T15:17:08Z

@secondsun

 @BuildStep
    void registerSPIClient(BuildProducer<ServiceProviderBuildItem> services) throws IOException {

        services.produce(
                new ServiceProviderBuildItem(ApicurioHttpClientProvider.class.getName(),
                        "io.apicurio.rest.client.JdkHttpClientProvider"));
    }

This code is not correct, it references the wrong client. Can you try using the Vert.x one?

secondsun · 2021-12-02T15:43:34Z

@cescoffier Above I've attatched a stack trace using the vertx client as you've suggested, I just hadn't committed it because it hasn't worked.

secondsun · 2021-12-02T16:36:01Z

I've worked with @carlesarnal and he's fixed the underlying bug in the registry. I've tested his snapshot with this PR and it fixes all of my issues as well. When central catches up with things we can bump the version of the registry and this should be good to merge. Thanks guys.

secondsun · 2021-12-06T14:56:59Z

@carlesarnal I saw the new registry was released, does that mean this is ready to merge?
@cescoffier I've implemented your feedback and requested a rereview.

cescoffier

LGTM, let's see what @carlesarnal and @geoand say.

geoand · 2021-12-06T15:30:16Z

extensions/apicurio-registry-avro/runtime/pom.xml

+      <dependency>
+        <groupId>org.apache.commons</groupId>
+        <artifactId>commons-lang3</artifactId>
+      </dependency>


Is this now needed for some reason?

Yes. We use it to simplify parsing here : https://github.com/quarkusio/quarkus/pull/21618/files#diff-a5c8e493cddd3831ed7120b5c27d8f03412a724cb8ca09f0828a08d2c752012bR88

carlesarnal · 2021-12-07T12:12:56Z

@carlesarnal I saw the new registry was released, does that mean this is ready to merge? @cescoffier I've implemented your feedback and requested a rereview.

No, unfortunately, I was on PTO and we missed that release, I'll try to get the fix into the next one. I'll ping you once we do a release so we can update this.

…ervices

secondsun · 2022-01-05T13:46:31Z

@carlesarnal the registry has been updated and this can be merged but right? Cc @cescoffier

cescoffier · 2022-01-05T17:24:33Z

LGTM, @geoand any comment before we click on the big red button?

secondsun · 2022-01-05T21:25:58Z

@cescoffier It looks like there is an outstanding change in your last review, can you clear that?

cescoffier · 2022-01-06T08:33:00Z

Thanks!

carlesarnal · 2022-01-11T09:15:21Z

@carlesarnal the registry has been updated and this can be merged but right? Cc @cescoffier

Sorry, PTO time for me. Yes, it has been updated and this should work just fine (I guess you already checked it).

secondsun changed the title ~~feat: service binding support for service registry~~ feat: Add support for service registry service binding Nov 22, 2021

secondsun changed the title ~~feat: Add support for service registry service binding~~ Add support for service registry service binding Nov 22, 2021

wtrocki reviewed Nov 22, 2021

View reviewed changes

...time/src/main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingConverter.java Outdated Show resolved Hide resolved

wtrocki reviewed Nov 22, 2021

View reviewed changes

...time/src/main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingConverter.java Outdated Show resolved Hide resolved

secondsun force-pushed the srs_binding_enhancement branch from 0937c8f to edc2559 Compare November 22, 2021 15:15

cescoffier requested review from cescoffier and geoand November 22, 2021 15:26

carlesarnal reviewed Nov 22, 2021

View reviewed changes

...time/src/main/java/io/quarkus/apicurio/registry/binding/ServiceRegistryBindingConverter.java Outdated Show resolved Hide resolved

carlesarnal approved these changes Nov 22, 2021

View reviewed changes

secondsun force-pushed the srs_binding_enhancement branch from edc2559 to 1e1e5ee Compare November 22, 2021 18:40

secondsun force-pushed the srs_binding_enhancement branch from 1e1e5ee to a1f0237 Compare November 22, 2021 19:22

cescoffier requested changes Nov 23, 2021

View reviewed changes

secondsun mentioned this pull request Nov 23, 2021

fix: changing srs service bindings to use only token endpoint redhat-developer/app-services-operator#292

Merged

secondsun force-pushed the srs_binding_enhancement branch from 5726377 to cc74e12 Compare November 24, 2021 22:39

secondsun requested a review from cescoffier November 24, 2021 22:40

cescoffier requested changes Nov 25, 2021

View reviewed changes

...eployment/src/main/java/io/quarkus/apicurio/registry/avro/ApicurioRegistryAvroProcessor.java Outdated Show resolved Hide resolved

cescoffier requested changes Nov 26, 2021

View reviewed changes

secondsun force-pushed the srs_binding_enhancement branch from cc74e12 to d171f13 Compare December 2, 2021 15:45

secondsun requested a review from cescoffier December 6, 2021 14:55

secondsun force-pushed the srs_binding_enhancement branch from d171f13 to 5be0e25 Compare December 6, 2021 14:56

cescoffier requested changes Dec 6, 2021

View reviewed changes

geoand reviewed Dec 6, 2021

View reviewed changes

feat: service binding support for service registry with application s…

7ce16d2

…ervices

secondsun force-pushed the srs_binding_enhancement branch from 5be0e25 to 7ce16d2 Compare January 4, 2022 15:09

geoand approved these changes Jan 5, 2022

View reviewed changes

secondsun requested a review from cescoffier January 5, 2022 21:25

cescoffier approved these changes Jan 6, 2022

View reviewed changes

cescoffier merged commit 9a40f35 into quarkusio:main Jan 6, 2022

quarkus-bot bot added this to the 2.7 - main milestone Jan 6, 2022

Add support for service registry service binding #21618

Add support for service registry service binding #21618

Conversation

secondsun commented Nov 22, 2021 • edited Loading

quarkus-bot bot commented Nov 22, 2021

secondsun commented Nov 22, 2021

quarkus-bot bot commented Nov 22, 2021 • edited Loading

Failing Jobs - Building 0937c8f

Failures

⚙️ Initial JDK 11 Build #

📦 extensions/apicurio-registry-avro/runtime

quarkus-bot bot commented Nov 22, 2021 • edited Loading

Failing Jobs - Building edc2559

Failures

⚙️ Initial JDK 11 Build #

📦 extensions/apicurio-registry-avro/runtime

carlesarnal left a comment

Choose a reason for hiding this comment

quarkus-bot bot commented Nov 22, 2021 • edited Loading

Failing Jobs - Building 1e1e5ee

Failures

⚙️ Initial JDK 11 Build #

📦 extensions/apicurio-registry-avro/runtime

cescoffier Nov 23, 2021

Choose a reason for hiding this comment

geoand Nov 23, 2021

Choose a reason for hiding this comment

cescoffier Nov 25, 2021

Choose a reason for hiding this comment

cescoffier Nov 23, 2021

Choose a reason for hiding this comment

geoand commented Nov 24, 2021

secondsun commented Nov 24, 2021

secondsun commented Nov 25, 2021

geoand commented Nov 26, 2021

cescoffier left a comment

Choose a reason for hiding this comment

secondsun commented Dec 2, 2021

geoand commented Dec 2, 2021

cescoffier commented Dec 2, 2021

secondsun commented Dec 2, 2021 • edited Loading

cescoffier commented Dec 2, 2021

carlesarnal commented Dec 2, 2021

secondsun commented Dec 2, 2021

cescoffier commented Dec 2, 2021

carlesarnal commented Dec 2, 2021

secondsun commented Dec 2, 2021

cescoffier commented Dec 2, 2021

secondsun commented Dec 2, 2021 • edited Loading

secondsun commented Dec 2, 2021

secondsun commented Dec 6, 2021

cescoffier left a comment

Choose a reason for hiding this comment

geoand Dec 6, 2021

Choose a reason for hiding this comment

secondsun Dec 6, 2021

Choose a reason for hiding this comment

carlesarnal commented Dec 7, 2021 • edited Loading

secondsun commented Jan 5, 2022

cescoffier commented Jan 5, 2022 • edited Loading

secondsun commented Jan 5, 2022

cescoffier commented Jan 6, 2022

carlesarnal commented Jan 11, 2022

secondsun commented Nov 22, 2021 •

edited

Loading

quarkus-bot bot commented Nov 22, 2021 •

edited

Loading

Failing Jobs - Building `0937c8f`

quarkus-bot bot commented Nov 22, 2021 •

edited

Loading

Failing Jobs - Building `edc2559`

quarkus-bot bot commented Nov 22, 2021 •

edited

Loading

Failing Jobs - Building `1e1e5ee`

secondsun commented Dec 2, 2021 •

edited

Loading

secondsun commented Dec 2, 2021 •

edited

Loading

carlesarnal commented Dec 7, 2021 •

edited

Loading

cescoffier commented Jan 5, 2022 •

edited

Loading