Support extension registry when adding or listing extensions in plugins #10062

gastaldi · 2020-06-16T19:33:15Z

This PR introduces support for querying the extension registry compiled from https://github.com/quarkusio/quarkus-extension-catalog.

It will fallback to the original behavior if the registry URL is not specified.

To test:

Make sure to change the Quarkus BOM dependency in the project to a released version which is also present in the registry(eg. 1.5.1.Final)

Maven

mvn quarkus:list-extensions -DsearchPattern=kubernetes -Dregistry=https://github.com/quarkusio/quarkus-extension-catalog/releases/download/v23/registry.json

Gradle

gradle listExtensions --searchPattern=mutiny --registry=https://github.com/quarkusio/quarkus-extension-catalog/releases/download/v23/registry.json

Fixes #9593

ia3andy

No tests?

I know it's a still draft :-)

devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusAddExtension.java

devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusListExtensions.java

devtools/maven/src/main/java/io/quarkus/maven/AddExtensionMojo.java

devtools/maven/src/main/java/io/quarkus/maven/ListExtensionsMojo.java

...-common/src/main/java/io/quarkus/devtools/commands/handlers/CreateProjectCommandHandler.java

.dependabot/config.yml

devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusAddExtension.java

maxandersen · 2020-06-24T12:31:04Z

I'm trying it out now using https://github.com/maxandersen/registrytest but I'm getting No extension found with this pattern using mvn quarkus:list-extensions -DsearchPattern=kubernetes -Dregistry=https://github.com/quarkusio/quarkus-extension-catalog/releases/download/v23/registry.json

what am I missing ?

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListExtensionsCommandHandler.java

gastaldi · 2020-06-24T13:32:50Z

I'm trying it out now using maxandersen/registrytest but I'm getting No extension found with this pattern using mvn quarkus:list-extensions -DsearchPattern=kubernetes -Dregistry=https://github.com/quarkusio/quarkus-extension-catalog/releases/download/v23/registry.json

what am I missing ?

For some reason the platform used in your project is detected as 1.5.2.Final, that's why it doesn't show anything. I'm investigating why that is happening

devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusListExtensions.java

aloubyansky · 2020-06-24T14:08:05Z

I don't see anything wrong here. I understand it's a work in progress.

devtools/maven/src/main/java/io/quarkus/maven/QuarkusProjectMojoBase.java

maxandersen · 2020-06-24T20:07:10Z

I don't see anything wrong here. I understand it's a work in progress.

well goal was to have it in 1.6 behind a feature flag

maxandersen · 2020-07-02T13:29:03Z

could we cut back on that massive registry.json ? dont think we need that big a file to test ;)

independent-projects/tools/registry-descriptor/README.md

maxandersen · 2020-07-02T14:58:09Z

suggestions from call 2020-07-02 👍

1) add "delta" info to platform and releases to capture what is different from the top level extension description
2) make sure semantics of which data ends up in top level extensions and platforms are well defined (like "latest release")
3) versions suggested to be "core-versions" and be a map instead of array.

aloubyansky · 2020-07-03T08:19:46Z

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListExtensionsCommandHandler.java

+                        "No extension found with pattern '%s' while searching in the configured extension registries (using Quarkus Core %s).\n"
+                                + "Falling back to existing platform extensions",
+                        search, quarkusVersion);
+                platformExtensions = searchExtensions(invocation, search);


What is happening here? What is this fallback about?

Why are we having two sources of extensions?

What is happening here? What is this fallback about?
Why are we having two sources of extensions?

The fallback happens to make sure that the search is also performed in the platform BOM that is associated to the project. This will likely happen if the specified registry does not list the Quarkus core version used in the project

The fact that they aren't unified is strange to me. Perhaps I'm misunderstood the use-case.

...dent-projects/tools/registry-descriptor/src/main/java/io/quarkus/registry/model/Release.java

...istry-descriptor/src/test/java/io/quarkus/registry/builder/ExtensionRegistryBuilderTest.java

gastaldi · 2020-07-11T01:09:47Z

While trying to unify the ExtensionRegistry (like requested in #10062 (comment)), it turns out that ExtensionManager needs to be revisited to perform the installation operations (BOMs, dependencies that belong to a platform, etc), that's why the tests are failing. I'm moving this PR to Draft state until I get this sorted out

gastaldi · 2020-07-19T02:02:34Z

Looks like I managed to sort out the ExtensionManager bugs and also handled the special Extension query cases

...s/devtools-common/src/main/java/io/quarkus/devtools/project/extensions/ExtensionManager.java

…latform BOM

Changes addressed

aloubyansky · 2020-07-21T13:41:49Z

devtools/maven/src/main/java/io/quarkus/maven/QuarkusProjectMojoBase.java

-        final DependencyManagement managed = project.getModel().getDependencyManagement();
-        return managed != null ? managed.getDependencies()
-                : Collections.emptyList();
+    private List<Dependency> getManagedDependencies(MavenArtifactResolver resolver) throws IOException {


FYI @ia3andy this is going to help collect platform BOMs from the parents.

devtools/maven/src/main/java/io/quarkus/maven/QuarkusProjectMojoBase.java

independent-projects/tools/devtools-common/pom.xml

Support ExtensionRegistry in AddExtensions Support multiple registries Introduced registry-descriptor and API Introduced ExtensionPredicate Add Extensions now use ExtensionRegistry.lookup Fixes #9593

gastaldi added the release/noteworthy-feature label Jun 16, 2020

gastaldi requested a review from maxandersen June 16, 2020 19:33

boring-cyborg bot added area/devtools Issues/PR related to maven, gradle, platform and cli tooling/plugins area/maven area/platform Issues related to definition and interaction with Quarkus Platform area/gradle Gradle area/dependencies Pull requests that update a dependency file labels Jun 16, 2020

ia3andy previously requested changes Jun 19, 2020

View reviewed changes

maxandersen requested a review from aloubyansky June 24, 2020 05:50

maxandersen reviewed Jun 24, 2020

View reviewed changes

.dependabot/config.yml Outdated Show resolved Hide resolved

maxandersen reviewed Jun 24, 2020

View reviewed changes

devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusAddExtension.java Outdated Show resolved Hide resolved

maxandersen reviewed Jun 24, 2020

View reviewed changes

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListExtensionsCommandHandler.java Outdated Show resolved Hide resolved

maxandersen reviewed Jun 24, 2020

View reviewed changes

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListExtensionsCommandHandler.java Outdated Show resolved Hide resolved

aloubyansky reviewed Jun 24, 2020

View reviewed changes

devtools/gradle/src/main/java/io/quarkus/gradle/tasks/QuarkusListExtensions.java Outdated Show resolved Hide resolved

aloubyansky reviewed Jun 24, 2020

View reviewed changes

devtools/maven/src/main/java/io/quarkus/maven/QuarkusProjectMojoBase.java Outdated Show resolved Hide resolved

gastaldi marked this pull request as ready for review June 29, 2020 18:36

gastaldi requested review from ia3andy, maxandersen and aloubyansky June 30, 2020 11:45

maxandersen reviewed Jul 2, 2020

View reviewed changes

independent-projects/tools/registry-descriptor/README.md Outdated Show resolved Hide resolved

maxandersen reviewed Jul 2, 2020

View reviewed changes

independent-projects/tools/registry-descriptor/README.md Outdated Show resolved Hide resolved

maxandersen reviewed Jul 2, 2020

View reviewed changes

independent-projects/tools/registry-descriptor/README.md Outdated Show resolved Hide resolved

aloubyansky reviewed Jul 3, 2020

View reviewed changes

...dent-projects/tools/registry-descriptor/src/main/java/io/quarkus/registry/model/Release.java Outdated Show resolved Hide resolved

aloubyansky reviewed Jul 3, 2020

View reviewed changes

...istry-descriptor/src/test/java/io/quarkus/registry/builder/ExtensionRegistryBuilderTest.java Outdated Show resolved Hide resolved

gastaldi requested review from aloubyansky and maxandersen July 9, 2020 17:50

gastaldi marked this pull request as draft July 11, 2020 01:40

gastaldi marked this pull request as ready for review July 19, 2020 02:01

gastaldi marked this pull request as draft July 19, 2020 12:43

ia3andy reviewed Jul 20, 2020

View reviewed changes

...s/devtools-common/src/main/java/io/quarkus/devtools/project/extensions/ExtensionManager.java Outdated Show resolved Hide resolved

gastaldi marked this pull request as ready for review July 20, 2020 18:47

gastaldi requested a review from ia3andy July 20, 2020 18:47

gastaldi added 2 commits July 21, 2020 01:26

Make ValueMap.getValue generic-friendly

bb4a531

Collect managed dependencies from POM hierarchy to resolve the used p…

a08b237

…latform BOM

aloubyansky reviewed Jul 21, 2020

View reviewed changes

devtools/maven/src/main/java/io/quarkus/maven/QuarkusProjectMojoBase.java Outdated Show resolved Hide resolved

aloubyansky reviewed Jul 21, 2020

View reviewed changes

independent-projects/tools/devtools-common/pom.xml Show resolved Hide resolved

Support ExtensionRegistry in ListExtensions

f7bafc6

Support ExtensionRegistry in AddExtensions Support multiple registries Introduced registry-descriptor and API Introduced ExtensionPredicate Add Extensions now use ExtensionRegistry.lookup Fixes #9593

aloubyansky approved these changes Jul 21, 2020

View reviewed changes

gastaldi merged commit 15c0e7d into quarkusio:master Jul 21, 2020

gastaldi deleted the registry branch July 21, 2020 23:28

gastaldi mentioned this pull request Jul 22, 2020

Remove standalone module #10902

Merged

gsmet modified the milestone: 1.7.0 - master Jul 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support extension registry when adding or listing extensions in plugins #10062

Support extension registry when adding or listing extensions in plugins #10062

gastaldi commented Jun 16, 2020 •

edited

Loading

ia3andy left a comment •

edited

Loading

maxandersen commented Jun 24, 2020

gastaldi commented Jun 24, 2020

aloubyansky commented Jun 24, 2020

maxandersen commented Jun 24, 2020

maxandersen commented Jul 2, 2020

maxandersen commented Jul 2, 2020 •

edited by gastaldi

Loading

aloubyansky Jul 3, 2020

aloubyansky Jul 3, 2020

gastaldi Jul 3, 2020

aloubyansky Jul 3, 2020

gastaldi commented Jul 11, 2020

gastaldi commented Jul 19, 2020

aloubyansky Jul 21, 2020

Support extension registry when adding or listing extensions in plugins #10062

Support extension registry when adding or listing extensions in plugins #10062

Conversation

gastaldi commented Jun 16, 2020 • edited Loading

To test:

Maven

Gradle

ia3andy left a comment • edited Loading

Choose a reason for hiding this comment

maxandersen commented Jun 24, 2020

gastaldi commented Jun 24, 2020

aloubyansky commented Jun 24, 2020

maxandersen commented Jun 24, 2020

maxandersen commented Jul 2, 2020

maxandersen commented Jul 2, 2020 • edited by gastaldi Loading

aloubyansky Jul 3, 2020

Choose a reason for hiding this comment

aloubyansky Jul 3, 2020

Choose a reason for hiding this comment

gastaldi Jul 3, 2020

Choose a reason for hiding this comment

aloubyansky Jul 3, 2020

Choose a reason for hiding this comment

gastaldi commented Jul 11, 2020

gastaldi commented Jul 19, 2020

aloubyansky Jul 21, 2020

Choose a reason for hiding this comment

gastaldi commented Jun 16, 2020 •

edited

Loading

ia3andy left a comment •

edited

Loading

maxandersen commented Jul 2, 2020 •

edited by gastaldi

Loading