"proxyMethod":"true" seems broken after upgrading to v0.5.1

[darioklapp]

Bug Report

After the upgrade from v0.4.1 to v0.5.1 a exception: java.lang.IllegalArgumentException: method GET must not have a request body is thrown if any method other than GET is used on a non-finite asset.

Describe the Bug

I am currently working on a demonstrator / proof of concept and am sharing an API via the HTTP consumer pull method. I recently upgraded from v0.4.1 to v0.5.1 and am now encountering an error when proxying HTTP verbs different to GET.

Before the upgrade, I was able to send data to the API interface shared via the connector using PATCH, for example. Now, with v0.5.1, it looks in the logs as if the consumer connector always tries a GET, no matter what method I specify.

For this bug report, let's assume that I share the placeholder Todo REST API from Typicode.

I started two identical connector instances in a Docker-Compose and created the data plane, the policy, the asset and the contract definition on the provider connector. I used the example project from Sampels as a guide.

When creating the asset, I added the fields proxyMethod, proxyBody and proxyQueryParams.

curl --location "http://${CONNECTOR_URL}:${MANAGEMENT_CTX_PORT}/management/v3/assets" \
--header "Content-Type: application/json" \
--data-raw "{
    \"@context\": {
        \"@vocab\": \"https://w3id.org/edc/v0.0.1/ns/\"
    },
    \"@id\": \"${ASSET_ID}\",
    \"properties\": {
        \"name\": \"Todo API\",
        \"content-type\": \"application/json\",
        \"foo\": \"bar\"
    },
    \"dataAddress\": {
        \"type\": \"HttpData\",
        \"name\": \"Todo API\",
        \"baseUrl\": \"https://jsonplaceholder.typicode.com/todos\",
        \"proxyPath\": \"true\",
        \"proxyQueryParams\": \"true\",
        \"proxyBody\": \"true\",
        \"proxyMethod\": \"true\"
    }
}

I then negotiated a contract on the consumer side and started the transfer. The EDR is then also received in a logger component:

I can then use the data from the Receive EDR to access the shared Todo API via the Connector

Request (executed in the consumer container):

curl --location "http://localhost:8185/public/1" \
--header "Authorization: {{AUTH_TOKEN}}"

Response:

{
    "userId": 1,
    "id": 1,
    "title": "delectus aut autem",
    "completed": false
}

So far so good. But if I try to change a todo using a PATCH :

Request (again executed in the consumer container):

curl --location --request PATCH "http://localhost:8185/public/1" \
--header "Authorization: {{AUTH_TOKEN}}" \
--header "Content-Type: application/json" \
--data "{
    \"completed\": true
}"

Response:
400 Bad Request (empty)

Expected Behavior

curl --location --request PATCH "http://localhost:8185/public/1" \
--header "Authorization: {{AUTH_TOKEN}}" \
--header "Content-Type: application/json" \
--data "{
    \"completed\": true
}"

Response:

{
    "userId": 1,
    "id": 1,
    "title": "delectus aut autem",
    "completed": true
}

This is exactly how it worked, at least in version v0.4.1. Unfortunately, this no longer works with the v0.5.1 version and I have no idea what the problem is. Am I doing something wrong or has something changed in the interfaces with the new release?

Observed Behavior

I get the 400 Error (Bad Request). The log suggests that the connector is not sending a PATCH but a GET request to the API.

2024-03-06 11:22:09 DEBUG 2024-03-06T10:22:09.115700632 TransferProcess 49d9da29-084c-4d4b-b5c6-d86cb51dccc2 is now in state STARTED
2024-03-06 11:22:09 DEBUG 2024-03-06T10:22:09.117903993 Sending EDR to http://logger:8080/receiver/urn:participant-2/dynamic/callback
2024-03-06 11:23:18 DEBUG 2024-03-06T10:23:18.605058738 Transferring from HttpData to AsyncStreaming.
2024-03-06 11:23:18 DEBUG 2024-03-06T10:23:18.60584789 Executing HTTP request: https://jsonplaceholder.typicode.com/todos/1
2024-03-06 11:23:39 DEBUG 2024-03-06T10:23:39.808685014 Transferring from HttpData to AsyncStreaming.
2024-03-06 11:23:39 SEVERE 2024-03-06T10:23:39.810524165 JerseyExtension: Unexpected exception caught
2024-03-06 11:23:39 java.lang.IllegalArgumentException: method GET must not have a request body.
2024-03-06 11:23:39     at okhttp3.Request$Builder.method(Request.kt:258)
2024-03-06 11:23:39     at org.eclipse.edc.connector.dataplane.http.params.HttpRequestFactory.toRequest(HttpRequestFactory.java:73)
2024-03-06 11:23:39     at org.eclipse.edc.connector.dataplane.http.params.HttpRequestFactory.toRequest(HttpRequestFactory.java:55)
2024-03-06 11:23:39     at org.eclipse.edc.connector.dataplane.http.pipeline.HttpDataSource.openPartStream(HttpDataSource.java:51)
2024-03-06 11:23:39     at org.eclipse.edc.connector.dataplane.util.sink.AsyncStreamingDataSink.transfer(AsyncStreamingDataSink.java:58)
2024-03-06 11:23:39     at org.eclipse.edc.connector.dataplane.framework.pipeline.PipelineServiceImpl.transfer(PipelineServiceImpl.java:113)
2024-03-06 11:23:39     at org.eclipse.edc.connector.dataplane.api.controller.DataPlanePublicApiController.handle(DataPlanePublicApiController.java:144)
2024-03-06 11:23:39     at org.eclipse.edc.connector.dataplane.api.controller.DataPlanePublicApiController.patch(DataPlanePublicApiController.java:92)
2024-03-06 11:23:39     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-03-06 11:23:39     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2024-03-06 11:23:39     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-03-06 11:23:39     at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2024-03-06 11:23:39     at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
2024-03-06 11:23:39     at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
2024-03-06 11:23:39     at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
2024-03-06 11:23:39     at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159)
2024-03-06 11:23:39     at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
2024-03-06 11:23:39     at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
2024-03-06 11:23:39     at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
2024-03-06 11:23:39     at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
2024-03-06 11:23:39     at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:261)
2024-03-06 11:23:39     at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
2024-03-06 11:23:39     at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
2024-03-06 11:23:39     at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
2024-03-06 11:23:39     at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
2024-03-06 11:23:39     at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
2024-03-06 11:23:39     at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
2024-03-06 11:23:39     at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:240)
2024-03-06 11:23:39     at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:697)
2024-03-06 11:23:39     at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
2024-03-06 11:23:39     at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
2024-03-06 11:23:39     at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:357)
2024-03-06 11:23:39     at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311)
2024-03-06 11:23:39     at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
2024-03-06 11:23:39     at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
2024-03-06 11:23:39     at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:529)
2024-03-06 11:23:39     at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
2024-03-06 11:23:39     at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1381)
2024-03-06 11:23:39     at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
2024-03-06 11:23:39     at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
2024-03-06 11:23:39     at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
2024-03-06 11:23:39     at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1303)
2024-03-06 11:23:39     at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
2024-03-06 11:23:39     at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:192)
2024-03-06 11:23:39     at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
2024-03-06 11:23:39     at org.eclipse.jetty.server.Server.handle(Server.java:563)
2024-03-06 11:23:39     at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)
2024-03-06 11:23:39     at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)
2024-03-06 11:23:39     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)
2024-03-06 11:23:39     at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287)
2024-03-06 11:23:39     at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
2024-03-06 11:23:39     at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
2024-03-06 11:23:39     at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
2024-03-06 11:23:39     at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)
2024-03-06 11:23:39     at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)
2024-03-06 11:23:39     at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)
2024-03-06 11:23:39     at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)
2024-03-06 11:23:39     at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
2024-03-06 11:23:39     at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
2024-03-06 11:23:39     at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
2024-03-06 11:23:39     at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
2024-03-06 11:23:39     at java.base/java.lang.Thread.run(Thread.java:833)

Steps to Reproduce

Build the connector unsing this build.gradle.kts

plugins {
    `java-library`
    id("application")
    alias(libs.plugins.shadow)
}

repositories {
    mavenCentral()
}

dependencies {
    runtimeOnly(libs.edc.control.plane.core)
    runtimeOnly(libs.edc.control.plane.api)
    runtimeOnly(libs.edc.control.plane.api.client)
    runtimeOnly(libs.edc.dsp)
    runtimeOnly(libs.edc.configuration.filesystem)
    runtimeOnly(libs.edc.vault.filesystem)
    runtimeOnly(libs.edc.iam.mock)
    runtimeOnly(libs.edc.management.api)
    runtimeOnly(libs.edc.data.plane.selector.core)
    runtimeOnly(libs.edc.data.plane.selector.api)
    runtimeOnly(libs.edc.data.plane.core)
    // runtimeOnly(libs.edc.data.plane.api) // for 0.4.1
    runtimeOnly(libs.edc.data.plane.control.api) // for 0.5.1
    runtimeOnly(libs.edc.data.plane.public.api) // for 0.5.1
    runtimeOnly(libs.edc.api.observability)
    runtimeOnly(libs.edc.transfer.data.plane)
    runtimeOnly(libs.edc.transfer.pull.http.dynamic.receiver)
}

application {
    mainClass.set("org.eclipse.edc.boot.system.runtime.BaseRuntime")
}

var distTar = tasks.getByName("distTar")
var distZip = tasks.getByName("distZip")

tasks.withType<com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar> {
    mergeServiceFiles()
    archiveFileName.set("connector.jar")
    dependsOn(distTar, distZip)
}

This is the content of the libs.versions.toml

[metadata]
format.version = "1.1"

[versions]
assertj = "3.25.1"
awaitility = "4.2.0"
edc = "0.5.1"
jakarta-json = "2.0.1"
junit-pioneer = "2.2.0"
jupiter = "5.10.1"
okhttp-mockwebserver = "5.0.0-alpha.12"
openTelemetry = "1.18.0"
restAssured = "5.4.0"
rsApi = "3.1.0"
testcontainers = "1.19.4"
kafkaClients = "3.6.1"


[libraries]
assertj = { module = "org.assertj:assertj-core", version.ref = "assertj" }
awaitility = { module = "org.awaitility:awaitility", version.ref = "awaitility" }
edc-api-core = { module = "org.eclipse.edc:api-core", version.ref = "edc" }
edc-api-observability = { module = "org.eclipse.edc:api-observability", version.ref = "edc" }
edc-auth-tokenbased = { module = "org.eclipse.edc:auth-tokenbased", version.ref = "edc" }
edc-boot = { module = "org.eclipse.edc:boot", version.ref = "edc" }
edc-build-plugin = { module = "org.eclipse.edc.edc-build:org.eclipse.edc.edc-build.gradle.plugin", version.ref = "edc" }
edc-configuration-filesystem = { module = "org.eclipse.edc:configuration-filesystem", version.ref = "edc" }
edc-connector-core = { module = "org.eclipse.edc:connector-core", version.ref = "edc" }
edc-control-plane-api-client = { module = "org.eclipse.edc:control-plane-api-client", version.ref = "edc" }
edc-control-plane-api = { module = "org.eclipse.edc:control-plane-api", version.ref = "edc" }
edc-control-plane-core = { module = "org.eclipse.edc:control-plane-core", version.ref = "edc" }
edc-control-plane-spi = { module = "org.eclipse.edc:control-plane-spi", version.ref = "edc" }
edc-data-plane-api = { module = "org.eclipse.edc:data-plane-api", version.ref = "edc" }
edc-data-plane-aws-s3 = { module = "org.eclipse.edc:data-plane-aws-s3", version.ref = "edc" }
edc-data-plane-azure-storage = { module = "org.eclipse.edc:data-plane-azure-storage", version.ref = "edc" }
edc-data-plane-client = { module = "org.eclipse.edc:data-plane-client", version.ref = "edc" }
edc-data-plane-core = { module = "org.eclipse.edc:data-plane-core", version.ref = "edc" }
edc-data-plane-http = { module = "org.eclipse.edc:data-plane-http", version.ref = "edc" }
edc-data-plane-kafka = { module = "org.eclipse.edc:data-plane-kafka", version.ref = "edc" }
edc-data-plane-selector-api = { module = "org.eclipse.edc:data-plane-selector-api", version.ref = "edc" }
edc-data-plane-selector-client = { module = "org.eclipse.edc:data-plane-selector-client", version.ref = "edc" }
edc-data-plane-selector-core = { module = "org.eclipse.edc:data-plane-selector-core", version.ref = "edc" }
edc-data-plane-spi = { module = "org.eclipse.edc:data-plane-spi", version.ref = "edc" }
edc-data-plane-util = { module = "org.eclipse.edc:data-plane-util", version.ref = "edc" }
edc-dsp = { module = "org.eclipse.edc:dsp", version.ref = "edc" }
edc-http = { module = "org.eclipse.edc:http", version.ref = "edc" }
edc-iam-mock = { module = "org.eclipse.edc:iam-mock", version.ref = "edc" }
edc-jersey-micrometer = { module = "org.eclipse.edc:jersey-micrometer", version.ref = "edc" }
edc-jetty-micrometer = { module = "org.eclipse.edc:jetty-micrometer", version.ref = "edc" }
edc-json-ld = { module = "org.eclipse.edc:json-ld", version.ref = "edc" }
edc-junit = { module = "org.eclipse.edc:junit", version.ref = "edc" }
edc-management-api = { module = "org.eclipse.edc:management-api", version.ref = "edc" }
edc-micrometer-core = { module = "org.eclipse.edc:micrometer-core", version.ref = "edc" }
edc-monitor-jdk-logger = { module = "org.eclipse.edc:monitor-jdk-logger", version.ref = "edc" }
edc-provision-aws-s3 = { module = "org.eclipse.edc:provision-aws-s3", version.ref = "edc" }
edc-runtime-metamodel = { module = "org.eclipse.edc:runtime-metamodel", version.ref = "edc" }
edc-transfer-data-plane = { module = "org.eclipse.edc:transfer-data-plane", version.ref = "edc" }
edc-transfer-process-api = { module = "org.eclipse.edc:transfer-process-api", version.ref = "edc" }
edc-transfer-pull-http-receiver = { module = "org.eclipse.edc:transfer-pull-http-receiver", version.ref = "edc" }
edc-transfer-pull-http-dynamic-receiver = { module = "org.eclipse.edc:transfer-pull-http-dynamic-receiver", version.ref = "edc" }
edc-util = { module = "org.eclipse.edc:util", version.ref = "edc" }
edc-vault-azure = { module = "org.eclipse.edc:vault-azure", version.ref = "edc" }
edc-vault-filesystem = { module = "org.eclipse.edc:vault-filesystem", version.ref = "edc" }
jakarta-rsApi = { module = "jakarta.ws.rs:jakarta.ws.rs-api", version.ref = "rsApi" }
jakartaJson = { module = "org.glassfish:jakarta.json", version.ref = "jakarta-json" }
junit-jupiter-api = { module = "org.junit.jupiter:junit-jupiter-api", version.ref = "jupiter" }
junit-jupiter-engine = { module = "org.junit.jupiter:junit-jupiter-engine", version.ref = "jupiter" }
junit-jupiter-params = { module = "org.junit.jupiter:junit-jupiter-params", version.ref = "jupiter" }
junit-pioneer = { module = "org.junit-pioneer:junit-pioneer", version.ref = "junit-pioneer" }
okhttp-mockwebserver = { module = "com.squareup.okhttp3:mockwebserver", version.ref = "okhttp-mockwebserver" }
opentelemetry-annotations = { module = "io.opentelemetry:opentelemetry-extension-annotations", version.ref = "openTelemetry" }
restAssured = { module = "io.rest-assured:rest-assured", version.ref = "restAssured" }
testcontainers = { module = "org.testcontainers:testcontainers", version.ref = "testcontainers" }
testcontainers-junit-jupiter = { module = "org.testcontainers:junit-jupiter", version.ref = "testcontainers" }
kafka-clients = { module = "org.apache.kafka:kafka-clients", version.ref = "kafkaClients" }
testcontainers-kafka = { module = "org.testcontainers:kafka", version.ref = "testcontainers" }
testcontainers-junit = { module = "org.testcontainers:junit-jupiter", version.ref = "testcontainers" }
opentelemetry = "io.opentelemetry.javaagent:opentelemetry-javaagent:2.0.0"
edc-jersey-core = { module = "org.eclipse.edc:jersey-core", version.ref = "edc" }
edc-catalog-core = { module = "org.eclipse.edc:catalog-core", version.ref = "edc" }
edc-federated-catalog-spi = { module = "org.eclipse.edc:federated-catalog-spi", version.ref = "edc" }
edc-federated-catalog-core = { module = "org.eclipse.edc:federated-catalog-core", version.ref = "edc" }
edc-federated-catalog-api = { module = "org.eclipse.edc:federated-catalog-api", version.ref = "edc" }
edc-crawler-spi = { module = "org.eclipse.edc:crawler-spi", version.ref = "edc" }
edc-data-plane-control-api = { module = "org.eclipse.edc:data-plane-control-api", version.ref = "edc" }
edc-data-plane-public-api = { module = "org.eclipse.edc:data-plane-public-api", version.ref = "edc" }

[plugins]
shadow = { id = "com.github.johnrengelman.shadow", version = "8.1.1" }

Please let me know if more information is needed.

Context Information

• EDC: v0.5.1
• OS: Ubuntu 22.10 (WSL)

Question

What am I doing wrong?
If this is not a bug, but the problem is the incorrect use of the EDC, please let me know.

Many thanks in advance!

[github-actions]

Thanks for your contribution 🔥 We will take a look asap 🚀

[wolf4ood]

Hi @darioklapp

thanks for raising this, i think this has already notified as issue here and fixed.

Thanks

[darioklapp]

@wolf4ood
Many thanks for the very quick reply!
My guess is that the fix will be included in version v0.5.2. Is there an approximate date when the release can be expected?

Btw, are there any SNAPSHOT artifacts anywhere?

[wolf4ood]

Hi @darioklapp

yes there are snapshot. You'd have to include sonatype snapshots as repository, i think the link is https://oss.sonatype.org/content/repositories/snapshots/

and use 0.5.2-SNAPSHOT as version