Skip to content

Commit

Permalink
Kubernetes: Provide properties to remote debugging
Browse files Browse the repository at this point in the history
This PR allows to easily configure the remote debugging and describes how to do this in the documentation.

I've also tried to expose another route to map the java agent port, but as intellij could handle this port from localhost (it threw a handshake error), I decided to not over-complicate this guide and simple do the port-forward bit (see my changes in documentation).

Fix quarkusio#17581
Fix quarkusio#23765
  • Loading branch information
Sgitario committed Mar 22, 2022
1 parent 93b93d1 commit 14dd231
Show file tree
Hide file tree
Showing 10 changed files with 256 additions and 2 deletions.
23 changes: 23 additions & 0 deletions docs/src/main/asciidoc/deploying-to-kubernetes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -729,6 +729,10 @@ The table below describe all the available configuration options.
| quarkus.kubernetes.resources.limits.cpu | String | |
| quarkus.kubernetes.resources.limits.memory | String | |
| quarkus.kubernetes.security-context | SecurityContext | | ( see SecurityContext )
| quarkus.kubernetes.remote-debug.enabled | boolean | | false
| quarkus.kubernetes.remote-debug.transport | String | | dt_socket
| quarkus.kubernetes.remote-debug.address-port | int | | 5005
| quarkus.kubernetes.remote-debug.suspend | String | | n
|====

Properties that use non-standard types, can be referenced by expanding the property.
Expand Down Expand Up @@ -1046,6 +1050,10 @@ The OpenShift resources can be customized in a similar approach with Kubernetes.
| quarkus.openshift.route.annotations | Map<String, String> | |
| quarkus.openshift.headless | boolean | | false
| quarkus.openshift.security-context | SecurityContext | | ( see SecurityContext )
| quarkus.openshift.remote-debug.enabled | boolean | | false
| quarkus.openshift.remote-debug.transport | String | | dt_socket
| quarkus.openshift.remote-debug.address-port | int | | 5005
| quarkus.openshift.remote-debug.suspend | String | | n
|====

[#knative]
Expand Down Expand Up @@ -1267,6 +1275,21 @@ In other words the extension will use whatever cluster `kubectl` uses. The same

At the moment no additional options are provided for further customization.

=== Remote Debugging

To remotely debug applications that are running on a kubernetes environment, we need to deploy the application as described in the previous section and add as new property: `quarkus.kubernetes.remote-debug.enabled=true`. This property will automatically configure the Java application to append the java agent configuration (for example: `-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005`) and also the service resource to listen using the java agent port.

After your application has been deployed with the debug enabled, next you need to tunnel the traffic from your local host machine to the specified port of the java agent:

[source,bash,subs=attributes+]
----
kubectl port-forward svc/<application name> 5005:5005
----

Using this command, you'll forward the traffic from the "localhost:5005" to the kubernetes service running the java agent using the port "5005" which is the one that the java agent uses by default for remote debugging. You can also configure another java agent port using the property `quarkus.kubernetes.remote-debug.address-port`.

Finally, all you need to do is to configure your favorite IDE to attach the java agent process that is forwarded to `localhost:5005` and start to debug your application. For example, in IntelliJ IDEA, you can follow https://www.jetbrains.com/help/idea/tutorial-remote-debug.html:[this tutorial] to debug remote applications.

== Using existing resources

Sometimes it's desirable to either provide additional resources (e.g. a ConfigMap, a Secret, a Deployment for a database etc) or provide custom ones that will be used as a `base` for the generation process.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
package io.quarkus.kubernetes.deployment;

import io.dekorate.kubernetes.config.Env;
import io.dekorate.kubernetes.config.EnvBuilder;
import io.dekorate.kubernetes.config.Port;
import io.quarkus.runtime.annotations.ConfigGroup;
import io.quarkus.runtime.annotations.ConfigItem;

@ConfigGroup
public class DebugConfig {

private static final String PORT_NAME = "debug";
private static final String JAVA_TOOL_OPTIONS = "JAVA_TOOL_OPTIONS";
private static final String AGENTLIB_FORMAT = "-agentlib:jdwp=transport=%s,server=y,suspend=%s,address=%s";

/**
* If true, the debug mode in pods will be enabled.
*/
@ConfigItem(defaultValue = "false")
boolean enabled;

/**
* The transport to use.
*/
@ConfigItem(defaultValue = "dt_socket")
String transport;

/**
* If enabled, it means the JVM will wait for the debugger to attach before executing the main class.
* If false, the JVM will immediately execute the main class, while listening for
* the debugger connection.
*/
@ConfigItem(defaultValue = "n")
String suspend;

/**
* It specifies the address at which the debug socket will listen.
*/
@ConfigItem(defaultValue = "5005")
Integer addressPort;

protected Env buildJavaToolOptionsEnv() {
return new EnvBuilder()
.withName(JAVA_TOOL_OPTIONS)
.withValue(String.format(AGENTLIB_FORMAT, transport, suspend, addressPort))
.build();
}

protected Port buildDebugPort() {
return Port.newBuilder()
.withName(PORT_NAME)
.withContainerPort(addressPort)
.build();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,11 @@ public enum DeploymentResourceKind {
@ConfigItem
SecurityContextConfig securityContext;

/**
* Debug configuration to be set in pods.
*/
DebugConfig remoteDebug;

public Optional<String> getPartOf() {
return partOf;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -503,6 +503,11 @@ public EnvVarsConfig getEnv() {
@ConfigItem
SecurityContextConfig securityContext;

/**
* Debug configuration to be set in pods.
*/
DebugConfig remoteDebug;

public Optional<String> getAppSecret() {
return this.appSecret;
}
Expand All @@ -522,9 +527,9 @@ public SecurityContextConfig getSecurityContext() {
}

public static boolean isOpenshiftBuildEnabled(ContainerImageConfig containerImageConfig, Capabilities capabilities) {
boolean implictlyEnabled = ContainerImageCapabilitiesUtil.getActiveContainerImageCapability(capabilities)
boolean implicitlyEnabled = ContainerImageCapabilitiesUtil.getActiveContainerImageCapability(capabilities)
.filter(c -> c.contains(OPENSHIFT) || c.contains(S2I)).isPresent();
return containerImageConfig.builder.map(b -> b.equals(OPENSHIFT) || b.equals(S2I)).orElse(implictlyEnabled);
return containerImageConfig.builder.map(b -> b.equals(OPENSHIFT) || b.equals(S2I)).orElse(implicitlyEnabled);
}

public DeploymentResourceKind getDeploymentResourceKind() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,11 @@ public List<ConfiguratorBuildItem> createConfigurators(ApplicationInfoBuildItem
});
result.add(new ConfiguratorBuildItem(new ApplyExpositionConfigurator(config.route)));

// Handle remote debug configuration for container ports
if (config.remoteDebug.enabled) {
result.add(new ConfiguratorBuildItem(new AddPortToOpenshiftConfig(config.remoteDebug.buildDebugPort())));
}

if (!capabilities.isPresent(Capability.CONTAINER_IMAGE_S2I)
&& !capabilities.isPresent("io.quarkus.openshift")
&& !capabilities.isPresent(Capability.CONTAINER_IMAGE_OPENSHIFT)) {
Expand Down Expand Up @@ -279,6 +284,13 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
new AddDockerImageStreamResourceDecorator(imageConfiguration, repositoryWithRegistry)));
});
}

// Handle remote debug configuration
if (config.remoteDebug.enabled) {
result.add(new DecoratorBuildItem(OPENSHIFT, new AddEnvVarDecorator(ApplicationContainerDecorator.ANY, name,
config.remoteDebug.buildJavaToolOptionsEnv())));
}

return result;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,12 @@ public List<ConfiguratorBuildItem> createConfigurators(KubernetesConfig config,
result.add(new ConfiguratorBuildItem(new AddPortToKubernetesConfig(value)));
});
result.add(new ConfiguratorBuildItem(new ApplyExpositionConfigurator((config.ingress))));

// Handle remote debug configuration for container ports
if (config.remoteDebug.enabled) {
result.add(new ConfiguratorBuildItem(new AddPortToKubernetesConfig(config.remoteDebug.buildDebugPort())));
}

return result;

}
Expand Down Expand Up @@ -180,6 +186,12 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
.findFirst().orElse(DEFAULT_HTTP_PORT);
result.add(new DecoratorBuildItem(KUBERNETES, new ApplyHttpGetActionPortDecorator(name, name, port)));

// Handle remote debug configuration
if (config.remoteDebug.enabled) {
result.add(new DecoratorBuildItem(KUBERNETES, new AddEnvVarDecorator(ApplicationContainerDecorator.ANY, name,
config.remoteDebug.buildJavaToolOptionsEnv())));
}

return result;
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package io.quarkus.it.kubernetes;

import static org.assertj.core.api.Assertions.assertThat;

import java.io.IOException;
import java.nio.file.Path;
import java.util.List;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.fabric8.kubernetes.api.model.EnvVar;
import io.fabric8.kubernetes.api.model.HasMetadata;
import io.fabric8.kubernetes.api.model.Service;
import io.fabric8.kubernetes.api.model.apps.Deployment;
import io.quarkus.test.ProdBuildResults;
import io.quarkus.test.ProdModeTestResults;
import io.quarkus.test.QuarkusProdModeTest;

public class KubernetesWithRemoteDebugTest {

@RegisterExtension
static final QuarkusProdModeTest config = new QuarkusProdModeTest()
.withApplicationRoot((jar) -> jar.addClasses(GreetingResource.class))
.setApplicationName("kubernetes-with-debug")
.setApplicationVersion("0.1-SNAPSHOT")
.withConfigurationResource("kubernetes-with-debug.properties");

@ProdBuildResults
private ProdModeTestResults prodModeTestResults;

@Test
public void assertGeneratedResources() throws IOException {
Path kubernetesDir = prodModeTestResults.getBuildDir().resolve("kubernetes");
List<HasMetadata> openshiftList = DeserializationUtil.deserializeAsList(
kubernetesDir.resolve("kubernetes.yml"));

assertThat(openshiftList).filteredOn(h -> "Deployment".equals(h.getKind())).singleElement().satisfies(h -> {
Deployment deployment = (Deployment) h;
assertThat(deployment.getSpec().getTemplate().getSpec().getContainers()).singleElement().satisfies(container -> {
List<EnvVar> envVars = container.getEnv();
assertThat(envVars).anySatisfy(envVar -> {
assertThat(envVar.getName()).isEqualTo("JAVA_TOOL_OPTIONS");
assertThat(envVar.getValue())
.isEqualTo("-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000");
});
});
});

assertThat(openshiftList).filteredOn(h -> "Service".equals(h.getKind())).singleElement().satisfies(h -> {
Service service = (Service) h;
assertThat(service.getSpec().getPorts()).anySatisfy(p -> {
assertThat(p.getName()).isEqualTo("http");
assertThat(p.getPort()).isEqualTo(80);
assertThat(p.getTargetPort().getIntVal()).isEqualTo(8080);
});

assertThat(service.getSpec().getPorts()).anySatisfy(p -> {
assertThat(p.getName()).isEqualTo("debug");
assertThat(p.getPort()).isEqualTo(8000);
assertThat(p.getTargetPort().getIntVal()).isEqualTo(8000);
});
});
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
package io.quarkus.it.kubernetes;

import static org.assertj.core.api.Assertions.assertThat;

import java.io.IOException;
import java.nio.file.Path;
import java.util.Collections;
import java.util.List;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.fabric8.kubernetes.api.model.EnvVar;
import io.fabric8.kubernetes.api.model.HasMetadata;
import io.fabric8.kubernetes.api.model.Service;
import io.fabric8.openshift.api.model.DeploymentConfig;
import io.quarkus.bootstrap.model.AppArtifact;
import io.quarkus.builder.Version;
import io.quarkus.test.ProdBuildResults;
import io.quarkus.test.ProdModeTestResults;
import io.quarkus.test.QuarkusProdModeTest;

public class OpenshiftWithRemoteDebugTest {

@RegisterExtension
static final QuarkusProdModeTest config = new QuarkusProdModeTest()
.withApplicationRoot((jar) -> jar.addClasses(GreetingResource.class))
.setApplicationName("openshift-with-debug")
.setApplicationVersion("0.1-SNAPSHOT")
.withConfigurationResource("openshift-with-debug.properties")
.setForcedDependencies(Collections.singletonList(
new AppArtifact("io.quarkus", "quarkus-openshift", Version.getVersion())));

@ProdBuildResults
private ProdModeTestResults prodModeTestResults;

@Test
public void assertGeneratedResources() throws IOException {
Path kubernetesDir = prodModeTestResults.getBuildDir().resolve("kubernetes");

assertThat(kubernetesDir)
.isDirectoryContaining(p -> p.getFileName().endsWith("openshift.json"))
.isDirectoryContaining(p -> p.getFileName().endsWith("openshift.yml"));
List<HasMetadata> openshiftList = DeserializationUtil.deserializeAsList(
kubernetesDir.resolve("openshift.yml"));

assertThat(openshiftList).filteredOn(h -> "DeploymentConfig".equals(h.getKind())).singleElement().satisfies(h -> {
DeploymentConfig deployment = (DeploymentConfig) h;
assertThat(deployment.getSpec().getTemplate().getSpec().getContainers()).singleElement().satisfies(container -> {
List<EnvVar> envVars = container.getEnv();
assertThat(envVars).anySatisfy(envVar -> {
assertThat(envVar.getName()).isEqualTo("JAVA_TOOL_OPTIONS");
assertThat(envVar.getValue())
.isEqualTo("-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005");
});
});
});

assertThat(openshiftList).filteredOn(h -> "Service".equals(h.getKind())).singleElement().satisfies(h -> {
Service service = (Service) h;
assertThat(service.getSpec().getPorts()).anySatisfy(p -> {
assertThat(p.getName()).isEqualTo("http");
assertThat(p.getPort()).isEqualTo(80);
assertThat(p.getTargetPort().getIntVal()).isEqualTo(8080);
});

assertThat(service.getSpec().getPorts()).anySatisfy(p -> {
assertThat(p.getName()).isEqualTo("debug");
assertThat(p.getPort()).isEqualTo(5005);
assertThat(p.getTargetPort().getIntVal()).isEqualTo(5005);
});
});
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
quarkus.kubernetes.remote-debug.enabled=true
quarkus.kubernetes.remote-debug.address-port=8000
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
quarkus.openshift.remote-debug.enabled=true

0 comments on commit 14dd231

Please sign in to comment.