Skip to content

Commit

Permalink
Fixes #6514 - How to warm up SslConnection. (#12151)
Browse files Browse the repository at this point in the history
Implemented "priming" of HTTP/1.1 connections using ConnectionPool.preCreateConnections(int) and HttpClientTransportOverHTTP.setInitializeConnections(true).

This sends `OPTIONS * HTTP/1.1` to the server.

I tried to implement this feature by forcing a write of 0 bytes from the layer above `SslConnection`, but it did not work when using TLS because in both WriteFlusher and SslConnection the fact that there are 0 bytes left to write is treated specially.

Other HTTP versions have no problems because they must initialize the connection by e.g. sending a SETTINGS frame, so they would also initialize TLS.

Signed-off-by: Simone Bordet <[email protected]>
  • Loading branch information
sbordet authored Aug 20, 2024
1 parent 877aaa5 commit 942e77c
Show file tree
Hide file tree
Showing 8 changed files with 267 additions and 11 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1049,6 +1049,31 @@ public void setConnectionPool() throws Exception
// end::setConnectionPool[]
}

public void preCreateConnections() throws Exception
{
// tag::preCreateConnections[]
HttpClient httpClient = new HttpClient();
httpClient.start();

// For HTTP/1.1, you need to explicitly configure to initialize connections.
if (httpClient.getTransport() instanceof HttpClientTransportOverHTTP http1)
http1.setInitializeConnections(true);

// Create a dummy request to the server you want to pre-create connections to.
Request request = httpClient.newRequest("https://host/");

// Resolve the destination for that request.
Destination destination = httpClient.resolveDestination(request);

// Pre-create, for example, half of the connections.
int preCreate = httpClient.getMaxConnectionsPerDestination() / 2;
CompletableFuture<Void> completable = destination.getConnectionPool().preCreateConnections(preCreate);

// Wait for the connections to be created.
completable.get(5, TimeUnit.SECONDS);
// end::preCreateConnections[]
}

public void unixDomain() throws Exception
{
// tag::unixDomain[]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ Jetty's client library provides the following `ConnectionPool` implementations:
* `DuplexConnectionPool`, historically the first implementation, only used by the HTTP/1.1 transport.
* `MultiplexConnectionPool`, the generic implementation valid for any transport where connections are reused with a most recently used algorithm (that is, the connections most recently returned to the connection pool are the more likely to be used again).
* `RoundRobinConnectionPool`, similar to `MultiplexConnectionPool` but where connections are reused with a round-robin algorithm.
* `RandomRobinConnectionPool`, similar to `MultiplexConnectionPool` but where connections are reused with an algorithm that chooses them randomly.
* `RandomConnectionPool`, similar to `MultiplexConnectionPool` but where connections are reused with an algorithm that chooses them randomly.

The `ConnectionPool` implementation can be customized for each destination in by setting a `ConnectionPool.Factory` on the `HttpClientTransport`:

Expand All @@ -167,6 +167,34 @@ The `ConnectionPool` implementation can be customized for each destination in by
include::code:example$src/main/java/org/eclipse/jetty/docs/programming/client/http/HTTPClientDocs.java[tags=setConnectionPool]
----

[[connection-pool-precreate-connections]]
=== Pre-Creating Connections

`ConnectionPool` offers the ability to pre-create connections by calling `ConnectionPool.preCreateConnections(int)`.

Pre-creating the connections saves the time and processing spent to establish the TCP connection, performing the TLS handshake (if necessary) and, for HTTP/2 and HTTP/3, perform the initial protocol setup.
This is particularly important for HTTP/2 because in the initial protocol setup the server informs the client of the maximum number of concurrent requests per connection (otherwise assumed to be just `1` by the client).

The scenarios where pre-creating connections is useful are, for example:

* Load testing, where you want to prepare the system with connections already created to avoid paying of cost of connection setup.
* Proxying scenarios, often in conjunction with the use of `RoundRobinConnectionPool` or `RandomConnectionPool`, where the proxy creates early the connections to the backend servers.

This is an example of how to pre-create connections:

[,java,indent=0]
----
include::code:example$src/main/java/org/eclipse/jetty/docs/programming/client/http/HTTPClientDocs.java[tags=preCreateConnections]
----

[NOTE]
====
Pre-creating connections for secure HTTP/1.1 requires you to call `HttpClientTransportOverHTTP.setInitializeConnections(true)`, otherwise only the TCP connection is established, but the TLS handshake is not initiated.
To initialize connections for secure HTTP/1.1, the client sends an initial `OPTIONS * HTTP/1.1` request to the server.
The server must be able to handle this request without closing the connection (in particular it must not add the `Connection: close` header in the response).
====

[[request-processing]]
== HttpClient Request Processing

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,49 @@ public class HttpClientConnectionFactory implements ClientConnectionFactory
/**
* <p>Representation of the {@code HTTP/1.1} application protocol used by {@link HttpClientTransportDynamic}.</p>
*/
public static final Info HTTP11 = new HTTP11(new HttpClientConnectionFactory());
public static final Info HTTP11 = new HTTP11();

private boolean initializeConnections;

/**
* @return whether newly created connections should be initialized with an {@code OPTIONS * HTTP/1.1} request
*/
public boolean isInitializeConnections()
{
return initializeConnections;
}

/**
* @param initialize whether newly created connections should be initialized with an {@code OPTIONS * HTTP/1.1} request
*/
public void setInitializeConnections(boolean initialize)
{
this.initializeConnections = initialize;
}

@Override
public org.eclipse.jetty.io.Connection newConnection(EndPoint endPoint, Map<String, Object> context)
{
HttpConnectionOverHTTP connection = new HttpConnectionOverHTTP(endPoint, context);
connection.setInitialize(isInitializeConnections());
return customize(connection, context);
}

private static class HTTP11 extends Info
/**
* <p>Representation of the {@code HTTP/1.1} application protocol used by {@link HttpClientTransportDynamic}.</p>
* <p>Applications should prefer using the constant {@link HttpClientConnectionFactory#HTTP11}, unless they
* need to customize the associated {@link HttpClientConnectionFactory}.</p>
*/
public static class HTTP11 extends Info
{
private static final List<String> protocols = List.of("http/1.1");

private HTTP11(ClientConnectionFactory factory)
public HTTP11()
{
this(new HttpClientConnectionFactory());
}

public HTTP11(ClientConnectionFactory factory)
{
super(factory);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@
import org.eclipse.jetty.client.DuplexConnectionPool;
import org.eclipse.jetty.client.Origin;
import org.eclipse.jetty.client.Request;
import org.eclipse.jetty.io.ClientConnectionFactory;
import org.eclipse.jetty.io.ClientConnector;
import org.eclipse.jetty.io.EndPoint;
import org.eclipse.jetty.util.ProcessorUtils;
Expand All @@ -37,7 +36,7 @@ public class HttpClientTransportOverHTTP extends AbstractConnectorHttpClientTran
public static final Origin.Protocol HTTP11 = new Origin.Protocol(List.of("http/1.1"), false);
private static final Logger LOG = LoggerFactory.getLogger(HttpClientTransportOverHTTP.class);

private final ClientConnectionFactory factory = new HttpClientConnectionFactory();
private final HttpClientConnectionFactory factory = new HttpClientConnectionFactory();
private int headerCacheSize = 1024;
private boolean headerCacheCaseSensitive;

Expand Down Expand Up @@ -79,25 +78,54 @@ public org.eclipse.jetty.io.Connection newConnection(EndPoint endPoint, Map<Stri
return connection;
}

@ManagedAttribute("The maximum allowed size in bytes for an HTTP header field cache")
/**
* @return the max size in bytes for the HTTP header field cache
*/
@ManagedAttribute("The maximum allowed size in bytes for the HTTP header field cache")
public int getHeaderCacheSize()
{
return headerCacheSize;
}

/**
* @param headerCacheSize the max size in bytes for the HTTP header field cache
*/
public void setHeaderCacheSize(int headerCacheSize)
{
this.headerCacheSize = headerCacheSize;
}

@ManagedAttribute("Whether the header field cache is case sensitive")
/**
* @return whether the HTTP header field cache is case-sensitive
*/
@ManagedAttribute("Whether the HTTP header field cache is case-sensitive")
public boolean isHeaderCacheCaseSensitive()
{
return headerCacheCaseSensitive;
}

/**
* @param headerCacheCaseSensitive whether the HTTP header field cache is case-sensitive
*/
public void setHeaderCacheCaseSensitive(boolean headerCacheCaseSensitive)
{
this.headerCacheCaseSensitive = headerCacheCaseSensitive;
}

/**
* @return whether newly created connections should be initialized with an {@code OPTIONS * HTTP/1.1} request
*/
@ManagedAttribute("Whether newly created connections should be initialized with an OPTIONS * HTTP/1.1 request")
public boolean isInitializeConnections()
{
return factory.isInitializeConnections();
}

/**
* @param initialize whether newly created connections should be initialized with an {@code OPTIONS * HTTP/1.1} request
*/
public void setInitializeConnections(boolean initialize)
{
factory.setInitializeConnections(initialize);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import java.util.concurrent.atomic.LongAdder;

import org.eclipse.jetty.client.Connection;
import org.eclipse.jetty.client.Destination;
import org.eclipse.jetty.client.HttpClientTransport;
import org.eclipse.jetty.client.HttpUpgrader;
import org.eclipse.jetty.client.Request;
Expand All @@ -40,6 +41,7 @@
import org.eclipse.jetty.client.transport.IConnection;
import org.eclipse.jetty.client.transport.SendFailure;
import org.eclipse.jetty.http.HttpHeader;
import org.eclipse.jetty.http.HttpMethod;
import org.eclipse.jetty.http.HttpVersion;
import org.eclipse.jetty.io.AbstractConnection;
import org.eclipse.jetty.io.EndPoint;
Expand All @@ -61,6 +63,7 @@ public class HttpConnectionOverHTTP extends AbstractConnection implements IConne
private final LongAdder bytesIn = new LongAdder();
private final LongAdder bytesOut = new LongAdder();
private long idleTimeout;
private boolean initialize;

public HttpConnectionOverHTTP(EndPoint endPoint, Map<String, Object> context)
{
Expand Down Expand Up @@ -159,12 +162,46 @@ public SendFailure send(HttpExchange exchange)
return delegate.send(exchange);
}

/**
* @return whether to initialize the connection with an {@code OPTIONS * HTTP/1.1} request.
*/
public boolean isInitialize()
{
return initialize;
}

/**
* @param initialize whether to initialize the connection with an {@code OPTIONS * HTTP/1.1} request.
*/
public void setInitialize(boolean initialize)
{
this.initialize = initialize;
}

@Override
public void onOpen()
{
super.onOpen();
fillInterested();
promise.succeeded(this);
boolean initialize = isInitialize();
if (initialize)
{
Destination destination = getHttpDestination();
Request request = destination.getHttpClient().newRequest(destination.getOrigin().asString())
.method(HttpMethod.OPTIONS)
.path("*");
send(request, result ->
{
if (result.isSucceeded())
promise.succeeded(this);
else
promise.failed(result.getFailure());
});
}
else
{
promise.succeeded(this);
}
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -703,7 +703,7 @@ public void testCountersSweepToStringThroughLifecycle(ConnectionPoolFactory fact
assertThat(connectionPool.toString(), not(nullValue()));
}

private static class ConnectionPoolFactory
public static class ConnectionPoolFactory
{
private final String name;
private final ConnectionPool.Factory factory;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,12 @@ protected SslContextFactory.Server newSslContextFactoryServer()
}

protected void startClient(Transport transport) throws Exception
{
prepareClient(transport);
client.start();
}

protected void prepareClient(Transport transport) throws Exception
{
QueuedThreadPool clientThreads = new QueuedThreadPool();
clientThreads.setName("client");
Expand All @@ -298,7 +304,6 @@ protected void startClient(Transport transport) throws Exception
client.setByteBufferPool(clientBufferPool);
client.setExecutor(clientThreads);
client.setSocketAddressResolver(new SocketAddressResolver.Sync());
client.start();
}

public AbstractConnector newConnector(Transport transport, Server server)
Expand Down
Loading

0 comments on commit 942e77c

Please sign in to comment.