Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: prom sink changed to PoolingHttpClientConnectionManager for idle connection eviction #195

Merged
merged 5 commits into from
Oct 27, 2022
Merged

fix: prom sink changed to PoolingHttpClientConnectionManager for idle connection eviction #195

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7e25f01
fix: prom sink changed to BasicHttpClientConnectionManager for idle c…
mayankrai09 Sep 28, 2022
0d6a15c
fix: update default SINK_PROM_MAX_CONNECTIONS
mayankrai09 Sep 29, 2022
0ce082d
fix: changed default value for max prom connection to use default Poo…
mayankrai09 Oct 10, 2022
5fbc623
fix: styling
mayankrai09 Oct 10, 2022
f146035
fix: added default value in doc
mayankrai09 Oct 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/docs/sinks/prometheus-sink.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@ Defines the connection timeout for the request in millis.
- Type: `required`
- Default value: `10000`

### `SINK_PROM_MAX_CONNECTIONS`

Defines the maximum number of HTTP connections with Prometheus.

- Example value: `10`
- Type: `optional`
- Default value: `default no more than 2 concurrent connections per given route and no more 20 connections`

### `SINK_PROM_RETRY_STATUS_CODE_RANGES`

Defines the range of HTTP status codes for which retry will be attempted.
Expand Down
3 changes: 3 additions & 0 deletions src/main/java/io/odpf/firehose/config/PromSinkConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ public interface PromSinkConfig extends AppConfig {
@DefaultValue("10000")
Integer getSinkPromRequestTimeoutMs();

@Key("SINK_PROM_MAX_CONNECTIONS")
Integer getSinkPromMaxConnections();

@Key("SINK_PROM_SERVICE_URL")
String getSinkPromServiceUrl();

Expand Down
9 changes: 7 additions & 2 deletions src/main/java/io/odpf/firehose/sink/prometheus/PromSinkFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.BasicHttpClientConnectionManager;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;

import java.util.Map;

Expand Down Expand Up @@ -67,7 +67,12 @@ private static CloseableHttpClient newHttpClient(PromSinkConfig promSinkConfig)
RequestConfig requestConfig = RequestConfig.custom().setSocketTimeout(promSinkConfig.getSinkPromRequestTimeoutMs())
.setConnectionRequestTimeout(promSinkConfig.getSinkPromRequestTimeoutMs())
.setConnectTimeout(promSinkConfig.getSinkPromRequestTimeoutMs()).build();
BasicHttpClientConnectionManager connectionManager = new BasicHttpClientConnectionManager();
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
if (promSinkConfig.getSinkPromMaxConnections() != null && promSinkConfig.getSinkPromMaxConnections() > 0) {
connectionManager.setMaxTotal(promSinkConfig.getSinkPromMaxConnections());
connectionManager.setDefaultMaxPerRoute(promSinkConfig.getSinkPromMaxConnections());
}

HttpClientBuilder builder = HttpClients.custom().setConnectionManager(connectionManager).setDefaultRequestConfig(requestConfig);

return builder.build();
Expand Down