Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[development] Promremote storage local setup #3791

Merged
merged 15 commits into from
Oct 6, 2021

Conversation

Antanukas
Copy link
Collaborator

@Antanukas Antanukas commented Sep 28, 2021

What this PR does / why we need it: Local dev setup to quickly test Prom Remote Write storage.

Special notes for your reviewer:

Refer to readme that describes this setup.

In order to generate some realistic metrics it's possible to scrape CAdvisor metrics from Kubernetes.
This is achieved by using kubectl proxy and the fact that each node exposes its metrics at localhost:8001/api/v1/nodes/${node}/proxy/metrics/cadvisor.

Does this PR introduce a user-facing and/or backwards incompatible change?:

None

Does this PR require updating code package or user-facing documentation?:

None

@Antanukas Antanukas changed the title Antanas/prom remote write backend local setup [local-dev] remote write backend local setup Sep 28, 2021
@Antanukas Antanukas force-pushed the antanas/prom-remote-write-backend-local-setup branch 2 times, most recently from 96ee123 to f9aa6ee Compare September 29, 2021 07:38
centai

[coordinator] docker integration tests for prom-remote storage

cleanup, wait for leader

[development] Local Docker setup for prom-remote storage

working config

fix test

revert
@Antanukas Antanukas force-pushed the antanas/prom-remote-write-backend-local-setup branch from f9aa6ee to 026f048 Compare September 29, 2021 07:39
@Antanukas Antanukas changed the base branch from master to antanas/better-error-handling September 29, 2021 07:39
@codecov
Copy link

codecov bot commented Sep 29, 2021

Codecov Report

Merging #3791 (31e82fe) into master (5b8f316) will decrease coverage by 0.0%.
The diff coverage is n/a.

❗ Current head 31e82fe differs from pull request most recent head 30e9218. Consider uploading reports for the commit 30e9218 to get more accurate results

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #3791     +/-   ##
========================================
- Coverage    56.8%   56.8%   -0.1%     
========================================
  Files         552     552             
  Lines       63077   63064     -13     
========================================
- Hits        35888   35834     -54     
- Misses      23986   24025     +39     
- Partials     3203    3205      +2     
Flag Coverage Δ
aggregator 63.3% <ø> (ø)
cluster ∅ <ø> (∅)
collector 58.4% <ø> (ø)
dbnode 60.4% <ø> (-0.2%) ⬇️
m3em 46.4% <ø> (ø)
metrics 19.7% <ø> (ø)
msg 74.2% <ø> (-0.1%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3270bf...30e9218. Read the comment docs.

@Antanukas Antanukas changed the title [local-dev] remote write backend local setup [local-dev] Remote write backend local setup Oct 1, 2021
@Antanukas Antanukas changed the title [local-dev] Remote write backend local setup [local-dev] Promremote storage local setup Oct 1, 2021
@Antanukas Antanukas changed the title [local-dev] Promremote storage local setup [development] Promremote storage local setup Oct 1, 2021
@Antanukas Antanukas marked this pull request as ready for review October 1, 2021 13:04
Base automatically changed from antanas/better-error-handling to master October 4, 2021 10:41
Copy link
Collaborator

@vpranckaitis vpranckaitis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a few nits regarding indentation. Though I don't know all the intricacies of running M3 locally, and this applies even more for Prometheus.

ATTEMPTS=10 MAX_TIMEOUT=4 TIMEOUT=1 retry_with_backoff \
'curl -vvvsSf localhost:7201/health'

echo "Running aggregator pipeline"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
echo "Running aggregator pipeline"
echo "Running aggregator pipeline"

M3AGGREGATOR_DEV_IMG=$(docker images m3aggregator:dev | fgrep -iv repository | wc -l | xargs)
M3COLLECTOR_DEV_IMG=$(docker images m3collector:dev | fgrep -iv repository | wc -l | xargs)

docker-compose -f docker-compose.yml up $DOCKER_ARGS etcd01
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
docker-compose -f docker-compose.yml up $DOCKER_ARGS etcd01
docker-compose -f docker-compose.yml up $DOCKER_ARGS etcd01

Comment on lines 63 to 71
"consumerService": {
"serviceId": {
"name": "m3aggregator",
"environment": "default_env",
"zone": "embedded"
},
"consumptionType": "REPLICATED",
"messageTtlNanos": "600000000000"
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
"consumerService": {
"serviceId": {
"name": "m3aggregator",
"environment": "default_env",
"zone": "embedded"
},
"consumptionType": "REPLICATED",
"messageTtlNanos": "600000000000"
}
"consumerService": {
"serviceId": {
"name": "m3aggregator",
"environment": "default_env",
"zone": "embedded"
},
"consumptionType": "REPLICATED",
"messageTtlNanos": "600000000000"
}

Comment on lines +111 to +113
"consumerService": {
"serviceId": {
"name": "m3coordinator",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"consumerService": {
"serviceId": {
"name": "m3coordinator",
"consumerService": {
"serviceId": {
"name": "m3coordinator",

Copy link
Collaborator

@robskillington robskillington left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Antanukas Antanukas enabled auto-merge (squash) October 6, 2021 07:23
@Antanukas Antanukas merged commit a6bf60d into master Oct 6, 2021
@Antanukas Antanukas deleted the antanas/prom-remote-write-backend-local-setup branch October 6, 2021 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants