Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed make docs #2611

Merged
merged 1 commit into from
May 14, 2020
Merged

Fixed make docs #2611

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
FROM quay.io/prometheus/busybox:latest
LABEL maintainer="The Thanos Authors"

COPY thanos /bin/thanos
COPY /thanos_tmp_for_docker /bin/thanos
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What issue does using this temporary thanos binary solve?

Copy link
Member Author

@bwplotka bwplotka May 14, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue:

  • We accidently were using wrong binary in genflagdocs vs used by docker. This is because make build was building to current dir ./thanos vs install-deps step is installing to GOBIN/thanos via go install

This is why I would love to be explicit (: and always use GOBIN. However either my dockerfile skills sucks or docker just cannot get anything outside current dir and recursive sub dirs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack got it 👍 your Docker skillz are not bad, it is an (intentional) limitation of the Docker build tooling


ENTRYPOINT [ "/bin/thanos" ]
12 changes: 7 additions & 5 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
PREFIX ?= $(shell pwd)
FILES_TO_FMT ?= $(shell find . -path ./vendor -prune -o -name '*.go' -print)

DOCKER_IMAGE_REPO ?= quay.io/thanos/thanos
Expand Down Expand Up @@ -176,8 +175,8 @@ react-app-start: $(REACT_APP_NODE_MODULES_PATH)
.PHONY: build
build: ## Builds Thanos binary using `promu`.
build: check-git deps $(PROMU)
@echo ">> building binaries $(GOBIN)"
@$(PROMU) build --prefix $(PREFIX)
@echo ">> building Thanos binary in $(GOBIN)"
@$(PROMU) build --prefix $(GOBIN)

.PHONY: crossbuild
crossbuild: ## Builds all binaries for all platforms.
Expand All @@ -193,8 +192,11 @@ deps: ## Ensures fresh go.mod and go.sum.
.PHONY: docker
docker: ## Builds 'thanos' docker with no tag.
docker: build
@echo ">> copying Thanos from $(GOBIN) to ./thanos_tmp_for_docker"
@cp $(GOBIN)/thanos ./thanos_tmp_for_docker
@echo ">> building docker image 'thanos'"
@docker build -t "thanos" .
@rm ./thanos_tmp_for_docker

.PHONY: docker-multi-stage
docker-multi-stage: ## Builds 'thanos' docker image using multi-stage.
Expand All @@ -212,13 +214,13 @@ docker-push:
.PHONY: docs
docs: ## Regenerates flags in docs for all thanos commands.
docs: $(EMBEDMD) build
@EMBEDMD_BIN="$(EMBEDMD)" SED_BIN="$(SED)" scripts/genflagdocs.sh
@EMBEDMD_BIN="$(EMBEDMD)" SED_BIN="$(SED)" THANOS_BIN="$(GOBIN)/thanos" scripts/genflagdocs.sh
@find . -type f -name "*.md" | SED_BIN="$(SED)" xargs scripts/cleanup-white-noise.sh

.PHONY: check-docs
check-docs: ## checks docs against discrepancy with flags, links, white noise.
check-docs: $(EMBEDMD) $(LICHE) build
@EMBEDMD_BIN="$(EMBEDMD)" SED_BIN="$(SED)" scripts/genflagdocs.sh check
@EMBEDMD_BIN="$(EMBEDMD)" SED_BIN="$(SED)" THANOS_BIN="$(GOBIN)/thanos" scripts/genflagdocs.sh check
@$(LICHE) --recursive docs --exclude "(couchdb.apache.org/bylaws.html|cloud.tencent.com|alibabacloud.com|zoom.us)" --document-root .
@$(LICHE) --exclude "goreportcard.com" --document-root . *.md
@find . -type f -name "*.md" | SED_BIN="$(SED)" xargs scripts/cleanup-white-noise.sh
Expand Down
3 changes: 1 addition & 2 deletions docs/components/compact.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,7 @@ In order to achieve this co-ordination, blocks are not deleted directly. Instead

## Flags

[embedmd]: # "flags/compact.txt $"

[embedmd]:# (flags/compact.txt $)
```$
usage: thanos compact [<flags>]

Expand Down
50 changes: 45 additions & 5 deletions docs/components/store.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ In general about 1MB of local disk space is required per TSDB block stored in th

## Flags

[embedmd]: # "flags/store.txt $"
[embedmd]:# (flags/store.txt $)
```$
usage: thanos store [<flags>]

Expand Down Expand Up @@ -137,11 +137,51 @@ Flags:
Prometheus relabel-config syntax. See format
details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
--consistency-delay=30m Minimum age of all blocks before they are being read.
--consistency-delay=0s Minimum age of all blocks before they are being
read. Set it to safe value (e.g 30m) if your
object storage is eventually consistent. GCS
and S3 are (roughly) strongly consistent.
--ignore-deletion-marks-delay=24h
Duration after which the blocks marked for deletion will be filtered out while fetching blocks.
The idea of ignore-deletion-marks-delay is to ignore blocks that are marked for deletion with some delay. This ensures store can still serve blocks that are meant to be deleted but do not have a replacement yet. If delete-delay duration is provided to compactor or bucket verify component, it will upload deletion-mark.json file to mark after what duration the block should be deleted rather than deleting the block straight away.
If delete-delay is non-zero for compactor or bucket verify component, ignore-deletion-marks-delay should be set to (delete-delay)/2 so that blocks marked for deletion are filtered out while fetching blocks before being deleted from bucket. Default is 24h, half of the default value for --delete-delay on compactor.
Duration after which the blocks marked for
deletion will be filtered out while fetching
blocks. The idea of ignore-deletion-marks-delay
is to ignore blocks that are marked for
deletion with some delay. This ensures store
can still serve blocks that are meant to be
deleted but do not have a replacement yet. If
delete-delay duration is provided to compactor
or bucket verify component, it will upload
deletion-mark.json file to mark after what
duration the block should be deleted rather
than deleting the block straight away. If
delete-delay is non-zero for compactor or
bucket verify component,
ignore-deletion-marks-delay should be set to
(delete-delay)/2 so that blocks marked for
deletion are filtered out while fetching blocks
before being deleted from bucket. Default is
24h, half of the default value for
--delete-delay on compactor.
--web.external-prefix="" Static prefix for all HTML links and redirect
URLs in the bucket web UI interface. Actual
endpoints are still served on / or the
web.route-prefix. This allows thanos bucket web
UI to be served behind a reverse proxy that
strips a URL sub-path.
--web.prefix-header="" Name of HTTP request header used for dynamic
prefixing of UI links and redirects. This
option is ignored if web.external-prefix
argument is set. Security risk: enable this
option only if a reverse proxy in front of
thanos is resetting the header. The
--web.prefix-header=X-Forwarded-Prefix option
can be useful, for example, if Thanos UI is
served via Traefik reverse proxy with
PathPrefixStrip option enabled, which sends the
stripped prefix value in X-Forwarded-Prefix
header. This allows thanos UI to be served on a
sub-path.

```

## Time based partitioning
Expand Down
7 changes: 4 additions & 3 deletions scripts/genflagdocs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ set -u

EMBEDMD_BIN=${EMBEDMD_BIN:-embedmd}
SED_BIN=${SED_BIN:-sed}
THANOS_BIN=${THANOS_BIN:-${GOBIN}/thanos}

function docs {
# If check arg was passed, instead of the docs generation verifies if docs coincide with the codebase.
Expand Down Expand Up @@ -38,17 +39,17 @@ CHECK=${1:-}

commands=("compact" "query" "rule" "sidecar" "store" "tools")
for x in "${commands[@]}"; do
./thanos "${x}" --help &> "docs/components/flags/${x}.txt"
${THANOS_BIN} "${x}" --help &> "docs/components/flags/${x}.txt"
done

toolsCommands=("bucket" "rules-check")
for x in "${toolsCommands[@]}"; do
./thanos tools "${x}" --help &> "docs/components/flags/tools_${x}.txt"
${THANOS_BIN} tools "${x}" --help &> "docs/components/flags/tools_${x}.txt"
done

toolsBucketCommands=("verify" "ls" "inspect" "web" "replicate" "downsample")
for x in "${toolsBucketCommands[@]}"; do
./thanos tools bucket "${x}" --help &> "docs/components/flags/tools_bucket_${x}.txt"
${THANOS_BIN} tools bucket "${x}" --help &> "docs/components/flags/tools_bucket_${x}.txt"
done

# Remove white noise.
Expand Down