distributed: major refactoring, use common templates #1520

AndrewChubatiuk · 2024-09-20T06:11:37Z

added default common.<component>.spec, which is merged with all *.<component>.spec, e.g: common.vmagent.spec is merged with availabilityZones[1].vmagent.spec
vmauthIngestGlobal was changed to write.global.vmauth
vmauthQueryGlobal was changed to read.global.vmauth
availabilityZones[*].allowIngest was changed to availabilityZones[*].write.allow
availabilityZones[*].allowRead was changed to availabilityZones[*].read.allow
availabilityZones[*].nodeSelector was moved to availabilityZones[*].common.spec.nodeSelector
availabilityZones[*].extraAffinity was moved to availabilityZones[*].common.spec.affinity
availabilityZones[*].topologySpreadConstraints was moved to availabilityZones[*].common.spec.topologySpreadConstraints
availabilityZones[*].vmauthIngest was moved to availabilityZones[*].write.vmauth
availabilityZones[*].vmauthQueryPerZone was moved to availabilityZones[*].read.perZone.vmauth
availabilityZones[*].vmauthCrossAZQuery was moved to availabilityZones[*].read.crossZone.vmauth

Haleygo · 2024-09-24T06:37:46Z

charts/victoria-metrics-distributed/CHANGELOG.md

+- `vmauthIngestGlobal` was changed to `write.global.vmauth`
+- `vmauthQueryGlobal` was changed to `read.global.vmauth`


How about moving them into .global as global.vmauthIngest and global.vmauthQuery?
Since there shouldn't be other fields under write.global or global.write.

global section in helm is special one, it's shared across all charts in release

Haleygo · 2024-09-24T06:58:44Z

charts/victoria-metrics-distributed/templates/per-az/vmagent.yaml

-  {{- $mergedSettings := merge (default (default dict) $zone.vmagent.spec.remoteWriteSettings) (dict "useMultiTenantMode" true) }}
-  remoteWriteSettings: {{ toYaml $mergedSettings | nindent 4 }}
-{{- end }}
+{{- $spec := mergeOverwrite (deepCopy ($zone.common).spec) (deepCopy ($zone.vmagent).spec) }}


Suggested change

{{- $spec := mergeOverwrite (deepCopy ($zone.common).spec) (deepCopy ($zone.vmagent).spec) }}

{{- $spec := mergeOverwrite (deepCopy $.Values.common.vmagent.spec) (deepCopy ($zone.common).spec) (deepCopy ($zone.vmagent).spec) }}

applied these changes

Haleygo · 2024-09-24T07:07:27Z

charts/victoria-metrics-distributed/values.yaml

+  -
+    # -- Availability zone name
+    name: zone-eu-1
+    common:


I think components shouldn't share other spec except nodeSelector/affinitytopologySpreadConstraints, which could lead to misuse and it's better to not lead to that way.
How about changing this as a plain commonSpec instead of having common.spec?

common.spec allows having then common.annotations and common.labels

Haleygo · 2024-09-24T07:09:34Z

charts/victoria-metrics-distributed/values.yaml

+    # Available parameters can be found [here](https://docs.victoriametrics.com/operator/api/index.html#vmagentspec)
+    spec:
+      remoteWriteSettings:
+        useMultiTenantMode: true


It's better to not set useMultiTenantMode=true by default, since it degrades vmagent performance.

updated this

Haleygo · 2024-09-24T07:10:06Z

charts/victoria-metrics-distributed/values.yaml

+      vminsert:
+        serviceSpec:
+          spec:
+            clusterIP: None


Any reason for this setting?

It was hardcoded in templates, looks like it's needed to provide to vmauth pod ips and move balancing to vmauth

Haleygo · 2024-09-24T07:26:02Z

charts/victoria-metrics-distributed/templates/per-az/vmauth-read-proxy.yaml

+        {{- $urls = append $urls (include "vm.url" $ctx)}}
+      {{- end }}
+    {{- end }}
+    url_prefix: {{ toYaml $urls | nindent 4 }}


Suggested change

url_prefix: {{ toYaml $urls | nindent 4 }}

{{- if $urls }}

url_prefix: {{ toYaml $urls | nindent 4 }}

{{- end }}

i would rather fail if there're no urls defined

added failing helm template if no urls defined for vmauth

Haleygo · 2024-09-24T07:27:49Z

charts/victoria-metrics-distributed/values.yaml

+            whenUnsatisfiable: ScheduleAnyway
+    write:
+      # -- Allow data ingestion to this zone
+      allow: true


How about unifying them to enable?

enable is for resource creation and allow is for traffic routing

zekker6 · 2024-10-07T16:37:36Z

charts/victoria-metrics-distributed/templates/_helpers.tpl

+  {{- $zones := (dict) -}}
+  {{- $commonClusterSpec := ((($Values.common).vmcluster).spec) | default dict -}}
+  {{- range $idx, $rolloutZone := $Values.availabilityZones -}}
+    {{- $commonSpec := $rolloutZone.spec | default dict -}}


Wdyt about moving $zone.spec.nodeSelector and the rest of common fields to $zone.common level?
That would make it easier to understand from user's PoV as having a spec in property path implies that there is some specification for this resource, while that it not the case.

it is a typo, it is actually $zone.common.spec, fixed it

AndrewChubatiuk requested review from zekker6, Haleygo and f41gh7 as code owners September 20, 2024 06:11

AndrewChubatiuk force-pushed the refactor-distributed-chart branch from 25855ca to 8426568 Compare September 22, 2024 19:01

github-actions bot added the logs-single label Sep 22, 2024

AndrewChubatiuk force-pushed the refactor-distributed-chart branch from 8426568 to d4bc969 Compare September 23, 2024 06:17

Haleygo reviewed Sep 24, 2024

View reviewed changes

AndrewChubatiuk force-pushed the refactor-distributed-chart branch from 73a18e9 to 56f579b Compare September 24, 2024 07:54

AndrewChubatiuk force-pushed the refactor-distributed-chart branch from 56f579b to 6bb6387 Compare October 2, 2024 06:39

zekker6 reviewed Oct 7, 2024

View reviewed changes

AndrewChubatiuk force-pushed the refactor-distributed-chart branch from b5e19ff to dcc3720 Compare October 7, 2024 18:50

distributed: major refactoring, use common templates

3c05054

AndrewChubatiuk force-pushed the refactor-distributed-chart branch from dcc3720 to 3c05054 Compare October 10, 2024 11:10

Merge branch 'master' into refactor-distributed-chart

b277434

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

distributed: major refactoring, use common templates #1520

distributed: major refactoring, use common templates #1520

AndrewChubatiuk commented Sep 20, 2024 •

edited

Loading

Haleygo Sep 24, 2024

AndrewChubatiuk Sep 24, 2024

Haleygo Sep 24, 2024

AndrewChubatiuk Sep 24, 2024

Haleygo Sep 24, 2024

AndrewChubatiuk Sep 24, 2024

Haleygo Sep 24, 2024

AndrewChubatiuk Sep 24, 2024 •

edited

Loading

Haleygo Sep 24, 2024

AndrewChubatiuk Sep 24, 2024 •

edited

Loading

Haleygo Sep 24, 2024

AndrewChubatiuk Sep 24, 2024

AndrewChubatiuk Oct 2, 2024

Haleygo Sep 24, 2024

AndrewChubatiuk Sep 24, 2024

zekker6 Oct 7, 2024

AndrewChubatiuk Oct 7, 2024

		- `vmauthIngestGlobal` was changed to `write.global.vmauth`
		- `vmauthQueryGlobal` was changed to `read.global.vmauth`

	{{- $spec := mergeOverwrite (deepCopy ($zone.common).spec) (deepCopy ($zone.vmagent).spec) }}
	{{- $spec := mergeOverwrite (deepCopy $.Values.common.vmagent.spec) (deepCopy ($zone.common).spec) (deepCopy ($zone.vmagent).spec) }}

distributed: major refactoring, use common templates #1520

Are you sure you want to change the base?

distributed: major refactoring, use common templates #1520

Conversation

AndrewChubatiuk commented Sep 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndrewChubatiuk Sep 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndrewChubatiuk Sep 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndrewChubatiuk commented Sep 20, 2024 •

edited

Loading

AndrewChubatiuk Sep 24, 2024 •

edited

Loading

AndrewChubatiuk Sep 24, 2024 •

edited

Loading