Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Dynamic function filter #13087

Closed
wants to merge 1 commit into from
Closed

WIP: Dynamic function filter #13087

wants to merge 1 commit into from

Conversation

akram
Copy link
Contributor

@akram akram commented Apr 3, 2023 •
edited
Loading

Passes a distribution function to the cluster filter function to allow sorting and distribution in a custom manner.

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Toolchain Guide
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • Optional. My organization is added to USERS.md.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.

Please see Contribution FAQs if you have questions about your pull-request.

@akram akram force-pushed the dynamic-function-filter branch from cf5aebd to 87423c6 Compare April 3, 2023 11:54
@akram akram changed the title Dynamic function filter WIP: Dynamic function filter Apr 3, 2023
anandf
anandf reviewed Apr 3, 2023
View reviewed changes
controller/appcontroller.go Outdated
@@ -222,16 +232,42 @@ func NewApplicationController(
return &ctrl, nil
}

func computeClusterToShardsPlacement(db db.ArgoDB) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function used ?

anandf
anandf reviewed Apr 3, 2023
View reviewed changes
controller/appcontroller.go
ctrl.projByNameCache.Delete(key)
return true
})
if ctrl != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this nil check required after this change ?

anandf
anandf reviewed Apr 3, 2023
View reviewed changes
controller/sharding/sharding.go Outdated
}

return func(c *v1alpha1.Cluster) int {
return clusterIndexdByClusterId[c.ID] % replicas
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a thought, can we do the mod operation in a common place ? The distribution function can just perform the hashing function returning the hash value for the cluster and the cluster filter can then do hash_value % replicas == shard to return the boolean value.

@akram akram marked this pull request as draft April 3, 2023 12:37
@akram akram force-pushed the dynamic-function-filter branch 12 times, most recently from db9bbcf to 26f52fe Compare April 3, 2023 15:16
github-advanced-security[bot]
github-advanced-security bot found potential problems Apr 3, 2023
View reviewed changes
cmd/argocd-application-controller/commands/argocd_application_controller.go
replicas := env.ParseNumFromEnv(common.EnvControllerReplicas, 0, 0, math.MaxInt32)
shard := env.ParseNumFromEnv(common.EnvControllerShard, -1, -math.MaxInt32, math.MaxInt32)
shard := int32(env.ParseNumFromEnv(common.EnvControllerShard, -1, -math.MaxInt32, math.MaxInt32))

Check failure

Code scanning / CodeQL

Incorrect conversion between integer types

Incorrect conversion of an integer with architecture-dependent bit size from [strconv.Atoi](1) to a lower bit size type int32 without an upper bound check.
controller/sharding/sharding.go
@@ -23,31 +33,79 @@
if err != nil {
return 0, fmt.Errorf("hostname should ends with shard number separated by '-' but got: %s", hostname)
}
return shard, nil
return int32(shard), nil

Check failure

Code scanning / CodeQL

Incorrect conversion between integer types

Incorrect conversion of an integer with architecture-dependent bit size from [strconv.Atoi](1) to a lower bit size type int32 without an upper bound check.
controller/sharding/sharding.go Fixed Show fixed Hide fixed
@akram akram force-pushed the dynamic-function-filter branch from 26f52fe to f6486da Compare April 3, 2023 15:58
@akram akram force-pushed the dynamic-function-filter branch from f6486da to 97f0af3 Compare April 3, 2023 16:02
@akram
Copy link
Contributor Author

akram commented Apr 3, 2023

/close

closing in favor of #13018 13018

github-advanced-security[bot]
github-advanced-security bot found potential problems Apr 3, 2023
View reviewed changes
controller/sharding/sharding.go
} else {
h := fnv.New32a()
_, _ = h.Write([]byte(id))
shard := int32(h.Sum32() % uint32(replicas))

Check failure

Code scanning / CodeQL

Incorrect conversion between integer types

Incorrect conversion of an integer with architecture-dependent bit size from [strconv.Atoi](1) to a lower bit size type uint32 without an upper bound check.
@blakepettersson
Copy link
Member

Alright, closing because you said so 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anandf anandf anandf left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@akram @blakepettersson @anandf