Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow configuring the CamundaExporter #22866

Closed
6 tasks done
Tracked by #20910
deepthidevaki opened this issue Sep 30, 2024 · 2 comments · Fixed by #23569
Closed
6 tasks done
Tracked by #20910

Allow configuring the CamundaExporter #22866

deepthidevaki opened this issue Sep 30, 2024 · 2 comments · Fixed by #23569
Assignees
Labels
component/exporter version:8.7.0-alpha1 Label that represents issues released on verions 8.7.0-alpha1

Comments

@deepthidevaki
Copy link
Contributor

deepthidevaki commented Sep 30, 2024

Most configurable parameters are already added as part of the respective PRs that add the feature. But there might be missing ones. This issue is to review and add any necessary configuration.

@mustafadagher mustafadagher self-assigned this Oct 9, 2024
github-merge-queue bot pushed a commit that referenced this issue Oct 9, 2024
## Description

Refactor exporter configs to match zeebe configs as possible

The goal is to avoid confusion when configuring the new exporter and
make it more fluent
- rename IndexSettings var to index
- move indexPrefix to be part of IndexSettings
- adjust usages

In addition, also made appendToFileSchemaSettings more concise

## Related issues

relates to #22866
@mustafadagher
Copy link
Contributor

mustafadagher commented Oct 15, 2024

I reviewed the configuration used in Operate as much as my knowledge could help (and in zeebe exporters too), and I couldn't find any configs that we missed except for two main things that exist in zeebe elastic search exporter

  1. indexSuffixDatePattern and I don't think we need this one now, unless you think otherwise.
  2. bulk.memoryLimit which we have now, yet left unused and I'll delete it according to the discussion in CamundaExporter should respect the memory limit in bulk configuration too when batching exported records #22869

I'll open a PR in a bit with some config validations that are ported from zeebe (I couldn't find anything relevant in operate code base).

EDIT: Operate also uses a refreshInterval in the indexSettings, I don't know if we should have this in the exporter as well.

@sdorokhova please advise if you see something that I might have missed.

@sdorokhova
Copy link
Contributor

EDIT: Operate also uses a refreshInterval in the indexSettings, I don't know if we should have this in the exporter as well.

@sdorokhova please advise if you see something that I might have missed.

I don't think that we need it.

Going over Operate code, it looks like we use it only in migration, which we won't have.

github-merge-queue bot pushed a commit that referenced this issue Oct 15, 2024
## Description

- Added some basic validations to exporter configurations
- Removed the `memoryLimit` from bulk configs (check the discussion in
#22869)

## Related issues

closes #22866
@camundait camundait added the version:8.7.0-alpha1 Label that represents issues released on verions 8.7.0-alpha1 label Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/exporter version:8.7.0-alpha1 Label that represents issues released on verions 8.7.0-alpha1
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants