Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations #6060

Merged
merged 45 commits into from
Oct 12, 2024
Merged

[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations #6060

merged 45 commits into from
Oct 12, 2024

Conversation

mahadzaryab1
Copy link
Collaborator

@mahadzaryab1 mahadzaryab1 commented Oct 6, 2024 •
edited
Loading

Which problem is this PR solving?

Description of the changes

How was this change tested?

  • Unit Tests and E2E Integration Tests

Checklist

@codecov Codecov
Copy link

codecov bot commented Oct 6, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.91%. Comparing base (b2f8be2) to head (bfd0f93).
Report is 1 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #6060      +/-   ##
==========================================
- Coverage   96.92%   96.91%   -0.01%     
==========================================
  Files         349      349              
  Lines       16598    16646      +48     
==========================================
+ Hits        16087    16133      +46     
- Misses        328      329       +1     
- Partials      183      184       +1
Flag Coverage Δ
badger_v1 7.98% <0.00%> (-0.01%) ⬇️
badger_v2 1.84% <0.00%> (+0.02%) ⬆️
cassandra-4.x-v1 15.75% <0.00%> (-0.04%) ⬇️
cassandra-4.x-v2 1.77% <0.00%> (+0.02%) ⬆️
cassandra-5.x-v1 15.75% <0.00%> (-0.04%) ⬇️
cassandra-5.x-v2 1.77% <0.00%> (+0.02%) ⬆️
elasticsearch-6.x-v1 18.95% <86.24%> (+0.25%) ⬆️
elasticsearch-7.x-v1 19.01% <86.24%> (+0.26%) ⬆️
elasticsearch-8.x-v1 19.20% <86.24%> (+0.25%) ⬆️
elasticsearch-8.x-v2 1.83% <0.00%> (+0.01%) ⬆️
grpc_v1 9.35% <0.00%> (-0.03%) ⬇️
grpc_v2 7.12% <0.00%> (+<0.01%) ⬆️
kafka-v1 9.68% <0.00%> (-0.02%) ⬇️
kafka-v2 1.84% <0.00%> (+0.02%) ⬆️
memory_v2 1.84% <0.00%> (+0.02%) ⬆️
opensearch-1.x-v1 19.06% <86.24%> (+0.26%) ⬆️
opensearch-2.x-v1 19.05% <86.24%> (+0.24%) ⬆️
opensearch-2.x-v2 1.83% <0.00%> (+0.01%) ⬆️
tailsampling-processor 0.49% <0.00%> (+0.03%) ⬆️
unittests 95.72% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

JaredTan95 and others added 10 commits October 6, 2024 10:48
@JaredTan95 @mahadzaryab1
Add configurable index data layout and rollover frequency
51bdde1 
Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
@JaredTan95 @mahadzaryab1
remove indexprefix
b3b0559 
Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
@JaredTan95 @mahadzaryab1
fix prefix
6dbb41e 
Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
@JaredTan95 @mahadzaryab1
fix prefix
44ea7cb 
Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
@JaredTan95 @mahadzaryab1
fix prefix
c0a3040 
Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
@yurishkuro @mahadzaryab1
fix-opensearch-config
4498844 
Signed-off-by: Yuri Shkuro <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
@mahadzaryab1
Add Function To Get Index Name--signoff
88928fb 
Signed-off-by: Mahad Zaryab <[email protected]>
@mahadzaryab1
Use New Function
60cc5a3 
Signed-off-by: Mahad Zaryab <[email protected]>
@mahadzaryab1
Remove Redundant Constants
aec39e5 
Signed-off-by: Mahad Zaryab <[email protected]>
@mahadzaryab1
Fix Test
4663d80 
Signed-off-by: Mahad Zaryab <[email protected]>
mahadzaryab1
mahadzaryab1 commented Oct 6, 2024
View reviewed changes
pkg/es/config/config.go Outdated Show resolved Hide resolved
mahadzaryab1
mahadzaryab1 commented Oct 6, 2024
View reviewed changes
pkg/es/config/config.go Outdated Show resolved Hide resolved
@mahadzaryab1 mahadzaryab1 changed the title [WIP] Refactor ElasticSearch/OpenSearch Storage Configurations Refactor ElasticSearch/OpenSearch Storage Configurations Oct 6, 2024
@mahadzaryab1 mahadzaryab1 marked this pull request as ready for review October 6, 2024 15:44
@mahadzaryab1 mahadzaryab1 requested a review from a team as a code owner October 6, 2024 15:44
@mahadzaryab1 mahadzaryab1 mentioned this pull request Oct 6, 2024
Closed
16 tasks
@yurishkuro yurishkuro mentioned this pull request Oct 6, 2024
Closed
@yurishkuro yurishkuro added the changelog:exprimental Change to an experimental part of the code label Oct 6, 2024
@mahadzaryab1
Add TODO
b4a0456 
Signed-off-by: Mahad Zaryab <[email protected]>
@mahadzaryab1
Copy link
Collaborator Author

@yurishkuro regarding #5790 (comment), are you referring to https://github.com/jaegertracing/jaeger/blob/main/plugin/storage/es/factory.go#L196-L210?

@yurishkuro
Copy link
Member

regarding #5790 (comment), are you referring to https://github.com/jaegertracing/jaeger/blob/main/plugin/storage/es/factory.go#L196-L210?

yes

@yurishkuro yurishkuro merged commit 563f3b0 into jaegertracing:main Oct 12, 2024
49 checks passed
Saumya40-codes pushed a commit to Saumya40-codes/jaeger that referenced this pull request Oct 14, 2024
@mahadzaryab1 @JaredTan95 @yurishkuro
[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations (j…
dff5ed5 
…aegertracing#6060)

## Which problem is this PR solving?
- Part of jaegertracing#6059

## Description of the changes
- Continuation of jaegertracing#5790
- Refactors the configurations for ElasticSearch and OpenSearch to
simplify the configuration by having more logical groupings

## How was this change tested?
- Unit Tests and E2E Integration Tests

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `yarn lint` and `yarn test`

---------

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
Co-authored-by: Jared Tan <[email protected]>
Co-authored-by: Yuri Shkuro <[email protected]>
Signed-off-by: Saumya Shah <[email protected]>
Saumya40-codes pushed a commit to Saumya40-codes/jaeger that referenced this pull request Oct 14, 2024
@mahadzaryab1 @JaredTan95
@yurishkuro @Saumya40-codes
[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations (j…
2b0237e 
…aegertracing#6060)

## Which problem is this PR solving?
- Part of jaegertracing#6059

## Description of the changes
- Continuation of jaegertracing#5790
- Refactors the configurations for ElasticSearch and OpenSearch to
simplify the configuration by having more logical groupings

## How was this change tested?
- Unit Tests and E2E Integration Tests

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `yarn lint` and `yarn test`

---------

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
Co-authored-by: Jared Tan <[email protected]>
Co-authored-by: Yuri Shkuro <[email protected]>
Signed-off-by: Saumya Shah <[email protected]>
chahatsagarmain pushed a commit to chahatsagarmain/jaeger that referenced this pull request Oct 23, 2024
@mahadzaryab1 @JaredTan95
@yurishkuro @chahatsagarmain
[jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations (j…
293819e 
…aegertracing#6060)

## Which problem is this PR solving?
- Part of jaegertracing#6059

## Description of the changes
- Continuation of jaegertracing#5790
- Refactors the configurations for ElasticSearch and OpenSearch to
simplify the configuration by having more logical groupings

## How was this change tested?
- Unit Tests and E2E Integration Tests

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `yarn lint` and `yarn test`

---------

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
Co-authored-by: Jared Tan <[email protected]>
Co-authored-by: Yuri Shkuro <[email protected]>
@mahadzaryab1 mahadzaryab1 deleted the es-layout branch October 31, 2024 22:35
@yurishkuro yurishkuro mentioned this pull request Nov 27, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yurishkuro yurishkuro yurishkuro approved these changes

@albertteoh albertteoh Awaiting requested review from albertteoh albertteoh is a code owner automatically assigned from jaegertracing/jaeger-maintainers

Assignees
No one assigned
Labels
changelog:exprimental Change to an experimental part of the code storage/elasticsearch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mahadzaryab1 @yurishkuro @JaredTan95