Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(elasticsearch): Elasticsearch improvements #6894

Merged
merged 59 commits into from
Feb 1, 2023
Merged

feat(elasticsearch): Elasticsearch improvements #6894

merged 59 commits into from
Feb 1, 2023

Conversation

david-leifker
Copy link
Collaborator

@david-leifker david-leifker commented Dec 29, 2022 •
edited
Loading

  • Stemming
  • Synonyms
  • URN (full or partial) search
  • Autocomplete improvements
  • Quoted search analyzer for exact & prefix match
  • Adds test fixture for search tests
  • Indices cleanup
  • Reindexing improvements and system update process
  • Updates ES to 7.10 from 7.9

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@david-leifker
feat(elasticsearch): Elasticsearch improvements
2120a4c 
* Stemming
* Synonyms
* URN (full or partial) search
* Autocomplete improvements
* Adds test fixture for search tests
* Updates ES to 7.10 from 7.9
@github-actions github-actions bot added devops PR or Issue related to DataHub backend & deployment ingestion PR or Issue related to the ingestion of metadata product PR or Issue related to the DataHub UI/UX labels Dec 29, 2022
david-leifker and others added 14 commits December 29, 2022 15:24
@david-leifker
Merge branch 'master' into feat/elasticsearch-optimization-ext
af4ca66
@david-leifker
quickstart updates (#6895)
668cebc
@david-leifker
David leifker/elasticsearch optimization ext (#6896)
60b88f5
@david-leifker
David leifker/elasticsearch optimization ext (#6897)
d34e756
@david-leifker
David leifker/elasticsearch optimization ext (#6898)
8234b0c
@david-leifker
David leifker/elasticsearch optimization ext (#6899)
2a09907
@RyanHolstien
feat(elasticsearchOptimization): typo fix (#6907)
1b15911
@RyanHolstien @david-leifker
feat(elasticsearch): add option to disable clone and fix urn issue (#…
a900d2f 
…6900)

Co-authored-by: david-leifker <[email protected]>
@RyanHolstien
actually fix war run (#6912)
98ce022
@RyanHolstien
fix(test): fix entity spec builder assumption (#6914)
abf81e0
@RyanHolstien
fix(bihe): fix consumer group name for docker (#6915)
49cb85c
@RyanHolstien
fix(quickstart): generate quickstart composes (#6916)
3298ef3
@RyanHolstien
fix(test): fix transformer test (#6917)
39d4710
@david-leifker
David leifker/elasticsearch optimization ext (#6913)
0af681a
@david-leifker david-leifker marked this pull request as draft December 31, 2022 06:20
@david-leifker david-leifker marked this pull request as ready for review January 3, 2023 18:22
david-leifker and others added 22 commits January 9, 2023 09:54
@david-leifker
Merge branch 'master' into feat/elasticsearch-optimization-ext
0548e77
@david-leifker
David leifker/elasticsearch optimization ext (#6982)
f44a92f
@david-leifker
David leifker/elasticsearch optimization ext (#6986)
3e9f74f
@david-leifker
David leifker/elasticsearch optimization ext (#6989)
8a7e215
@david-leifker @anshbansal
David leifker/elasticsearch optimization ext (#6996)
0d62465 
Co-authored-by: Aseem Bansal <[email protected]>
@RyanHolstien
feat(eso): add docs (#6981)
33eef5e
@david-leifker
Merge branch 'master' into feat/elasticsearch-optimization-ext
7283080
@RyanHolstien
feast(eso): rename topic (#7029)
f3a5ad2
@nmbryant
feat(elastic): adds function to retrieve orphaned indices (#7110)
ac04055
@david-leifker
David leifker/elasticsearch optimization ext (#7152)
5288c5d
@david-leifker @anshbansal
@maggiehays @hsheth2 @siddiquebagwan @siddiquebagwan-gslab @jjoyce0510 @yangjiandan @treff7es @Rajasekhar-Vuppala @vishalkSimplify @bda618 @mayurinehate @jx2lee @ngamanda @chriscollins3456 @aditya-radhakrishnan @pedro93 @looppi @remisalmon @bossenti @shirshanka @DSchmidtDev @RyanHolstien @szalai1 @ruedigerblock @feldjay @wngus606 @tonycsoka @nmbryant @ssilb4 @mmmeeedddsss @dependabot @Masterchen09 @laulpogan @tinolyu @tinolyuu @kdongho
David leifker/elasticsearrch optimization ext merge (#7155)
f5b1793 
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Aseem Bansal <[email protected]>
Co-authored-by: Maggie Hays <[email protected]>
Co-authored-by: Harshal Sheth <[email protected]>
Co-authored-by: mohdsiddique <[email protected]>
Co-authored-by: MohdSiddique Bagwan <[email protected]>
Co-authored-by: John Joyce <[email protected]>
Co-authored-by: Yang Jiandan <[email protected]>
Co-authored-by: yangjd33 <[email protected]>
Co-authored-by: Tamas Nemeth <[email protected]>
Co-authored-by: Rajasekhar-Vuppala <[email protected]>
Co-authored-by: Vishal <[email protected]>
Co-authored-by: VISHAL KUMAR <[email protected]>
Co-authored-by: Dmitry Bryazgin <[email protected]>
Co-authored-by: Mayuri Nehate <[email protected]>
Co-authored-by: jx2lee <[email protected]>
Co-authored-by: jj.lee <[email protected]>
Co-authored-by: Amanda Ng <[email protected]>
Co-authored-by: Chris Collins <[email protected]>
Co-authored-by: Aditya Radhakrishnan <[email protected]>
Co-authored-by: Pedro Silva <[email protected]>
Co-authored-by: Teppo Naakka <[email protected]>
Co-authored-by: Remi <[email protected]>
Co-authored-by: Tim <[email protected]>
Co-authored-by: Shirshanka Das <[email protected]>
Co-authored-by: David Schmidt <[email protected]>
Co-authored-by: RyanHolstien <[email protected]>
Co-authored-by: Peter Szalai <[email protected]>
Co-authored-by: ruedigerblock <[email protected]>
Co-authored-by: J Feldman <[email protected]>
Co-authored-by: seoju <[email protected]>
Co-authored-by: tony csoka <[email protected]>
Co-authored-by: Nate Bryant <[email protected]>
Co-authored-by: Fully <[email protected]>
Co-authored-by: Mert Tunç <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Felix Lüdin <[email protected]>
Co-authored-by: Paul Logan <[email protected]>
Co-authored-by: (((((Al))))) <[email protected]>
Co-authored-by: Tino Lyu <[email protected]>
Co-authored-by: kdongho <[email protected]>
@david-leifker
Merge branch 'master' into feat/elasticsearch-optimization-ext
79dbc23
@david-leifker
Delete junit.quick.xml
620de70
@david-leifker
David leifker/elasticsearch optimization ext (#7163)
9d686e5
@david-leifker
Merge branch 'master' into feat/elasticsearch-optimization-ext
16b28ac
@david-leifker
David leifker/elasticsearch optimization ext (#7164)
9d806a5
@david-leifker
David leifker/elasticsearch optimization ext (#7165)
16eb729
@david-leifker
Update smoke.sh
00a062e
@david-leifker
Delete junit.quick.xml
123d74d
@david-leifker @anshbansal
David leifker/elasticsearch optimization ext (#7166)
90048b8 
Co-authored-by: Aseem Bansal <[email protected]>
@david-leifker
David leifker/elasticsearch optimization ext (#7201)
36d97af
@david-leifker
Merge branch 'master' into feat/elasticsearch-optimization-ext
a532388
@david-leifker david-leifker merged commit 39920bb into master Feb 1, 2023
@david-leifker david-leifker deleted the feat/elasticsearch-optimization-ext branch February 1, 2023 00:44
danielcmessias pushed a commit to danielcmessias/datahub that referenced this pull request Feb 1, 2023
@david-leifker @danielcmessias
feat(elasticsearch): Elasticsearch improvements (datahub-project#6894)
61339fe
hsheth2 added a commit to hsheth2/datahub that referenced this pull request Feb 1, 2023
@hsheth2
ci(): fix datahub-upgrade quickstart regression
57f8b61 
Fixes datahub-project#6894.
@hsheth2 hsheth2 mentioned this pull request Feb 1, 2023
Merged
5 tasks
jjoyce0510
jjoyce0510 reviewed Feb 1, 2023
View reviewed changes
datahub-upgrade/src/main/java/com/linkedin/datahub/upgrade/system/SystemUpdate.java


@Slf4j
public class SystemUpdate implements Upgrade {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty cool how we are adding these into here....

ericyomi pushed a commit to ericyomi/datahub that referenced this pull request Feb 8, 2023
@david-leifker
feat(elasticsearch): Elasticsearch improvements (datahub-project#6894)
be6364e
oleg-ruban pushed a commit to RChygir/datahub that referenced this pull request Feb 28, 2023
@david-leifker
feat(elasticsearch): Elasticsearch improvements (datahub-project#6894)
37b1092
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jjoyce0510 jjoyce0510 jjoyce0510 left review comments

@RyanHolstien RyanHolstien RyanHolstien approved these changes

Assignees
No one assigned
Labels
devops PR or Issue related to DataHub backend & deployment ingestion PR or Issue related to the ingestion of metadata product PR or Issue related to the DataHub UI/UX
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@david-leifker @RyanHolstien @jjoyce0510 @anshbansal @nmbryant