Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ingest/unity): use fully qualified catalog/schema patterns #7900

Merged
merged 3 commits into from
May 2, 2023

Conversation

hsheth2
Copy link
Collaborator

@hsheth2 hsheth2 commented Apr 24, 2023

In our logs, we already display the fully qualified names for catalogs/schemas whenever we drop something. This changes the actual patterns to use the things that it's already printing out, and hence yields more explainable behavior.

This also updates some stuff in the UI ingestion form which just filled the wrong values into certain fields.

Closes #7489.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added ingestion PR or Issue related to the ingestion of metadata product PR or Issue related to the DataHub UI/UX labels Apr 24, 2023
Copy link
Collaborator

@asikowitz asikowitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple documentation questions but overall implementation looks good

@@ -138,7 +138,7 @@ export const UNITY_TABLE_ALLOW: RecipeField = {

const tableDenyFieldPath = 'source.config.metastore_id_pattern.deny';
export const UNITY_TABLE_DENY: RecipeField = {
name: 'catalog_pattern.allow',
name: 'table_pattern.allow',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be table_pattern.deny? Also, do we need to change the field path?. Same goes for tableAllowFieldPath

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@hsheth2 hsheth2 merged commit bf86235 into datahub-project:master May 2, 2023
@hsheth2 hsheth2 deleted the unity-filters branch May 2, 2023 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ingestion PR or Issue related to the ingestion of metadata product PR or Issue related to the DataHub UI/UX
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect field name on Unity Catalog ingestion page
2 participants