fix(ingest): support airflow mapped operators #6738
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
ingestion
cccs-seb
force-pushed
the
support-mapped-operators
branch
from
7b51359
to
01b224a
Compare
cccs-seb
force-pushed
the
support-mapped-operators
branch
from
01b224a
to
755c6f3
Compare
|
Addressed some linting issues + rebased |
hsheth2
reviewed
Dec 12, 2022
hsheth2
changed the title
jjoyce0510
reviewed
Dec 13, 2022
anshbansal
added
the
community-contribution
hsheth2
approved these changes
Dec 13, 2022
szalai1
pushed a commit
to szalai1/datahub
that referenced
this pull request
Dec 22, 2022
cccs-Dustin
pushed a commit
to CybercentreCanada/datahub
that referenced
this pull request
Feb 1, 2023
cccs-Dustin
added a commit
to CybercentreCanada/datahub
that referenced
this pull request
Feb 3, 2023
* chore(ci): update base ingestion image requirements file (datahub-project#6687) * fix(ci): reduce warnings due to deprecated action (datahub-project#6686) * refactor(ui): Adding caching for users, groups, and roles (datahub-project#6673) * fix(ci): revert confluent kafka in base image (datahub-project#6690) * fix(security): version bump to latest minor python image (datahub-project#6694) * docs(ingest/salesforce): list required permissions (datahub-project#6610) * feat(ingest): bigquery - option to set on behalf project (datahub-project#6660) * ci: stop commenting unit test results on PR (datahub-project#6700) The results will still be surfaced under the "Test Results" action workflow, but the results won't be commented on the PR itself. * fix(publish): Attempting to fix publish for auth-api (datahub-project#6695) * build(deps): bump qs from 6.5.2 to 6.5.3 in /smoke-test/tests/cypress (datahub-project#6663) * build(deps): bump express from 4.17.1 to 4.18.2 in /datahub-web-react (datahub-project#6665) * fix(ingest/tableau): support ssl_verify flag properly (datahub-project#6682) * fix(config): unify the handling of boolean environment variables (datahub-project#6684) * fix(ui): fix search on policy builder (datahub-project#6703) * build(deps): bump qs from 6.5.2 to 6.5.3 in /datahub-web-react (datahub-project#6664) * fix(ingest): cleanup config extra usage (datahub-project#6699) * docs(logos): Update Great Expectations logo (datahub-project#6698) * fix(security): play framework upgrade (datahub-project#6626) * fix(security): play framework upgrade * fix(ingest/sagemaker): handle missing ProcessingInputs field (datahub-project#6697) Fixes datahub-project#6360. * build: add retries to gradle wrapper download in ingestion docker (datahub-project#6704) * test(quickstart): add debugging to quickstart test (datahub-project#6718) * fix(setup): Bump setup images to alpine 3.14 with arch based on machine OS. (datahub-project#6612) * fix(setup): Bump setup images to alpine 3.14 with arch based on machine OS. * fix(ingest): fix bug in auto_status_aspect (datahub-project#6705) Co-authored-by: Tamas Nemeth <[email protected]> * fix(security): commons-text in frontend, hadoop-commons in datahub-upgrade (datahub-project#6723) * fix(build): rename conflicting module `auth-api` (datahub-project#6728) * fix(build): rename conflicting module `auth-api` * docs(aws): edit markdown link (datahub-project#6706) * fix(ingest): mysql - fix mysql ingestion issue with non-lowercase database (datahub-project#6713) * feat(ingest): redact configs reported in ingestion_run_summary (datahub-project#6696) * fix(ingest): bigquery - rectify filter for BigQuery external tables (datahub-project#6691) * feat(ingest): snowflake - add separate config for include_column_lineage in snowflake (datahub-project#6712) * fix(ci): flakiness due to onboarding tour in add user test (datahub-project#6734) * feat(ui): Support DataBricks Unity Catalog Source in Ui Ingestion (datahub-project#6707) * feat(ingest/iceberg): add stateful ingestion (datahub-project#6344) * doc(restore): document restore indices API endpoint (datahub-project#6737) * feat(): Views Feature Milestone 1 (datahub-project#6666) * feat(ingest): bigquery - external url support and a small profiling filter fix (datahub-project#6714) * test(ingest): make hive/trino test more reliable (datahub-project#6741) * Initial commit for bigquery ingestion guide (datahub-project#6587) * Initial commit for bigquery ingestion guide * Addressing PR review comments * Fixing lint error * Shorten titles * Removing images * update copy on overview.md * update to setup steps with additional roles * update configuration.md * lowcasing overview.md filename * lowcasing setup.md * lowcasing configuration.md * update reference to setup.md * update reference to setup.md * update reference to configuration.md * lowcase bigquery ingestion guide filenames * Update location of ingestion guides in sidebar * renaming ingestion quickstart guide sidebar * remove old files * Update docs-website/sidebars.js * tweak Co-authored-by: Maggie Hays <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> * fix(ci): remove warnings due to deprecated action (datahub-project#6735) * feat(ingest): add stateful ingestion to the ldap source (datahub-project#6127) Co-authored-by: Harshal Sheth <[email protected]> * fix(ingest): fix serde for empty dicts in unions with null (datahub-project#6745) The code changes in acryldata/avro_gen#16, but tests are written here. * feat(ingest): start simplifying stateful ingestion state (datahub-project#6740) * fix(): Add auth-api as compileOnly dependency (datahub-project#6747) Co-authored-by: MohdSiddique Bagwan <[email protected]> * fix(elasticsearch): build in resilience against IO exceptions on httpclient (datahub-project#6680) * fix(elasticsearch): build in resilience against IO exceptions on http client * ci: fix ingestion gradle retry (datahub-project#6752) * fix(ingest): support airflow mapped operators (datahub-project#6738) * fix(actions): fix mistype slack/teams base url (datahub-project#6754) * fix(smoke-test): fix stateful ingestion test regression (datahub-project#6753) * fix(auth): Renames metadata-auth archive name to not conflict with other modules. (datahub-project#6749) Co-authored-by: david-leifker <[email protected]> * fix(ingest/lookml): fix directory handling and a github_info resolution bug (datahub-project#6751) * refactor(ingest): bigquery-lineage - allow tables and datasets in uppercase (datahub-project#6739) * refactor(ux): Misc UX Improvements (tutorial copy, caching, filters) (datahub-project#6743) * docs(): Added build failed yarn error (datahub-project#6757) docs: add build failed yarn error message and how to deal with it I encountere this error and with the help of the community i could deal with it (https://datahubspace.slack.com/archives/CV2UVAPPG/p1670608619404699). * feat(ingest): remove source config from DatahubIngestionCheckpoint (datahub-project#6722) * fix(python-sdk): DataHubGraph get_aspect should accept empty responses (datahub-project#6760) * fix(): Fix the datahub-web-react build (datahub-project#6764) * docs(ingest/airflow): clarify Airflow 1.x docs for airflow plugin (datahub-project#6761) * feat(ingest): simplify more stateful ingestion state (datahub-project#6762) * fix(ingest): bigquery - handling custom sql errors as warning (datahub-project#6777) * docs(docker): add section for adding community images (datahub-project#6770) * docs(ingest): fix error in custom tags transformer example (datahub-project#6767) * feat(ingest): add `datahub state inspect` command (datahub-project#6763) * refactor(ui): Caching Ingestion Secrets (datahub-project#6772) * docs(snowflake) Snowflake quick ingestion guide (datahub-project#6750) * Optimize kafka setup (datahub-project#6778) * fix(kafka-setup): parallelize topic creation * feat(ingest): lookml - add unreachable views to report (datahub-project#6779) * feat(ci): adding github security reporting to trivy scans (datahub-project#6773) * fix(smoke-test): remove stateful ingestion config check (datahub-project#6781) * fix(ingest): correct external url for account identifier with account name (datahub-project#6715) * fix(tutorial): skip getting steps if there is no user (datahub-project#6786) * fix(kafka-setup): fix return code check (datahub-project#6782) * fix(kafka-setup): parallelize topic creation * Remove -setup from docker compose (not services) * fix(ui): Fixing minor issues with Ingestion forms (datahub-project#6790) * fix(ingest): prevent NullPointerException when non-jdbc SaveIntoDataSourceCommand (datahub-project#6803) * fix(docs): edit text to link (datahub-project#6798) * fix(ingest/dbt): remove unsupported usage indicator (datahub-project#6805) * refactor(ui): Miscellaneous caching improvements (datahub-project#6796) * fix(ingest): bigquery - sharded table support improvements (datahub-project#6789) * chore(ingest): pin black version (datahub-project#6807) * refactor(ingest/stateful): remove most remaining state classes (datahub-project#6791) * fix(bigquery-legacy): Fix for TypeError related failures in legacy plugin (datahub-project#6806) Co-authored-by: John Joyce <[email protected]> * Update Grafana Dashboard (datahub-project#6076) * Add Datasource as variable in dashboard (cherry picked from commit e75b3f7) * Update datahub_dashboard.json (cherry picked from commit 7015926) * Bump docker compose version to 3.8 (cherry picked from commit ff6a97b) * Update grafana image tag from latest to 9.1.4 (cherry picked from commit 2c88e2a) * Update old metric name in datahub_dashboard.json (cherry picked from commit 21b502e) * Add panel for new metrics (cherry picked from commit 1944527) Co-authored-by: Peter Szalai <[email protected]> * refactor(ingest/stateful): remove `IngestionJobStateProvider` (datahub-project#6792) * chore(ingest): bump python package dependencies to resolve vulns (datahub-project#6384) Co-authored-by: John Joyce <[email protected]> * refactor(ingest/stateful): remove `get_last_state` method (datahub-project#6794) * fix(ui): URL encode urns for ownership entity links (datahub-project#6814) * fix(posts): add deletePost GraphQL endpoint (datahub-project#6813) * fix(policies): resolve the associated domain for a domain as the domain itself (datahub-project#6812) * feat(lineage) Adds ability to edit lineage manually from the UI (datahub-project#6816) * fix(ui): change caching to happen post server-response when creating a UI ingestion recipe (datahub-project#6815) * feat(ingest/stateful): remove platform_instance_id from state urn (datahub-project#6795) * feat(ui): Adding DBT Cloud support for UI ingestion (datahub-project#6804) * feat(kafka): expose default kafka producer mechanism (datahub-project#6381) * Expose Kafka Sender Retry Parameters * Implement KafkaHealthChecker * feat(kafka): expose default kafka producer mechanism * feat(ingest): add failure/warning counts to ingest_stats (datahub-project#6823) * refactor(ingest): clean up pipeline init error handling (datahub-project#6817) * fix(ingest): exclude ztsd from uber jar to prevent jni conflicts with spark (datahub-project#6787) Co-authored-by: Tamas Nemeth <[email protected]> * feat(ingest/bigquery): add option to enable/disable legacy sharded table support (datahub-project#6822) Co-authored-by: Harshal Sheth <[email protected]> Co-authored-by: John Joyce <[email protected]> * fix(ingest): support patches in `auto_status_aspect` (datahub-project#6827) Patches generate a raw MCP because MCPW doesn't support patches right now, so we need to handle that correctly downstream. * fix(ci): reduce flakiness views select test (datahub-project#6821) * refactor(ingest): clean up exception types (datahub-project#6818) * fix(ingest): fixed snowflake oauth ingestion not using role attribute from recipe (datahub-project#6825) * refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker (datahub-project#6002) * fix(lineage) Fix lineage viz with multiple siblings (datahub-project#6826) * fix(pac4j-oidc): add verifier parameter (datahub-project#6835) * fix(pac4j-oidc): add verifier parameter * feat(ingest): extract kafka topic config properties as customProperties (datahub-project#6783) * docs: Incorrect import statement fixed in example (datahub-project#6838) * feat(ingestion): spark - support lineage for delta lake writes (datahub-project#6834) * feat(ui): Support adding custom id when creating term and term group (datahub-project#6830) * feat(ci): add cypress test ui based ingestion (datahub-project#6769) * feat(ui): sortable domain list (datahub-project#6736) * fix(ci): add labels based on more folders (datahub-project#6840) * fix(ingest): kafka ingest task hand up with error bootstrap server (datahub-project#6820) * fix(ingest): Fixing lint (datahub-project#6844) * fix(ingestion) Inject pipeline_name into recipes at runtime (datahub-project#6833) * feat(ingest): add db/schema properties hook to SQL common (datahub-project#6847) * fix(oidc): fix oidc authentication loop (datahub-project#6848) * fix(oidc): fix oidc authentication loop * docs(confluent): add details for actions pod for confluent (datahub-project#6810) * feat(ingestion): Business Glossary# Add domain support in GlossaryTerm ingestion (datahub-project#6829) * lint fix * domain in term * domain in term * review comments * add todo Co-authored-by: MohdSiddique Bagwan <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> * fix(ingest/looker): handle missing `label` fields (datahub-project#6849) * refactor(ui): Misc domains improvements (datahub-project#6850) * feat(ingest): add pydantic helper for removed fields (datahub-project#6853) * chore(0.9.5): Bump defaults for release v0.9.5 (datahub-project#6856) * Revert "fix(ci): remove warnings due to deprecated action (datahub-project#6735)" (datahub-project#6857) This reverts commit 1da27ed. * refactor(restli-mce-consumer) (datahub-project#6744) * fix(security): commons-text in frontend * refactor(restli): set threads based on cpu cores feat(mce-consumers): hit local restli endpoint * testing docker build * Add retry configuration options for entity client * Kafka debugging * fix(kafka-setup): parallelize topic creation * Adjust docker build * Docker build updates * WIP * fix(lint): metadata-ingestion lint * fix(gradle-docker): fix docker frontend dep * fix(elastic): fix race condition between gms and mae for index creation * Revert "fix(elastic): fix race condition between gms and mae for index creation" This reverts commit 9629d12. * fix(test): fix datahub frontend test for clean/test cycle * fix(test): datahub-frontend missing assets in test * fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer * gitingore update * fix(docker): remove platform on docker base image, set by buildx * refactor(kafka-producer): update kafka producer tracking/logging * updates per PR feedback * Add documentation around mce standalone consumer Kafka consumer concurrency to follow thread count for restli & sql connection pool Co-authored-by: leifker <[email protected]> Co-authored-by: Pedro Silva <[email protected]> * fix(ci): reduce smoke test run time (datahub-project#6841) * fix(security): require signed/encrypted jwt tokens (datahub-project#6565) * fix(security): require unsigned/encrypted jwt tokens * Add import Co-authored-by: Pedro Silva <[email protected]> * feat(ingest): update profiling to fetch configurable number of sample values (datahub-project#6859) * feat(ingest/airflow): support raw dataset urns in airflow lineage (datahub-project#6854) * feat(ingest/airflow): support dataset Urns in airflow lineage This PR also - resolves a reported circular import issue - refactors the Airflow tests to reduce duplication * fix test * refactor(graphql): make graphqlengine easier to use (datahub-project#6865) * fix(kafka): datahub-upgrade job (datahub-project#6864) * feat(ingest): pass timeout config in kafka admin client api calls (datahub-project#6863) * chore(ingest): loosen requirements file (datahub-project#6867) * feat(ingest): upgrade pydantic version (datahub-project#6858) This PR also removes the requirement on docker-compose v1 and makes our tests use v2 instead. Co-authored-by: Harshal Sheth <[email protected]> * fix(elasticsearch): fixes out of order runId writes (datahub-project#6845) Co-authored-by: leifker <[email protected]> Co-authored-by: Pedro Silva <[email protected]> * chore(ingest): loosen additional requirements (datahub-project#6868) * feat(ingest): bigquery/snowflake - Store last profile date in state (datahub-project#6832) * docs(google-analytics): Correct grammatical error in README.md (datahub-project#6870) * feat(CI): add venv caching (datahub-project#6843) Co-authored-by: Harshal Sheth <[email protected]> * feat(ingest/snowflake): handle failures gracefully and raise permission failures (datahub-project#6748) * fix(runid): always update runid, except when queued (datahub-project#6876) * fix(ingest): conditionally include env in assertion guid (datahub-project#6811) * chore(ci): update dependencies docs-website (datahub-project#6871) * feat(ui) - Add a custom error message for bulk edit to add clarity (datahub-project#6775) Co-authored-by: Kamalaseshan Radha <[email protected]> * docs(adding users): Refreshing the docs for adding new DataHub Users (datahub-project#6879) * test(mce-consumer): mockbeans (datahub-project#6878) * feat(ingest): avoid embedding serialized json in metadata files (datahub-project#6742) * refactor(gradle): move the local docker registry to common location (datahub-project#6881) * refactor(smoke): use env variables (datahub-project#6866) * fix(lint): pin pydantic version (datahub-project#6886) * refactor(docs): Correctly spell elasticsearch in docs (datahub-project#6880) * fix(ingest): okta undefined variable error (datahub-project#6882) * fix(ci): reduce flakiness in add_users, siblings smoke test (datahub-project#6883) * fix(ingest): trino - fall back to default table comment method for all Trino query errors (datahub-project#6873) * test(misc): misc test updates (datahub-project#6890) * deprecate(ingest): bigquery - Removing bigquery-legacy source (datahub-project#6851) Co-authored-by: John Joyce <[email protected]> * chore(ingest): remove inferred args to MCPW, part 1 (datahub-project#6819) * test(ingest/kafka-connect): make docker setup more reliable (datahub-project#6902) * fix(ingest): profiling (bigquery) - Address biquery profiling query error due to timestamp vs data mismatch (datahub-project#6874) * fix(cli): Make datahub quickstart work with latest docker compose in M1 (datahub-project#6891) Co-authored-by: Harshal Sheth <[email protected]> * fix(cli): fix delete urn cli bug + stricter type annotations (datahub-project#6903) * fix(ingest/airflow): reorder imports to avoid cyclical dependencies (datahub-project#6719) Co-authored-by: Harshal Sheth <[email protected]> * feat: remove jq requirement + tweak modeldocgen args (datahub-project#6904) Co-authored-by: Tamas Nemeth <[email protected]> * chore(ingest): loosen pyspark and pydeequ deps (datahub-project#6908) * docs(ingest/looker): fix typos + update lookml github action example (datahub-project#6910) * fix(ingest/metabase): use card_id in dashboard to chart lineage (datahub-project#6583) Co-authored-by: 陈城 <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> * fix(es-setup): create data stream on non-aws (datahub-project#6926) * docs(): Adding missing Platform logos (datahub-project#6892) * feat(ingestion): PowerBI# Improve PowerBI source ingestion (datahub-project#6549) Co-authored-by: MohdSiddique Bagwan <[email protected]> * fix(kafka-setup): Fix compose context for kafka-setup (datahub-project#6923) * feat(backend): Supporting Embeddable Previews for Dashboards, Charts, Datasets (datahub-project#6875) * chore(deps): bump json5 from 2.2.1 to 2.2.3 in /docs-website (datahub-project#6930) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump json5 from 1.0.1 to 1.0.2 in /datahub-web-react (datahub-project#6931) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Aseem Bansal <[email protected]> * fix(ci): managed ingestion test fix (datahub-project#6946) * feat(ingest): add `include_table_location_lineage` flag for SQL common (datahub-project#6934) * feat(ingest): allow extracting snowflake tags (datahub-project#6500) * chore(ingest): unpin pydantic dep (datahub-project#6909) * chore(ingest): partially revert pyspark dep from datahub-project#6908 (datahub-project#6954) * fix(ingest): use branch info when cloning git repos (datahub-project#6937) * chore(ingest): remove inferred args to MCPW, part 2 (datahub-project#6905) * fix(ingest/unity): simplify MCP generation and reporting (datahub-project#6911) Co-authored-by: John Joyce <[email protected]> * chore(ci): parallelise build and test workflow to reduce time (datahub-project#6949) * fix(frontend): sasl.client.callback.handler.class (datahub-project#6962) * chore(react): remove outdated cypress tests and dependency (datahub-project#6948) * fix(ci): restrict GE to fix build issues (datahub-project#6967) * feat(queries): [Experimental] Allow customization of # of queries in Query tab via env var (datahub-project#6964) * feat(ingest/postgres): emit lineage for postgres views (datahub-project#6953) * feat(ingest/vertica): support projections and lineage in vertica (datahub-project#6785) Co-authored-by: mraman2512 <[email protected]> Co-authored-by: Aman.Kumar <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> * fix(ingest): add missing dep for powerbi (datahub-project#6969) * Docs fixes week of 12 22 (datahub-project#6963) Co-authored-by: John Joyce <[email protected]> * fix(ingest): unfreeze bigquery/snowflake column dataclass (datahub-project#6921) * chore(frontend) Remove unused dependencies from package.json (datahub-project#6974) * chore: misc fixes (datahub-project#6966) * feat(ingest/glue): emit s3 lineage for s3a and s3n schemes (datahub-project#6788) Co-authored-by: Harshal Sheth <[email protected]> * fix(kafka-setup): Make kafka-setup run with multiple threads (datahub-project#6970) * feat(ingest): mark database_alias and env as deprecated (datahub-project#6901) * fix(docs): Updating Tag, Glossary Term docs to point to correct GraphQL methods (datahub-project#6965) * chore(deps): bump certifi from 2020.12.5 to 2022.12.7 in /metadata-ingestion/src/datahub/ingestion/source/feast_image (datahub-project#6979) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: John Joyce <[email protected]> * fix(ingest): profiling - Fixing issue with the wrong timestamp stored in check (datahub-project#6978) Co-authored-by: Harshal Sheth <[email protected]> * config(quickstart): enable auto-reindex for quickstart (datahub-project#6983) * feat(privileges) - Create a privilege to manage glossary children recursively (datahub-project#6731) Co-authored-by: Kamalaseshan Radha <[email protected]> Co-authored-by: John Joyce <[email protected]> * chore(ingest): finish removing feast-legacy (datahub-project#6985) * feat(ingest): add import descriptions of two or more nested messages (datahub-project#6959) Co-authored-by: 서주현[G플레이스데이터개발] <[email protected]> * feat(docs) Add feature guide for Manual Lineage (datahub-project#6933) Co-authored-by: John Joyce <[email protected]> * docs(rfc): Serialising GMS Updates with Preconditions (datahub-project#5818) * fix(ingest): kafka-connect - support newer version of debezium (datahub-project#6943) Co-authored-by: Mayuri Nehate <[email protected]> Co-authored-by: John Joyce <[email protected]> * fix(docs): build and broken snowflake docs fix (datahub-project#6997) * fix(ingest): bigquery - views in case more than 1 datasets with views (datahub-project#6995) Co-authored-by: Tamas Nemeth <[email protected]> * fix(docs): Renaming Business Glossary Doc (datahub-project#7001) * fix(ingest/snowflake): fix type annotations + refactor get_connect_args (datahub-project#7004) * fix(docs): Changing the platform event topic name in kafka custom topic docs (datahub-project#7007) * fix(docs): fix name of privilege referenced in posts doc (datahub-project#7002) * fix(SSO): Correctly redirect to originally requested URL in SSO (datahub-project#7011) * fix(ingest): remove dead code from tests (datahub-project#7005) Co-authored-by: John Joyce <[email protected]> * feat(ingestion): Tableau # Embed links (datahub-project#6994) * feat(auth) Update auth cookies to have same-site none for chrome extension (datahub-project#6976) * docs(website): DPG WIP (datahub-project#6998) Co-authored-by: Jeff Merrick <[email protected]> * docs: resize datahub logo (datahub-project#7014) * fix(kafka-setup): Remove reference to non-existing topic (datahub-project#7019) * fix(ingest): powerbi # use display name field as title for powerbi report page (datahub-project#7017) * feat(auth): Allow session ttl to be configurable by env variable (datahub-project#7022) * fix(ui): URL Encode all Entity Profile URLs (datahub-project#7023) * fix(ui ingest): Fix test connection when stateful ingest is enabled (datahub-project#7013) * docs(sso) move root user warning to earlier in SSO guides (datahub-project#7028) * fix(ingest/looker): add clarity in chart input parsing logs (datahub-project#7003) * chore(ingest): remove duplicate data_platform.json file (datahub-project#7026) * feat(ingestion): PowerBI # Remove corpUserInfo aspect ingestion (datahub-project#7034) Co-authored-by: MohdSiddique Bagwan <[email protected]> * fix(metadata-models): remove unnecessary bin folder (datahub-project#7035) * fix(docs): fixing typos (datahub-project#7030) * feat(ingest): Ingest Previews for Looker Charts, Dashboards, and Explores (datahub-project#6941) * fix(graphql):fix issue: autorender aspect could not be displayed on t… (datahub-project#6993) Co-authored-by: yangjd33 <[email protected]> * fix(config): adding quotes (datahub-project#7038) * fix(config): adding quotes (datahub-project#7040) * fix(ingest/bigquery): Turning some usage warning message to debug log as it caused confusion (datahub-project#7024) * feat(ingest/vertica): Adding Vertica as source in Datahub UI (datahub-project#7010) Co-authored-by: Vishal <[email protected]> Co-authored-by: VISHAL KUMAR <[email protected]> Co-authored-by: John Joyce <[email protected]> * fix(): Removed a double set for two fields (datahub-project#7037) * fix(secret-service): fix default encrypt key (datahub-project#7074) * Fix versioning file * Modify SourceConfig and SourceReport * Instantiate StaleEntityRemovalHandler in __init__ method of source * Add entities from current run to the state object * Emitting soft-delete workunits associated with stale entities * Run 'black' tests * Run 'isort' tests * Run 'flake8' and 'mypy' tests * Update status method of all entities * Add draft of integration test for Iceberg source * Add a new integration test for the stateful feature of the iceberg source * Fixing automatic merge * Add steps to build datahub's front-end and back-end to the pipeline * Modified command to build front-end to be consistent with what DataHub (upstream) uses --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Aseem Bansal <[email protected]> Co-authored-by: John Joyce <[email protected]> Co-authored-by: david-leifker <[email protected]> Co-authored-by: orlandine <[email protected]> Co-authored-by: Tamas Nemeth <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Felix Lüdin <[email protected]> Co-authored-by: Aditya Radhakrishnan <[email protected]> Co-authored-by: Maggie Hays <[email protected]> Co-authored-by: Pedro Silva <[email protected]> Co-authored-by: jx2lee <[email protected]> Co-authored-by: Mayuri Nehate <[email protected]> Co-authored-by: Jan Hicken <[email protected]> Co-authored-by: Dmitry Bryazgin <[email protected]> Co-authored-by: mohdsiddique <[email protected]> Co-authored-by: MohdSiddique Bagwan <[email protected]> Co-authored-by: RyanHolstien <[email protected]> Co-authored-by: cccs-seb <[email protected]> Co-authored-by: fully <[email protected]> Co-authored-by: Patrick Franco Braz <[email protected]> Co-authored-by: jakobhanna <[email protected]> Co-authored-by: Shirshanka Das <[email protected]> Co-authored-by: danielli-ziprecruiter <[email protected]> Co-authored-by: Monica Senapati <[email protected]> Co-authored-by: Navin Sharma <[email protected]> Co-authored-by: Peter Szalai <[email protected]> Co-authored-by: raysaka <[email protected]> Co-authored-by: Chris Collins <[email protected]> Co-authored-by: djordje-mijatovic <[email protected]> Co-authored-by: Dago Romer <[email protected]> Co-authored-by: Mirko R <[email protected]> Co-authored-by: Teppo Naakka <[email protected]> Co-authored-by: wangsaisai <[email protected]> Co-authored-by: leifker <[email protected]> Co-authored-by: cccs-eric <[email protected]> Co-authored-by: Meenakshi Kamalaseshan Radha <[email protected]> Co-authored-by: Kamalaseshan Radha <[email protected]> Co-authored-by: Marvin Rösch <[email protected]> Co-authored-by: Stijn De Haes <[email protected]> Co-authored-by: cc <[email protected]> Co-authored-by: 陈城 <[email protected]> Co-authored-by: Fredrik Sannholm <[email protected]> Co-authored-by: Gabe Lyons <[email protected]> Co-authored-by: Lucas Roesler <[email protected]> Co-authored-by: VISHAL KUMAR <[email protected]> Co-authored-by: mraman2512 <[email protected]> Co-authored-by: Aman.Kumar <[email protected]> Co-authored-by: Paul Logan <[email protected]> Co-authored-by: seoju <[email protected]> Co-authored-by: 서주현[G플레이스데이터개발] <[email protected]> Co-authored-by: Matt Matravers <[email protected]> Co-authored-by: 서재권(Data Platform) <[email protected]> Co-authored-by: Thosan Girisona <[email protected]> Co-authored-by: Jeff Merrick <[email protected]> Co-authored-by: Yang Jiandan <[email protected]> Co-authored-by: yangjd33 <[email protected]> Co-authored-by: Rajasekhar-Vuppala <[email protected]> Co-authored-by: Vishal <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Airflow plugin was crashing when trying to apply task policies to dynamically generated tasks (i.e., MappedOperators). This fix will support current versions of Airflow and future versions when the following issue is addressed: apache/airflow#24547.
Related Datahub issue: #6606
Checklist