Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty strings are treated as null when syncing from Snowflake to Postgres, causing failures on non-null constraints #349

Closed
nicklausroach opened this issue Jul 30, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@nicklausroach
Copy link

Issue Description

  • Description of the issue: After upgrading to v1.2.14, our syncs from Snowflake to Postgres are failing. This seems to be due to empty strings in a field being treated as nulls, resulting in a failure when hitting our non-null constraints.

  • Sling version (sling --version): 1.2.14

  • Operating System (linux, mac, windows): linux

  • Replication Configuration: Generated by custom Dagster resource

  • Log Output (please run command with -d):

fatal:
~ Error executing: insert into "public"."usage_metrics_daily_jobs_aggregated" ("organization_id", "agent_type", "job_name", "repository_name", "job_day", "materializations", "step_duration_mins", "steps", "run_duration_mins", "runs", "steps_credits") select "organization_id", "agent_type", "job_name", "repository_name", "job_day"::date as "job_day", "materializations", "step_duration_mins", "steps", "run_duration_mins", "runs", "steps_credits" from "public"."usage_metrics_daily_jobs_aggregated_tmp"
pq: null value in column "repository_name" of relation "usage_metrics_daily_jobs_aggregated" violates not-null constraint
@flarco
Copy link
Collaborator

flarco commented Jul 30, 2024

Thanks for reporting.

@flarco flarco added the bug Something isn't working label Jul 30, 2024
@flarco
Copy link
Collaborator

flarco commented Aug 14, 2024

Should be good now with 9644af3
Closing.

@flarco flarco closed this as completed Aug 14, 2024
@flarco flarco mentioned this issue Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants