Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse invalid dates #109

Closed

Conversation

ivanovyordan
Copy link

@ivanovyordan ivanovyordan commented Nov 4, 2020

Description of change

Resolves #94
Another solution: #95

Python can only parse dates up to 9999-12-31. When we try to work with a date past the maximum supported, a ValueError is being thrown.

These dates are invalid anyway, so I decided to just hardcode them to the maximum supported date.

QA steps

  • automated tests passing
  • manual qa steps passing (list below)

I actually couldn't make the logical replication slot working. Other than that, all unit tests pass.

Manual test:

  1. Create a Postgres table
create table test_table (id serial primary key, created_date date);
  1. Insert some rows:
insert into test_table(created_date) values('1989-02-27'), ('9999-12-31'), ('10000-01-01');
  1. Ran the tap and confirmed it fails
  2. Added a Snowflake target
  3. Ran the tap against the target and confirmed it fails again
  4. Made the change
  5. Ran the tap confirmed it pass
  6. Ran the tap against the target and confirmed it pass again
  7. Ran the INSERT query again
  8. Ran the tap against the target and confirmed it pass again

Result value in Snowflake - 9999-12-31 00:00:00.000

Risks

We'd obviously have some data loss when we have dates 8000 years from now.

Rollback steps

  • revert this branch

Python can only parse dates up to 9999-12-31. When we try to work with a
date past the maximum supported, a ValueError is being thrown.

These dates are invalid anyway, so I decided to just hardcode them to
the maximum supported date.
@cmerrick
Copy link

cmerrick commented Nov 4, 2020

Hi @ivanovyordan, thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes.

@cmerrick
Copy link

cmerrick commented Nov 4, 2020

You did it @ivanovyordan!

Thank you for signing the Singer Contribution License Agreement.

@ivanovyordan
Copy link
Author

Sorry, this PR just doesn't have the needed quality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Minimum date range not supported
2 participants