Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meltano supported target-postgres #36

Open
6 of 12 tasks
pnadolny13 opened this issue Jan 24, 2023 · 4 comments
Open
6 of 12 tasks

Meltano supported target-postgres #36

pnadolny13 opened this issue Jan 24, 2023 · 4 comments

Comments

@pnadolny13
Copy link
Contributor

pnadolny13 commented Jan 24, 2023

@pnadolny13
Copy link
Contributor Author

pnadolny13 commented Jan 24, 2023

@tayloramurphy I created this to define the scope of development for these new plugins, track status, and decide when its ready to recommend. Can you help fill in the description with the features (specific to this source) we expect the initial release to support? It will help us and users understand what done is and when we're there.

cc @visch @aaronsteers

@visch
Copy link
Member

visch commented Feb 1, 2023

I don't know when we want to make the swap for the default variant the MeltanoLabs variant.

What I can say is what features does transferwise have that we don't have. Note that because transferwise has been used for a lot longer time they have a lot of "longtail" fixes that we just haven't hit yet with the meltanolabs variant. I also haven't been trying to build at feature parity. Maybe we could take the tests from their variant just to verify everything works as expected.

"Important" features missing from meltanolabs variant. I'm not sure these should hold up the swap to default variant recommendation. The target does work today in a lot of use cases:

  1. Activate Version - feat: Activate Version with example test target-postgres#89 (Done, just needs to get merged)
  2. anyOf - Support for anyOf in property type declarations target-postgres#86
  3. SSL - Support for SSL target-postgres#71 (Should be easy)
  4. Append Only - Support for empty key properties target-postgres#54
  5. Allow Column Alters - allow_column_alter should be True target-postgres#2
  6. Parallelism - Test Parallelism  target-postgres#3 (Should be easy)
  7. Native Batch Support - Native Postgres Batch imports target-postgres#73

@aaronsteers has prioritized these for me generally in the order from top to botom (Not including anything Parallelism or below)

@aaronsteers
Copy link

Thanks, @visch! If SSH (MeltanoLabs/target-postgres#71) becomes difficult, we can push it back. And for anyOf, we can keep the scope there nice and small by focusing on the "nullable" use cases - and not yet handling more complex data type intersections.

With those two notes, I think the above list looks great. Thanks again.

@pnadolny13
Copy link
Contributor Author

@aaronsteers @visch this is great! I updated the issue checklist with what I think is the consensus in/out of scope items. Definitely feel free to change them if more stuff comes up thats blocking us recommending this variant.

@aaronsteers @visch I did ^^^ the same thing as I mentioned in #35 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants