SSH for Postgres Destination #5743

cgardens · 2021-08-30T22:18:43Z

What

Follows on work from SSH Postgres Source SSH for Postgres Source #5742
@Phlair as far as I can tell, all tests are passing except those that require normalization. Can you rebase onto this branch so that we can get to green and release this.

How

Slightly different approach than I took on the Postgres Source. I use a decorate to wrap the destination methods in SSH tunnels. I think I like this pattern better. Curious to hear opinions.

Pre-merge Checklist

verify normalization works
publish version
update public docs and change log
update developer logs

cgardens · 2021-08-30T22:18:57Z

/test connector=connectors/destination-postgres

🕑 connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1183945013
❌ connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1183945013

airbyte-jenny

The decorator wrapping approach seems clever and pretty clean. Code makes sense overall.

michel-tricot · 2021-08-30T22:44:06Z

...n-jdbc/src/main/java/io/airbyte/integrations/destination/jdbc/SshWrappedJdbcDestination.java

+
+  private final Destination delegate;
+
+  public SshWrappedJdbcDestination(final Destination delegate) {


According to the name of the class, if it is only for JDBC destination, shoudn't the argument be of type JdbcDestination or a less permissive type?

Good call. I actually think the name of this class should be more permissive. Just call it SshWrappedDestination. Ultimately there's nothing here going on that can't work for anything that we might want to access via ssh tunnel.

sherifnada

I have the same question about transparently replacing hostname, but otherwise looks mergable

cgardens · 2021-09-03T14:52:43Z

/test connector=connectors/destination-postgres

🕑 connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1198429643
❌ connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1198429643

Co-authored-by: George Claireaux <[email protected]>

cgardens · 2021-09-07T21:58:35Z

/test connector=connectors/destination-postgres

🕑 connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1211125237
❌ connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1211125237

cgardens · 2021-09-07T22:20:23Z

/publish connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1211182403
✅ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1211182403

sherifnada

mostly reviewed the normalization changes, lgtm

docs/integrations/sources/postgres.md

Co-authored-by: Sherif A. Nada <[email protected]>

cgardens · 2021-09-07T23:18:36Z

/publish connector=connectors/destination-postgres

🕑 connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1211306660
✅ connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1211306660

cgardens requested review from jrhizor, sherifnada, Phlair and airbyte-jenny August 30, 2021 22:18

github-actions bot added the area/connectors Connector related issues label Aug 30, 2021

jrhizor temporarily deployed to more-secrets August 30, 2021 22:21 Inactive

airbyte-jenny approved these changes Aug 30, 2021

View reviewed changes

michel-tricot reviewed Aug 30, 2021

View reviewed changes

sherifnada reviewed Aug 30, 2021

View reviewed changes

jrhizor approved these changes Sep 1, 2021

View reviewed changes

cgardens force-pushed the cgardens/pg_ssh branch from 37f685d to 437fb14 Compare September 2, 2021 13:50

Base automatically changed from cgardens/pg_ssh to master September 2, 2021 18:32

cgardens force-pushed the cgardens/pg_ssh_destination branch from 817f1fa to d43b500 Compare September 3, 2021 14:38

github-actions bot added the area/documentation Improvements or additions to documentation label Sep 3, 2021

jrhizor temporarily deployed to more-secrets September 3, 2021 14:54 Inactive

Phlair mentioned this pull request Sep 6, 2021

ssh tunnel for postgres normalization #5818

Closed

1 task

cgardens added 4 commits September 7, 2021 14:12

wip

447ad55

update change log

f1cbeeb

update to use new config schema

8b3af24

fix tests to clean themselves up

7df041f

cgardens force-pushed the cgardens/pg_ssh_destination branch from e63a5a3 to 7df041f Compare September 7, 2021 21:12

ssh tunnel for postgres normalization (#5897)

86145b1

Co-authored-by: George Claireaux <[email protected]>

github-actions bot added area/worker Related to worker normalization labels Sep 7, 2021

jrhizor temporarily deployed to more-secrets September 7, 2021 22:01 Inactive

cgardens added 2 commits September 7, 2021 15:12

format

17536e5

fix alpha

1695ed4

bump normalization version

f2f7337

jrhizor temporarily deployed to more-secrets September 7, 2021 22:23 Inactive

sherifnada approved these changes Sep 7, 2021

View reviewed changes

docs/integrations/sources/postgres.md Outdated Show resolved Hide resolved

cgardens and others added 2 commits September 7, 2021 15:48

Update docs/integrations/sources/postgres.md

2841e24

Co-authored-by: Sherif A. Nada <[email protected]>

fix schema

54bfc88

jrhizor temporarily deployed to more-secrets September 7, 2021 23:10 Inactive

cgardens added 2 commits September 7, 2021 16:15

change log for destination

df2f840

bump postgres version

6ca1a29

jrhizor temporarily deployed to more-secrets September 7, 2021 23:21 Inactive

cgardens merged commit 8ad43af into master Sep 8, 2021

cgardens deleted the cgardens/pg_ssh_destination branch September 8, 2021 00:06

sherifnada mentioned this pull request Sep 8, 2021

[EPIC] SSH tunnel support for OLTP DB sources & destinations #5901

Closed

7 tasks

jrhizor mentioned this pull request Sep 8, 2021

Bump Airbyte version from 0.29.15-alpha to 0.29.16-alpha #5925

Merged

This was referenced Sep 10, 2021

Add support & test cases for Oracle Destination via SSH tunnels #5969

Closed

Add support & test cases for MSSQL Destination via SSH tunnels #5970

Closed

alexandr-shegeda mentioned this pull request Sep 20, 2021

Add support & test cases for MySQL Destination via SSH tunnels #6316

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SSH for Postgres Destination #5743

SSH for Postgres Destination #5743

cgardens commented Aug 30, 2021 •

edited

Loading

cgardens commented Aug 30, 2021 •

edited by github-actions bot

Loading

airbyte-jenny left a comment

michel-tricot Aug 30, 2021

cgardens Aug 31, 2021

sherifnada left a comment

cgardens commented Sep 3, 2021 •

edited by github-actions bot

Loading

cgardens commented Sep 7, 2021 •

edited by github-actions bot

Loading

cgardens commented Sep 7, 2021 •

edited by github-actions bot

Loading

sherifnada left a comment

cgardens commented Sep 7, 2021 •

edited by github-actions bot

Loading


		private final Destination delegate;

		public SshWrappedJdbcDestination(final Destination delegate) {

SSH for Postgres Destination #5743

SSH for Postgres Destination #5743

Conversation

cgardens commented Aug 30, 2021 • edited Loading

What

How

Recommended reading order

Pre-merge Checklist

cgardens commented Aug 30, 2021 • edited by github-actions bot Loading

airbyte-jenny left a comment

Choose a reason for hiding this comment

michel-tricot Aug 30, 2021

Choose a reason for hiding this comment

cgardens Aug 31, 2021

Choose a reason for hiding this comment

sherifnada left a comment

Choose a reason for hiding this comment

cgardens commented Sep 3, 2021 • edited by github-actions bot Loading

cgardens commented Sep 7, 2021 • edited by github-actions bot Loading

cgardens commented Sep 7, 2021 • edited by github-actions bot Loading

sherifnada left a comment

Choose a reason for hiding this comment

cgardens commented Sep 7, 2021 • edited by github-actions bot Loading

cgardens commented Aug 30, 2021 •

edited

Loading

cgardens commented Aug 30, 2021 •

edited by github-actions bot

Loading

cgardens commented Sep 3, 2021 •

edited by github-actions bot

Loading

cgardens commented Sep 7, 2021 •

edited by github-actions bot

Loading

cgardens commented Sep 7, 2021 •

edited by github-actions bot

Loading

cgardens commented Sep 7, 2021 •

edited by github-actions bot

Loading