🎉 Redshift Source and Destination set SSL as default option #7234

VitaliiMaltsev · 2021-10-21T07:40:36Z

What

We want to create secure-only versions of connectors that can be used in the Airbyte cloud. The idea is that these connectors inherently prevent certain insecure connections such as connecting to a database over the public internet without encryption.

How

Modified the connectors spec to hide any options which allow the user to disable TLS. Changed the connector to enable TLS by default if the TLS option is not specified

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

Updating a connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

Connector Generator

Issue acceptance criteria met
PR name follows PR naming conventions
If adding a new generator, add it to the list of scaffold modules being tested
The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
Documentation which references the generator is updated as needed.

CLAassistant · 2021-10-21T07:41:42Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

vmaltsev seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

alexandr-shegeda

LGTM

sherifnada · 2021-10-21T16:19:19Z

docs/integrations/destinations/redshift.md

@@ -112,6 +112,7 @@ See [docs](https://docs.aws.amazon.com/redshift/latest/dg/r_Character_types.html

 | Version | Date       | Pull Request | Subject |
 | :------ | :--------  | :-----       | :------ |
+| 0.3.18  | 2021-10-21 | [7234](https://github.com/airbytehq/airbyte/pull/7234) | Allow SSL traffic only |


we should also add a line in both .md documents that makes it clear to the user that we only use encrypted traffic

sherifnada · 2021-10-21T16:19:41Z

.../java/io/airbyte/integrations/destination/redshift/RedshiftDestinationAcceptanceTestSSL.java

-import com.fasterxml.jackson.databind.JsonNode;
-import com.fasterxml.jackson.databind.node.ObjectNode;
-import io.airbyte.commons.json.Jsons;
-import io.airbyte.commons.string.Strings;
 import io.airbyte.db.Database;
 import io.airbyte.db.Databases;

 public class RedshiftDestinationAcceptanceTestSSL extends RedshiftCopyDestinationAcceptanceTest {


We can remove the SSL test since all tests would be using SSL

VitaliiMaltsev · 2021-10-22T07:01:15Z

/test connector=connectors/source-redshift

🕑 connectors/source-redshift https://github.com/airbytehq/airbyte/actions/runs/1371111676
✅ connectors/source-redshift https://github.com/airbytehq/airbyte/actions/runs/1371111676
No Python unittests run

VitaliiMaltsev · 2021-10-22T07:15:21Z

/test connector=connectors/destination-redshift

🕑 connectors/destination-redshift https://github.com/airbytehq/airbyte/actions/runs/1371158597
✅ connectors/destination-redshift https://github.com/airbytehq/airbyte/actions/runs/1371158597
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     77    46%
	 normalization/transform_catalog/destination_name_transformer.py     120      6    95%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 370    218    41%
	 normalization/transform_catalog/table_name_registry.py              174     34    80%
	 normalization/transform_catalog/transform.py                         45     26    42%
	 normalization/transform_catalog/utils.py                             33      7    79%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         140     29    79%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1062    403    62%

VitaliiMaltsev · 2021-10-22T07:58:36Z

/publish connector=connectors/source-redshift

🕑 connectors/source-redshift https://github.com/airbytehq/airbyte/actions/runs/1371283187
✅ connectors/source-redshift https://github.com/airbytehq/airbyte/actions/runs/1371283187

VitaliiMaltsev · 2021-10-22T07:59:02Z

/publish connector=connectors/destination-redshift

🕑 connectors/destination-redshift https://github.com/airbytehq/airbyte/actions/runs/1371284586
❌ connectors/destination-redshift https://github.com/airbytehq/airbyte/actions/runs/1371284586

VitaliiMaltsev · 2021-10-22T08:46:20Z

/publish connector=connectors/destination-redshift

🕑 connectors/destination-redshift https://github.com/airbytehq/airbyte/actions/runs/1371442011
✅ connectors/destination-redshift https://github.com/airbytehq/airbyte/actions/runs/1371442011

…q#7234) * Redshift Source and Restination set SSL as default option * add changelog * remove SSL test| add more documentation * bump new version * bump new version Co-authored-by: vmaltsev <[email protected]>

Redshift Source and Restination set SSL as default option

3b5f512

github-actions bot added the area/connectors Connector related issues label Oct 21, 2021

VitaliiMaltsev temporarily deployed to more-secrets October 21, 2021 07:41 Inactive

add changelog

3c47edf

github-actions bot added the area/documentation Improvements or additions to documentation label Oct 21, 2021

VitaliiMaltsev temporarily deployed to more-secrets October 21, 2021 07:49 Inactive

jrhizor temporarily deployed to more-secrets October 21, 2021 07:51 Inactive

jrhizor temporarily deployed to more-secrets October 21, 2021 07:52 Inactive

VitaliiMaltsev changed the title ~~Redshift Source and Restination set SSL as default option~~ 🎉 Redshift Source and Restination set SSL as default option Oct 21, 2021

VitaliiMaltsev requested review from alexandr-shegeda and sherifnada October 21, 2021 08:15

VitaliiMaltsev mentioned this pull request Oct 21, 2021

🎉 Redshift Source: create secure-only version #7040

Closed

38 tasks

alexandr-shegeda changed the title ~~🎉 Redshift Source and Restination set SSL as default option~~ 🎉 Redshift Source and Destination set SSL as default option Oct 21, 2021

alexandr-shegeda approved these changes Oct 21, 2021

View reviewed changes

This was linked to issues Oct 21, 2021

Redshift Source: create secure-only version #6973

Closed

Redshift Destination: create secure-only version #6974

Closed

sherifnada approved these changes Oct 21, 2021

View reviewed changes

vmaltsev added 2 commits October 22, 2021 09:49

Merge branch 'master' into vmaltsev/redshift-ssl-only

b85f051

remove SSL test| add more documentation

fddc7c1

VitaliiMaltsev temporarily deployed to more-secrets October 22, 2021 07:01 Inactive

jrhizor temporarily deployed to more-secrets October 22, 2021 07:03 Inactive

jrhizor temporarily deployed to more-secrets October 22, 2021 07:17 Inactive

bump new version

23be2ea

VitaliiMaltsev temporarily deployed to more-secrets October 22, 2021 07:59 Inactive

jrhizor temporarily deployed to more-secrets October 22, 2021 08:01 Inactive

bump new version

3c4b847

VitaliiMaltsev temporarily deployed to more-secrets October 22, 2021 08:45 Inactive

jrhizor temporarily deployed to more-secrets October 22, 2021 08:48 Inactive

VitaliiMaltsev merged commit 0c93274 into master Oct 22, 2021

VitaliiMaltsev deleted the vmaltsev/redshift-ssl-only branch October 22, 2021 09:28

jrhizor mentioned this pull request Oct 25, 2021

Bump Airbyte version from 0.30.22-alpha to 0.30.23-alpha #7320

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 Redshift Source and Destination set SSL as default option #7234

🎉 Redshift Source and Destination set SSL as default option #7234

VitaliiMaltsev commented Oct 21, 2021 •

edited

Loading

CLAassistant commented Oct 21, 2021

alexandr-shegeda left a comment

sherifnada Oct 21, 2021

sherifnada Oct 21, 2021

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

🎉 Redshift Source and Destination set SSL as default option #7234

🎉 Redshift Source and Destination set SSL as default option #7234

Conversation

VitaliiMaltsev commented Oct 21, 2021 • edited Loading

What

How

Pre-merge Checklist

Community member or Airbyter

Airbyter

Community member or Airbyter

Airbyter

CLAassistant commented Oct 21, 2021

alexandr-shegeda left a comment

Choose a reason for hiding this comment

sherifnada Oct 21, 2021

Choose a reason for hiding this comment

sherifnada Oct 21, 2021

Choose a reason for hiding this comment

VitaliiMaltsev commented Oct 22, 2021 • edited by github-actions bot Loading

VitaliiMaltsev commented Oct 22, 2021 • edited by github-actions bot Loading

VitaliiMaltsev commented Oct 22, 2021 • edited by github-actions bot Loading

VitaliiMaltsev commented Oct 22, 2021 • edited by github-actions bot Loading

VitaliiMaltsev commented Oct 22, 2021 • edited by github-actions bot Loading

VitaliiMaltsev commented Oct 21, 2021 •

edited

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading

VitaliiMaltsev commented Oct 22, 2021 •

edited by github-actions bot

Loading