CDK: Add initial Destination abstraction and tests

[sherifnada]

What

Title

How

The interesting part here is the write method. It has the following signature:

@abstractmethod
 def write(
            self,
            config: Mapping[str, Any],
            configured_catalog: ConfiguredAirbyteCatalog,
            input_messages: Iterable[AirbyteMessage]
    ) -> Iterable[AirbyteMessage]:

this pretty much maps to the protocol's write method while abstracting away the fact that we're reading from stdin, by passing in input_messages as an opaque iterable of messages.

Since this class inherits Connector, as usual a dev will need to override check and spec (the latter by providing a spec.json file) to complete a connector. All in all a simple interface. Will probably provide helper methods in a following PR to ease state management or other concerns. Started playing around with a Google Sheets destination to verify.

Recommended reading order

1. destination.py
2. destination_test.py

[davinchia]

Build is failing format. Code changes themselves look solid and inline with what we currently do!

[keu]

Suggested change

	assert destination.spec.call_count == 1 # type: ignore
	destination.spec.assert_called_once() # type: ignore

[sherifnada]

np, done, but is there a benefit to this?

[keu]

Suggested change

	spec_message = next(iter(destination.run_cmd(parsed_args)))
	spec_message = next(destination.run_cmd(parsed_args))

is there any reason for it?

[sherifnada]

run_cmd returns an Iterable so MyPy is not convinced next works without iter

[keu]

Suggested change

	other_iter = other if isinstance(other, Iterator) else iter(other)
	self_iter = self if isinstance(self, Iterator) else iter(self)
	other_iter = iter(other)
	self_iter = iter(self)

[keu]

in general this is overkill,
just use

return list(self) == list(other)

[sherifnada]

done

[keu]

see my proposal above, with it you can do:

destination.spec.return_value = expected_spec

[keu]

not sure why this is here

[sherifnada]

because I forgot to remove it :P

[AetherUnbound]

I know my review is of little consequence, but the ABC is straightforward and easy for me to follow!

[sherifnada]

thanks @AetherUnbound ! Hopefully you'll get to play around with very soon 💪🏼

[sherifnada]

/publish-cdk

🕑 https://github.com/airbytehq/airbyte/actions/runs/1028280016
✅ https://github.com/airbytehq/airbyte/actions/runs/1028280016