Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allow skipping child streams by returning an empty child context from parent stream #1597

Closed
edgarrmondragon opened this issue Apr 10, 2023 · 0 comments · Fixed by #1601
Assignees
Labels
kind/Feature New feature or request valuestream/SDK

Comments

@edgarrmondragon
Copy link
Collaborator

Feature scope

Taps (catalog, state, stream maps, etc.)

Description

Some pseudocodish approach would be:

class ParentStream:
  # Custom tap implementation
  def get_child_context(self, record, context):
    if not record["has_children_records"]:

  # Part of the SDK
  def _sync_children(self, child_context):
    if not child_context:
      logger.warning(
        "Context for child streams of '%s' is empty, skipping sync of any child streams",
        self.name,
      )
      return
    for child_stream in self.child_streams:
      if child_stream.selected or child_stream.has_selected_descendents:
        child_stream.sync(context=child_context)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/Feature New feature or request valuestream/SDK
Projects
None yet
1 participant