Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: state_type incorrect reference #11

Merged
merged 1 commit into from
Feb 28, 2024
Merged

Conversation

sabino
Copy link
Contributor

@sabino sabino commented Feb 16, 2024

Hi!

I've been trying to implement this tap for some time now, as we want to use the CDC implementation with the default pgoutput configuration. Using the postgres extractor definition here: https://hub.meltano.com/extractors/tap-postgres--airbyte/

But I've faced an issue where the state was not being picked up for some reason.
Eventually I figured it was missing the metadata definition to set the replication-method:

    metadata:
      '*':
        replication-method: LOG_BASED

But I've also noticed something else on the log:

2024-02-16T18:31:13.432770Z [info     ]     self.airbyte_state = unpacked_state cmd_type=elb consumer=False name=raw__REDACTED producer=True stdio=stderr string_id=raw__REDACTED
2024-02-16T18:31:13.433219Z [info     ] UnboundLocalError: local variable 'unpacked_state' referenced before assignment cmd_type=elb consumer=False name=raw__REDACTED producer=True stdio=stderr string_id=raw__REDACTED
2024-02-16T18:31:13.433660Z [info     ]                                cmd_type=elb consumer=False name=raw__REDACTED producer=True stdio=stderr string_id=raw__REDACTED
2024-02-16T18:31:13.434099Z [info     ] During handling of the above exception, another exception occurred: cmd_type=elb consumer=False name=raw__REDACTED producer=True stdio=stderr string_id=raw__REDACTED
2024-02-16T18:31:13.434545Z [info     ]                                cmd_type=elb consumer=False name=raw__REDACTED producer=True stdio=stderr string_id=raw__REDACTED
2024-02-16T18:31:13.434978Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=False name=raw__REDACTED producer=True stdio=stderr string_id=raw__REDACTED

After some inspection I saw these statements on the code and after changing them to what this PR proposal, it started working as expected.

I'm wondering if this was not an issue for someone at some point.

@alexmaras
Copy link

alexmaras commented Feb 28, 2024

I've also found that the incremental state wasn't working. I've had to make one further change, branching off of this:

https://github.com/alexmaras/tap-airbyte-wrapper

I'll happily PR my own once this PR is merged - it allows the config to fall back to default_cursor_field in airbyte, which is used if a cursor_field isn't specified. cursor_field doesn't seem to be set anywhere by the tap wrapper, so I'm not totally clear on how you'd even get that into the config.

@z3z1ma z3z1ma merged commit fc42fbd into MeltanoLabs:main Feb 28, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants