Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCP Pubsub Publish Operator / Preprocessor Output - Bytes not supported generating DAGs #79

Open
brighton1101 opened this issue Aug 5, 2020 · 0 comments · May be fixed by #124
Open

Comments

@brighton1101
Copy link
Contributor

With the addition of the GCP Pubsub Publish Operator, the preprocessor for this converts message data property to bytes. However, bytes are not supported when generating the actual Python DAGs. The below is the error seen:

...
  File "/home/jenkins/workspace/bigdataconf-test-pr/venv/lib/python3.7/site-packages/boundary_layer/builders/util.py", line 132, in format_value
    type(value)))
Exception: Cannot format value `b'U1VDQ0VTUw=='`: no handler for type <class 'bytes'>

The solution for this likely lies in boundary_layer/builders/util.py in the function format_value. However, I know that byte literals are treated differently in Python 2 and Python 3, so this might be a little more involved of a change. Regardless, there may be future instances when generating a DAG with byte literals is required as well, so it might be worth looking into.

@brighton1101 brighton1101 linked a pull request Jul 22, 2021 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant