-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Preserve __alias__
when mapping streams with repeated schema messages
#1524
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1524 +/- ##
=======================================
Coverage 85.50% 85.51%
=======================================
Files 57 57
Lines 4692 4693 +1
Branches 802 802
=======================================
+ Hits 4012 4013 +1
Misses 489 489
Partials 191 191
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @DanilJr!
Thanks for the PR. Just one question. Otherwise this looks good 🙂
__alias__
is when mapping streams with repeated schema messages
I would like to have automated tests but at least manual testing confirms that the patch works as intended: 1. Singer input{"type": "SCHEMA", "stream": "example", "schema": {"properties": {"id": {"type": "integer"}}}}
{"type": "RECORD", "stream": "example", "record": {"id": 1}}
{"type": "SCHEMA", "stream": "example", "schema": {"properties": {"id": {"type": "integer"}}}}
{"type": "RECORD", "stream": "example", "record": {"id": 2}} 2. Stream config{
"stream_maps": {
"example": {
"__alias__": "example_v2"
}
}
} 3. Mapper output{"type": "SCHEMA", "stream": "example_v2", "schema": {"properties": {"id": {"type": "integer"}}}, "key_properties": [], "bookmark_properties": []}
{"type": "RECORD", "stream": "example_v2", "record": {"id": 1}, "time_extracted": "2023-03-24T19:41:37.953424+00:00"}
{"type": "SCHEMA", "stream": "example_v2", "schema": {"properties": {"id": {"type": "integer"}}}, "key_properties": [], "bookmark_properties": []}
{"type": "RECORD", "stream": "example_v2", "record": {"id": 2}, "time_extracted": "2023-03-24T19:41:37.953966+00:00"} I can also confirm that using a explicit shallow copy ( |
__alias__
is when mapping streams with repeated schema messages__alias__
when mapping streams with repeated schema messages
It relates to this issue
Closes #1521
📚 Documentation preview 📚: https://meltano-sdk--1524.org.readthedocs.build/en/1524/