fixes #567: ( Impact on BC/DR - Data loss ) - High priority, Nodes with PointValue property cannot be successfully sink to Neo4j With Neo4j Streams Plugin #583
Conversation
| @@ -33,6 +54,42 @@ fun Point.toStreamsPoint(): StreamsPoint { | |||
| } | |||
| } | |||
|
|
|||
| fun Map<String, Any>.toMapValue(): MapValue { | |||
There was a problem hiding this comment.
should we maybe use org.neo4j.values.virtual.MapValueBuilder instead?
| override fun deserialize(parser: JsonParser, context: DeserializationContext): EVENT { | ||
| val root: JsonNode = parser.codec.readTree(parser) | ||
| val meta = JSONUtils.convertValue<Meta>(root["meta"]) | ||
| val schema = JSONUtils.convertValue<Schema>(root["schema"]) |
There was a problem hiding this comment.
am I correct to think that if the user did not include schema information, this still won't work? If so, what would the user do to create point values from messages generated by non-neo4j applications? Regardless, I think we need to extend the documentation accordingly.
There was a problem hiding this comment.
I would not advocate to the users to use this message created by themselves otherwise we need to document all the possible permutations of schema and so on. So for me, this is about the CDC message which contains the info. If it doesn't it does mean that the message is not well-formed.
Maybe is worth adding docs or at least a link to the CDC doc page (https://neo4j.com/docs/kafka-streams/producer/#_transaction_event_handler) in the Kafka Connect PR where there is no direct reference to the message structure.
There was a problem hiding this comment.
Yes, looks like we are also missing the type for geo properties on the sample messages on the page you linked. Would you please update that page and also a separate page (probably best to avoid links) that includes similar sample messages for the kafka connect documentation?
pom.xml
Outdated
| @@ -43,7 +43,7 @@ | |||
| <modules> | |||
| <module>common</module> | |||
| <module>test-support</module> | |||
| <module>kafka-connect-neo4j</module> | |||
| <!-- <module>kafka-connect-neo4j</module>--> | |||
There was a problem hiding this comment.
should we remove this and even the source folder, maybe as a separate PR?
There was a problem hiding this comment.
Yes I commented this to open a discussion about it. As we don't need it anymore we can get rid of it in this branch
The Streams Transaction Event Handler does not produce messages as AVRO format. |
conker84
force-pushed
the
issue_567
branch
4 times, most recently
from
bc12284 to
4f092fb
Compare
conker84
force-pushed
the
issue_567
branch
17 times, most recently
from
7341d9d to
f8e65d7
Compare
…ity, Nodes with PointValue property cannot be successfully sink to Neo4j With Neo4j Streams Plugin
Fixes #567
One sentence summary of the change.
Proposed Changes (Mandatory)
A brief list of proposed changes in order to fix the issue:
5.0for that