Skip to content

Commit

Permalink
fix: remove mapValues that excluded ROWTIME and ROWKEY columns (#4066)
Browse files Browse the repository at this point in the history 
* fix: remove mapValues that excluded ROWTIME and ROWKEY columns

With the changes introduced in #4052 it is no longer possible for the output schema of persistent queries to have reserved column names, i.e. `ROWKEY` or `ROWTIME` and hence the code to exclude these, both from the schema and from the output, is no longer needed and can be removed.

`ROWKEY` and `ROWTIME` are still copied into the value schema to ensure they are easily available. However, the final `ProjectNode` will never include them in the output as they are not allowed.
  • Loading branch information
@big-andy-coates
big-andy-coates authored Dec 9, 2019
1 parent 89e059b commit a6982bd
Show file tree
Hide file tree
Showing 461 changed files with 984 additions and 2,575 deletions.
53 changes: 0 additions & 53 deletions ksql-execution/src/main/java/io/confluent/ksql/execution/util/SinkSchemaUtil.java
View file

This file was deleted.

66 changes: 0 additions & 66 deletions ksql-execution/src/test/java/io/confluent/ksql/execution/util/SinkSchemaUtilTest.java
View file

This file was deleted.

7 changes: 2 additions & 5 deletions .../0_6_0-pre/arraycontains_-_filter_rows_where_the_ARRAY_column_contains_a_specified_STRING
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...re/arraycontains_-_filter_rows_where_the_STRUCT-_ARRAY_column_contains_a_specified_STRING
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...t/resources/expected_topology/0_6_0-pre/arrayindex_-_select_the_first_element_of_an_Array
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,8 @@ Topologies:
--> Project
<-- KSTREAM-SOURCE-0000000000
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000003
--> KSTREAM-SINK-0000000003
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: KSTREAM-MAPVALUES-0000000003 (stores: [])
--> KSTREAM-SINK-0000000004
Sink: KSTREAM-SINK-0000000003 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000003

7 changes: 2 additions & 5 deletions ...sources/expected_topology/0_6_0-pre/arrayindex_-_select_the_last_element_of_an_Array_(-1)
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,8 @@ Topologies:
--> Project
<-- KSTREAM-SOURCE-0000000000
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000003
--> KSTREAM-SINK-0000000003
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: KSTREAM-MAPVALUES-0000000003 (stores: [])
--> KSTREAM-SINK-0000000004
Sink: KSTREAM-SINK-0000000003 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000003

7 changes: 2 additions & 5 deletions ...c/test/resources/expected_topology/0_6_0-pre/asarray_-_construct_a_list_from_two_elements
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,8 @@ Topologies:
--> Project
<-- KSTREAM-SOURCE-0000000000
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000003
--> KSTREAM-SINK-0000000003
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: KSTREAM-MAPVALUES-0000000003 (stores: [])
--> KSTREAM-SINK-0000000004
Sink: KSTREAM-SINK-0000000003 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000003

7 changes: 2 additions & 5 deletions ...ts/src/test/resources/expected_topology/0_6_0-pre/asmap_-_create_map_from_key_value_lists
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,8 @@ Topologies:
--> Project
<-- KSTREAM-SOURCE-0000000000
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000003
--> KSTREAM-SINK-0000000003
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: KSTREAM-MAPVALUES-0000000003 (stores: [])
--> KSTREAM-SINK-0000000004
Sink: KSTREAM-SINK-0000000003 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000003

7 changes: 2 additions & 5 deletions ...tional-tests/src/test/resources/expected_topology/0_6_0-pre/average-udaf_-_average_double
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,8 @@ Topologies:
--> KTABLE-TOSTREAM-0000000006
<-- Aggregate-Aggregate-ToOutputSchema
Processor: KTABLE-TOSTREAM-0000000006 (stores: [])
--> KSTREAM-MAPVALUES-0000000007
--> KSTREAM-SINK-0000000007
<-- Aggregate-Project
Processor: KSTREAM-MAPVALUES-0000000007 (stores: [])
--> KSTREAM-SINK-0000000008
Sink: KSTREAM-SINK-0000000007 (topic: OUTPUT)
<-- KTABLE-TOSTREAM-0000000006
Sink: KSTREAM-SINK-0000000008 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000007

7 changes: 2 additions & 5 deletions ...unctional-tests/src/test/resources/expected_topology/0_6_0-pre/average-udaf_-_average_int
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,8 @@ Topologies:
--> KTABLE-TOSTREAM-0000000006
<-- Aggregate-Aggregate-ToOutputSchema
Processor: KTABLE-TOSTREAM-0000000006 (stores: [])
--> KSTREAM-MAPVALUES-0000000007
--> KSTREAM-SINK-0000000007
<-- Aggregate-Project
Processor: KSTREAM-MAPVALUES-0000000007 (stores: [])
--> KSTREAM-SINK-0000000008
Sink: KSTREAM-SINK-0000000007 (topic: OUTPUT)
<-- KTABLE-TOSTREAM-0000000006
Sink: KSTREAM-SINK-0000000008 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000007

7 changes: 2 additions & 5 deletions ...nctional-tests/src/test/resources/expected_topology/0_6_0-pre/average-udaf_-_average_long
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,8 @@ Topologies:
--> KTABLE-TOSTREAM-0000000006
<-- Aggregate-Aggregate-ToOutputSchema
Processor: KTABLE-TOSTREAM-0000000006 (stores: [])
--> KSTREAM-MAPVALUES-0000000007
--> KSTREAM-SINK-0000000007
<-- Aggregate-Project
Processor: KSTREAM-MAPVALUES-0000000007 (stores: [])
--> KSTREAM-SINK-0000000008
Sink: KSTREAM-SINK-0000000007 (topic: OUTPUT)
<-- KTABLE-TOSTREAM-0000000006
Sink: KSTREAM-SINK-0000000008 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000007

7 changes: 2 additions & 5 deletions ...ests/src/test/resources/expected_topology/0_6_0-pre/average_-_calculate_average_in_select
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,8 @@ Topologies:
--> KTABLE-TOSTREAM-0000000006
<-- Aggregate-Aggregate-ToOutputSchema
Processor: KTABLE-TOSTREAM-0000000006 (stores: [])
--> KSTREAM-MAPVALUES-0000000007
--> KSTREAM-SINK-0000000007
<-- Aggregate-Project
Processor: KSTREAM-MAPVALUES-0000000007 (stores: [])
--> KSTREAM-SINK-0000000008
Sink: KSTREAM-SINK-0000000007 (topic: AVG)
<-- KTABLE-TOSTREAM-0000000006
Sink: KSTREAM-SINK-0000000008 (topic: AVG)
<-- KSTREAM-MAPVALUES-0000000007

7 changes: 2 additions & 5 deletions .../test/resources/expected_topology/0_6_0-pre/between_-_test_BETWEEN_with_array_dereference
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...l-tests/src/test/resources/expected_topology/0_6_0-pre/between_-_test_BETWEEN_with_bigint
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...src/test/resources/expected_topology/0_6_0-pre/between_-_test_BETWEEN_with_floating_point
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...tests/src/test/resources/expected_topology/0_6_0-pre/between_-_test_BETWEEN_with_integers
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...esources/expected_topology/0_6_0-pre/between_-_test_BETWEEN_with_integers_and_expressions
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...rces/expected_topology/0_6_0-pre/between_-_test_BETWEEN_with_integers_and_variable_values
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions .../src/test/resources/expected_topology/0_6_0-pre/between_-_test_BETWEEN_with_string_values
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...rces/expected_topology/0_6_0-pre/between_-_test_BETWEEN_with_string_values_with_substring
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

7 changes: 2 additions & 5 deletions ...s/src/test/resources/expected_topology/0_6_0-pre/between_-_test_NOT_BETWEEN_with_integers
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,8 @@ Topologies:
--> Project
<-- KSTREAM-TRANSFORMVALUES-0000000001
Processor: Project (stores: [])
--> KSTREAM-MAPVALUES-0000000004
--> KSTREAM-SINK-0000000004
<-- WhereFilter
Processor: KSTREAM-MAPVALUES-0000000004 (stores: [])
--> KSTREAM-SINK-0000000005
Sink: KSTREAM-SINK-0000000004 (topic: OUTPUT)
<-- Project
Sink: KSTREAM-SINK-0000000005 (topic: OUTPUT)
<-- KSTREAM-MAPVALUES-0000000004

Loading

0 comments on commit a6982bd

Please sign in to comment.