Skip to content

Commit

Permalink
Merge partition sections (#2165)
Browse files Browse the repository at this point in the history
  • Loading branch information
JimGalasyn authored Nov 16, 2018
1 parent 77f73a4 commit d28a7fb
Showing 1 changed file with 12 additions and 18 deletions.
30 changes: 12 additions & 18 deletions docs/developer-guide/partition-data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ Co-partitioning Requirements
****************************

* The input records for the join must have the :ref:`same keying scheme <keys-have-same-keying-scheme>`
and the :ref:`same number of partitions <keys-have-same-number-of-partitions>` on both sides.
* The key for the join must be in the :ref:`same partition <records-are-in-the-same-partition>` on both sides.
* The input records must have the :ref:`same number of partitions <keys-have-same-number-of-partitions>` on both sides.
* Both sides of the join must have the :ref:`same partitioning strategy <records-have-same-partitioning-strategy>`.

When your inputs are co-partitioned, records with the same key, from both
Expand All @@ -22,8 +21,8 @@ them <ensure-co-partitioning>` by using the PARTITION BY clause.

.. _keys-have-same-keying-scheme:

Same Keying Scheme
==================
Records Have the Same Keying Scheme
===================================

The input records for the join must have the same keying scheme, which means
that the join must use the same key field on both sides.
Expand All @@ -34,8 +33,8 @@ The join won't match if the key fields don't have the same name and type.

.. _keys-have-same-number-of-partitions:

Number of Partitions is Equal
=============================
Records Have the Same Number of Partitions
==========================================

The input records for the join must have the same number of partitions on both
sides.
Expand All @@ -47,17 +46,6 @@ Use the ``<path-to-confluent>/bin/kafka-topics`` CLI tool
with the ``--describe`` option to see the number of partitions for the
Kafka topics that correspond with your streams and tables.

.. _records-are-in-the-same-partition:

Records Are In The Same Partition
=================================

The input records for the join must be in the same partition on both sides.

For example, in a stream-table join, if a ``userId`` key with the value ``alice123``
is in Partition 1 for the stream, but ``alice123`` is in Partition 2 for the table,
the join won't match, even though both sides are keyed by ``userId``.

.. _records-have-same-partitioning-strategy:

Records Have the Same Partitioning Strategy
Expand All @@ -71,7 +59,13 @@ But if the producer applications for your records have custom partitioners
specified in `configuration <http://kafka.apache.org/documentation/#producerconfigs>`__,
the same custom partitioner logic must be used for records on both sides of the join.
The applications that write to the join inputs must have the same partitioning
strategy so that records with the same key are delivered to same partition number.
strategy, so that records with the same key are delivered to same partition number.

This means that the input records must be in the same partition on both sides
of the join. For example, in a stream-table join, if a ``userId`` key with the
value ``alice123`` is in Partition 1 for the stream, but ``alice123`` is in
Partition 2 for the table, the join won't match, even though both sides are
keyed by ``userId``.

KSQL can't verify whether the partitioning strategies are the same for
both join inputs, so you must ensure this.
Expand Down

0 comments on commit d28a7fb

Please sign in to comment.