From d64b4e7d245438bd6db0617f7eee619bbf7d7eee Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 8 Jun 2021 13:44:06 -0500 Subject: [PATCH 1/4] Add `message.topic` to gossipsub `message-id` in Altair --- specs/altair/p2p-interface.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/specs/altair/p2p-interface.md b/specs/altair/p2p-interface.md index d6f8a0dfee..49082452fc 100644 --- a/specs/altair/p2p-interface.md +++ b/specs/altair/p2p-interface.md @@ -74,6 +74,15 @@ New topics are added in Altair to support the sync committees and the beacon blo The specification around the creation, validation, and dissemination of messages has not changed from the Phase 0 document. +The derivation of the `message-id` has changed starting with Altair to incorporate the message topic along with the message data. +The `message-id` MUST be the following 20 byte value computed from the message: +* If `message.data` has a valid snappy decompression, set `message-id` to the first 20 bytes of the `SHA256` hash of + the concatenation of `MESSAGE_DOMAIN_VALID_SNAPPY` with the snappy decompressed message data and the topic name, + i.e. `SHA256(MESSAGE_DOMAIN_VALID_SNAPPY + snappy_decompress(message.data) + message.topic)[:20]`. +* Otherwise, set `message-id` to the first 20 bytes of the `SHA256` hash of + the concatenation of `MESSAGE_DOMAIN_INVALID_SNAPPY` with the raw message data and the topic name, + i.e. `SHA256(MESSAGE_DOMAIN_INVALID_SNAPPY + message.data + message.topic)[:20]`. + The new topics along with the type of the `data` field of a gossipsub message are given in this table: | Name | Message Type | From 25a2e3463ecc0fceb2677deda5f9456c445bd559 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 8 Jun 2021 12:06:02 -0700 Subject: [PATCH 2/4] Update specs/altair/p2p-interface.md Co-authored-by: Diederik Loerakker --- specs/altair/p2p-interface.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/altair/p2p-interface.md b/specs/altair/p2p-interface.md index 49082452fc..4021cbbd04 100644 --- a/specs/altair/p2p-interface.md +++ b/specs/altair/p2p-interface.md @@ -74,7 +74,7 @@ New topics are added in Altair to support the sync committees and the beacon blo The specification around the creation, validation, and dissemination of messages has not changed from the Phase 0 document. -The derivation of the `message-id` has changed starting with Altair to incorporate the message topic along with the message data. +The derivation of the `message-id` has changed starting with Altair to incorporate the message `topic` along with the message `data`. These are fields of the `Message` Protobuf, and interpreted as empty byte strings if missing. The `message-id` MUST be the following 20 byte value computed from the message: * If `message.data` has a valid snappy decompression, set `message-id` to the first 20 bytes of the `SHA256` hash of the concatenation of `MESSAGE_DOMAIN_VALID_SNAPPY` with the snappy decompressed message data and the topic name, From f60f13964c3c71196f5ea138ffb2edc6fa7c2db9 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 8 Jun 2021 14:15:57 -0500 Subject: [PATCH 3/4] Harden topic serialization --- specs/altair/p2p-interface.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/specs/altair/p2p-interface.md b/specs/altair/p2p-interface.md index 4021cbbd04..ccc31a2804 100644 --- a/specs/altair/p2p-interface.md +++ b/specs/altair/p2p-interface.md @@ -77,11 +77,13 @@ The specification around the creation, validation, and dissemination of messages The derivation of the `message-id` has changed starting with Altair to incorporate the message `topic` along with the message `data`. These are fields of the `Message` Protobuf, and interpreted as empty byte strings if missing. The `message-id` MUST be the following 20 byte value computed from the message: * If `message.data` has a valid snappy decompression, set `message-id` to the first 20 bytes of the `SHA256` hash of - the concatenation of `MESSAGE_DOMAIN_VALID_SNAPPY` with the snappy decompressed message data and the topic name, - i.e. `SHA256(MESSAGE_DOMAIN_VALID_SNAPPY + snappy_decompress(message.data) + message.topic)[:20]`. + the concatenation of the following data: `MESSAGE_DOMAIN_VALID_SNAPPY`, the length of the topic byte string (encoded as little-endian `uint64`), + the topic byte string, and the snappy decompressed message data: + i.e. `SHA256(MESSAGE_DOMAIN_VALID_SNAPPY + uint_to_bytes(uint64(len(message.topic))) + message.topic + snappy_decompress(message.data)[:20]`. * Otherwise, set `message-id` to the first 20 bytes of the `SHA256` hash of - the concatenation of `MESSAGE_DOMAIN_INVALID_SNAPPY` with the raw message data and the topic name, - i.e. `SHA256(MESSAGE_DOMAIN_INVALID_SNAPPY + message.data + message.topic)[:20]`. + the concatenation of the following data: `MESSAGE_DOMAIN_INVALID_SNAPPY`, the length of the topic byte string (encoded as little-endian `uint64`), + the topic byte string, and the raw message data: + i.e. `SHA256(MESSAGE_DOMAIN_INVALID_SNAPPY + uint_to_bytes(uint64(len(message.topic))) + message.topic + message.data)[:20]`. The new topics along with the type of the `data` field of a gossipsub message are given in this table: From a343680fd41f74556e5381aca63412d7eaa0dad9 Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 8 Jun 2021 14:29:08 -0500 Subject: [PATCH 4/4] Add implementation note on message id fn --- specs/altair/p2p-interface.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/specs/altair/p2p-interface.md b/specs/altair/p2p-interface.md index ccc31a2804..8de85188d1 100644 --- a/specs/altair/p2p-interface.md +++ b/specs/altair/p2p-interface.md @@ -85,6 +85,12 @@ The `message-id` MUST be the following 20 byte value computed from the message: the topic byte string, and the raw message data: i.e. `SHA256(MESSAGE_DOMAIN_INVALID_SNAPPY + uint_to_bytes(uint64(len(message.topic))) + message.topic + message.data)[:20]`. +Implementations may need to carefully handle the function that computes the `message-id`. In particular, messages on topics with the Phase 0 +fork digest should use the `message-id` procedure specified in the Phase 0 document. +Messages on topics with the Altair fork digest should use the `message-id` procedure defined here. +If an implementation only supports a single `message-id` function, it can define a switch inline; +for example, `if topic in phase0_topics: return phase0_msg_id_fn(message) else return altair_msg_id_fn(message)`. + The new topics along with the type of the `data` field of a gossipsub message are given in this table: | Name | Message Type |