[RFC] Maskbook Post Binary Payload Format v1

[neruthes]

Metadata

Field	Value
Manifest	Meta/Article-3
Purpose	Documentation
Feedback deadline	2019-12-13 12:00:00 UTC
Explicitly designated reviewers	...

Abstract

Maskbook Post Bitstream Payload Format is designed to provide a compact method to convey post payloads over size-sensitive transportation infrastructures, especially image steganography.

Structure

The structure of a post payload consists of 11 major parts:

Section Name	Length	Description
Magic Header	18 B	31 AB 24 6E 8A 24 FC 49 DC AF 2A 6D 79 DF 8F A2 CB 7F, which will become Maskbook/Encrypted+Post/ after Base64 encoding
Version	2 B	UInt16, starting from 0x0000
Blob Positions	10 B	Array<UInt16>[5]
PostIV	16 B	IV of AES, working as PostUUID
Public Share	1 B	Bool, 0x00 or 0x01
Cipher Message	Dynamic	Blob
Self-Recipient Key Hint	Dynamic	Blob (PostKey, ECDH-encrypted to the author)
List of Receiving Groups	Dynamic	Array<String>.join(0x7C7C)
Extension Data Public	Dynamic	Serialized Object, UTF-8
Extension Data Private	Dynamic	Serialized Object, UTF-8, AES-encrypted with PostKey and PostIV
Checksum	32 B	Blob (SHA2-256 hash)

Sections

Blob Positions

This part consists of 5 numbers (UInt16), which respectively indicate the length of the sections since Cipher Message.

This is designed to optimize for using Blob.prototype.slice to select the respective parts without reading through the whole blob.

PostIV

// TODO

Cipher Message

// TODO

Self-Recipient Key Hint

// TODO

List of Receiving Groups

// TODO

Extension Data Public

This is a serialized JSON object. Reserved for further definition.

Extension Data Private

This is an AES-encrypted blob of a serialized JSON object. Reserved for further definition.

Checksum

The SHA2-256 hash of all bytes afore.

[neruthes]

It is possible that @guanbinrui may want to employ this format for #316.

[Jack-Works]

do we realy need to make a binary format by ourself?

[Tedko]

+1. Better not do this now or start this early.

…

On Fri, Nov 15, 2019 at 22:14 Jack Works ***@***.***> wrote: do we realy need to make a binary format by ourself? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#329?email_source=notifications&email_token=ABTAVTMQDOLMRB5SQQ3ELGTQT2VCVA5CNFSM4JFW6USKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEFRIVI#issuecomment-554374229>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTAVTKEJIC62IFASXIUEJLQT2VCVANCNFSM4JFW6USA> .

[neruthes]

Trivialization can be harmful. Now on Twitter we have the #198 format, and on Facebook we have the original payload. Headers, encoding methods, and separators all differ. If we adopt a binary version, we will only need to implement network-specific interfaces, instead of network-specific payloads.

[neruthes]

This RFC will open for a longer period of time as we work closely on other components.

The discussion here is not blocking #373. The discussion here is not making amendments to #198.

[neruthes]

This RFC will open for a longer period of time as we work closely on other components.

The discussion here is not blocking #373. The discussion here is not making amendments to #198.

If #373 and #198 are subject to amendments, they will be updated independently.

[neruthes]

Update:

This RFC should draw to conclusion on 2019-12-13.

[Tedko]

Love to hear opinions from @Jack-Works . But pls don't prioritize this for now.

Remember #410 is much more important for our growth. Prioritize that RFC pls

[Jack-Works]

There are many binary JSON format like http://bsonspec.org/ and it will be easier to encode/decode by format like bson

[neruthes]

Love to hear opinions from @Jack-Works . But pls don't prioritize this for now.

Remember #410 is much more important for our growth. Prioritize that RFC pls

Without a new payload format, Red Packet feature will become a giant historic burden.

[neruthes]

There are many binary JSON format like http://bsonspec.org/ and it will be easier to encode/decode by format like bson

I proposed using JSON-based format in the past and the feedbacks were pure negation.

We may change to Base64-encoded BSON under #225.

[yisiliu]

bson is good enough

[neruthes]

I will start updating this to a BSON-based format today.

[Jack-Works]

I will start updating this to a BSON-based format today.

We can send our JSON format to bson library and it becomes binary! 🧙‍♂️

[neruthes]

This RFC is dismissed because we will introduce a new format upon #502, after #502 is ratified.