unixfs v1.5 metadata support

UNIXFS.md

@@ -46,10 +46,15 @@ message Data {
 
 	optional uint64 hashType = 5;
 	optional uint64 fanout = 6;
+
+	repeated Metadata metadata = 7;
+	optional Metadata defaultMetadata = 8;
 }
 
 message Metadata {
-	optional string MimeType = 1;
+	optional string mimeType = 1;
+	optional uint32 mode = 2;
+	optional int64 mtime = 3;
 }
 ```
 
@@ -59,6 +64,61 @@ For files that are comprised of more than a single block, the 'Type' field will
 
 This data is serialized and placed inside the 'Data' field of the outer merkledag protobuf, which also contains the actual links to the child nodes of this object.
 
+## Metadata
+
+UnixFS currently supports three metadata fields:
+
+* `mimeType` -- The mime-type of the file. This generally shouldn't be used.
+* `mode` -- The `mode` is for optionally persisting the [file permissions in numeric notation](https://en.wikipedia.org/wiki/File_system_permissions#Numeric_notation) \[[spec](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_stat.h.html)\]. It defaults to 0755 if unspecified.
+* `mtime` -- The modification time in seconds since the epoch. This defaults to the unix epoch if unspecified.
+
+There are three ways to specify file metadata:
+
+### Embedded in the directory
+
+Each entry in the repeated metadata field corresponds to the linked file/directory at the same offset in the Links section of the outer dag-pb. This field is appropriate in nodes with type `Directory` and `HAMTShard`. However, metadata should _not_ be specified for links that point to other HAMT shards.
+
+For some context, this solution was chosen over embedding the metadata in the file itself because:
+
+1. It allows accessing the metadata without downloading the target files.
+2. More importantly, it avoids changing the root hash of the target files to allow for better deduplication.
+
+### Using an intermediate node
+
+DEPRECATED
+
+Metadata can be applied to a single file/directory with an intermediate "metadata node":
+
+1. A `Type` of `Metadata`.
+2. A `Data` containing an encoded `Metadata` message.
+3. A single `Link` (in the outer dag-pb node) pointing to the actual file/directory.
+
+This solution has been _deprecated_ because it requires an additional intermediate node just for the metadata. It is also poorly supported.
+
+### Default metadata
+
+The `defaultMetadata` field can be used in conjunction with the repeated `metadata` field to specify metadata "defaults".
+
+1. The first `defaultMetadata` field encountered when traversing a sharded directory takes precedence.
+2. Default metadata and explicit per-file metadata are merged field-wise.
+
+Given the `defaultMetdata` field, the _actual_ default metadata is determined as follows:
+
+* For directories:
+  * The default mode is `defaultMetadata.mode | 0111` (sets the execute bit).
+  * The default mime type is ignored.
+  * The default mtime is as specified in `defaultMetadata`.
+* For symlinks, the defaults are ignored.
+* For regular files, the defaults are as specified in `defaultMetadata`.
+
+To determine the metadata for a file:
+
+1. The per-file metadata is looked up in the `metadata` list as usual.
+2. For each field in the file's metadata:
+  1. If the field is specified, use it.
+  2. If the field is unspecified but a default is specified, use it.
+  3. Otherwise, use the global default.
+
 ## Importing
 
 Importing a file into unixfs is split up into two parts. The first is chunking, the second is layout.