From 172846d13353ca8db0d23de234165006e28f620f Mon Sep 17 00:00:00 2001 From: achingbrain Date: Thu, 14 Nov 2019 17:13:13 +0000 Subject: [PATCH] docs: adds spec sections for mode and mtime metadata I didn't include the pros and cons of the other options because I don't think they should be included in the spec. That is, if I'm implementing this, I read the spec to understand what I should implement and how it should behave, I don't want to read a bunch of exposition on how we got to where we are. The decision log is important though, should it go in `ipfs/notes`? Or maybe we could have accompanying `*-notes.md` files? Follows on from/supersedes #223 --- UNIXFS.md | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/UNIXFS.md b/UNIXFS.md index 4a071d8a4..28ff8f151 100644 --- a/UNIXFS.md +++ b/UNIXFS.md @@ -43,9 +43,10 @@ message Data { optional bytes Data = 2; optional uint64 filesize = 3; repeated uint64 blocksizes = 4; - optional uint64 hashType = 5; optional uint64 fanout = 6; + optional uint32 mode = 7; + optional int64 mtime = 8; } message Metadata { @@ -59,6 +60,32 @@ For files that are comprised of more than a single block, the 'Type' field will This data is serialized and placed inside the 'Data' field of the outer merkledag protobuf, which also contains the actual links to the child nodes of this object. +For files comprised of a single block, the 'Type' field will be set to 'File', 'filesize' will be set to the total number of bytes in the file and the file data will be stored in the 'Data' field. + +The serialized size of a UnixFS node must not exceed 256KiB in order to work will with the [Bitswap] protocol. + +## Metadata + +UnixFS currently supports three optional metadata fields: + +* `mode` -- The `mode` is for persisting the [file permissions in numeric notation](https://en.wikipedia.org/wiki/File_system_permissions#Numeric_notation) \[[spec](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_stat.h.html)\]. If unspecified this defaults to `0755` for directories/HAMT shards and `0644` for all other types where applicable. +* `mtime` -- The modification time in seconds since the epoch. This defaults to the unix epoch if unspecified. +* `MimeType` -- The mime type of the file. This field is deprecated, as is the `Metadata` message and neither will be present in `UnixFSv2` because mime types are not typically properties of a file system. + +### Inheritance + +When traversing down through a UnixFSv1.5 directory, child entries without metadata fields will inherit those of their direct ascendants. + +### Deduplication and inlining + +Where the file data is small it would normally be stored in the `Data` field of the UnixFS `File` node. + +To aid in deduplication of data even for small files, file data can be stored in a separate node linked to from the `File` node in order for the data to have a constant [CID] regardless of the metadata associated with it. + +As a further optimization, if the `File` node's serialized size is small, it may be inlined into it's v1 [CID] by using the [`identity`](https://github.com/multiformats/multicodec/blob/master/table.csv) [multihash]. + +Such [CID]s must consist of 23 bytes or fewer in order for them to fit inside the 63 character limit for a DNS label when encoded in base32 (see [RFC1035 Section 2.3.1](https://tools.ietf.org/html/rfc1035#section-2.3.1)). + ## Importing Importing a file into unixfs is split up into two parts. The first is chunking, the second is layout. @@ -86,3 +113,7 @@ If there is only a single chunk, no intermediate unixfs file nodes are created, ## Exporting To read the file data out of the unixfs graph, perform an in order traversal, emitting the data contained in each of the leaves. + +[multihash]: https://tools.ietf.org/html/draft-multiformats-multihash-00 +[CID]: https://docs.ipfs.io/guides/concepts/cid/ +[Bitswap]: https://github.com/ipfs/specs/blob/master/BITSWAP.md \ No newline at end of file