Skip to content

Commit

Permalink
docs: adds spec sections for mode and mtime metadata
Browse files Browse the repository at this point in the history
I didn't include the pros and cons of the other options because I
don't think they should be included in the spec.  That is, if I'm
implementing this, I read the spec to understand what I should
implement and how it should behave, I don't want to read a bunch
of exposition on how we got to where we are.

The decision log is important though, should it go in
`ipfs/notes`?  Or maybe we could have accompanying `*-notes.md`
files?

Follows on from/supersedes #223
  • Loading branch information
achingbrain committed Nov 14, 2019
1 parent 51577eb commit 172846d
Showing 1 changed file with 32 additions and 1 deletion.
33 changes: 32 additions & 1 deletion UNIXFS.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,10 @@ message Data {
optional bytes Data = 2;
optional uint64 filesize = 3;
repeated uint64 blocksizes = 4;
optional uint64 hashType = 5;
optional uint64 fanout = 6;
optional uint32 mode = 7;
optional int64 mtime = 8;
}
message Metadata {
Expand All @@ -59,6 +60,32 @@ For files that are comprised of more than a single block, the 'Type' field will

This data is serialized and placed inside the 'Data' field of the outer merkledag protobuf, which also contains the actual links to the child nodes of this object.

For files comprised of a single block, the 'Type' field will be set to 'File', 'filesize' will be set to the total number of bytes in the file and the file data will be stored in the 'Data' field.

The serialized size of a UnixFS node must not exceed 256KiB in order to work will with the [Bitswap] protocol.

## Metadata

UnixFS currently supports three optional metadata fields:

* `mode` -- The `mode` is for persisting the [file permissions in numeric notation](https://en.wikipedia.org/wiki/File_system_permissions#Numeric_notation) \[[spec](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_stat.h.html)\]. If unspecified this defaults to `0755` for directories/HAMT shards and `0644` for all other types where applicable.
* `mtime` -- The modification time in seconds since the epoch. This defaults to the unix epoch if unspecified.
* `MimeType` -- The mime type of the file. This field is deprecated, as is the `Metadata` message and neither will be present in `UnixFSv2` because mime types are not typically properties of a file system.

### Inheritance

When traversing down through a UnixFSv1.5 directory, child entries without metadata fields will inherit those of their direct ascendants.

### Deduplication and inlining

Where the file data is small it would normally be stored in the `Data` field of the UnixFS `File` node.

To aid in deduplication of data even for small files, file data can be stored in a separate node linked to from the `File` node in order for the data to have a constant [CID] regardless of the metadata associated with it.

As a further optimization, if the `File` node's serialized size is small, it may be inlined into it's v1 [CID] by using the [`identity`](https://github.com/multiformats/multicodec/blob/master/table.csv) [multihash].

Such [CID]s must consist of 23 bytes or fewer in order for them to fit inside the 63 character limit for a DNS label when encoded in base32 (see [RFC1035 Section 2.3.1](https://tools.ietf.org/html/rfc1035#section-2.3.1)).

## Importing

Importing a file into unixfs is split up into two parts. The first is chunking, the second is layout.
Expand Down Expand Up @@ -86,3 +113,7 @@ If there is only a single chunk, no intermediate unixfs file nodes are created,
## Exporting

To read the file data out of the unixfs graph, perform an in order traversal, emitting the data contained in each of the leaves.

[multihash]: https://tools.ietf.org/html/draft-multiformats-multihash-00
[CID]: https://docs.ipfs.io/guides/concepts/cid/
[Bitswap]: https://github.com/ipfs/specs/blob/master/BITSWAP.md

0 comments on commit 172846d

Please sign in to comment.