Skip to content
This repository has been archived by the owner on Mar 10, 2020. It is now read-only.

docs: add mode and mtime and .touch/.chmod mfs commands #572

Merged
merged 12 commits into from
Jan 9, 2020
129 changes: 123 additions & 6 deletions SPEC/FILES.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ The regular, top-level API for add, cat, get and ls Files on IPFS
The Files API, aka MFS (Mutable File System)

_Explore the Mutable File System through interactive coding challenges in our [ProtoSchool tutorial](https://proto.school/#/mutable-file-system/)._
- [files.chmod](#fileschmod)
- [files.cp](#filescp)
- [files.flush](#filesflush)
- [files.ls](#filesls)
Expand All @@ -36,6 +37,7 @@ _Explore the Mutable File System through interactive coding challenges in our [P
- [files.readReadableStream](#filesreadreadablestream)
- [files.rm](#filesrm)
- [files.stat](#filesstat)
- [files.touch](#filestouch)
- [files.write](#fileswrite)

### ⚠️ Note
Expand All @@ -58,6 +60,8 @@ Where `data` may be:
{
path: '/tmp/myfile.txt', // The file path
content: <data> // A Buffer, Readable Stream, Pull Stream or File with the contents of the file
mode: '0755' // optional string or integer mode to store the entry with. strings will be interpreted as a base 8 number
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I think about a unix mode as a string I think “rwx” “r--“ etc. Why not use this format as our stringy mode?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah? I have to admit I don't when setting modes, only when viewing them with ls -l or whatever.

In my head at least, the choice was between a literal base 10 number (e.g. 493) or a literal base 8 number (e.g. 0755). Linters don't seem to like literal numbers in anything other than base 10 so that's why it's a string '0755' above.

It's good to be flexible though, maybe something we could add later?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it helps to get this stuff merged faster then sure 👍 We’ll be supporting rwx format in chmod so I reckon people will expect to be able to use it here too.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We support Symbolic Notation in chmod (e.g. a=rwx, ug-x, etc) which are modifications to an existing mode.

I guess if you want to use it here you could apply the modification to the default mode for files/directories?

mtime: <time> // A Date object, an object with `{ sec, nsecs }` properties or the output of `process.hrtime()` or `process.hrtime.bigint()`
achingbrain marked this conversation as resolved.
Show resolved Hide resolved
}
```
If no `content` is passed, then the path is treated as an empty directory
Expand Down Expand Up @@ -98,6 +102,8 @@ an array of objects is returned, each of the form:
{
path: '/tmp/myfile.txt',
hash: 'QmHash', // base58 encoded multihash
mode: Number,
mtime: { secs: Number, nsecs: Number },
size: 123
}
```
Expand Down Expand Up @@ -130,16 +136,20 @@ const results = await ipfs.add(files)

The `results` array:

```json
```javascript
[
{
"path": "tmp",
"hash": "QmWXdjNC362aPDtwHPUE9o2VMqPeNeCQuTBTv1NsKtwypg",
"mode": 493,
"mtime": { secs: Number, nsecs: Number },
"size": 67
},
{
"path": "/tmp/myfile.txt",
"hash": "QmNz1UBzpdd4HfZ3qir3aPiRdX5a93XwTuDNyXRc6PKhWW",
"mode": 420,
"mtime": { secs: Number, nsecs: Number },
"size": 11
}
]
Expand Down Expand Up @@ -179,6 +189,8 @@ stream.on('data', function (file) {
// {
// path: '/tmp/myfile.txt',
// hash: 'QmHash' // base58 encoded multihash
// mode: Number,
// mtime: { secs: Number, nsecs: Number },
// size: 123
// }
})
Expand Down Expand Up @@ -207,6 +219,8 @@ Returns a Pull Stream, where objects can be written of the forms
{
path: '/tmp/myfile.txt', // The file path
content: <data> // A Buffer, Readable Stream, Pull Stream or File with the contents of the file
mode: '0755' // optional string or integer mode to store the entry with. strings will be interpreted as a base 8 number
`mtime` Either a ` Date` object, an object with `{ sec, nsecs }` properties or the output of `process.hrtime()` (default: now)
achingbrain marked this conversation as resolved.
Show resolved Hide resolved
}
```

Expand All @@ -233,6 +247,8 @@ pull(
// {
// path: '/tmp/myfile.txt',
// hash: 'QmHash' // base58 encoded multihash
// mode: Number
// mtime: { secs: Number, nsecs: Number }
// size: 123
// }
})
Expand Down Expand Up @@ -264,6 +280,8 @@ an array of objects is returned, each of the form:
{
path: 'test-folder',
hash: 'QmRNjDeKStKGTQXnJ2NFqeQ9oW23WcpbmvCVrpDHgDg3T6',
mode: Number
mtime: Date
size: 123
}
```
Expand Down Expand Up @@ -318,6 +336,8 @@ an array of objects is returned, each of the form:
{
path: '/tmp/myfile.txt',
hash: 'QmHash', // base58 encoded multihash
mode: Number,
mtime: { secs: Number, nsecs: Number },
size: 123
}
```
Expand Down Expand Up @@ -511,7 +531,9 @@ the yielded objects are of the form:
```js
{
path: '/tmp/myfile.txt',
content: <Readable stream>
content: <Readable stream>,
mode: Number,
mtime: { secs: Number, nsecs: Number }
}
```

Expand Down Expand Up @@ -563,7 +585,9 @@ the yielded objects are of the form:
```js
{
path: '/tmp/myfile.txt',
content: <Pull Stream>
content: <Pull Stream>,
mode: Number,
mtime: { secs: Number, nsecs: Number }
}
```

Expand Down Expand Up @@ -624,7 +648,9 @@ an array of objects is returned, each of the form:
path: 'QmVvjDy7yF7hdnqE8Hrf4MHo5ABDtb5AbX6hWbD3Y42bXP/alice.txt',
size: 11696,
hash: 'QmZyUEQVuRK3XV7L9Dk26pg6RVSgaYkiSTEdnT2kZZdwoi',
type: 'file'
type: 'file',
mode: Number,
mtime: { secs: Number, nsecs: Number }
}
```

Expand Down Expand Up @@ -674,7 +700,9 @@ the yielded objects are of the form:
path: 'QmVvjDy7yF7hdnqE8Hrf4MHo5ABDtb5AbX6hWbD3Y42bXP/alice.txt',
size: 11696,
hash: 'QmZyUEQVuRK3XV7L9Dk26pg6RVSgaYkiSTEdnT2kZZdwoi',
type: 'file'
type: 'file',
mode: Number,
mtime: { secs: Number, nsecs: Number }
}
```

Expand Down Expand Up @@ -727,7 +755,9 @@ the yielded objects are of the form:
path: 'QmVvjDy7yF7hdnqE8Hrf4MHo5ABDtb5AbX6hWbD3Y42bXP/alice.txt',
size: 11696,
hash: 'QmZyUEQVuRK3XV7L9Dk26pg6RVSgaYkiSTEdnT2kZZdwoi',
type: 'file'
type: 'file',
mode: Number,
mtime: { secs: Number, nsecs: Number }
}
```

Expand Down Expand Up @@ -759,6 +789,47 @@ A great source of [examples][] can be found in the tests for this API.

The Mutable File System (MFS) is a virtual file system on top of IPFS that exposes a Unix like API over a virtual directory. It enables users to write and read from paths without having to worry about updating the graph. It enables things like [ipfs-blob-store](https://github.com/ipfs/ipfs-blob-store) to exist.

#### `files.chmod`

> Change mode for files and directories

##### `ipfs.files.chmod(path, mode, [options])`

Where:

- `path` is the path to the entry to modify. It might be:
- An existing MFS path to a file or a directory (e.g. `/my-dir/my-file.txt`)
- An IPFS path (e.g. `/ipfs/QmWGeRAEgtsHW3ec7U4qW2CyVy7eA2mFRVbk1nb24jFyks`)
- A [CID][cid] instance (e.g. `new CID('QmWGeRAEgtsHW3ec7U4qW2CyVy7eA2mFRVbk1nb24jFyks')`)
- `mode` is the new file mode. It might be:
- A string octal, e.g. `'0755'`
- A string modification of the existing mode, e.g. `'+x'`, `'-gw'`, etc
achingbrain marked this conversation as resolved.
Show resolved Hide resolved
- An integer, e.g. the returned value from `parseInt('0755', 8)`
achingbrain marked this conversation as resolved.
Show resolved Hide resolved
- `options` is an optional Object that might contain the following keys:
- `recursive` is a Boolean value that indicates if `mode` should be applied to all sub files/directories of `path` (default: false)
- `hashAlg` is which algorithm to use when creating CIDs for modified entries. (default: `sha2-256`) [The list of all possible values]( https://github.com/multiformats/js-multihash/blob/master/src/constants.js#L5-L343)
- `flush` is a Boolean value to decide whether or not to immediately flush MFS changes to disk (default: true)
- `cidVersion`: the CID version to use for any updated entries (integer, default 0)

**Returns**

| Type | Description |
| -------- | -------- |
| `Promise<void>` | If action is successfully completed. Otherwise an error will be thrown |

**Example:**

```JavaScript
// To give a file -rwxrwxrwx permissions
await ipfs.files.chmod('/path/to/file.txt', parseInt('0777', 8))

// Alternatively
await ipfs.files.chmod('/path/to/file.txt', '+rwx')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?+rwx?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow - what does the leading ? mean in ?+rwx? I can't see any reference to it in man chmod and when I try to do this locally I get an error:

$ chmod ?+rwx foo.txt
chmod: Invalid file mode: ?+rwx

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, nevermind. I didn't realize +rwx was equivalent to a+rwx.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, it will support u+rwx, g+r, o+w, a-w etc. though right? We should specify what subset of https://en.wikipedia.org/wiki/Modes_(Unix)#Format we do support.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it'll support all of those.

Though now you mention it there's a testing gap for a...

..and maybe not X right now, though it can be added.


// You can omit the leading `0` too
await ipfs.files.chmod('/path/to/file.txt', '777')
```

#### `files.cp`

> Copy files.
Expand Down Expand Up @@ -821,6 +892,8 @@ Where:
- `format` is what type of nodes to write any newly created directories as (default: `dag-pb`)
- `hashAlg` is which algorithm to use when creating CIDs for newly created directories (default: `sha2-256`) [The list of all possible values]( https://github.com/multiformats/js-multihash/blob/master/src/constants.js#L5-L343)
- `flush` is a Boolean value to decide whether or not to immediately flush MFS changes to disk (default: true)
- `mode`: optional UnixFS mode to create the directory with - a number or a string that will be interpreted as a base 8 number
- `mtime`: A Date object, an object with `{ sec, nsecs }` properties or the output of `process.hrtime()` or `process.hrtime.bigint()

**Returns**

Expand Down Expand Up @@ -884,6 +957,42 @@ console.log(stats)
// }
```

#### `files.touch`

> Update the mtime of a file or directory

##### `ipfs.files.touch(path, [options])`

Where:

- `path` is the path to the file or directory to update. It might be:
- An existing MFS path to a file or directory (e.g. `/my-dir/a.txt`)
- An IPFS path (e.g. `/ipfs/QmWGeRAEgtsHW3ec7U4qW2CyVy7eA2mFRVbk1nb24jFyks`)
- A [CID][cid] instance (e.g. `new CID('QmWGeRAEgtsHW3ec7U4qW2CyVy7eA2mFRVbk1nb24jFyks')`)
- `options` is an optional Object that might contain the following keys:
- `mtime` Either a ` Date` object, an object with `{ sec, nsecs }` properties or the output of `process.hrtime()` (default: now)
- `hashAlg` is which algorithm to use when creating CIDs for modified entries. (default: `sha2-256`) [The list of all possible values]( https://github.com/multiformats/js-multihash/blob/master/src/constants.js#L5-L343)
- `flush` is a Boolean value to decide whether or not to immediately flush MFS changes to disk (default: true)
- `cidVersion`: the CID version to use for any updated entries (integer, default 0)

**Returns**

| Type | Description |
| -------- | -------- |
| `Promise<void>` | If action is successfully completed. Otherwise an error will be thrown |

**Example:**

```JavaScript
// set the mtime to the current time
await ipfs.files.touch('/path/to/file.txt')

// set the mtime to a specific time
await ipfs.files.touch('/path/to/file.txt', {
mtime: new Date('May 23, 2014 14:45:14 -0700')
})
```

#### `files.rm`

> Remove a file or directory.
Expand Down Expand Up @@ -1036,6 +1145,8 @@ Where:
- `length` is an Integer with the maximum number of bytes to read (default: Read all bytes from `content`)
- `rawLeaves`: if true, DAG leaves will contain raw file data and not be wrapped in a protobuf (boolean, default false)
- `cidVersion`: the CID version to use when storing the data (storage keys are based on the CID, including its version) (integer, default 0)
- `mode`: optional UnixFS mode to create or update the file with - a number or a string that will be interpreted as a base 8 number
- `mtime`: A Date object, an object with `{ sec, nsecs }` properties or the output of `process.hrtime()` or `process.hrtime.bigint()`

**Returns**

Expand Down Expand Up @@ -1144,6 +1255,8 @@ each object contains the following keys:
- `type` which is the object's type (`directory` or `file`)
- `size` the size of the file in bytes
- `hash` the hash of the file
- `mode` the UnixFS mode as a Number
- `mtime` an objects with numeric `secs` and `nsecs` properties

**Example:**

Expand Down Expand Up @@ -1188,6 +1301,8 @@ the yielded objects contain the following keys:
- `type` which is the object's type (`directory` or `file`)
- `size` the size of the file in bytes
- `hash` the hash of the file
- `mode` the UnixFS mode as a Number
- `mtime` an object with numeric `secs` and `nsecs` properties

**Example:**

Expand Down Expand Up @@ -1230,6 +1345,8 @@ the yielded objects contain the following keys:
- `type` which is the object's type (`directory` or `file`)
- `size` the size of the file in bytes
- `hash` the hash of the file
- `mode` the UnixFS mode as a Number
- `mtime` an object with numeric `secs` and `nsecs` properties

**Example:**

Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@
"get-stream": "^5.1.0",
"hat": "0.0.3",
"ipfs-block": "~0.8.0",
"ipfs-unixfs": "~0.1.16",
"ipfs-utils": "~0.4.0",
"ipfs-unixfs": "~0.2.0",
"ipfs-utils": "^0.4.1",
"ipld-dag-cbor": "~0.15.0",
"ipld-dag-pb": "^0.18.1",
"is-ipfs": "~0.6.1",
Expand Down
62 changes: 62 additions & 0 deletions src/files-mfs/chmod.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
/* eslint-env mocha */
'use strict'

const hat = require('hat')
const { getDescribe, getIt, expect } = require('../utils/mocha')

module.exports = (common, options) => {
const describe = getDescribe(options)
const it = getIt(options)

describe('.files.chmod', function () {
this.timeout(40 * 1000)

let ipfs

async function testMode (mode, expectedMode) {
const testPath = `/test-${hat()}`

await ipfs.files.write(testPath, Buffer.from('Hello, world!'), {
create: true
})
await ipfs.files.chmod(testPath, mode)

const stat = await ipfs.files.stat(testPath)
expect(stat).to.have.property('mode').that.equals(expectedMode)
}

before(async () => {
ipfs = (await common.spawn()).api
})

after(() => common.clean())

it('should change file mode', async function () {
const mode = parseInt('544', 8)
await testMode(mode, mode)
})

it('should change file mode as string', async function () {
const mode = parseInt('544', 8)
await testMode('544', mode)
})

it('should change file mode to 0', async function () {
const mode = 0
await testMode(mode, mode)
})

it('should change directory mode', async function () {
const testPath = `/test-${hat()}`
const mode = parseInt('544', 8)

await ipfs.files.mkdir(testPath, {
create: true
})
await ipfs.files.chmod(testPath, mode)

const stat = await ipfs.files.stat(testPath)
expect(stat).to.have.property('mode').that.equals(mode)
})
})
}
Loading