Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MORE docs + tests #28

Merged
merged 7 commits into from
Jan 9, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 105 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,57 @@
# ssb-config

Configuration module used by [`scuttlebot`](https://github.com/ssbc/scuttlebot).
Configuration module useful for starting an [`ssb-server`](https://github.com/ssbc/ssb-server).

## example
## example usage

``` js
var Server = require('ssb-server')
var config = require('ssb-config')

//if you want to set up a test network, that
//doesn't collide with main ssb pass the name of that network in.
var server = Server(config)
server.whoami((err, feed) => {
console.log(feed)

var test_config = require('ssb-config/inject')('testnet', {port: 9999})
//you can also pass a second argument, which overrides the default defaults.
server.close(() => console.log('closing the server!'))
})
```

Custom configutation (e.g. set up a test network that doesn't collide with main ssb network):
```js
// if you want to customise, e.g.
//

var Server = require('ssb-server')
var Config = require('ssb-config/inject')
var config = Config('testnet', { port: 9999 })

var server = Server(config)
server.whoami((err, feed) => {
console.log(feed)

server.close(() => console.log('closing the server!'))
})
```

## API

### `require('ssb-config')`

returns you the stock standard config for starting an `ssb-server`

### `require('ssb-config/inject')(appName, opts) => Object`

A function which takes:
- `appName` *(string)* which declares where to look for further config, where to read and write databases. Stores data in `~/.${appName}`, defaults to `ssb` (so data in `~/.ssb`).
- `opts` *(object)* an object which is fed into the config generation as a bunch of defaults (see 'Configuration' below)

## Configuration

There are some configuration options for the sysadmins out there. All configuration is loaded via [`rc`](https://github.com/dominictarr/rc). You can pass any configuration value in as cli arg, env var, or in a file.
All configuration is loaded via [`rc`](https://github.com/dominictarr/rc). This means the final config is a result of config collected from `opts` passed into the inject method, cli args, env var, and config (e.g. `~/.ssb/config`). See the rc repo for full details.

* `host` *(string)* The domain or ip address for `sbot`. Defaults to your public ip address.
* `port` *(string|number)* The port for `sbot`. Defaults to `8008`.
Options:
* `connections` *(object)* Details `incoming` and `outgoing` connections behaviour (see below)
* `remote` ... TODO ... a multisever address for ... (in the future this may be deprecated / derived from `connections`
* `timeout`: *(number)* Number of milliseconds a replication stream can idle before it's automatically disconnected. Defaults to `30000`.
* `pub` *(boolean)* Replicate with pub servers. Defaults to `true`.
* `local` *(boolean)* Replicate with local servers found on the same network via `udp`. Defaults to `true`.
Expand All @@ -30,53 +62,98 @@ There are some configuration options for the sysadmins out there. All configurat
* `master` *(array)* Pubkeys of users who, if they connect to the Scuttlebot instance, are allowed to command the primary user with full rights. Useful for remotely operating a pub. Defaults to `[]`.
* `logging.level` *(string)* How verbose should the logging be. Possible values are error, warning, notice, and info. Defaults to `notice`.

### `connections`
Deprecated Options:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea! Probably a good idea to warn about that if they are overwritten.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean that the raw values are being removed from the final config? I've added a note about that now

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given what dinosaur said about env variables scratch my comment.

* `host` *(string)* The domain or ip address for `sbot`. Defaults to your public ip address.
* `port` *(string|number)* The port for `sbot`. Defaults to `8008`.
* `ws` TODO

Two objects to specify `incoming` and `outgoing` transports and transformations for connections.
You should use `connections` to more explicitly configure connections.
These values are currently only used to generate `connections.incoming` if that option isn't provided. The raw options are no longer returned in the final config - this is to ensure we don't have multiple places where different `host` / `port` / `ws` are being set!

The default is the following. It specifies the default TCP `net`work transport for incoming and outging connections, using secret-handshake/boxstream ([shs](https://github.com/auditdrivencrypto/secret-handshake)) for authentication and encryption.
### `connections`

An object with two required properties: `incoming` and `outgoing` to specify transports and transformations for connections.
Defaults to the following:
```json
"connections": {
{
"incoming": {
"net": [{ "port": 8008, "scope": "public", "transform": "shs" }]
},
"outgoing": {
"net": [{ "transform": "shs" }]
}
},
}
```

If you want to use [Tor](https://torproject.org) to create outgoing connections you can specify your `outgoing` like this. It will use `localhost:9050` as the socks server for creating this.
It specifies the default TCP `net`work transport for incoming and outging connections, using secret-handshake/boxstream ([shs](https://github.com/auditdrivencrypto/secret-handshake)) for authentication and encryption.

A **transport** is a vehicle or avenue for communication. The following transports are currently supported:
- `net` - TCP based
- `unix` - socket based
- `onion` - TOR based
- `ws` - websocket based

Each transport can have an array of different configurations passed to it, these are objects with properties:
- `transform` *(string)* determines whether traffic is encrypted, and if so how.
- `shs` - secret handashake
- `noauth` - no encryption
- `port` *(integer)*
- `host` *(string)* only relevant for ... TODO
- `scope` *(string)* ... TODO
- `private` - TODO
- `public` - TODO
- `local` - TODO
- `device` - TODO
- `external` *(array of strings)* ... TODO
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for leaving TODOs, this makes it much easier to figure out what needs to be documented in the future.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tunnel: is also a type of transport that may occur, see https://github.com/ssbc/ssb-tunnel . If it's still a valid plugin, I kind of want to see it used more.

I have also seen 'device' and 'local' used in scopes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for these additions @hamnox , I'll add them in as stubs!

- `server` .... TODO ??

---

### Example `connnections` configurations

If you only want to use [Tor](https://torproject.org) to create outgoing connections you can specify your `outgoing` like this. It will use `localhost:9050` as the socks server for creating this.

```json
"connections": {
{
"incoming": {
"net": [{ "port": 8008, "scope": "public", "transform": "shs" }]
},
"outgoing": {
"onion": [{ "transform": "shs" }]
}
},
}
```

If you want to run a peer behind NAT or other kind of proxy but still want sbot to be able to create invites for the outside addres, you can specify a `public` scope as your `incoming.net` by defining the `external` parameter like this:

```json
"incoming": {
"net": [
{ "scope": "public", "external": ["cryptop.home"], "transform": "shs", "port": 8008 },
{ "scope": "private", "transform": "shs", "port": 8008, "host": "internal1.con.taine.rs" },
]
},
{
"incoming": {
"net": [
{ "scope": "public", "external": ["cryptop.home"], "transform": "shs", "port": 8008 },
{ "scope": "private", "transform": "shs", "port": 8008, "host": "internal1.con.taine.rs" },
]
},
"outgoing": {
"net": [{ "transform": "shs" }]
}
}
```

One thing to notice is that you _need_ `incoming` connections for Apps (like patchwork or git-ssb) to function. By default they use the same authentication mechanism (shs) to grant access to the database, choosing access levels depending on the keypair that opens the connection. If you connect to yourself, you get full access (query and publish). If a remote peer connects, it can only replicate. So be sure to have **at least one** `incoming` connection.

That beeing said, the overhead of encryption for local applications can be very high, especially on low-powered devices. For this use-case there is a `noauth` transform which by-passes the authentication and grants full access to anybody that can connect to it. **hint:** *This is risky! it might expose private messages or enables people to publish as you!* Therefore be sure to bind the listener to `localhost` or use the `unix` socket. The `unix` file socket is creted as `$HOME/.ssb/socket` by default and has permissions such that only the user running `sbot server` can open it, just like the `.ssb/secret` file.
That being said, the overhead of encryption for local applications can be very high, especially on low-powered devices. For this use-case there is a `noauth` transform which by-passes the authentication and grants full access to anybody that can connect to it. **hint:** *This is risky! it might expose private messages or enables people to publish as you!* Therefore be sure to bind the listener to `localhost` or use the `unix` socket. The `unix` file socket is creted as `$HOME/.ssb/socket` by default and has permissions such that only the user running `sbot server` can open it, just like the `.ssb/secret` file.

```json
"incoming": {
"unix": [{ "scope":"device", "transform":"noauth" }],
"net": [{ "scope": "device", "transform": "noauth", "port": 8009, "host": "localhost" }]
},
{
"incoming": {
"unix": [{ "scope":"device", "transform":"noauth" }],
"net": [{ "scope": "device", "transform": "noauth", "port": 8009, "host": "localhost" }]
},
"outgoing": {
"net": [{ "transform": "shs" }]
}
}
```

The local plugin inside scuttlebot will use the first incoming connection of either public or private scope.
Expand Down
59 changes: 32 additions & 27 deletions inject.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,53 +7,54 @@ var merge = require('deep-extend')
var RC = require('rc')

var SEC = 1e3
var MIN = 60*SEC
var MIN = 60 * SEC

module.exports = function (name, override) {
name = name || 'ssb'
var HOME = home() || 'browser' //most probably browser
var result = RC(name || 'ssb', merge({
//just use an ipv4 address by default.
//there have been some reports of seemingly non-private
//ipv6 addresses being returned and not working.
//https://github.com/ssbc/scuttlebot/pull/102
var HOME = home() || 'browser' // most probably browser

var result = RC(name, merge({
// just use an ipv4 address by default.
// there have been some reports of seemingly non-private
// ipv6 addresses being returned and not working.
// https://github.com/ssbc/scuttlebot/pull/102
path: path.join(HOME, '.' + name),
party: true,
host: nonPrivate.v4 || '',
port: 8008,
timeout: 0,
pub: true,
local: true,
friends: {
dunbar: 150,
hops: 3
},
ws: {
port: 8989
},
gossip: {
connections: 3
},
// *** LEGACY *** (used to generate default connections.incoming)
host: nonPrivate.v4 || '',
port: 8008,
ws: { port: 8989 },
// **************
connections: {
outgoing: {
net: [{ transform: "shs" }]
net: [{ transform: 'shs' }]
}
},
path: path.join(HOME, '.' + name),
timers: {
connection: 0,
reconnect: 5*SEC,
ping: 5*MIN,
handshake: 5*SEC
reconnect: 5 * SEC,
ping: 5 * MIN,
handshake: 5 * SEC
},
//change these to make a test network that will not connect to the main network.
// change these to make a test network that will not connect to the main network.
caps: {
//this is the key for accessing the ssb protocol.
//this will be updated whenever breaking changes are made.
//(see secret-handshake paper for a full explaination)
//(generated by crypto.randomBytes(32).toString('base64'))
// this is the key for accessing the ssb protocol.
// this will be updated whenever breaking changes are made.
// (see secret-handshake paper for a full explaination)
// (generated by crypto.randomBytes(32).toString('base64'))
shs: '1KHLiKZvAvjbY1ziZEHMXawbCEIM6qwjCDm3VYRan/s=',

//used to sign messages
// used to sign messages
sign: null
},
master: [],
Expand All @@ -62,11 +63,15 @@ module.exports = function (name, override) {

if (!result.connections.incoming) {
result.connections.incoming = {
net: [{ host: result.host, port: result.port, scope: "public", "transform": "shs" }],
ws: [{ host: result.host, port: result.ws.port, scope: "device", "transform": "shs" }]
net: [{ host: result.host, port: result.port, scope: 'public', 'transform': 'shs' }],
ws: [{ host: result.host, port: result.ws.port, scope: 'device', 'transform': 'shs' }]
}
}
return result
}

// *** LEGACY TIDYUP ***
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ummm .. I did this, which might be bad (naughty mix). To my eye, if we have config.connections we shouldn't really have suspicious config.port wandering around waiting to be accidentally used when there's a different config.connections.incoming.net[0].port been set. RIGHT?

This should really be a separate PR, but life is messy you know?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dig this. I don't have a solid grasp on which modules use config.port and config.host, but this would require a major version bump and some release notes, yeah?

Copy link

@ahdinosaur ahdinosaur Jan 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config.port and config.host are useful configurations because you can set them with environment variables ssb_port and ssb_host. removing these means we can't configure pubs in the standard way. that being said, my plan for PeachCloud is to avoid using these parts of the jsbot stack altogether, but recent changes are currently breaking ahdinosaur/ssb-pub: ahdinosaur/ssb-pub#15 (comment)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, my above comment is probably in the wrong place, deleting config.host from the returned object is good, not being able to set ssb_host to configure the external host is meh.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, so that sounds like a 👍 from @ahdinosaur which is good enough for me.
What I've got supports old inputs, but also removes a footgun!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

turns out this was needed by ssb-server/bin. It's not documented anywhere, sure. and even I had forgotten that was used. But there generally tend to be things like this. The best way to catch things like this is to symlink a module into it's dependants and then run those tests.

cd ssb-server; cd node_modules; rm -rf ssb-keys; ln -s ../../ssb-keys; cd ..; npm test
This is the last line of defence against releasing broken stuff, and it would make me very happy if people making PRs would run this before they submit it for review.

delete result.host
delete result.port
delete result.ws

return result
}
Loading