mergable network config

[dominictarr]

as stated here I think we need a more flexible network config. we need to have default settings that may be disabled or overridden in a granular fashion. the current pattern only sets the connections defaults if the user hasn't set any thing, and if they set anything doesn't set any defaults. This causes the user to copy the default config and edit it, which means if we want to change the default config it still needs to work with the copied config. It would be better if the user updates only overrid the parts they wanted to change

connections: {
  incoming:
  // <name>: {transport,scope,port,host,transform}
    ws_device: {
      transport: 'ws', scope: 'device', port: 9000, //since scope is device, binds loopback interface
      transform: 'shs'
    },
    ws_local: {
      transport: 'ws', scope: 'local', port: 9000,//since scope is local, binds local (private) network interface
      transform: 'shs'
    },
    ws_public: {
      transport: 'ws', scope: 'public', port: 9000, //since scope is local, binds local (private) network interface
      transform: 'shs'
    }
  }
}

other considerations

running more than one protocol instance per scope

you may wish to have more than one server on the same scope, with different protocols or different shs settings. those will go under a separate name, and you can disable them by setting --connections.incoming.ws_public=

other scopes

ssb-device-address lets you announce an address for a scope. it can be encrypted, to only some recipients too, for example, you might not want the whole of the scuttleverse connecting to your ssb-tunnel but do want your friends to connect, so put that in a scope "friends". "work" might be another obvious alternative scope.

Also, bluetooth, and other local network IoT stuff might expand the set of scopes to non-ip protocols.

[dominictarr]

@arj03 @regular what are your thoughts?

[dominictarr]

another possibility is to have a layer for scope.

connections: {
  incoming: {
   <scope>: { <name>: {transport,scope,... ,transform},... },...
  }
}

that would group every thing in the same scope together, which would make it nice and easy to merge.
it would also allow you to temporarily disable an entire scope. (maybe you wish to disable local scope while in a crowded cafe, for example)

[dominictarr]

maybe... the solution is to allow an address to exist in more than one scope?
then I can say scope: ["device", "local"] and lets that binds to loopback and local network addresses...
and also returns an address when called on either device or loopback. also, it may use a different address on each - say net:localhost:8000~... for device and net:192.168...:8000~ on local, etc

hmm, I feel this might do it

[arj03]

Yeah, like the multiple scopes approach. In general liking this PR.

[dominictarr]

Hmm - scopes is starting to seem overloaded.
I'm proposing scopes to map directly to type of ip address, and also have additional ones that are not on an ip address at all. What if "work" uses a connection that accepts peers via wifi (and so needs to be bound to private ip addresses, but with say, different shs settings) so it needs to bind like local but be publishe to "work". in ssb-tunnel I have addresses that are not bound to a ip type at all, such an address is "public" in the sense it's accessable, but it's not literally a public range address.

[regular]

+1 for scope feeling overloaded. I'd even say it's the wrong abstraction.

With the config, the user wants to say

• on the bluetooth interface, I want shs
• on 192.168.x.x I want blah
• on 10.10.x.x I want blah
• on 178.124.43.12 I want blah
• on localhost I want noauth

the scopes public and private are to broad and don't allow to express the above (because there can be more than one lan interface and more than one interface with a public ip), scope is just not a good match for this!

And with getAddress {scope} the user wants to say

• give me an address that works for people on 192.168.x.x
• give me an address that works for people on the public internet
• give me an address that works for bluetooth devices

Should we talk about interfaces in the connection configuration instead? I'd argue that the user should be able to define named interfaces and these interface names (we might as well call them 'servers') can then be used in getAddress.

For convenience, and so that sbot works out of the box, we could offer symbolic names for the first public and the first private address we can find. This would work like if the user had entries for public and private in etc/hosts

  interfaces: {
  // <name>: {transport,port,host,transform}
    ws_device: {
      host: 'localhost',
      transport: 'ws',
      port: 9000,
      transform: 'shs'
    },
    ws_local: {
      transport: 'ws',
      host: 'private',
      port: 9000,
      transform: 'shs'
    },
    ws_public: {
      transport: 'ws',
      host: 'public',
      port: 9000
    }
  }

and then:

sbot getAddress ws_public

Since we are talking breaking change now: port and host should really be properties of the net and ws transport, because the transport defines the semantics of these properties, there will be different properties for different transports. Also, we discovered the need for the transform to have properties as well (I don't remember where) and you might want to have multiple transforms.

 interfaces: {
  // <name>: {transport,transforms}
    ws_device: {
      transport: {
         type: 'ws',
         host: 'localhost',
         port: 9000
     },
      transforms: [[
        type: 'shs'
     }]
    }
...

[dominictarr]

That example you give has a lot of duplication, what if it was:

  websocket: { //just a name of this server, not a scope
    type: 'ws'
    scope: {
      //<scope>: <host>,...
      device: 'localhost',
      local: 'private',
      public: 'public' //or internet: public?
      //how the <host> is interpreted is up to the type.
      //non-ip protocols don't need that.
    },
    port: 9000,
    transform: 'shs'
  }

for transforms with properties, I suggest this structure:

{
  transport: 'net',
  ...,
  transform: {  //as linked list, remains mergable.
    type: 'gzip',
    transform: 'shs' //allow transform:name if it doesn't need config
  }
}

{
  transport: 'net',
  ...,
  transform: {  //transform with more non-default config
    type: 'shs',
    cap: <alt_cap>,
    seed: <seed> //always connect to me with this private key, for semi anonymous connections
  }
}

Yeah, I'm also thinking of maybe

[dominictarr]

I want to emphasize: It's important to be able to return multiple addresses: A pub should expose a net and a ws address (so that it's possible to connect from a browser). A public peer that isn't able to connect to a net address can just ignore that part, same goes for upgrading shs.

I guess the thing is there is really two aspects here: groups of addresses that you want to share in different contexts (which was how I interpreted scopes - actually, arj came up with scopes because we wanted to make the pluggable transports stuff work with tor) and what interfaces to bind to - which became a problem once we introduced noauth. interfaces to bind to is specifically a tcp problem (okay, maybe udp too)

[dominictarr]

so maybe interfaces to bind to is something that is handled by the specific plugin?

[regular]

@dominictarr

      device: 'localhost',
      local: 'private',
      public: 'public' //or internet: public?

D: uhm .... That's very high up on the list of utterly confusing key/value triplets! :)

I like the linked list idea though ... (yes, arrays and merge don't go together well)

Yes, groups! That's a much better term! You want to group the connection methods that are available for processes on your device, people onboard your boat, and people on the Internet into three different groups. But IMHO you dont want to specify the tcp host indirectly by giving the name of that group.

That example you give has a lot of duplication

does it though? To be honest, I don't see a single duplication? "ws_device" is just a label/name without semantics.

[dominictarr]

I'm not sure that "group" and "scope" really mean anything different, they are both so generic to seem interchangable to me.

@regular each row duplicates:

<name>: {
  type: "ws",
  ..., //host line which is different
  port: 9000,
  transform: 'shs'
}

I realized a thing just a second ago: I want to advertise a tunnel address - but not mix it with a public scoped net server. I could just make up a new scope - and announce that instead of "public".
then, binding to :: is fine, as long as getAddress is smart about returning the right address, and not :: because that's useless for another peer on wifi.
but noauth must have a way to listen strictly on localhost.

Okay, maybe you had already figured that out... but I still need a way for my net server to give an address in multiple scopes. I notice there is a weird exception that includes public scopes in private addresses: https://github.com/ssbc/multiserver/blob/master/index.js#L49 (I don't think there should be a special case there)

And remember, several of the plugins were reporting their scopes incorrectly, like, ignoring config, which was fixed here: https://github.com/ssbc/multiserver/blob/master/index.js#L49

[regular]

Ah, you mean it's redundant, when you want to listen on all interfaces? I see.

the tunnel thing: In my world you would define your own named interface and advertise that.

About https://github.com/ssbc/multiserver/blob/master/index.js#L49 I guess the reasoning behind this code (and I am guessing, since I'm not the author), is that when you are in the private scope, you can still reach the public scope. This way, for a normal setup, you only need a server in one scope: public. It can be reached from local, private and public. But the fact that you don't think that code should be there and someone else obviously thought it should be there, illustrates my main concern about the term "scope": there doesn't seem to be consent on the details of how they are supposed to work. Different people write code with different definitions in their heads, it seems to me.

I would not say that "group" and "scope" are interchangeable terms. "group" defines a set of things, while "scope" is about the visibility of things. I want to define a group of interfaces that I can refer to in getAddress <groupname> to get a list of alternative ways to connect to this particular subset of servers inside my sbot. Now, you could chose to use group names like "public" and "private", which would pretty much get you to what we have right now, BUT the semantics is up to you! The code doesn't assume anything about it (and therefor cannot be wrong about it). It's simpler, and maybe it's sufficient to cover all use cases?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.