Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New more flexible address format code #440

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
319 changes: 319 additions & 0 deletions lncore/addr.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,319 @@
package lncore

import (
"fmt"
"regexp"
"strings"
"strconv"
"encoding/hex"

"github.com/mit-dci/lit/bech32"
"github.com/mit-dci/lit/crypto/fastsha256"
)

/*
* Ok so there's a lot of stuff going on here. Lit natively supports addresses
* that are just the pkh of the pubkey encoded in bech32. And that's pretty
* cool, you look up the IP+port in a tracker (or DHT) and connected with
* Noise_XX, then verify that the pubkey they provide you matches the pkh you
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So how lit should work after #437 is that if we haven't connected to the peer before, we still go ahead with NOISE_XX and get the pubkey of the host. But if we are connecting to a host whose pubkey we know already, we use NOISE_XK, since we already know the remote node's pubkey and theres no requirement to ask again. This is similar to how we using ssh asks you if you want to remember and I guess an implementing UI could ask the user for his preference as well

* already have. Also you can provide IP+port if you want.
*
* But in mainstream BOLT nodes they use the full pubkey hex-encoded. They have
* a DHT where you look up the IP+port (if not provided) and connect with
* Noise_XK, which assumes you already have the pubkey (since you do) and that's
* all fine.
*
* This tries to unify all of them with formats like this under 1 parser:
*
* <bech32 pkh>[@<ip>[:<port>]
* or
*
* <hex of bech32 pubkey>[@<ip>[:<port>]
*
* or fully
*
* <bech32 pkh>:<pubkey>[@<ip>:<port>]
*/

// LnAddr is just a bech32-encoded pubkey.
// TODO Move this to another package so it's more obviously not *just* IO-related.
type LnAddr string

// ToString returns the LnAddr as a string. Right now it just unwraps it but it
// might do something more eventually.
func (lnaddr LnAddr) ToString() string {
return string(lnaddr)
}

// LnDefaultPort is from BOLT1.
const LnDefaultPort = 9735
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could rename this to BOLTDefaultPort or just plain DefaultPort since Ln is implied :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RPC though? Although that is in a different package so maybe it doesn't matter.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.


// PkhBech32Prefix if for addresses lit already uses.
const PkhBech32Prefix = "ln"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another small issue, but this is just lit's bech32 prefix and it would be better named that way

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, along with the port const naming.


// PubkeyBech32Prefix is for encoding the full pubkey in bech32.
const PubkeyBech32Prefix = "lnpk"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would we need this though, can't we parse with just the length of the resultant bech32 string?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's true, I suppose I wanted to make the difference more obvious. Especially since technically ln1dfasdfasdf:lnpk1asdfsadfasdf is a valid address, it'd look a little weird to me if it was ln1:ln1, and it'd be less obvious for the average joe to know how to pick out the "short" version of their address, which we want to make the pkh.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good thanks! Although I wonder why people would want to use bech32 pks (since lit right now supports only bech32 pkh and BOLT supports hex pks). Would be nice if some implementations decide to change though, so having this does make sense.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, it doesn't really matter. They get a little bit of error correction (maybe), but since most of the time addresses are going to be copy/pasted I don't think it matters. I added it to quell arguments about "well hex might be bad but bech32 is better" since this is demonstrably not that much shorter and not worth the extra effort.


// LnAddressData is all of the data that can be encoded in a parsed address.
type LnAddressData struct {
// Pkh is the pubkey hash as bech32.
Pkh *string

// Pubkey is the pubkey, as bytes, decoded from source format.
Pubkey []byte

// IPAddr is the IP address, if present.
IPAddr *string

// Port should always be defined, default is LnDefaultPort.
Port uint16
}

// HasFullAddress returns whether or not we need to do a tracker/DHT lookup in
// order to resolve the connection address for this address data. Basically, it
// checks if there's an IP address defined.
func (d *LnAddressData) HasFullAddress() bool {
return d.IPAddr != nil
}

// AddrFmtFull is <bech32 pkh>:<hex pk>@<ip>:<port>
const AddrFmtFull = "full"

// AddrFmtFullBech32Pk is <bech32 pkh>:<bech32 pk>@<ip>:<port>
const AddrFmtFullBech32Pk = "full_bech32_pk"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// AddrFmtFullBech32Pk is <bech32 pkh>:<bech32 pk>@<ip>:<port>

in the above, if we have the bech32 pk, we could easily derive the bech32 kph (LitAddr). Maybe we could just have the bech32 pk in full and then use the relevant methods where we need the pkh?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes you can definitely derive the pkh from the pk. I hope nobody uses this format except when making fully-qualified addresses for whatever reason. But in practical use I don't really see why anyone would use it. I left it (as with the variant with the hex pk) just as "one option".


// AddrFmtFullNoPort is <bech32 pkh>:<hex pk>@<ip>
const AddrFmtFullNoPort = "full_no_port"

// AddrFmtFullBech32PkNoPort is <bech32 pkh>:<bech32 pk>@<ip>
const AddrFmtFullBech32PkNoPort = "full_bech32_pk_no_port"

// AddrFmtBech32 is just <bech32 pkh>
const AddrFmtBech32 = "bech32_pkh"

// AddrFmtLit is just AddrFmtBech32
const AddrFmtLit = "lit"

// AddrFmtPubkey is <hex pubkey>
const AddrFmtPubkey = "hex_pk"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, unless we're using this in some internal parsing, the hex_pk on its own is not that useful (since there's no tracker that lnd uses). How do you envision using this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In c-lightning you can connect to nodes if you just have the pubkey, I'm not sure how it looks up the IP address but it is a valid "thing" you can connect with.


// AddrFmtBech32Pubkey is <bech32 pk>
const AddrFmtBech32Pubkey = "bech32_pk"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somewhat similar to the comment above in that the bech32 pk can be converted to a bech32 pkh and then used instead of parsing pks and pkh's separately. I assume that requiring the full bech32 pk would not be needed at all (except for internal parsing, where we could use defined methods / functions)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it's not really necessary but I thought I'd include it "just because".


// AddrFmtPubkeyIP is <hex pk>@<ip>
const AddrFmtPubkeyIP = "hex_pk_ip"

// AddrFmtPubkeyIPPort is <hex pk>@<ip>:<port>
const AddrFmtPubkeyIPPort = "hex_pk_ip_port"

// AddrFmtLitFull is <bech32 pkh>@<ip>:<port>
const AddrFmtLitFull = "lit_full"

// AddrFmtLitIP is <bech32 pkh>@<ip>
const AddrFmtLitIP = "lit_ip"

// DumpAddressFormats returns the addresses in all of the formats that fully represent it.
func DumpAddressFormats(data *LnAddressData) (map[string]string, error) {

ret := make(map[string]string)

if data.Port != LnDefaultPort && data.IPAddr == nil {
return ret, fmt.Errorf("nondefault port specified but IP not specified")
}

// Full.
if data.Pkh != nil && data.Pubkey != nil && data.IPAddr != nil {
ret[AddrFmtFull] = fmt.Sprintf("%s:%X@%s:%d", *data.Pkh, data.Pubkey, *data.IPAddr, data.Port)
ret[AddrFmtFullBech32Pk] = fmt.Sprintf("%s:%s@%s:%d", *data.Pkh, bech32.Encode(PubkeyBech32Prefix, data.Pubkey), *data.IPAddr, data.Port)
}

// Minimal Lit.
if data.Pkh != nil {
ret[AddrFmtLit] = *data.Pkh
}

// Long Lit.
if data.Pkh != nil && data.IPAddr != nil {
if data.Port == LnDefaultPort {
ret[AddrFmtLitIP] = fmt.Sprintf("%s@%s", *data.Pkh, *data.IPAddr)
}
ret[AddrFmtLitFull] = fmt.Sprintf("%s@%s:%d", *data.Pkh, *data.IPAddr, data.Port)
}

// Hex pubkey stuff
if data.Pubkey != nil {
ret[AddrFmtPubkey] = fmt.Sprintf("%X", data.Pubkey)
if data.IPAddr != nil {
if data.Port == LnDefaultPort {
ret[AddrFmtPubkeyIP] = fmt.Sprintf("%X@%s", data.Pubkey, *data.IPAddr)
}
ret[AddrFmtPubkeyIPPort] = fmt.Sprintf("%X@%s:%d", data.Pubkey, *data.IPAddr, data.Port)
}
}

// TODO more
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these formats are enough actually, we just need two formats (maybe three, if you count just the pkh as one format).

  1. Lit Addr (pkh) with Port
  2. BOLT Addr (hex pubkey) with Port
    (3). Lit Addr (pkh)

Again, dependent on how we would use this since displaying addresses requires only the two formats above unless I'mm missing something here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah people would probably only end up using those 3 formats often, but since people do weird things I thought it makes sense to include all possible variations? Especially since it's not hard for a person to manually convert most formats to simpler versions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I guess with your overall rationale in the other comments, I can see why you would want to include other formats here.


return ret, nil

}

// ParseLnAddrData takes a LnAddr and parses the internal data. Assumes it's
// not been force-casted as full checks are done in the ParseLnAddr function.
func ParseLnAddrData(addr LnAddr) (*LnAddressData, error) {

addrdata := new(LnAddressData)

parts := strings.SplitN(string(addr), "@", 2) // using cast here because reasons

// First process the identify information.
pkdata := strings.SplitN(parts[0], ":", 2)
if len(pkdata) == 2 {

// The pkh is already in the right format for internal storage.
addrdata.Pkh = &pkdata[0]

// Now figure out how to parse the full pubkey info.
if strings.HasPrefix(pkdata[1], PubkeyBech32Prefix + "1") {
_, data, err := bech32.Decode(pkdata[1])
if err != nil {
return nil, err
}
addrdata.Pubkey = data
} else {
data, err := hex.DecodeString(pkdata[1])
if err != nil {
return nil, err
}
addrdata.Pubkey = data
}

} else {

// If there's only 1 part then there's 3 mutually exclusive options.
if strings.HasPrefix(pkdata[0], PkhBech32Prefix + "1") {
addrdata.Pkh = &pkdata[0]
} else if strings.HasPrefix(pkdata[0], PubkeyBech32Prefix + "1") {
_, data, err := bech32.Decode(pkdata[0])
if err != nil {
return nil, err
}
addrdata.Pubkey = data
} else {
data, err := hex.DecodeString(pkdata[0])
if err != nil {
return nil, err
}
addrdata.Pubkey = data
}

// Now we figure out what the pkh should be if the full pubkey is specified.
if addrdata.Pkh == nil && addrdata.Pubkey != nil {
pkh := ConvertPubkeyToBech32Pkh(addrdata.Pubkey)
addrdata.Pkh = &pkh
}

}

// Now parse the location information if it's there.
if len(parts) > 1 {
ipdata := strings.SplitN(parts[1], ":", 2)

// The IP address is already in the right format, probably.
addrdata.IPAddr = &ipdata[0]

// Parse the port if it's there.
if len(ipdata) > 1 {
port, err := strconv.Atoi(ipdata[1])
if err != nil {
return nil, err
}
if port > 65535 {
return nil, fmt.Errorf("port number %d not in range", port)
}
addrdata.Port = uint16(port)
} else {
addrdata.Port = LnDefaultPort
}

}

return addrdata, nil // TODO
}

/*
+0 q p z r y 9 x 8
+8 g f 2 t v d w 0
+16 s 3 j n 5 4 k h
+24 c e 6 m u a 7 l
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could omit this? (Or add this as a comment below the bech32 regex).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that was to help explain the regex.


const bech32bodyregex = `[qpzry9x8gf2tvdw0s3jn54khce6mua7l]{4,}`
const ipaddrregex = `[0-9]{1,3}(\.[0-9]{1,3}){3}`
const pkhregex = `ln1` + bech32bodyregex
const pubkeyregex = `(lnpk1` + bech32bodyregex + `|` + `[0-9a-fA-F]{64})`
const fulladdrregex = `(` + pkhregex + `|` + pubkeyregex + `|` + pkhregex + `:` + pubkeyregex + `)(@` + ipaddrregex + `(:[0-9]{1,5})?)?`

// ParseLnAddr will verify that the string passed is a valid LN address of any format.
func ParseLnAddr(m string) (LnAddr, error) {

r := regexp.MustCompile(fulladdrregex)

if !r.Match([]byte(m)) {
return "", fmt.Errorf("address doesn't match overall format")
}

parts := strings.SplitN(m, "@", 2)
pkdata := strings.SplitN(parts[0], ":", 2)
if len(pkdata) == 2 {
// This means there's both a pkh and a pubkey.

pkhprefix, _, pkherr := bech32.Decode(pkdata[0])
if pkherr != nil {
return "", fmt.Errorf("invalid pkh bech32")
}
if pkhprefix != PkhBech32Prefix {
return "", fmt.Errorf("pkh bech32 prefix incorrect")
}

pkprefix, _, pkerr := bech32.Decode(pkdata[1])
if pkerr != nil {
// Maybe it's hex.
_, err := hex.DecodeString(pkdata[1])
if err != nil {
return "", fmt.Errorf("pubkey not valid bech32 or hex ('%s' and '%s')", pkerr.Error(), err.Error())
}
}
if pkprefix != PubkeyBech32Prefix {
return "", fmt.Errorf("pubkey bech32 prefix incorrect")
}

} else {
// This means there's *either* a pubkey or pkh, in some format.

prefix, _, err := bech32.Decode(pkdata[0])
if err != nil {
// Maybe it's hex.
_, err2 := hex.DecodeString(pkdata[0])
if err2 != nil {
return "", fmt.Errorf("pubkey not valid bech32 or hex ('%s' and '%s')", err.Error(), err2.Error())
}
}

if err == nil && prefix != PkhBech32Prefix && prefix != PubkeyBech32Prefix {
return "", fmt.Errorf("pubkey (or phk) bech32 prefix incorrect")
}
}

// Should we make sure that the pubkey and pkh match if both are present?

return LnAddr(m), nil // should be the only place we cast to this type

}

// ConvertPubkeyToBech32Pkh converts the pubkey into the bech32 representation
// of the PKH.
func ConvertPubkeyToBech32Pkh(pubkey []byte) string {
hash := fastsha256.Sum256(pubkey)
enc := bech32.Encode("ln", hash[:20])
return enc
}
Loading