Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for the PROXY protocol #184

Draft
wants to merge 130 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
130 commits
Select commit Hold shift + click to select a range
1b82d33
Create a new config format so we can expand listener configuration fo…
sbruens May 31, 2024
a4c2007
Remove unused `fakeAddr`.
sbruens May 31, 2024
72b27d7
Split `startPort` up between TCP and UDP.
sbruens Jun 3, 2024
fddfc57
Use listeners to configure TCP and/or UDP services as needed.
sbruens Jun 3, 2024
c1ee12f
Remove commented out line.
sbruens Jun 3, 2024
354301e
Use `ElementsMatch` to compare the services irrespective of element o…
sbruens Jun 3, 2024
751d164
Do not ignore the `keys` field if `services` is used as well.
sbruens Jun 12, 2024
6297304
Add some more tests for failure scenarios and empty files.
sbruens Jun 12, 2024
0ac0a72
Remove unused `GetPort()`.
sbruens Jun 12, 2024
794f860
Move `ResolveAddr` to config.go.
sbruens Jun 12, 2024
01b7e8a
Remove use of `net.Addr` type.
sbruens Jun 12, 2024
87a1565
Pull listener creation into its own function.
sbruens Jun 12, 2024
51a13a7
Move listener validation/creation to `config.go`.
sbruens Jun 14, 2024
f8d7aa5
Use a custom type for listener type.
sbruens Jun 14, 2024
1952036
Fix accept handler.
sbruens Jun 14, 2024
7212265
Add doc comment.
sbruens Jun 14, 2024
6e2068d
Fix tests still supplying the port.
sbruens Jun 14, 2024
7114434
Move old config parsing to `loadConfig`.
sbruens Jun 14, 2024
1b2dd42
Lowercase `readConfig`.
sbruens Jun 14, 2024
744b2cf
Add support for PROXY protocol on TCP.
sbruens Jun 17, 2024
c5fb99a
Create a `Listen` func to create proxy or direct listeners based on t…
sbruens Jun 17, 2024
802e689
Update config test.
sbruens Jun 17, 2024
e085d4c
Wrap direct listener inside proxy protocol listener.
sbruens Jun 17, 2024
98ccbce
Move listeners into `tcp.go`.
sbruens Jun 17, 2024
d83e6ba
Fix tests.
sbruens Jun 17, 2024
870b379
Rename `IClientStreamConn` to `ClientStreamConn`.
sbruens Jun 17, 2024
00d484b
Rename `WrapStreamAccepter` to `WrapStreamListener`.
sbruens Jun 17, 2024
4ce06f0
Use `Config` suffix for config types.
sbruens Jun 21, 2024
8660032
Remove the IP version specifiers from the `newListener` config handling.
sbruens Jun 21, 2024
26b9100
refactor: remove use of port in proving metric
sbruens Jun 21, 2024
1b8e903
Fix tests.
sbruens Jun 21, 2024
442b927
Merge branch 'sbruens/absorb-port' into sbruens/proxy
sbruens Jun 21, 2024
4216ce3
Add a TODO comment to allow short-form direct listener config.
sbruens Jun 21, 2024
35c828d
Make legacy key config name consistent with type.
sbruens Jun 21, 2024
1322f2d
Move config validation out of the `loadConfig` function.
sbruens Jun 21, 2024
adc11f2
Remove unused port from bad merge.
sbruens Jun 21, 2024
3084dfd
Add comment describing keys.
sbruens Jun 24, 2024
7e5aae5
Move validation of listeners to config's `Validate()` function.
sbruens Jun 24, 2024
b136c79
Introduce a `NetworkAdd` to centralize parsing and creation of listen…
sbruens Jun 25, 2024
4bf9c27
Use `net.ListenConfig` to listen.
sbruens Jun 24, 2024
b7bb65b
Simplify how we create new listeners.
sbruens Jun 25, 2024
af3ca31
Do not use `io.Closer`.
sbruens Jun 28, 2024
fc72593
Use an inline error check.
sbruens Jul 1, 2024
b24a339
Use shared listeners and packet connections.
sbruens Jul 1, 2024
3bc76bc
Close existing listeners once the new ones are serving.
sbruens Jul 1, 2024
f71b13d
Elevate failure to stop listeners to `ERROR` level.
sbruens Jul 1, 2024
6893e2a
Merge remote-tracking branch 'origin/master' into sbruens/proxy
sbruens Jul 2, 2024
32cc180
Be more lenient in config validation to allow empty listeners or keys.
sbruens Jul 2, 2024
640f80f
Ensure the address is an IP address.
sbruens Jul 3, 2024
22638c7
Use `yaml.v3`.
sbruens Jul 3, 2024
2631b87
Move file reading back to `main.go`.
sbruens Jul 8, 2024
d76efd2
Do not embed the `net.Listener` type.
sbruens Jul 8, 2024
b8c5ab8
Use a `Service` object to abstract away some of the complex logic of …
sbruens Jul 8, 2024
5ac0f46
Fix how we deal with legacy services.
sbruens Jul 8, 2024
1f097be
Remove commented out lines.
sbruens Jul 8, 2024
80b25b1
Use `tcp` and `udp` types for direct listeners.
sbruens Jul 8, 2024
2070d40
Use a `ListenerManager` instead of globals to manage listener state.
sbruens Jul 8, 2024
eacfa0e
Add validation check that no two services have the same listener.
sbruens Jul 8, 2024
dc1075a
Use channels to notify shared listeners they need to stop acceoting.
sbruens Jul 10, 2024
2a343e2
Pass TCP timeout to service.
sbruens Jul 10, 2024
e58b79d
Move go routine call up.
sbruens Jul 10, 2024
c7465fb
Allow inserting single elements directly into the cipher list.
sbruens Jul 11, 2024
43fa0d6
Add the concept of a listener set to track existing listeners and clo…
sbruens Jul 11, 2024
cf9b7d2
Refactor how we create listeners.
sbruens Jul 11, 2024
ae7f41d
Update comments.
sbruens Jul 11, 2024
120db8e
`go mod tidy`.
sbruens Jul 11, 2024
5cbeb54
Merge branch 'sbruens/shared-listeners' into sbruens/proxy
sbruens Jul 11, 2024
d705603
refactor: don't link the TCP handler to a specific listener
sbruens Jul 16, 2024
2fb4a6b
Merge branch 'sbruens/remove-listener-dependency' into sbruens/shared…
sbruens Jul 16, 2024
d2ef46e
Protect new cipher handling methods with mutex.
sbruens Jul 16, 2024
ab07400
Move `listeners.go` under `/service`.
sbruens Jul 16, 2024
71d7140
Use callback instead of passing in key and manager.
sbruens Jul 16, 2024
9dfa4e2
Move config start into a go routine for easier cleanup.
sbruens Jul 16, 2024
0a63f5c
Make a `StreamListener` type.
sbruens Jul 19, 2024
f018d17
Rename `closeFunc` to `onCloseFunc`.
sbruens Jul 19, 2024
4295c45
Rename `globalListener`.
sbruens Jul 19, 2024
e6963f6
Don't track usage in the shared listeners.
sbruens Jul 19, 2024
7113f02
Add `getAddr()` to avoid some duplicate code.
sbruens Jul 19, 2024
e4d679f
Move listener set creation out of the inner function.
sbruens Jul 22, 2024
be5f9b0
Remove `PushBack()` from `CipherList`.
sbruens Jul 22, 2024
343e412
Move listener set to `main.go`.
sbruens Jul 22, 2024
7f86ff1
Close the accept channel with an atomic value.
sbruens Jul 22, 2024
e80b2c5
Update comment.
sbruens Jul 22, 2024
b1428ed
Address review comments.
sbruens Jul 22, 2024
1c16de8
Close before deleting key.
sbruens Jul 22, 2024
ebc7053
`server.Stop()` does not return a value
sbruens Jul 22, 2024
67fc7fb
Add a comment for `StreamListener`.
sbruens Jul 22, 2024
7a15e7d
Do not delete the listener from the manager until the last user has c…
sbruens Jul 22, 2024
499829e
Consolidate usage counting inside a `listenAddress` type.
sbruens Jul 22, 2024
f165dbd
Remove `atomic.Value`.
sbruens Jul 22, 2024
2a2420a
Add some missing comments.
sbruens Jul 22, 2024
8178d78
Merge branch 'master' into sbruens/shared-listeners
sbruens Jul 25, 2024
cccba1a
address review comments
sbruens Jul 25, 2024
da4ccaa
Add type guard for `sharedListener`.
sbruens Jul 25, 2024
d47f612
Stop the existing config in a goroutine.
sbruens Jul 25, 2024
a928e2c
Add a TODO to wait for all handlers to be stopped.
sbruens Jul 25, 2024
98cc3a0
Run `stopConfig` in a goroutine in `Stop()` as well.
sbruens Jul 25, 2024
48d0931
Create a `TCPListener` that implements a `StreamListener`.
sbruens Jul 25, 2024
2dec847
Track close functions instead of the entire listener, which is not ne…
sbruens Jul 25, 2024
ab22e47
Delegate usage tracking to a reference counter.
sbruens Jul 30, 2024
3c2a3ef
Remove the `Get()` method from `refCount`.
sbruens Jul 31, 2024
5e282f1
Return immediately.
sbruens Jul 31, 2024
547e9e6
Rename `shared` to `virtual` as they are not actually shared.
sbruens Jul 31, 2024
c6774c8
Simplify `listenAddr`.
sbruens Jul 31, 2024
df2f9d0
Fix use of the ref count.
sbruens Jul 31, 2024
c678372
Add simple test case for early closing of stream listener.
sbruens Jul 31, 2024
e41abab
Add tests for creating stream listeners.
sbruens Jul 31, 2024
b626a1c
Merge branch 'sbruens/shared-listeners' into sbruens/proxy
sbruens Jul 31, 2024
f9432d2
Create handlers on demand.
sbruens Jul 31, 2024
f43fbec
Merge branch 'sbruens/proxy' into sbruens/proxy-protocol
sbruens Aug 1, 2024
6b11f4f
Refactor create methods.
sbruens Aug 2, 2024
3e03394
Merge branch 'sbruens/shared-listeners' into sbruens/proxy
sbruens Aug 2, 2024
37678cf
Merge branch 'sbruens/proxy' into sbruens/proxy-protocol
sbruens Aug 2, 2024
9924713
Use `errors.Is()`.
sbruens Aug 2, 2024
c8d8332
Fix proxy post-merge.
sbruens Aug 2, 2024
827be3c
Check if the command is UNKNOWN (v1) or LOCAL (v2).
sbruens Aug 5, 2024
2622b5e
Add test scenarios for client addr.
sbruens Aug 5, 2024
fe8bbdd
Address review comments.
sbruens Aug 5, 2024
36a0a1d
Use a mutex to ensure another user doesn't acquire a new closer while…
sbruens Aug 5, 2024
aeb2652
Move mutex up.
sbruens Aug 6, 2024
8873b10
Manage the ref counting next to the listener creation.
sbruens Aug 6, 2024
899d13d
Do the lazy initialization inside an anonymous function.
sbruens Aug 6, 2024
80e5d49
Fix concurrent access to `acceptCh` and `closeCh`.
sbruens Aug 7, 2024
aa00f2e
Use `/` in key instead of `-`.
sbruens Aug 7, 2024
e658b90
Return error from stopping listeners.
sbruens Aug 7, 2024
fede4d8
Use channels to ensure `virtualPacketConn`s get closed.
sbruens Aug 7, 2024
4730d74
Add more test cases for packet listeners.
sbruens Aug 7, 2024
30bbdfa
Merge branch 'sbruens/shared-listeners' into sbruens/proxy
sbruens Aug 7, 2024
57d46a2
Merge branch 'sbruens/proxy' into sbruens/proxy-protocol
sbruens Aug 7, 2024
baea2a2
Move PROXY proto logic to its own `proxyproto.go` file.
sbruens Aug 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 143 additions & 0 deletions cmd/outline-ss-server/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
// Copyright 2024 Jigsaw Operations LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
"errors"
"fmt"
"io"
"net"
"net/url"
"os"

"github.com/Jigsaw-Code/outline-ss-server/service"
"gopkg.in/yaml.v2"
)

type Service struct {
Listeners []Listener
Keys []Key
}

type ListenerType string

const (
listenerTypeDirect ListenerType = "direct"
listenerTypeProxy ListenerType = "proxy_protocol"
)

type Listener struct {
sbruens marked this conversation as resolved.
Show resolved Hide resolved
Type ListenerType
Address string
}

type Key struct {
ID string
Cipher string
Secret string
}

type LegacyKeyService struct {
Key `yaml:",inline"`
Port int
}

type Config struct {
Services []Service

// Deprecated: `keys` exists for backward compatibility. Prefer to configure
// using the newer `services` format.
Keys []LegacyKeyService
}

// readConfig attempts to read a config from a filename and parses it as a [Config].
func readConfig(filename string) (*Config, error) {
config := Config{}
configData, err := os.ReadFile(filename)
if err != nil {
return nil, fmt.Errorf("failed to read config: %w", err)
}
err = yaml.Unmarshal(configData, &config)
if err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err)
}
return &config, nil
}

// validateListener asserts that a listener URI conforms to the expected format.
func validateListener(u *url.URL) error {
if u.Opaque != "" {
return errors.New("URI cannot have an opaque part")
}
if u.User != nil {
return errors.New("URI cannot have an userdata part")
}
if u.RawQuery != "" || u.ForceQuery {
return errors.New("URI cannot have a query part")
}
if u.Fragment != "" {
return errors.New("URI cannot have a fragement")
}
if u.Path != "" && u.Path != "/" {
return errors.New("URI path not allowed")
}
return nil
}

// newListener creates a new listener from a URL-style address specification.
//
// Example addresses:
//
// tcp4://127.0.0.1:8000
// udp://127.0.0.1:9000
func newListener(addr string) (io.Closer, error) {
u, err := url.Parse(addr)
if err != nil {
return nil, err
}

switch u.Scheme {
case "tcp", "tcp4", "tcp6":
if err := validateListener(u); err != nil {
return nil, fmt.Errorf("invalid listener `%s`: %v", u, err)
}
return net.Listen(u.Scheme, u.Host)
case "udp", "udp4", "udp6":
if err := validateListener(u); err != nil {
return nil, fmt.Errorf("invalid listener `%s`: %v", u, err)
}
return net.ListenPacket(u.Scheme, u.Host)
default:
return nil, fmt.Errorf("unsupported protocol: %s", u.Scheme)
}
}

// Listen creates a new listener based on a given [Listener] config.
func Listen(config Listener) (io.Closer, error) {
sbruens marked this conversation as resolved.
Show resolved Hide resolved
listener, err := newListener(config.Address)
if err != nil {
return nil, err
}
switch ln := listener.(type) {
case net.Listener:
streamListener := &service.StreamListener{Listener: ln}
if config.Type == listenerTypeProxy {
return &service.ProxyListener{StreamListener: *streamListener}, err
}
return streamListener, err
default:
return listener, err
}
}
15 changes: 15 additions & 0 deletions cmd/outline-ss-server/config_example.deprecated.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
keys:
- id: user-0
port: 9000
cipher: chacha20-ietf-poly1305
secret: Secret0

- id: user-1
port: 9000
cipher: chacha20-ietf-poly1305
secret: Secret1

- id: user-2
port: 9001
cipher: chacha20-ietf-poly1305
secret: Secret2
40 changes: 26 additions & 14 deletions cmd/outline-ss-server/config_example.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,27 @@
keys:
- id: user-0
port: 9000
cipher: chacha20-ietf-poly1305
secret: Secret0
services:
- listeners:
- type: direct
address: "tcp://[::]:9000"
- type: direct
address: "udp://[::]:9000"
- type: proxy_protocol
sbruens marked this conversation as resolved.
Show resolved Hide resolved
address: "tcp://[::]:9010"
- type: proxy_protocol
address: "udp://[::]:9010"
keys:
- id: user-0
cipher: chacha20-ietf-poly1305
secret: Secret0
- id: user-1
cipher: chacha20-ietf-poly1305
secret: Secret1

- id: user-1
port: 9000
cipher: chacha20-ietf-poly1305
secret: Secret1

- id: user-2
port: 9001
cipher: chacha20-ietf-poly1305
secret: Secret2
- listeners:
- type: direct
address: "tcp://[::]:9001"
- type: direct
address: "udp://[::]:9001"
keys:
- id: user-2
cipher: chacha20-ietf-poly1305
secret: Secret2
103 changes: 103 additions & 0 deletions cmd/outline-ss-server/config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// Copyright 2024 Jigsaw Operations LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
"os"
"testing"

"github.com/stretchr/testify/require"
)

func TestReadConfig(t *testing.T) {
config, err := readConfig("./config_example.yml")

require.NoError(t, err)
expected := Config{
Services: []Service{
Service{
Listeners: []Listener{
Listener{Type: listenerTypeDirect, Address: "tcp://[::]:9000"},
Listener{Type: listenerTypeDirect, Address: "udp://[::]:9000"},
Listener{Type: listenerTypeProxy, Address: "tcp://[::]:9010"},
Listener{Type: listenerTypeProxy, Address: "udp://[::]:9010"},
},
Keys: []Key{
Key{"user-0", "chacha20-ietf-poly1305", "Secret0"},
Key{"user-1", "chacha20-ietf-poly1305", "Secret1"},
},
},
Service{
Listeners: []Listener{
Listener{Type: listenerTypeDirect, Address: "tcp://[::]:9001"},
Listener{Type: listenerTypeDirect, Address: "udp://[::]:9001"},
},
Keys: []Key{
Key{"user-2", "chacha20-ietf-poly1305", "Secret2"},
},
},
},
}
require.Equal(t, expected, *config)
}

func TestReadConfigParsesDeprecatedFormat(t *testing.T) {
config, err := readConfig("./config_example.deprecated.yml")

require.NoError(t, err)
expected := Config{
Keys: []LegacyKeyService{
LegacyKeyService{
Key: Key{ID: "user-0", Cipher: "chacha20-ietf-poly1305", Secret: "Secret0"},
Port: 9000,
},
LegacyKeyService{
Key: Key{ID: "user-1", Cipher: "chacha20-ietf-poly1305", Secret: "Secret1"},
Port: 9000,
},
LegacyKeyService{
Key: Key{ID: "user-2", Cipher: "chacha20-ietf-poly1305", Secret: "Secret2"},
Port: 9001,
},
},
}
require.Equal(t, expected, *config)
}

func TestReadConfigFromEmptyFile(t *testing.T) {
file, _ := os.CreateTemp("", "empty.yaml")

config, err := readConfig(file.Name())

require.NoError(t, err)
require.ElementsMatch(t, Config{}, config)
}

func TestReadConfigFromNonExistingFileFails(t *testing.T) {
config, err := readConfig("./foo")

require.Error(t, err)
require.ElementsMatch(t, nil, config)
}

func TestReadConfigFromIncorrectFormatFails(t *testing.T) {
file, _ := os.CreateTemp("", "empty.yaml")
file.WriteString("foo")

config, err := readConfig(file.Name())

require.Error(t, err)
require.ElementsMatch(t, Config{}, config)
}
Loading