docs: add TCP forwarding docs

README.md

@@ -19,39 +19,36 @@ Piko is a reverse proxy that provides a secure way to connect to services that
 aren’t publicly routable, known as tunneling. Instead of sending traffic
 directly to your services, your upstream services open outbound-only
 connections (tunnels) to Piko, then Piko forwards traffic to your services via
-their outbound connections.
+their established connections.
 
 Piko has two key design goals:
-* Serve production traffic: To serve production traffic Piko may run as a
-cluster of nodes for fault tolerance, horizontal scaling and zero-downtime
-deployments
-* Simple to host: Piko is designed to be simple to self-host, particularly on
-Kubernetes
+* Built to serve production traffic by running as a cluster of nodes fault
+tolerance, horizontal scaling and zero-downtime deployments
+* Simple to host behind a HTTP(S) load balancer on Kubernetes
+
+Therefore Piko can be used as an open-source alternative to
+[Ngrok](https://ngrok.com/).
 
 Such as you may use Piko to expose services in a customer network, a bring your
 own cloud (BYOC) service, or to connect to user devices.
 
-Therefore Piko can be used as an open-source alternative to
-[Ngrok](https://ngrok.com/).
+## Features
 
 ### Reverse Proxy
 
 In a traditional reverse proxy, you configure routing rules describing how to
-route incoming traffic to upstream services. The proxy will then open
+route incoming traffic to your upstream services. The proxy will then open
 connections to your services and forward incoming traffic. This means your
 upstream services must be discoverable and have an exposed port that's
 accessible from the proxy.
 
-With Piko, rather than configuring routing rules in the proxy, instead your
-upstreams configure their own routing rules and open a secure outbound-only
-connection to Piko. Piko then forwards incoming traffic to the correct upstream
-via its outbound connection.
+Whereas with Piko, your upstreams open outbound-only connections to the
+[Piko server](./docs/server/server.md) and specify what endpoint they are
+listening on. Piko then forwards incoming traffic to the correct upstream via
+its outbound connection.
 
 Therefore your services may run anywhere without requiring a public route, as
-long as they can open a connection to Piko. This enables accessing services in
-private environments, such as an external customer network or your local
-network. It can also be used to simplify your infrastructure as you don’t need
-to set up firewall rules, DNS, certificates, load balancers…
+long as they can open a connection to the Piko server.
 
 ### Endpoints
 
@@ -63,6 +60,29 @@ same endpoint, requests are load balanced among the available upstreams.
 No static configuration is required to configure endpoints, upstreams can
 listen on any endpoint they choose.
 
+You can open an upstream listener using the
+[Piko agent](./docs/agent/agent.md), which supports both HTTP and TCP
+upstreams. Such as to listen on endpoint `my-endpoint` and forward traffic to
+`localhost:3000`:
+```
+# HTTP listener.
+$ piko agent http my-endpoint 3000
+
+# TCP listener.
+$ piko agent tcp my-endpoint 3000
+```
+
+You can also use the [Go SDK](./docs/sdk/go-sdk.md) to listen directly from
+your application using a standard `net.Listener`.
+
+<p align="center">
+  <img src="assets/images/overview.png" alt="overview" width="80%"/>
+</p>
+
+### HTTP(S)
+
+Piko acts as a transparent HTTP(S) reverse proxy.
+
 Incoming HTTP(S) requests identify the target endpoint to connect to using
 either the `Host` header or `x-piko-endpoint` header.
 
@@ -71,21 +91,38 @@ Such as if your hosting Piko with a wildcard domain at `*.piko.example.com`,
 sending a request to `foo.piko.example.com` will be routed to an upstream
 listening on endpoint `foo`.
 
-To avoid having to set up a wildcard domain you can instead use
-`x-piko-endpoint`, such as if Piko is hosted at `piko.example.com`, you can
-send requests to endpoint `foo` using header `x-piko-endpoint: foo`.
+To avoid having to set up a wildcard domain you can instead use the
+`x-piko-endpoint` header, such as if Piko is hosted at `piko.example.com`, you
+can send requests to endpoint `foo` using header `x-piko-endpoint: foo`.
 
-<p align="center">
-  <img src="assets/images/overview.png" alt="overview" width="80%"/>
-</p>
+### TCP
+
+Piko supports proxying TCP traffic, though unlike HTTP it requires using either
+[Piko forward](./docs/forward/forward.md) or the
+[Go SDK](./docs/sdk/go-sdk.md) to map the desired local TCP port to the target
+endpoint (as there's no way to identify the target endpoint using raw TCP).
+
+[Piko forward](./docs/forward/forward.md) is basically the opposite of
+[Piko agent](./docs/agent/agent.md). Instead of listening on an endpoint and
+forwarding to a local port on the upstream, Piko forward runs on the client and
+listens on a TCP port then forwards connections to the configured upstream
+endpoint.
+
+Such as to listen on port `3000` and forward connections to endpoint
+`my-endpoint`:
+```
+piko forward 3000 my-endpoint
+```
 
+You can also use the [Go SDK](./docs/sdk/go-sdk.md) to open a `net.Conn` that's
+connected to the configured endpoint.
 
 ## Design Goals
 
 ### Production Traffic
 
-Piko is built to serve production traffic, which means the Piko server must run
-as a cluster of nodes to be fault tolerant, scale horizontally and support zero
+Piko is built to serve production traffic by running the Piko server as a
+cluster of nodes to be fault tolerant, scale horizontally and support zero
 downtime deployments.
 
 Say an upstream is listening for traffic on endpoint E and connects to node N.
@@ -110,7 +147,7 @@ Upstream services and downstream clients may connect to any node in the cluster
 via the load balancer, then the cluster manages routing traffic to the
 appropriate upstream.
 
-See [Kubernetes](./docs/manage/kubernetes.md) for details.
+See [Kubernetes](./docs/server/kubernetes.md) for details.
 
 ## Getting Started
 
@@ -131,10 +168,12 @@ ask questions, get help, or suggest ideas.
 - Tutorials
   - [Getting Started](./docs/getting-started.md)
   - [Install](./docs/tutorials/install.md)
+  - [TCP Forwarding](./docs/tutorials/tcp-forwarding.md)
 - [Server](./docs/server/server.md)
   - [Observability](./docs/server/observability.md)
   - [Kubernetes](./docs/server/kubernetes.md)
 - [Agent](./docs/agent/agent.md)
+- [Forward](./docs/forward/forward.md)
 - [Go SDK](./docs/sdk/go-sdk.md)
 
 ## Contributing

docs/forward/forward.md

@@ -0,0 +1,74 @@
+# Forward
+
+Piko forward listens on a local port and forwards connections to the configured
+upstream endpoint.
+
+Such as you may listen on port 3000 and forward connections to endpoint
+'my-endpoint'.
+
+When using HTTP(S), you can connect to the Piko server directly using the
+`Host` or `x-piko-endpoint` header to identify the endpoint ID, though when
+using raw TCP you must first proxy the connection on the client to connect
+to the configured endpoint.
+
+Such as `piko forward tcp 3000 my-endpoint` listens locally on port `3000` then
+forwards connections to endpoint `my-endpoint`.
+
+<p align="center">
+  <img src="../../assets/images/forward.png" alt="overview" width="60%"/>
+</p>
+
+## Configuration
+
+Piko supports both YAML configuration and command-line flags.
+
+The YAML file path can be set using `--config.path`.
+
+See `piko agent -h` for the available configuration options.
+
+## YAML Configuration
+
+Piko forward supports the following YAML configuration (where most parameters
+have corresponding command line flags):
+
+```
+# Ports contains the set of ports to listen on and the endpoint to forward to.
+ports:
+- addr: "3000"
+  endpoint_id: my-endpoint
+
+connect:
+  # The Piko server URL to connect to. Note this must be configured to use the
+  # Piko server 'proxy' port.
+  url: http://localhost:8000
+
+  # Timeout attempting to connect to the Piko server.
+  timeout: 30s
+
+  tls:
+    # A path to a certificate PEM file containing root certificiate authorities to
+    # validate the TLS connection to the Piko server.
+    #
+    # Defaults to using the host root CAs.
+    root_cas: ""
+
+log:
+    # Minimum log level to output.
+    #
+    # The available levels are 'debug', 'info', 'warn' and 'error'.
+    level: info
+
+    # Each log has a 'subsystem' field where the log occured.
+    #
+    # '--log.subsystems' enables all log levels for those given subsystems. This
+    # can be useful to debug a particular subsystem without having to enable all
+    # debug logs.
+    #
+    # Such as you can enable 'gossip' logs with '--log.subsystems gossip'.
+    subsystems: []
+```
+
+### TlS
+
+To specify a custom root CA to validate the TLS connection to the Piko server,
+use `--connect.tls.root-cas`.

docs/how-piko-works.md

@@ -2,12 +2,12 @@
 
 This document provides an overview of how Piko works.
 
-The [Piko server](./server/server.md) is a HTTP(S) reverse proxy that forwards
-requests to upstream listeners. Unlike a traditional reverse proxy, Piko never
-opens a connection directly to your upstream. Instead upstreams listeners open
-outbound-only connections to the server and listen on a particular endpoint.
-The server then forwards requests to a listening upstream via its outbound
-connection to the server.
+The [Piko server](./server/server.md) is a reverse proxy that forwards
+connections to upstream listeners. Unlike a traditional reverse proxy, Piko
+never opens a connection directly to your upstream. Instead upstreams listeners
+open outbound-only connections to the server and listen on a particular
+endpoint. The server then forwards requests to a listening upstream via its
+outbound connection to the server.
 
 This means upstreams can run anywhere without requiring a public route, as long
 as they can open a connection to the Piko server.

docs/server/server.md

@@ -1,6 +1,6 @@
 # Server
 
-The Piko server is responsible for forwarding requests from downstream clients
+The Piko server is responsible for forwarding traffic from downstream clients
 to upstream listeners. See [How Piko Works](../how-piko-works.md) for details.
 
 The server is designed to be hosted as a cluster of nodes behind a HTTP load

docs/tutorials/tcp-forwarding.md

@@ -0,0 +1,108 @@
+# TCP Forwarding
+
+This tutorial shows you how to use Piko to forward TCP traffic to upstream
+listeners.
+
+Piko supports proxying TCP traffic, though unlike HTTP it requires using either
+[Piko forward](../forward/forward.md) or the
+[Go SDK](../sdk/go-sdk.md) to map the desired local TCP port to the target
+endpoint (as there's no way to identify the target endpoint using raw TCP).
+
+As an example, this runs an upstream Redis server then connects to it via
+Piko.
+
+### Prerequisites
+
+* [Install Piko](./install.md)
+* [Install Redis](https://redis.io/docs/latest/operate/oss_and_stack/install/install-redis/)
+
+## Piko Server
+
+Start a [Piko server](../server/server.md) node:
+```bash
+$ piko server
+```
+
+See the server [documentation](../server/server.md) for details.
+
+## Redis Upstream
+
+Start the Redis server:
+```bash
+$ redis-server
+```
+
+This will listen for connections on port `6379` by default.
+
+Next run the [Piko agent](../agent/agent.md) alongside the server to register a
+Piko endpoint, `my-redis-endpoint`, and forward incoming TCP connections to
+port `6379`:
+```bash
+$ piko agent tcp my-redis-endpoint 6379
+```
+
+To keep the tutorial simple, you can run the Redis server and Piko agent
+locally, though using Piko you could run them anywhere, as long as the agent
+can open an outbound connection to the Piko server. Such as they could be
+running behind a firewall or NAT blocking all incoming traffic.
+
+Instead of using the Piko agent, you could also use the
+[Go SDK](../sdk/go-sdk.md):
+```go
+var opts []piko.Option
+// ...
+
+client := piko.New(opts...)
+if err := client.ListenAndForward(
+    context.Background(),
+    "my-redis-endpoint",
+    "localhost:6379",
+);  err != nil {
+    panic("forward: " + err.Error())
+}
+```
+
+## Connect
+
+Finally you can open a TCP connection to the endpoint `my-redis-endpoint` using
+[Piko forward](../forward/forward.md).
+
+`piko forward` listens on a local TCP port and forwards connections to the
+configured endpoint. Run `piko forward` to listen on port `7000` and forward to
+endpoint `my-redis-endpoint`:
+```bash
+$ piko forward tcp 7000 my-redis-endpoint
+```
+
+You can then connect to port `7000` and your connection will be forwarded to
+your registered upstream listener:
+```bash
+$ redis-cli -p 7000 PING
+PONG
+```
+
+Alternatively you can connect to Redis directly from your application using
+the Go SDK. The Piko client has a
+`Dial(ctx context.Context, endpointID string) (net.Conn, error)` method
+that returns a `net.Conn`. Such as connecting with the
+[Redis Go](https://github.com/redis/go-redis) client:
+```go
+var opts []piko.Option
+// ...
+
+client := piko.New(opts...)
+
+// Use a custom dialer that connects to the upstream endpoint via Piko.
+dialer := func(ctx context.Context, network, addr string) (net.Conn, error) {
+    return client.Dial(ctx, addr)
+}
+// github.com/redis/go-redis/v9
+rdb := redis.NewClient(&redis.Options{
+    Addr:   "my-redis-endpoint",
+    Dialer: dialer,
+})
+
+if err := rdb.Ping(context.Background()).Err(); err != nil {
+    panic("ping: " + err.Error())
+}
+```