Chisel Kubernetes Operator ⚒️

Use a VPS (or any other machine) as a reverse proxy for your Kubernetes cluster, without paying the extra 25$ a month!

This is a Kubernetes operator for Chisel. It allows you to use Chisel as a LoadBalancer provider for your Kubernetes cluster, similar to inlets-operator

View the documentation: https://chisel.fyralabs.com

Features

• Automatic provisioning of exit nodes on cloud providers
• Free and open-source
• TCP and UDP support
• INFINITE tunnels! (You can create as many tunnels as you want, as long as you have enough bandwidth)
• Use any machine as an exit node
• Hybrid cloud support (You can use multiple cloud providers at once)

TODO

• Authentication
• Multiple tunnel services per exit node (so you don't have to pay for multiple VMs)
• Extra configuration options
• TCP/UDP support
• Multiple IPs per exit node
• Multiple exit nodes support
• Cloud provisioner (like inletsctl/inlets-operator)

Why?

The issue

If you want to expose a service to the internet, you need a public IP address.

However, if you're running a cluster inside a NATed network (like a home network), you can't just expose your service to the internet. You need to port forward your service to the internet. This might be fine, but then there's another problem:

The world's running out of IPv4 addresses. This means that ISPs are starting to charge extra for public IP addresses, and most home networks are locked behind a CGNAT, and requires you to pay extra for a public IP address.

You could just use an IPv6 address, but most ISPs don't support IPv6 yet, and K8s with IPv6 is kind of a pain to set up.

Disclaimer: We are not responsible for any problems arising from abuse of this software. Please proxy responsibly. See LICENSE.md for more details.

The other issue

You might say, "What about Inlets?" Inlets is a great solution, but it comes with a couple caveats:

• You need to pay for an inlets PRO license to even use it (It's a proprietary solution)
• You still need to pay for the exit node on top of the inlets PRO license

The solution

Introducing the Fyra Labs Chisel Operator! This operator provides a replacement for inlets, but free and open-source!

This operator makes use of the Chisel tunnel to expose your services to the internet through SSH. And you can use any machine as an exit node!

Since Chisel routes traffic through SSH, all traffic is encrypted and secure. The Chisel Operator also supports automatic provisioning of exit nodes on cloud providers, so you get basically the same functionality, but free!

---

While this code is free and open-source, we still accept donations! If you really like this project, please consider donating to us on GitHub Sponsors :3

How does it work?

This operator works similarly to inlets-operator. It watches for LoadBalancer services, then allocates an exit node's IP address for that service and creates a Chisel client deployment on that node. The Chisel client will then connect to the Chisel server running on the exit node, and the service will be exposed on the exit node's IP address.

Alternatives

SaaS solutions

• Cloudflare (Argo) Tunnel - Cloudflare's solution to this problem. It is free and also open-source, but it only works on Cloudflare-managed domains and requires you to use Cloudflare's DNS service. But it comes with a couple caveats:
  • Only HTTP and HTTPS is supported for exposing services. If you want to expose a TCP service, you must connect to it through Cloudflare Tunnel on the client.
  • According to Cloudflare's Terms of Service, you are not allowed to use Cloudflare's proxies to stream video or audio content. This means that you cannot use Cloudflare Tunnel to expose your Plex or Jellyfin server, or any other media streaming service. This is also the reason I started this project.
• ngrok - ngrok is a proprietary solution that allows you to expose your local service to the internet. It is free to use, but it comes with a couple caveats:
  • Only HTTP and HTTPS is supported for exposing services. TCP traffic is supported through a paid plan.
  • Limited bandwidth
  • Custom domains are only available on a paid plan

Self-hosted solutions

• Run Chisel manually on your exit node - This is the most straightforward solution. You can simply run Chisel manually on your exit node without using this operator. However, this solution is hard to automate, which is the point of this project.
• frp - Fast reverse proxy, requires manual configuration of server and client.
• inlets - Bite the bullet and pay for an inlets PRO license. inlets-pro allows you to automatically provision exit nodes on cloud providers, but it is a proprietary solution and requires you to pay a monthly fee.
• rathole - Similar to frp, written in Rust.

VPNs and overlay networks

• Tailscale - VPN solution that allows you to connect your devices in one big overlay network. Also has Funnel, a reverse proxy solution that allows you to expose your local service to the internet. Self-hostable control plane is available, but default is to use Tailscale's hosted control plane.
• ZeroTier - Similar to Tailscale, Under BSD license, Can connect to multiple networks at once.

---

Find more alternatives here

How do I use it?

There are two ways to use this operator:

• Manually provision the exit (reverse proxy) node, and let the operator manage the Chisel client deployment
• Let the operator provision exit nodes on a cloud provider of your choice. The operator currently supports the following cloud providers:
  • DigitalOcean
  • Linode (Currently only on regions with Metadata services)
  • AWS

Cluster Installation

Install using the Kustomize config from the stable branch:

kubectl apply -k https://github.com/FyraLabs/chisel-operator?ref=stable

Or if you would like to go straight to the latest commit:

kubectl apply -k https://github.com/FyraLabs/chisel-operator

A Helm chart will be available soon.

Usage

Operator-managed exit nodes

This operator can automatically provision exit nodes on cloud providers.

To use this feature, you must first create a ExitNodeProvisioner resource. This resource contains the configuration for the cloud provider, and the operator will use this resource to provision exit nodes.

apiVersion: chisel-operator.io/v1
kind: ExitNodeProvisioner
metadata:
  name: digitalocean
  namespace: default
spec:
  # Cloud provider configuration, must be one per resource
  # Valid values are DigitalOcean, Linode, AWS
  DigitalOcean:
    # Reference to a secret containing the DigitalOcean API token
    # with key `DIGITALOCEAN_TOKEN`
    # Must be in the same namespace as the ExitNodeProvisioner
    auth: digitalocean
    region: sgp1

Now, you can go with one of the two routes:

Automatic provisioning per service

Chisel Operator can automatically allocate cloud exit nodes for services, if you set an annotation on a LoadBalancer service.

apiVersion: v1
kind: Service
metadata:
  name: whoami
  annotations:
    # If the provisioner is in another namespace, you need to specify that
    # for example, with a namespace of example, the value would be example/digitalocean
    chisel-operator.io/exit-node-provisioner: "digitalocean"
spec:
  selector:
    app: whoami
  ports:
    - port: 80
      targetPort: 80
  type: LoadBalancer

This will create a new ExitNode resource named after the service, and the operator will automatically allocate an exit node for that service.

This is useful if you want to just allocate an entire exit node for a single service.

Manually-allocated, but operator-managed exit nodes

You can also manually allocate exit nodes, but still let the operator manage the Chisel client deployment. This is useful if you want to allocate a single exit node for multiple services, in case you're on a budget and don't want to pay for multiple exit nodes for each service.

To do this, create an ExitNode resource with the annotation chisel-operator.io/exit-node-provisioner set to the name (and namespace if the provisioner is in a different namespace) of the ExitNodeProvisioner resource.

apiVersion: chisel-operator.io/v1
kind: ExitNode
metadata:
  name: my-exit-node
  namespace: default
  annotations:
    # If the provisioner is in another namespace, you need to specify that
    # for example, with a namespace of example, the value would be example/digitalocean
    chisel-operator.io/exit-node-provisioner: "digitalocean"
spec:
  # IP address of exit node
  # In this case, we will leave this as a blank string, and let the operator allocate an IP address for us
  host: ""
  # Control plane socket port
  port: 9090
  # Name of the secret containing the auth key
  # This is not required, but recommended
  # If not set, the operator will automatically generate a secret for you
  # auth: cloud-test-auth

Now, to use this exit node, you can create a LoadBalancer service with the annotation chisel-operator.io/exit-node-name set to the name of the ExitNode resource.

apiVersion: v1
kind: Service
metadata:
  name: whoami
  annotations:
    chisel-operator.io/exit-node-name: "cloud-test"
spec:
  selector:
    app: whoami
  ports:
    - port: 80
      targetPort: 80
  type: LoadBalancer

NOTE: You can also use this for manually-provisioned exit nodes

NOTE: If you do not specify the annotation, the operator will allocate the first available exit node for you.

Provisioning Chisel manually

NOTE: You can skip this step if you're using the cloud provisioner.

To install Chisel, install the Chisel binary on the machine using this script:

curl https://i.jpillora.com/chisel! | bash

OPTIONAL: You should create a systemd service for Chisel so it can run in the background. Create a file called /etc/systemd/system/chisel.service with the following contents:

[Unit]
Description=Chisel Tunnel
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=0

[Install]
WantedBy=multi-user.target

[Service]
Restart=always
RestartSec=1
User=root
# You can add any additional flags here
# This example uses port 9090 for the tunnel socket. `--reverse` is required for our use case.
# `foo:bar` is an example of the authentication credentials.
# The format is `username:password`.
# You may also create an auth file with the `--authfile` flag.
ExecStart=/usr/local/bin/chisel server --port=9090 --reverse --auth foo:bar

Then run systemctl daemon-reload and systemctl enable --now chisel.service to enable and start the service.

Best Practices

You should always secure your Chisel server with a username and password. You can authenticate to the server by creating a secret in the same namespace as the ExitNode with a key called auth, and setting the auth field in the ExitNode to the name of the secret. The secret should be a string of username:password in plain text.

Currently, you should use the public IP address of your exit node as the host field in the ExitNode resource. This is because the operator currently does not support using a domain name as the host field. This will be fixed in the future.

Exposing services

It is recommended you use an Ingress controller to expose your services. This greatly simplifies the process for exposing other services, as you only need to expose the Ingress controller's HTTP(S) ports.

How do I contribute?

Feel free to open a pull request or an issue if you have any suggestions or improvements. We're open to any ideas!

Legal

Fyra Labs disclaims all liability related to usage of chisel-operator. Please proxy responsibly. See LICENSE.md for additional details. Contact [email protected] with complaints.