Vedran

Polkadot chain load balancer.

Architecture

Vedran loadbalancer is used in conjunction with Vedran daemon. Suppose the node owner wants to register to loadbalancer, than it is required to install and run Vedran daemon. Daemon executes the registration process and starts providing all relevant information (ping, metrics) to the Vedran loadbalancer. Please check Vedran daemon repo for more details on the daemon itself.

Demo

This is dockerized demo of entire setup with loadbalancer, node and daemon

Requirements

• Install Docker Engine
• Install Docker Compose

Run demo with docker-compose up

After all components have been started and node has sent first valid metrics report (after 30 seconds), you can invoke RPC methods on localhost:4000 using HTTP requests or on localhost:4000/ws using WebSocket request Metrics can be seen on localhost:3000 hosted grafana under vedran-dashboard

You can check available rpc methods with:

curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "rpc_methods"}' http://localhost:4000

This demo starts five separate dockerized components:

• Polkadot node (repository)
• Vedran daemon (repository)
• Vedran loadbalancer (port: 4000)
• Prometheus server (port: 9090) - scrapes metrics from vedran's /metrics endpoint
• Grafana (port: 3000) - Visualizes metrics Example dashboard

Trigger Manual Payout

Our compose setup runs dev chain, and our load balancer uses Allice account to do payout, so you don't have to obtain dev DOTs. Polkadot node operator is Bob (he received payout from Allice). Load balancer in this setup runs payout daily, if you don't want to wait, you can run following command which will create an additional container (in compose network) and trigger payout from Allice account:

docker run --network vedran_default nodefactory/vedran:latest payout --private-key 0xe5be9a5092b81bca64be81d212e7f2f9eba183bb7a90954f7b76361f6edb5c0a --payout-reward 100 --load-balancer-url "http://vedran:4000/ws"

Get vedran binary releases

Download prebuild binary from releases. Be careful to chose appropriate binary depending on your OS. For more details on how to run vedran loadbalancer see Starting loadbalancer part.

Get vedran package

Alternatively, it is possible to get vedran golang package:

1. Install Golang 1.15 or greater
2. Run the command below

go get github.com/NodeFactoryIo/vedran

3. Run vedran from your Go bin directory. For linux systems it will likely be:

~/go/bin/vedran

Note that if you need to do this, you probably want to add your Go bin directory to your $PATH to make things easier!

Starting loadbalancer

First download latest prebuilt binaries from releases and unzip it.

Load balancer is started by invoking start command.

For example ./vedran start --auth-secret=supersecret --private-key=lb-wallet-private-key.

For more information you can always run vedran with --help flag. For list of all commands run vedran --help or for list of all options for specific command run vedran start --help.

Load balancer will expose Polkadot RPC API on port 80 by default (can be changed using flag --server-port)

Vedran loadbalancer supports both HTTP and Websockets protocols for Polkadot RPC API.

• HTTP - available on root path /
• WS - available on separate path /ws

For production use certificates (e.g. https://certbot.eff.org/) should be generated and passsed via flags: --key-file, --cert-file and port changed to 443

Start command will start application on 2 ports that need to be exposed to public:

1. RPC entrypoint to nodes and API for nodes to register to load balancer (default: 80)
2. HTTP tunnel server for creating tunnels between the node and load balancer so node operators don't to have expose nodes to public network (default: 5223)

Required flags

--auth-secret - authentication secret used for generating tokens

--private-key - loadbalancers wallet private key, used for sending founds on payout

Most important flags

Flag	Description	Default value
--server-port	port on which RPC API is exposed	80
--public-ip	public IP address of loadbalancer	uses multiple services to find out public IP
--cert-file	path to SSL certification file	uses HTTP
--key-file	path to SSL private key file	uses HTTP
--tunnel-port	port on which tunnel server is listening for connect requests	5223
--tunnel-port-range	range of ports that will be used for creating tunnels	20000:30000

Other flags

Flag	Description	Default value
--name	public name for load balancer	autogenerated name is used
--capacity	maximum number of nodes allowed to connect	unlimited capacity
--whitelist	comma separated list of node id-s, if provided only these nodes will be allowed to connect. This flag can't be used together with --whitelist-file flag, only one option for setting whitelisted nodes can be used	all nodes are whitelisted
--whitelist-file	path to file with node id-s in each line, if provided only these nodes will be allowed to connect. This flag can't be used together with --whitelist flag, only one option for setting whitelisted nodes can be used	all nodes are whitelisted
--fee	value between 0-1 representing fixed fee percentage that loadbalancer will take	0.1 (10%)
--selection	type of selection that is used for selecting nodes on new request, valid values are round-robin and random	round-robin
--payout-interval	automatic payout interval specified as number of days, for more details see payout instructions	-
--payout-reward	defined reward amount that will be distributed on the payout (amount in Planck), for more details see payout instructions	-
--lb-payout-address	address on which load balancer fee will be sent	-
--log-level	log level (debug, info, warn, error)	error
--log-file	path to file in which logs will be saved	stdout
--root-dir	root directory for all generated files (e.g. database file, log file)	uses current directory

Obtaining DOTs

If you want to do anything on Polkadot, Kusama, or Westend, then you'll need to get an account and some DOT, KSM, or WND tokens, respectively. When initializing payout, you will provide loadbalancer with created account and from this account rewards will be sent to connected nodes on payout.

For Westend's WND tokens, see the faucet instructions on the Wiki.

Payouts

Automatic payout

When starting vedran loadbalancer it is possible to configure automatic payout by providing these flags:

--private-key - loadbalancers wallet private key (string representation of hex value prefixed with 0x), used for sending rewards on the payout

--payout-interval - automatic payout interval specified as number of days

--payout-reward - defined total reward amount that will be distributed on the payout (amount in Planck). If omitted, the entire balance of lb wallet will be used as a total reward, and in this case --lb-payout-fee-address must be set

--lb-payout-address - address on which load balancer fee will be sent. If omitted, load balancer fee will be left on load balancer wallet after payout. This flag is required if --payout-reward is not set (or set to -1)

If all flags have been provided, then each {payout-interval} days automatic payout will be started.

Manual payout

It is possible to run payout script at any time by invoking vedran payout command through the console.

--private-key - loadbalancers wallet private key (string representation of hex value prefixed with 0x), used for sending rewards on the payout

--payout-reward - defined total reward amount that will be distributed on the payout (amount in Planck). If omitted, the entire balance of lb wallet will be used as a total reward, and in this case --lb-payout-fee-address must be set

--lb-payout-fee-address - address on which load balancer fee will be sent. If omitted, load balancer fee will be left on load balancer wallet after payout. This flag is required if --payout-reward is not set (or set to -1)

Additionally, it is possible to change URL on which payout script will connect with loadbalancer when executing transactions by setting flag (default value will be http://localhost:80)

--load-balancer-url - loadbalancer URL

Get private key

You can use subkey tool to get private key for your wallet.

After installing subkey tool call subkey inspect "insert your mnemonic here". You can find private key as Secreet seed. See example output of subkey command:

  Secret seed:      0x1a84771145cdcee05e49142aaff2e5d669ce4b29344a09b973b751ae661acabf
  Public key (hex): 0xa4548fa9b3b15dc4d1c59789952f0ccf6138dd63faf802637895c941f0522d35
  Account ID:       0xa4548fa9b3b15dc4d1c59789952f0ccf6138dd63faf802637895c941f0522d35
  SS58 Address:     5FnAq6wrMzri5V6jLfKgBkbR2rSAMkVAHVYWa3eU7TAV5rv9

Monitoring

Monitoring is done via grafana and prometheus which are expected to be installed.

Installation

• Grafana installation
• Prometheus installation

Configuration

• Grafana

  Should be configured to fetch data from prometheus server as data source Tutorial.

  Should have a dashboard that visualizes data scraped from prometheus server. Example configuration can be found here and can be imported like this.

• Prometheus

  Prometheus should be configured to scrape metrics from vedran's /metrics endpoint via prometheus .yml configuration. Example of which can be found here

Stats collection intervals

It is possible to change default calculation intervals for specific statistic categories. These intervals define how often will these statistics be recalculated.

Valid values are time intervals such as "5s", "1.5h" or "2h45m". Valid time units are "ms", "s", "m", "h"

ENV	Description	Default value
PROM_FEE_STATS_INTERVAL	payout fees for nodes and load balancer	12 hours
PROM_NODE_STATS_INTERVAL	active and penalized nodes	15 seconds
PROM_REQUEST_STATS_INTERVAL	successful and failed requests	15 seconds
PROM_PAYOUT_STATS_INTERVAL	payout distribution	1 minute

Vedran loadbalancer API

POST api/v1/nodes

Register node to loadbalancer. Body should contain details about node:

{
  "id": "string",
  "config_hash": "string",
  "payout_address": "string"
}

Returns token used for invoking rest of API and tunnel_server_address on which daemon can open tunnel toward loadbalancer.

{
  "token": "string",
  "tunnel_server_address": "string"
}

---

POST api/v1/nodes/pings

Ping loadbalancer from node. Auth token should be in header as X-Auth-Header.

---

PUT api/v1/nodes/metrics

Send metrics for node. Auth token should be in header as X-Auth-Header. Body should contain metrics as:

{
  "peer_count": "int32",
  "best_block_height": "int64",
  "finalized_block_height": "int64",
  "target_block_height": "int64",
  "ready_transaction_count": "int32"
}

---

GET api/v1/stats

Returns statistics for all nodes (mapped on node payout address).

{
  "stats": {
    "node_1_payout_address": {
      "total_pings": "float64",
      "total_requests": "float64"
    },
    "node_2_payout_address": {
      "total_pings": "float64",
      "total_requests": "float64"
    }
  }
}

---

GET api/v1/stats/lb

Returns statistics on reward distribution between load balancer and nodes.

{
  "lb_fee": "string",
  "nodes_fee": "string"
}

Development

Clone

git clone [email protected]:NodeFactoryIo/vedran.git

Lint

Golangci-lint is expected to be installed.

make lint

Build

make build

Test

make test

License

This project is licensed under Apache 2.0:

• Apache License, Version 2.0, (LICENSE-APACHE)