Initial commit

.gitignore

@@ -0,0 +1,10 @@
+/build
+/vendor
+.idea
+*.iml
+.terraform/
+.DS_Store
+*.plan
+*.tfvars
+*.tfstate
+*.code-workspace

CONTRIBUTING.md

@@ -0,0 +1,13 @@
+# Contributing
+
+Thank you for considering contributing to this project!
+
+## Contribution steps
+1. If you need to push a trivial change, go ahead, and create a PR.
+2. For bigger changes, it's generally best to open an issue describing the bug or feature request you are intending to solve.
+3. To avoid any duplicated work, claim any open issue by putting your name in the body of the issue and mention that you want to take this up; in case there is no open issue yet, open one and claim it (or not).
+
+## Pull Request Checklist
+1. Fork this repository, create a feature branch, and when you are done subsequently create an upstream PR.
+2. Make sure you leave an good explaintatory description so people understand the <i>what/why</i>, and <i>how</i>
+3. Make sure your PR passes all the tests

Makefile

@@ -0,0 +1,30 @@
+NAME := klaabu
+
+BUILD  := ${CURDIR}/build
+BIN    := ${BUILD}/bin/${NAME}
+
+.PHONY: clean tidy compile test run
+.DEFAULT_GOAL := build
+
+clean:
+	rm -rf ${BUILD}
+
+tidy:
+	go mod tidy -v
+
+fmt:
+	go fmt github.com/erikkn/klaabu/...
+
+vendor:
+	go mod vendor
+
+compile:
+	go build -mod=readonly -o ${BIN} cli/*.go
+
+test:
+	go test -v ./...
+
+build: tidy fmt vendor compile
+
+run: build
+	${BIN}

README.md

@@ -0,0 +1,77 @@
+# Klaabu
+
+Klaabu is an IP network documenting tool, usually referred to as an IPAM (IP Address Management) tool, with a very strong focus on simplicity, convenience, and operating in Cloud-Native environments.
+
+## Preface
+A big THANKS to **Taras Burko**, for mentoring me, and for all your time & effort in writing this program with me.
+
+A big THANKS to **Taavi Tuisk** for mentoring & helping me these past years.
+
+## Why invent something new?
+There are a ton of different IPAM tools available out there, but with Klaabu you don’t have to <i>host</i> and <i>maintain</i> yet another tool. NetBox, for instance, is a great tool, but in case NetBox is your source-of-truth, you will need to have some sort of an SLA, strong observability, DR plan, and more.
+Klaabu tries to solve this unnecessary TOIL by using a file-based schema that you can store in Git and a powerful `validate` function to make sure your schema is in the right state. The validate function can be used in your CI pipeline, or standalone, to make sure you don’t have duplicated, overlapping CIDRs, or some other schema mistakes. The combination of storing your schema in Git and the validate function offers exactly the kind of workflow you are already used to, with code reviews, audit trails, and more.
+
+### Why another markup language?
+The first couple of iterations of the Klaabu program were using a YAML-based schema. However, after using Klaabu in a live environment (with 100 VPCs, ~500 subnets, BGP ASNs, and more) we decided that this is not the right format. Using a YAML-based schema in such an environment will result in hundreds of lines, which becomes very hard to read. We believe that the ‘Klaabu Markup Language’ (`kml`) is much easier and more convenient to the user.
+
+### Terraform
+Apart from some other powerful (CLI) functions, Klaabu offers the `export-terraform` function which exports your schema to a Terraform supported module. In turn, you can import this module from any other Terraform module and lookup the CIDRs, attributes, or labels. Having a single source-of-truth that also works natively with Terraform is very convenient and will simplify the usage of your VPC, Subnet, SG, and other Terraform modules a lot.
+
+## Installation
+With GO installed:
+
+```bash
+go get github.com/erikkn/klaabu
+```
+
+Alternatively, you can also import the package directly:
+
+```bash
+import “github.com/erikkn/klaabu”
+```
+
+In case you want to build the package yourself
+
+```bash
+make build
+```
+
+Last but not least, you can also just download the binary directly from the release page.
+
+## CLI usage
+```
+Usage: klaabu <command> [args]
+```
+
+At the moment the Klaabu CLI supports the following commands:
+* `find`: recursively search the schema for any object that matches your search pattern.
+* `get`: in contrast to the `find` command, `get` retrieves a single object in the schema.
+* `list`: the `list` command shows all the child objects of a certain instance.
+* `space`: use this command to see the available IP space within a certain prefix/object.
+* `init`: initializes a new schema.
+* `validate`: validates your schema, including the actual content of the objects (e.g. valid CIDR, no overlapping CIDRs, and more).
+* `fmt`: used to rewrite Klaabu configuration files to a canonical format and style.
+* `export-terraform`: exports your schema to a valid Terraform module; your schema is stored in a file with the `tf.json` notation, which is a valid input module for Terraform.
+
+### Examples
+The examples assume your schema lives in the current working directory / you have an environment variable set pointing to the location of your schema.
+
+```bash
+klaabu space 192.168.0.0/20
+```
+
+```bash
+klaabu find -label az=euc1-az1
+
+klaabu find -label vpc=foobar,env=production
+```
+
+## Workflow
+- Create a new private repository and store your `schema.kml` in there
+- Use your IDE to add a new CIDR to your schema
+- Run `klaabu fmt` to produce configuration files that conform to the imposed style
+- Run `klaabu validate` to make sure your schema is in a valid state and that you don’t have overlapping CIDRs for instance
+- Follow your personal/company’s process for committing and merging your changes. You probably want to follow the traditional code review process with a CI pipeline that also uses the validate function.
+
+## Contributing
+Check out the [CONTRIBUTING](./CONTRIBUTING.md/) guide if you want to contribute.

cli/find.go

@@ -0,0 +1,78 @@
+package main
+
+import (
+	"flag"
+	"log"
+	"os"
+	"strings"
+
+	"github.com/erikkn/klaabu/klaabu"
+)
+
+// The purpose of the 'find' command is more to 'explore' the schema and searching for something in a broad way. 'GET', however, is used if you already know what you need to lookup and do a specific search.
+
+func findCommand() {
+	find := flag.NewFlagSet("find", flag.ExitOnError)
+
+	schemaFileName := find.String("schema", "schema.kml", "(Optional) Schema file path (default './schema.kml')")
+	termsString := find.String("label", "", "(Optional) Filter by labels. Example: 'az=euc1-az1,type=vpc'. No filtering when empty.")
+	err := find.Parse(os.Args[2:])
+
+	parentId := find.Arg(0)
+
+	if err != nil || len(os.Args) < 2 {
+		log.Printf("Usage: klaabu find [OPTIONS] [PARENT_ID] \n\n Subcommands: \n")
+		find.PrintDefaults()
+		os.Exit(1)
+	}
+
+	schema, err := klaabu.LoadSchemaFromKmlFile(*schemaFileName)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	var parent *klaabu.Prefix
+	if parentId == "" {
+		parent = schema.Root
+	} else {
+		parent = schema.PrefixById(parentId)
+		if parent == nil {
+			log.Fatalln(err)
+		}
+	}
+
+	terms := make([]*klaabu.LabelSearchTerm, 0)
+
+	if *termsString != "" {
+		termStrings := strings.Split(*termsString, ",")
+		for _, termString := range termStrings {
+			termSlice := strings.Split(termString, "=")
+			var term klaabu.LabelSearchTerm
+			if len(termSlice) == 2 {
+				term.Key = strings.TrimSpace(termSlice[0])
+				value := strings.TrimSpace(termSlice[1])
+				term.Value = &value
+			} else if len(termSlice) == 1 {
+				term.Key = strings.TrimSpace(termSlice[0])
+				term.Value = nil
+			}
+			terms = append(terms, &term)
+		}
+	}
+
+	println(">>> num terms: ", len(terms))
+	for _, term := range terms {
+		println(">>> term: ", term.Key, term.Value)
+	}
+
+	cidrs := parent.FindPrefixesByLabelTerms(terms)
+
+	for _, v := range cidrs {
+		labels := make([]string, 0, len(v.Labels))
+		for k, v := range v.Labels {
+			labels = append(labels, k+"="+v)
+		}
+
+		log.Printf("%s: %s [%s]\n", string(v.Cidr), strings.Join(v.Aliases, "|"), strings.Join(labels, ","))
+	}
+}

cli/fmt.go

@@ -0,0 +1,56 @@
+package main
+
+import (
+	"flag"
+	"io/ioutil"
+	"log"
+	"os"
+
+	"github.com/erikkn/klaabu/klaabu"
+)
+
+func fmtCommand() {
+	flags := flag.NewFlagSet("fmt", flag.ExitOnError)
+
+	schemaFileName := flags.String("schema", "schema.kml", "Schema file path")
+	flags.Parse(os.Args[2:])
+
+	node, err := klaabu.LoadKmlFromFile(*schemaFileName)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	schema, err := klaabu.KmlToSchema(node)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	err = schema.Validate()
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	originalBytes, err := ioutil.ReadFile(*schemaFileName)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	wf, err := os.OpenFile(*schemaFileName, os.O_WRONLY, 0)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	err = klaabu.MarshalKml(node, wf)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	updatedBytes, err := ioutil.ReadFile(*schemaFileName)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	if string(originalBytes) != string(updatedBytes) {
+		log.Println(*schemaFileName)
+	}
+}

cli/get.go

@@ -0,0 +1,35 @@
+package main
+
+import (
+	"flag"
+	"github.com/erikkn/klaabu/klaabu"
+	"log"
+	"os"
+)
+
+func getCommand() {
+	get := flag.NewFlagSet("get", flag.ExitOnError)
+	schemaFileName := get.String("schema", "schema.kml", "(Optional) Schema file path (defaults to ./schema.kml)")
+	err := get.Parse(os.Args[2:])
+
+	prefixId := get.Arg(0)
+
+	if err != nil || len(os.Args) < 3 {
+		log.Printf("Usage: klaabu get [OPTIONS] PREFIX_ID \n\n Subcommands: \n")
+		get.PrintDefaults()
+		os.Exit(1)
+	}
+
+	schema, err := klaabu.LoadSchemaFromKmlFile(*schemaFileName)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	prefix := schema.PrefixById(prefixId)
+	if prefix == nil {
+		log.Fatalf("not found: %s", prefixId)
+	}
+
+	// TODO: Output all the individual fields, might change again so a `todo` for later.
+	log.Println(prefix)
+}

cli/init.go

@@ -0,0 +1,54 @@
+package main
+
+import (
+	"flag"
+	"log"
+	"os"
+	"strings"
+
+	"github.com/erikkn/klaabu/klaabu"
+)
+
+func initCommand() {
+	init := flag.NewFlagSet("init", flag.ExitOnError)
+
+	// TODO change to schema.kml
+	schemaFileName := init.String("schema", "schema-new.kml", "Schema file path")
+	version := init.String("version", "v1", "Schema version")
+	labelsString := init.String("labels", "", "(Optional) Schema labels, e.g. 'org=acme,env=staging'")
+
+	init.Parse(os.Args[2:])
+
+	if len(os.Args) < 3 {
+		log.Printf("Usage: klaabu init [OPTIONS] \n\n Subcommands: \n")
+		init.PrintDefaults()
+		os.Exit(1)
+	}
+
+	if *version != "v1" {
+		log.Printf("%s is not a valid schema version \n\n", *version)
+		log.Printf("Usage: klaabu init [OPTIONS] \n\n Subcommands: \n")
+		init.PrintDefaults()
+	}
+
+	labels := make(map[string]string)
+	if *labelsString != "" {
+		for _, v := range strings.Split(*labelsString, ",") {
+			pair := strings.Split(v, "=")
+			if len(pair) != 2 {
+				log.Fatalln("error with labels")
+			}
+
+			labels[strings.TrimSpace(pair[0])] = strings.TrimSpace(pair[1])
+		}
+	}
+
+	schema := klaabu.NewSchema(labels)
+
+	err := klaabu.WriteSchemaToFile(schema, schemaFileName)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	log.Printf("Successfully created your new schema '%s' \n", *schemaFileName)
+}