Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial project setup and provider interface definition #23

Merged
merged 4 commits into from
Jul 28, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: lint

on: [pull_request]

jobs:
lint:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Lint
run: make lint
12 changes: 12 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: test

on: [pull_request]

jobs:
test:
name: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Test
run: make test
32 changes: 32 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
run:
timeout: 10m
tests: false
allow-parallel-runners: true
skip-dirs:
- pkg/provider/mock

issues:
max-same-issues: 0
max-issues-per-linter: 0
exclude-rules:
- linters: [golint]
text: "should not use dot imports|don't use an underscore in package name"

linters-settings:
funlen:
lines: 110
statements: 60
staticcheck:
go: "1.16"
stylecheck:
go: "1.16"

linters:
enable-all: true
disable:
- gochecknoglobals
- gochecknoinits
- wsl
- tagliatelle
- exhaustivestruct
- godox # remove in the future
86 changes: 85 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,9 +1,93 @@
# Build Information
build_date := $(shell date +%Y-%m-%dT%H:%M:%SZ)
git_commit := $(shell git rev-parse --short HEAD)
OS := $(shell go env GOOS)
ARCH := $(shell go env GOARCH)
UNAME := $(shell uname -s)

# versions
BUF_VERSION := v0.43.2

# Directories
TOOLS_DIR := hack/tools
TOOLS_BIN_DIR := $(TOOLS_DIR)/bin
TOOLS_SHARE_DIR := $(TOOLS_DIR)/share
TEST_E2E_DIR := test/e2e

.PHONY: test-e2e ## Run e2e tests
PATH := $(abspath $(TOOLS_BIN_DIR)):$(PATH)

$(TOOLS_BIN_DIR):
mkdir -p $@

$(TOOLS_SHARE_DIR):
mkdir -p $@

# Binaries
GOLANGCI_LINT := $(TOOLS_BIN_DIR)/golangci-lint
GINKGO := $(TOOLS_BIN_DIR)/ginkgo
BUF := $(TOOLS_BIN_DIR)/buf
MOCKGEN:= $(TOOLS_BIN_DIR)/mockgen

.DEFAULT_GOAL := help

##@ Generate

.PHONY: generate
generate: $(BUF) $(MOCKGEN) ## Generate code
go generate ./...

##@ Linting

.PHONY: lint
lint: $(GOLANGCI_LINT) ## Lint
$(GOLANGCI_LINT) run -v --fast=false

##@ Testing

.PHONY: test
test: ## Run unit tests
go test ./...

.PHONY: test-e2e
test-e2e: ## Run e2e tests
go test -timeout 30m -p 1 -v -tags=e2e ./test/e2e/...

.PHONY: compile-e2e
compile-e2e: # Test e2e compilation
go test -c -o /dev/null -tags=e2e ./test/e2e

##@ Tools binaries

$(GOLANGCI_LINT): hack/tools/go.mod # Get and build golangci-lint
cd $(TOOLS_DIR); go build -tags=tools -o $(subst hack/tools/,,$@) github.com/golangci/golangci-lint/cmd/golangci-lint

$(GINKGO): hack/tools/go.mod # Get and build gginkgo
cd $(TOOLS_DIR); go build -tags=tools -o $(subst hack/tools/,,$@) github.com/onsi/ginkgo/ginkgo

$(MOCKGEN): hack/tools/go.mod # Get and build mockgen
cd $(TOOLS_DIR); go build -tags=tools -o $(subst hack/tools/,,$@) github.com/golang/mock/mockgen

BUF_TARGET := buf-Linux-x86_64.tar.gz

ifeq ($(OS), darwin)
BUF_TARGET := buf-Darwin-x86_64.tar.gz
endif


BUF_SHARE := $(TOOLS_SHARE_DIR)/buf.tar.gz
$(BUF_SHARE): $(TOOLS_SHARE_DIR)
curl -sL -o $(BUF_SHARE) "https://github.com/bufbuild/buf/releases/download/$(BUF_VERSION)/$(BUF_TARGET)"

$(BUF): $(TOOLS_BIN_DIR) $(BUF_SHARE)
tar xfvz $(TOOLS_SHARE_DIR)/buf.tar.gz -C $(TOOLS_SHARE_DIR) buf/bin
cp $(TOOLS_SHARE_DIR)/buf/bin/* $(TOOLS_BIN_DIR)
rm -rf $(TOOLS_SHARE_DIR)/buf


.PHONY: help
help: ## Display this help. Thanks to https://suva.sh/posts/well-documented-makefiles/
ifeq ($(OS),Windows_NT)
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make <target>\n"} /^[a-zA-Z_-]+:.*?##/ { printf " %-40s %s\n", $$1, $$2 } /^##@/ { printf "\n%s\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
else
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_-]+:.*?##/ { printf " \033[36m%-40s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
endif
Empty file added api/.gitkeep
Empty file.
8 changes: 8 additions & 0 deletions buf.gen.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: v1beta1
plugins:
- name: go
out: api
opt: paths=source_relative
- name: go-grpc
out: api
opt: paths=source_relative,require_unimplemented_servers=false
18 changes: 18 additions & 0 deletions buf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
version: v1beta1
deps:
- buf.build/beta/googleapis
build:
roots:
- api
lint:
use:
- BASIC
- FILE_LOWER_SNAKE_CASE
except:
- ENUM_NO_ALLOW_ALIAS
- IMPORT_NO_PUBLIC
- PACKAGE_AFFINITY
- PACKAGE_DIRECTORY_MATCH
- PACKAGE_SAME_DIRECTORY
breaking:
use:
16 changes: 16 additions & 0 deletions docs/adr/0000-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# 0. Title
<!-- A short and clear title which is prefixed with the ADR number -->

* Status: [proposed | rejected | accepted | deprecated | … | superseded by [ADR-0005](0005-example.md)] <!-- mandatory -->
* Date: 2020-10-29 [YYY-MM-DD - date of the decision] <!-- mandatory -->
* Authors: [list of GitHub handles for the authors]
* Deciders: [list of GitHub handles for those that made the decision] <!-- mandatory -->

## Context
<!-- What is the context of the decision and whats the motivation -->

## Decision
<!-- What is the decision that has been made -->

## Consequences
<!-- Whats the result or impact of this decision. Does anything need to change and are new GitHub issues created as a result -->
23 changes: 23 additions & 0 deletions docs/adr/0001-use-adrs-for-decisions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# 1. Use ADRs to record decisions

* Status: accepted
* Date: 2021-07-08
* Authors: @richardcase
* Deciders: @richardcase

## Context

Decisions that affect the development of reignite that are not captured via a proposal need to be captured in some way. We need a method that is lightweight and easy to discover the decision that have been made. The record of decisions will help future contributors to the project to understand why something has been implemented or is done a certain way.

## Decision

The project will use [Architectural Decision Records (ADR)](https://adr.github.io/) to record decisions that are made outside of a proposal.

A [template](./0000-template.md) has been created based on prior work:

* https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions
* https://adr.github.io/madr/

## Consequences

When decisions are made that affect the entire project then a new ADR needs to be created. Likewise, if a decision has been superceded then we need to caputure this as a new ADR and mark the previous ADR as superceded. Maintainers and contributors will need to decide when an ADR is to be created.
22 changes: 22 additions & 0 deletions docs/adr/0002-use-go.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# 2. Use Go for the implementation

* Status: accepted
* Date: 2021-06-29
* Authors: @richardcase
* Deciders: @mnowster @pfzero

## Context

For the implemenetation of reignite we need to consider whether its implemented in Go or should we consider Rust. Go is the language used predominatly at Weaveworks and within the Kubernetes ecosystem. However, we need to consider Rust for reignite for the following reasons:

* [AWS](https://aws.amazon.com/blogs/opensource/innovating-with-rust/) / [Microsoft](https://msrc-blog.microsoft.com/2019/07/22/why-rust-for-safe-systems-programming/) are publicly advocating Rust for system programming
* Firecracker & Open Hypervisor are borth written in Rust. There are benefits in terms of contributing to these projects
* WASM support is mature
* Rust is very efficient and [fast](https://benchmarksgame-team.pages.debian.net/benchmarksgame/which-programs-are-fastest.html)

## Decision

The decision is to use Go for the implementation of reignite due to the available skillsets of engineers. We can reconsider this in the future.

## Consequences
N/A
91 changes: 91 additions & 0 deletions docs/comparison.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Firecracker & Cloud Hypervisor Comparisson

## Actions

| Action | Firecracker | Cloud Hypervisor |
| ------------- | ------ | ------------- |
| Start | Yes | Yes (boot) |
| Flush Metrics | Yes | ? |
| CtrlAltDel | Yes | ? |
| Pause | Yes | Yes |
| Resume | Yes | Yes |
| Delete | No | Yes |
| Shutdown | ? | Yes |
| Reboot | No | Yes |
| Resize | No | Yes |


## Machine

| Data | Firecracker | Cloud Hypervisor |
| --------------- | -------- | ----- |
| Boot Args | Yes | Yes |
| Initrd | Yes | Yes |
| Kernel Image | Yes | Yes |
| CPU Template | Yes | ? |
| Hyper Threading | Yes | Yes |
| Mem Size | Yes | Yes |
| Track Dirty Pg | Yes | ? |
| vcpu | Yes | Yes |
| max vcps | No | Yes |
| cpu topology | No | Yes |
| hotplug size | No | Yes |
| Shared Memory | No | Yes |
| Huge Pages | No | Yes |


## Drive

| Data | Firecracker| Cloud Hypervisor |
| ------------- | ------- | ----- |
| Is Root | Yes | ? |
| Is Read Only | Yes | Yes |
| Cache Type | Yes | ? |
| Partition ID | Yes | ? |
| Host Path | Yes | Yes |
| RX Rate Limit | Yes | Yes |
| TX rate Limit | Yes | ? |
| Direct | ? | Yes |
| iommu | ? | Yes |
| Queues (size) | ? | Yes |
| vhost | ? | Yes |


## Network Interface

| Data | Firecracker | Cloud Hypervisor |
| -------- | -------- | -------- |
| Query MMDS | yes | ? |
| Guest MAC | Yes | Yes |
| Host Dev | Yes | Yes |
| Iface id | Yes | Yes |
| - TAP | Yes | Yes |
| - macvtap | Yes | Yes |
| RX Rate Limit | Yes | Yes |
| TX rate Limit | Yes | Yes |
| iommu | ? | Yes |
| Queues (size) | ? | Yes |
| vhost | ? | Yes |
| IP Address | ? | Yes |
| Netmask | ? | Yes |


## Additional Features

| Feature | Firecracker | Cloud Hypervisor |
| -------- | -------- | -------- |
| Metadata Service | Yes | ? |
| Logging | Yes | ? |
| Metrics | Yes | Yes |
| Snapshot | Yes | Yes |
| vsock | Yes | Yes |
| Baloon Device | Yes | Yes |
| Random Num gen | ? | Yes |
| fs (dir share) | ? | Yes |
| Serial | Yes | Yes |
| Console | ? | Yes |
| Devices??? | ? | Yes |
| SGX | ? | Yes |
| Numa | Yes | Yes |
| pmem | ? | Yes |
| Live migration | ? | Yes |
9 changes: 8 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,13 @@ module github.com/weaveworks/reignite
go 1.16

require (
github.com/go-playground/universal-translator v0.17.0 // indirect
github.com/go-playground/validator v9.31.0+incompatible // indirect
github.com/golang/mock v1.6.0 // indirect
github.com/gruntwork-io/terratest v0.36.3 // indirect
github.com/onsi/gomega v1.13.0 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/onsi/gomega v1.14.0 // indirect
go.uber.org/atomic v1.8.0 // indirect
go.uber.org/multierr v1.7.0 // indirect
go.uber.org/zap v1.18.1 // indirect
)
Loading