feat: Oracle packages and Github verification realm

examples/gno.land/p/demo/gnorkle/README.md

@@ -0,0 +1,41 @@
+# gnorkle
+
+gnorkle is an attempt at a generalized oracle implementation. The gnorkle `Instance` definitions lives in the `gnorkle` directory. It can be effectively initialized by using the `gnorkle.NewInstance` constructor. The owner of the instance is then able to add feeds with or without whitelists, manage the instance's own whitelist, remove feeds, get the latest feed values, and handle incoming message requests that result in some type of action taking place.
+
+An example of gnorkle in action can be found in the `examples/gno.land/r/gnoland/ghverify` realm. Any realm that wishes to integrate the minimum oracle functionality should include functions that relay incoming messages to the oracle instance's message handler and that can produce the latest feed values.
+
+## Feeds
+
+Feeds that are added to an oracle must implement the `gnorkle.Feed` interface. A feed can be thought of as a mechanism that takes in off-chain data and produces data to be consumed on-chain. The three important components of a feed are:
+- Tasks: a feed is composed of one or more `feed.Task` instances. These tasks ultimately instruct an agent what it needs to do to provide data to a feed for ingestion.
+- Ingester: a feed should have a `gnorkle.Ingester` instance that handles incoming data from agents. The ingester's role is to perform the correct action on the incoming data, like adding it to an aggregate or storing it for later evaluation.
+- Storage: a feed should have a `gnorkle.Storage` instance that manages the state of the data a feed publishes -- data to be consumed on-chain.
+
+A single oracle instance may be composed of many feeds, each of which has a unique ID.
+
+Here is an example of what differences may exist amongst feed implementations:
+- Static: a static feed is one that only needs to produce a value once. It ingests values and then publishes the result. Once a single value is published, the state of the feed becomes immutable.
+- Continuous: a continuous feed can accept and ingest data, continously adding and changing its own internal state based on the data received. It can then publish values on demand based on its current state.
+- Periodic: a periodic feed may give all whitelisted agents the opportunity to send data for ingestion within a bounded period of time. After this window closes, the results can be committed and a value is pubished. The process then begins again for the next period.
+
+The only feed currently implemented is the `feeds/static.Feed` type.
+
+## Tasks
+
+It's not hard to be a task -- just implement the one method `feed.Task` interface. On-chain task definitions should not do anything other than store data and be able to marshal that data to JSON. Of course, it is also useful if the task marshal's a `type` field as part of the JSON object so an agent is able to know what type of task it is dealing with and what data to expect in the payload.
+
+## Ingesters
+
+An ingester's primary role is to receive data provided by agents and figure out what to do with it. Ingesters must implement the `gnorkle.Ingester` interface. There are currently two message function types that an ingester may want to handle, `message.FuncTypeIngest` and `message.FuncTypeCommit`. The former message type should result in the ingester accumulating data in its own data store, while the latter should use what it has in its data store to publish a feed value to the `gnorkle.Storage` instance provided to it.
+
+The only ingester currently implemented is the `ingesters/single.ValueIngester` type.
+
+## Storage
+
+Storage types are responsible for storing values produced by a feed's ingester. A storage type must implement `gnorkle.Storage`. This type should be able add values to the storage, retrieve the latest value, and retrieve a set of historical values. It is probably a good idea to make the storage bounded.
+
+The only storage currently implemented is the `storage/simple.Storage` type.
+
+## Whitelists
+
+Whitelists are optional but they can be set on both the oracle instance and the feed levels. The absence of a whitelist definition indicates that ALL addresses should be considered to be whitelisted. Otherwise, the presence of a defined whitelist indicates the callers address MUST be in the whitelist in order for the request to succeed. A feed whitelist has precedence over the oracle instance's whitelist. If a feed has a whitelist and the caller is not on it, the call fails. If a feed doesn't have a whitelist but the instance does and the caller is not on it, the call fails. If neither have a whitelist, the call succeeds. The whitlist logic mostly lives in `gnorkle/whitelist.gno` while the only current `gnorkle.Whitelist` implementation is the `agent.Whitelist` type. The whitelist is not owned by the feeds they are associated with in order to not further pollute the `gnorkle.Feed` interface.

examples/gno.land/p/demo/gnorkle/agent/gno.mod

@@ -0,0 +1,3 @@
+module gno.land/p/demo/gnorkle/agent
+
+require gno.land/p/demo/avl v0.0.0-latest

examples/gno.land/p/demo/gnorkle/agent/whitelist.gno

@@ -0,0 +1,50 @@
+package agent
+
+import "gno.land/p/demo/avl"
+
+// Whitelist manages whitelisted agent addresses.
+type Whitelist struct {
+	store *avl.Tree
+}
+
+// ClearAddresses removes all addresses from the whitelist and puts into a state
+// that indicates it is moot and has no whitelist defined.
+func (m *Whitelist) ClearAddresses() {
+	m.store = nil
+}
+
+// AddAddresses adds the given addresses to the whitelist.
+func (m *Whitelist) AddAddresses(addresses []string) {
+	if m.store == nil {
+		m.store = avl.NewTree()
+	}
+
+	for _, address := range addresses {
+		m.store.Set(address, struct{}{})
+	}
+}
+
+// RemoveAddress removes the given address from the whitelist if it exists.
+func (m *Whitelist) RemoveAddress(address string) {
+	if m.store == nil {
+		return
+	}
+
+	m.store.Remove(address)
+}
+
+// HasDefinition returns true if the whitelist has a definition. It retuns false if
+// `ClearAddresses` has been called without any subsequent `AddAddresses` calls, or
+// if `AddAddresses` has never been called.
+func (m Whitelist) HasDefinition() bool {
+	return m.store != nil
+}
+
+// HasAddress returns true if the given address is in the whitelist.
+func (m Whitelist) HasAddress(address string) bool {
+	if m.store == nil {
+		return false
+	}
+
+	return m.store.Has(address)
+}

examples/gno.land/p/demo/gnorkle/feed/gno.mod

@@ -0,0 +1 @@
+module gno.land/p/demo/gnorkle/feed

examples/gno.land/p/demo/gnorkle/feed/task.gno

@@ -0,0 +1,7 @@
+package feed
+
+// Task is a unit of work that can be part of a `Feed` definition. Tasks
+// are executed by agents.
+type Task interface {
+	MarshalJSON() ([]byte, error)
+}

examples/gno.land/p/demo/gnorkle/feed/type.gno

@@ -0,0 +1,16 @@
+package feed
+
+// Type indicates the type of a feed.
+type Type int
+
+const (
+	// TypeStatic indicates a feed cannot be changed once the first value is committed.
+	TypeStatic Type = iota
+	// TypeContinuous indicates a feed can continuously ingest values and will publish
+	// a new value on request using the values it has ingested.
+	TypeContinuous
+	// TypePeriodic indicates a feed can accept one or more values within a certain period
+	// and will proceed to commit these values at the end up each period to produce an
+	// aggregate value before starting a new period.
+	TypePeriodic
+)

examples/gno.land/p/demo/gnorkle/feed/value.gno

@@ -0,0 +1,9 @@
+package feed
+
+import "time"
+
+// Value represents a value published by a feed. The `Time` is when the value was published.
+type Value struct {
+	String string
+	Time   time.Time
+}

examples/gno.land/p/demo/gnorkle/feeds/static/feed.gno

@@ -0,0 +1,143 @@
+package static
+
+import (
+	"bufio"
+	"bytes"
+
+	"gno.land/p/demo/gnorkle/feed"
+	"gno.land/p/demo/gnorkle/gnorkle"
+	"gno.land/p/demo/gnorkle/ingesters/single"
+	"gno.land/p/demo/gnorkle/message"
+	"gno.land/p/demo/gnorkle/storage/simple"
+	"gno.land/p/demo/ufmt"
+)
+
+// Feed is a static feed.
+type Feed struct {
+	id            string
+	isLocked      bool
+	valueDataType string
+	ingester      gnorkle.Ingester
+	storage       gnorkle.Storage
+	tasks         []feed.Task
+}
+
+// NewFeed creates a new static feed.
+func NewFeed(
+	id string,
+	valueDataType string,
+	ingester gnorkle.Ingester,
+	storage gnorkle.Storage,
+	tasks []feed.Task,
+) *Feed {
+	return &Feed{
+		id:            id,
+		valueDataType: valueDataType,
+		ingester:      ingester,
+		storage:       storage,
+		tasks:         tasks,
+	}
+}
+
+// NewSingleValueFeed is a convenience function  for creating a static feed
+// that autocommits a value after a single ingestion.
+func NewSingleValueFeed(
+	id string,
+	valueDataType string,
+	tasks []feed.Task,
+) *Feed {
+	return NewFeed(
+		id,
+		valueDataType,
+		&single.ValueIngester{},
+		simple.NewStorage(1),
+		tasks,
+	)
+}
+
+// ID returns the feed's ID.
+func (f *Feed) ID() string {
+	return f.id
+}
+
+// Type returns the feed's type.
+func (f *Feed) Type() feed.Type {
+	return feed.TypeStatic
+}
+
+// Ingest ingests a message into the feed. It either adds the value to the ingester's
+// pending values or commits the value to the storage.
+func (f *Feed) Ingest(funcType message.FuncType, msg, providerAddress string) {
+	if f.isLocked {
+		panic("feed locked")
+	}
+
+	switch funcType {
+	case message.FuncTypeIngest:
+		// Autocommit the ingester's value if it's a single value ingester
+		// because this is a static feed and this is the only value it will ever have.
+		if canAutoCommit := f.ingester.Ingest(msg, providerAddress); canAutoCommit {
+			f.ingester.CommitValue(f.storage, providerAddress)
+			f.isLocked = true
+		}
+
+	case message.FuncTypeCommit:
+		f.ingester.CommitValue(f.storage, providerAddress)
+		f.isLocked = true
+
+	default:
+		panic("invalid message function " + string(funcType))
+	}
+}
+
+// Value returns the feed's latest value, it's data type, and whether or not it can
+// be safely consumed. In this case it uses `f.isLocked` because, this being a static
+// feed, it will only ever have one value; once that value is committed the feed is locked
+// and there is a valid, non-empty value to consume.
+func (f *Feed) Value() (feed.Value, string, bool) {
+	return f.storage.GetLatest(), f.valueDataType, f.isLocked
+}
+
+// MarshalJSON marshals the components of the feed that are needed for
+// an agent to execute tasks and send values for ingestion.
+func (f *Feed) MarshalJSON() ([]byte, error) {
+	buf := new(bytes.Buffer)
+	w := bufio.NewWriter(buf)
+
+	w.Write([]byte(
+		`{"id":"` + f.id +
+			`","type":"` + ufmt.Sprintf("%d", int(f.Type())) +
+			`","value_type":"` + f.valueDataType +
+			`","tasks":[`),
+	)
+
+	first := true
+	for _, task := range f.tasks {
+		if !first {
+			w.WriteString(",")
+		}
+
+		taskJSON, err := task.MarshalJSON()
+		if err != nil {
+			return nil, err
+		}
+
+		w.Write(taskJSON)
+	}
+
+	w.Write([]byte("]}"))
+	w.Flush()
+
+	return buf.Bytes(), nil
+}
+
+// Tasks returns the feed's tasks. This allows task consumers to extract task
+// contents without having to marshal the entire feed.
+func (f *Feed) Tasks() []feed.Task {
+	return f.tasks
+}
+
+// IsActive returns true if the feed is accepting ingestion requests from agents.
+func (f *Feed) IsActive() bool {
+	return !f.isLocked
+}

examples/gno.land/p/demo/gnorkle/feeds/static/gno.mod

@@ -0,0 +1,10 @@
+module gno.land/p/demo/gnorkle/feeds/static
+
+require (
+	gno.land/p/demo/gnorkle/feed v0.0.0-latest
+	gno.land/p/demo/gnorkle/gnorkle v0.0.0-latest
+	gno.land/p/demo/gnorkle/ingesters/single v0.0.0-latest
+	gno.land/p/demo/gnorkle/message v0.0.0-latest
+	gno.land/p/demo/gnorkle/storage/simple v0.0.0-latest
+	gno.land/p/demo/ufmt v0.0.0-latest
+)

examples/gno.land/p/demo/gnorkle/gnorkle/feed.gno

@@ -0,0 +1,24 @@
+package gnorkle
+
+import (
+	"gno.land/p/demo/gnorkle/feed"
+	"gno.land/p/demo/gnorkle/message"
+)
+
+// Feed is an abstraction used by a gnorkle `Instance` to ingest data from
+// agents and provide data feeds to consumers.
+type Feed interface {
+	ID() string
+	Type() feed.Type
+	Value() (value feed.Value, dataType string, consumable bool)
+	Ingest(funcType message.FuncType, rawMessage, providerAddress string)
+	MarshalJSON() ([]byte, error)
+	Tasks() []feed.Task
+	IsActive() bool
+}
+
+// FeedWithWhitelist associates a `Whitelist` with a `Feed`.
+type FeedWithWhitelist struct {
+	Feed
+	Whitelist
+}

examples/gno.land/p/demo/gnorkle/gnorkle/gno.mod

@@ -0,0 +1,9 @@
+module gno.land/p/demo/gnorkle/gnorkle
+
+require (
+	gno.land/p/demo/avl v0.0.0-latest
+	gno.land/p/demo/gnorkle/agent v0.0.0-latest
+	gno.land/p/demo/gnorkle/feed v0.0.0-latest
+	gno.land/p/demo/gnorkle/ingester v0.0.0-latest
+	gno.land/p/demo/gnorkle/message v0.0.0-latest
+)

examples/gno.land/p/demo/gnorkle/gnorkle/ingester.gno

@@ -0,0 +1,11 @@
+package gnorkle
+
+import "gno.land/p/demo/gnorkle/ingester"
+
+// Ingester is the abstraction that allows a `Feed` to ingest data from agents
+// and commit it to storage using zero or more intermediate aggregation steps.
+type Ingester interface {
+	Type() ingester.Type
+	Ingest(value, providerAddress string) (canAutoCommit bool)
+	CommitValue(storage Storage, providerAddress string)
+}