Merge branch 'master' of github.com:gnolang/gno into dev/morgan/go-ty…

…pes-typecheck

examples/gno.land/p/demo/gnorkle/README.md

@@ -0,0 +1,112 @@
+# gnorkle
+
+gnorkle is an attempt at a generalized oracle implementation. The gnorkle `Instance` definitions lives in the `gnorkle` directory. It can be effectively initialized by using the `gnorkle.NewInstance` constructor. The owner of the instance is then able to add feeds with or without whitelists, manage the instance's own whitelist, remove feeds, get the latest feed values, and handle incoming message requests that result in some type of action taking place.
+
+An example of gnorkle in action can be found in the `examples/gno.land/r/gnoland/ghverify` realm. Any realm that wishes to integrate the minimum oracle functionality should include functions that relay incoming messages to the oracle instance's message handler and that can produce the latest feed values.
+
+## Feeds
+
+Feeds that are added to an oracle must implement the `gnorkle.Feed` interface. A feed can be thought of as a mechanism that takes in off-chain data and produces data to be consumed on-chain. The three important components of a feed are:
+- Tasks: a feed is composed of one or more `feed.Task` instances. These tasks ultimately instruct an agent what it needs to do to provide data to a feed for ingestion.
+- Ingester: a feed should have a `gnorkle.Ingester` instance that handles incoming data from agents. The ingester's role is to perform the correct action on the incoming data, like adding it to an aggregate or storing it for later evaluation.
+- Storage: a feed should have a `gnorkle.Storage` instance that manages the state of the data a feed publishes -- data to be consumed on-chain.
+
+A single oracle instance may be composed of many feeds, each of which has a unique ID.
+
+Here is an example of what differences may exist amongst feed implementations:
+- Static: a static feed is one that only needs to produce a value once. It ingests values and then publishes the result. Once a single value is published, the state of the feed becomes immutable.
+	- Example: a realm wants to integrate football match results for its users. It may embed a static oracle that allows it to publish the match results to the chain. A static feed is a good choice for this because the match results will never change.
+- Continuous: a continuous feed can accept and ingest data, continously adding and changing its own internal state based on the data received. It can then publish values on demand based on its current state.
+	- Example: a realm wants to provide a verifiable random function. It embeds an oracle that defines tasks and whitelists a select group of trusted agents to provide data that gets combined to produce a random value. A continuous feed is a good choice for this because data may be accepted continously and there is no single static result.
+- Periodic:  periodic feed may give all whitelisted agents the opportunity to send data for ingestion within a bounded period of time. After this window closes, the results can be committed and a value is pubished. The process then begins again for the next period.
+	- Example: a realm wants to provide weather information to its users. It may choose a group of trusted agents to publish weather data for each half hour interval. This periodic feed can finalize the weather data for each postal code at the end of each interval using the aggregation function defined by the owner of the oracle.
+
+The only feed currently implemented is the `feeds/static.Feed` type.
+
+## Tasks
+
+It's not hard to be a task -- just implement the one method `feed.Task` interface. On-chain task definitions should not do anything other than store data and be able to marshal that data to JSON. Of course, it is also useful if the task marshal's a `type` field as part of the JSON object so an agent is able to know what type of task it is dealing with and what data to expect in the payload.
+
+### Example use case
+Imagine there is a public API out there. The oracle defines a task with a type of `HTTPGET`. The agent interacting with the oracle knows how to extract the task's type and extract the rest of the data to complete the task
+```go
+type apiGet struct {
+	taskType   string
+	url        string
+}
+```
+So the data might look something like:
+```json
+apiGet {
+	"task_type": "HTTPGET",
+	"url": "http://example.com/api/latest"
+}
+```
+The agent can use this data to make the request and publish the results to the oracle. Tasks can have structures as complex or simple as is required.
+
+## Ingesters
+
+An ingester's primary role is to receive data provided by agents and figure out what to do with it. Ingesters must implement the `gnorkle.Ingester` interface. There are currently two message function types that an ingester may want to handle, `message.FuncTypeIngest` and `message.FuncTypeCommit`. The former message type should result in the ingester accumulating data in its own data store, while the latter should use what it has in its data store to publish a feed value to the `gnorkle.Storage` instance provided to it.
+
+The only ingester currently implemented is the `ingesters/single.ValueIngester` type.
+
+### Example use case
+The `Ingester` interface has two main methods that must be implemented, `Ingest` and `CommitValue`, as defined here:
+```go
+type Ingester interface {
+	Type() ingester.Type
+	Ingest(value, providerAddress string) (canAutoCommit bool)
+	CommitValue(storage Storage, providerAddress string)
+}
+```
+Consider an oracle that provides a price feed. This price feed would need an ingester capable of ingesting incoming values and producing a final result at the end of a given amount of time. So we may have something that looks like:
+```go
+type MultiValueIngester struct {
+	agentAddresses []string
+	prices         []uint64
+}
+
+func (i *MultiValueIngester) Ingest(value, providerAddress string) bool {
+	price, err := strconv.ParseUint(value, 10, 64)
+	if err != nil {
+		panic("invalid value type")
+	}
+
+	i.agentAddresses = append(i.agentAddresses, providerAddress)
+	i.prices = append(i.prices, price)
+
+	// This value cannot be autocommitted to storage because the ingester expects
+	// multiple values and it doesn't know if this is the final value.
+	return false
+}
+
+func (i *MultiValueIngester) CommitValue(storage Storage, providerAddress string) {
+	priceAggregate := i.aggregatePrices()
+	storage.Put(priceAggregate)
+
+	// Reset to prepare for the next price period.
+	i.agentAddresses = []string{}
+	i.prices = []uint64{}
+}
+```
+An ingester is highly customizable and should be used to do any necessary aggregations.
+
+## Storage
+
+Storage types are responsible for storing values produced by a feed's ingester. A storage type must implement `gnorkle.Storage`. This type should be able add values to the storage, retrieve the latest value, and retrieve a set of historical values. It is probably a good idea to make the storage bounded.
+
+The only storage currently implemented is the `storage/simple.Storage` type.
+
+### Example use case
+In most cases the storage implementation will be a key value store, so its use case is fairly generic. The `Put` method is used to store finalized data and the `GetLatest` method can be used by consumers of the data. `Storage` is an interface in order to allow for memory management throughout the lifetime of the oracle writing data to it. The `GetHistory` method can be used, if desired, to keep a fixed historical window of published data points.
+```go
+type Storage interface {
+	Put(value string)
+	GetLatest() feed.Value
+	GetHistory() []feed.Value
+}
+```
+
+## Whitelists
+
+Whitelists are optional but they can be set on both the oracle instance and the feed levels. The absence of a whitelist definition indicates that ALL addresses should be considered to be whitelisted. Otherwise, the presence of a defined whitelist indicates the callers address MUST be in the whitelist in order for the request to succeed. A feed whitelist has precedence over the oracle instance's whitelist. If a feed has a whitelist and the caller is not on it, the call fails. If a feed doesn't have a whitelist but the instance does and the caller is not on it, the call fails. If neither have a whitelist, the call succeeds. The whitlist logic mostly lives in `gnorkle/whitelist.gno` while the only current `gnorkle.Whitelist` implementation is the `agent.Whitelist` type. The whitelist is not owned by the feeds they are associated with in order to not further pollute the `gnorkle.Feed` interface.

examples/gno.land/p/demo/gnorkle/agent/gno.mod

@@ -0,0 +1,3 @@
+module gno.land/p/demo/gnorkle/agent
+
+require gno.land/p/demo/avl v0.0.0-latest

examples/gno.land/p/demo/gnorkle/agent/whitelist.gno

@@ -0,0 +1,50 @@
+package agent
+
+import "gno.land/p/demo/avl"
+
+// Whitelist manages whitelisted agent addresses.
+type Whitelist struct {
+	store *avl.Tree
+}
+
+// ClearAddresses removes all addresses from the whitelist and puts into a state
+// that indicates it is moot and has no whitelist defined.
+func (m *Whitelist) ClearAddresses() {
+	m.store = nil
+}
+
+// AddAddresses adds the given addresses to the whitelist.
+func (m *Whitelist) AddAddresses(addresses []string) {
+	if m.store == nil {
+		m.store = avl.NewTree()
+	}
+
+	for _, address := range addresses {
+		m.store.Set(address, struct{}{})
+	}
+}
+
+// RemoveAddress removes the given address from the whitelist if it exists.
+func (m *Whitelist) RemoveAddress(address string) {
+	if m.store == nil {
+		return
+	}
+
+	m.store.Remove(address)
+}
+
+// HasDefinition returns true if the whitelist has a definition. It retuns false if
+// `ClearAddresses` has been called without any subsequent `AddAddresses` calls, or
+// if `AddAddresses` has never been called.
+func (m Whitelist) HasDefinition() bool {
+	return m.store != nil
+}
+
+// HasAddress returns true if the given address is in the whitelist.
+func (m Whitelist) HasAddress(address string) bool {
+	if m.store == nil {
+		return false
+	}
+
+	return m.store.Has(address)
+}

examples/gno.land/p/demo/gnorkle/agent/whitelist_test.gno

@@ -0,0 +1,47 @@
+package agent_test
+
+import (
+	"testing"
+
+	"gno.land/p/demo/gnorkle/agent"
+)
+
+func TestWhitelist(t *testing.T) {
+	var whitelist agent.Whitelist
+
+	if whitelist.HasDefinition() {
+		t.Error("whitelist should not be defined initially")
+	}
+
+	whitelist.AddAddresses([]string{"a", "b"})
+	if !whitelist.HasAddress("a") {
+		t.Error(`whitelist should have address "a"`)
+	}
+	if !whitelist.HasAddress("b") {
+		t.Error(`whitelist should have address "b"`)
+	}
+
+	if !whitelist.HasDefinition() {
+		t.Error("whitelist should be defined after adding addresses")
+	}
+
+	whitelist.RemoveAddress("a")
+	if whitelist.HasAddress("a") {
+		t.Error(`whitelist should not have address "a"`)
+	}
+	if !whitelist.HasAddress("b") {
+		t.Error(`whitelist should still have address "b"`)
+	}
+
+	whitelist.ClearAddresses()
+	if whitelist.HasAddress("a") {
+		t.Error(`whitelist cleared; should not have address "a"`)
+	}
+	if whitelist.HasAddress("b") {
+		t.Error(`whitelist cleared; should still have address "b"`)
+	}
+
+	if whitelist.HasDefinition() {
+		t.Error("whitelist cleared; should not be defined")
+	}
+}

examples/gno.land/p/demo/gnorkle/feed/errors.gno

@@ -0,0 +1,5 @@
+package feed
+
+import "errors"
+
+var ErrUndefined = errors.New("undefined feed")

examples/gno.land/p/demo/gnorkle/feed/gno.mod

@@ -0,0 +1 @@
+module gno.land/p/demo/gnorkle/feed

examples/gno.land/p/demo/gnorkle/feed/task.gno

@@ -0,0 +1,7 @@
+package feed
+
+// Task is a unit of work that can be part of a `Feed` definition. Tasks
+// are executed by agents.
+type Task interface {
+	MarshalJSON() ([]byte, error)
+}

examples/gno.land/p/demo/gnorkle/feed/type.gno

@@ -0,0 +1,16 @@
+package feed
+
+// Type indicates the type of a feed.
+type Type int
+
+const (
+	// TypeStatic indicates a feed cannot be changed once the first value is committed.
+	TypeStatic Type = iota
+	// TypeContinuous indicates a feed can continuously ingest values and will publish
+	// a new value on request using the values it has ingested.
+	TypeContinuous
+	// TypePeriodic indicates a feed can accept one or more values within a certain period
+	// and will proceed to commit these values at the end up each period to produce an
+	// aggregate value before starting a new period.
+	TypePeriodic
+)

examples/gno.land/p/demo/gnorkle/feed/value.gno

@@ -0,0 +1,9 @@
+package feed
+
+import "time"
+
+// Value represents a value published by a feed. The `Time` is when the value was published.
+type Value struct {
+	String string
+	Time   time.Time
+}