Skip to content

Latest commit

 

History

History
284 lines (252 loc) · 10.6 KB

File metadata and controls

284 lines (252 loc) · 10.6 KB

Golem — current architecture

Authors: Witold Dzięcioł, Przemysław Rekucki, Marek Dopiera, <YOUR NAME GOES HERE>
Reviewers: Maciej Maciejowski, Paweł Burgchardt
Status: WIP

About this document

The goal of this document is to describe present Golem architecture in enough detail for an outside person to understand how it works under the hood. The intended audience is assumed to be technical but not necessarily have deep expertise in the crypto world. The level of detail stops short of describing code and its organization but describes key technologies used in order to implement the needed functionalities.

The aim is not to dive deep into every integration, but to capture architectural decisions and their consequences.

Functional view

This section describes what comprises Golem network, namely the actors, technical artifacts and activities they actors may perform on those activities. The objective of this paragraph is to tie together all the terms and provide a very high level description of what they are.

What Golem is

What Golem is not

Design principles

  • Golem is protocol for selling and using resources in Golem Network

  • Protocol is open for being extended by it's users (community)

    • It provides basic specification how Nodes should communicate within Golem Network
    • Protocol allows for creating sub-protocols on top of Golem, meaning that developers can define specifications that add specific bahaviors to Nodes, Provider/Requestor agents, SDKs etc.
  • Golem Factory creates specification that are guiding reference implementation of yagna daemon and Agents applications

  • This doesn't close the option to reimplement those components in complately different way

  • Example: mid-agreement payments

    • On the lowest protocol level Golem defines market exchange of Offers/Demands artifacts, property constraint langauge and Payment artifacts (DebitNotes, Invoices)
    • Golem Factory creates specification of protocol defining what should be included in Offer/Demand to use mid-agreement payments and how should Agent application bahave in relation to sending DebitNotes and when payments should happen
    • Bahavior is implemented in reference implementation
    • Other developers are allowed not to use this protocol when implementic their own Nodes
    • They can prepare their own alternative specification of mid-agreement payments as well

Actors

This section describes the actors using Golem Network and their role in the system.

Provider

Requestor

Note: We use term Requestor not Requester.

Developer

End User

Consumer of resources can be different person than Requestor. For example we can have web service which forwards computationally expensive jobs to Golem Network. Example: Deposits

Service Owner

Funder

Spender

Activities

This section describes what actors can do to the system. The descriptions are only as detailed as to explain how the actors control the artifacts. The goal is to give the reader an overview of the terms introduced by Golem without any details and establish a glossary to ensure consistency within the document.

Provisioning a provider

Setting up a requester

Finding a provider

Fulfilling an agreement

Creating an image

Starting a VM

Setting up a VPN

Artifacts/Terms

This section describes the artifacts, i.e. the terms introduced in Golem Network on which actors can act. They are organized by respective aspects of Golem Network. The descriptions describe their function rather than their implementation.

Section should serve as dictionary to be linked by other chapters.

Participating entities

Yagna daemon

Provider agent

Requester agent

Marketplace

Offer

Demand

Subscription

This word is used to describe Offer/Demand put on market, so we should mention it.

Proposal

Negotiation

Agreement

Execution system

Activity

Execution environment (ExeUnit)

ExeUnit Batch
ExeUnit Command
VM
VM Image
WASM
WASM image

VPN

Network

Payment System

Payment Driver

Payment Platform

Token

Wallet

Allocation

Debit Note

Invoice

Payment

Transaction (on blockchain)

Key architectural decisions

GLM is built on XYZ

GLM is used for clearing

No centralized offer matching rules

Only providers' offers are propagated

Agreements are not stored on the blockchain

Offline requestors are not supported

Local storage (TODO: what role does the local DB play?)

Technical view - architecture layers

Bottom-up description of different layers of Golem that impose structure on how we split components and facilitate communication between them. Golem is not monolithic and division of responsibility is not arbitrary.

The goal of this chapter is to sketch general idea, rather than describe specifics.

Network layer

GSB layer

RPC-like communication layer providing a way to call other modules or other nodes in unified way.

Specialized modules layer (market, payment, activity, vpn etc.)

Market negotiation protocols

Offers/Demands property and constraint language is used to build subprotocols for negotiating certain aspects of Agreements.

Execution environments

There are a few ways to implement an execution environment. We have a few APIs here:

  • Bind and handle GSB messages directly
  • Implement RuntimeSDK and use ExeUnit binary for common functionalities
  • Command specification which can be executed by ExeUnit is flexible enough that higher level protocols could be created here.

API layer

Each specialized module exposes some APIs to users. It is either REST API, yagna cli or set of GSB endpoints for communication between modules and nodes.

REST API

CLI

GSB API (communication between modules)

SDKs layer (python and js SDK)

Agent application (Requestor script, Provider Agent)

Higher level SDKs

Golem compose

Ray on Golem

Technical view - functional aspects

This section dives into aspects of Golem network architecture to explain how they work under the hood. The level of detail does not include code structure, interfaces nor data structures but names key components and interaction between them.

Market interactions

Discovery and Offers/Demand matching

  • Offer/Demand properties and constraint langauge
  • Yagna is property agnostic - doesn't understand semantic of properties, only agent do
  • Some examples of properties and costraints and how it works
  • Links to more detailed docs for properties langauge and properties sepcification (?)

Offer propagation

  • Link to design decision
  • Algorithm overview

Process of negotiations and making an agreement

  • Initial Proposal
  • Countering Proposal
  • What can change in counter proposal (protocols based on property langauge)?
  • Provider Agent possible Proposal responses (counter, reject)
  • Requestor Agent possible Proposal responses (counter, reject, propose Agreement)
  • Provider Agent possible Agreement responses (accept, reject)
  • Requestor possbility of Agreement Proposal cancallation
  • Restarting negotiations (who can, who can't and how?)

Agreement termination

  • Who is allowed to terminate? In what situation?
  • What is specified by protocol and what is left to future specifications?
  • Termination reason concept

Communication between nodes

Central net

Hybrid net

  • Identification
  • Relay
  • Discovering Nodes
  • P2P communication
  • Relayed communication
  • Cryptography
    • Node identity verification (challenges)
    • Communication encryption

GSB communication through Net

  • Communication is independent from net implementation
  • GSB addressing

Payments & Clearing

Payments models

  • Describe generic model which is open for new implementations
  • Payment model specification in Offer/Demand language
  • Linear Payment model as an example

Payments flow during Agreement

  • Negotiating payment platform and other payment details
  • Testnet(s) vs. mainnet(s)
  • Tokens
  • Partial payments vs. payments after Agreement finish
  • DebitNotes/Invoices interactions (acceptance, rejection, cancellation)
  • How DebitNote/Invoice acceptance relates to payment on blockchain?
  • Payment settlement and payment confirmation for Provider

Payment drivers

  • Abstract concept (independance from underlying payment mechanisms)
  • How payment platform relates to payment driver?
  • Examples: erc20 driver, zksync (?)

Payments batching

Deposits payments

  • Overview of the concept
  • Link to external documentation describing details

ExeUnits

Abstract concept

  • ExeUnit concept is generic enough to sell any kind of computation resources
  • Generic ExeUnits (for example VM, WASM etc.) vs. specialized ExeUnits for specific tasks like:
  • Interaction with yagna through GSB
  • Control flow between Requestor and ExeUnit
  • Extensible commands list (ExeUnit implementation dependent)
Controlling ExeUnit (basic concepts)
  • Spawning ExeUnit (contract between Provider Agent and ExeUnit)
    • Self-test
    • Offer template
  • Binding to GSB (addressing based on activity id)
  • Requestor state control
  • Commands and batches:
    • Deploy, Start, Transfer, Run, Terminate
    • Querying command/batch state, receiving results
    • Transfer methods (GFTP, http)

ExeUnit Supervisor

  • Why splitting Supervisor and Runtime?
  • Common functionalities provided by Supervisor

ExeUnit Runtime

GFTP

VM runtime

  • Virtual machine desciption (so the reader knows what is there, but not details)
  • Functionalities (outbound, VPN, process output capturing)
  • VM images, gvmkit-build etc

WASM runtime

  • WASM supported execution engines
  • WASM images

VPN

Processes and the responsibility split between them

IPC

Reputation management

Key architectural shortcomings

This section contains known shortcomings of the implemented architecture — irrespective of whether they were intentional or unintentional.

Preexisting two categories of actors

The preexisting categories of actors (providers and requesters) and their asymmetric roles are limiting in certain scenarios. FIXME FIXME FIME

TODO