Module Hooks Implementation

[grisha87]

Following the plugin API implementation, I want to consult the idea for a hook API implementation that can be used in combination with the previously introduced plugin API.

My intent is to discuss the general idea and the API design. A working PoC implementation of this idea is already done. If we agree to proceed with it (in this or a modified form), I can start listing down "hook/integration" points in the most beneficial places of the current golem-js module structure and implement a few such integration points :)

General considerations

Regarding hooks:

1. They can be sync or async.
2. They allow the user to influence what a low-level module (like Market or Payment) should do, while the general algorithm/processing pipeline remains unchanged.
3. Users influence the processing of entities (Offer, Invoice, DebitNote) by updating a provided use-case object (ReceivedInitialOffer, ReceivedInvoice, ReceivedDebitNote).
4. Initially, they are scoped to the low-level modules, but this does not mean GolemNetwork could not have its own dictionary of hooks.

Regarding use-case objects:

1. They carry contextual information (e.g., ReceivedDebitNote contains a .data field with the DebitNote DTO).
2. They are mutable and fully synchronous—side effect free.
3. They implement any interaction rules and constraints that we can foresee.

Usage example

Consider the following code:

const someModule = new ExampleMarketModule();

someModule.addHook('onOffer', (offer) => {
  console.log("AMOUNT HOOK > Executing amount check hook", offer)

  if (offer.data.amount < 1000) {
    console.log("AMOUNT HOOK > The offer is too small, rejecting it");
    offer.reject();
  } else {
    console.log("AMOUNT HOOK > The offer is big enough, continue");
  }
});

someModule.addHook("onOffer", (offer) => {
  console.log("PRICE HOOK > Executing price check hook", offer);

  if (offer.data.price > 1000) {
    console.log("PRICE HOOK > The offer is too expensive, rejecting it");
    offer.reject();
  } else {
    console.log("PRICE HOOK > The offer is cheap enough, continue");
  }
})

someModule.work();

Key points:

• Users can provide input by defining a handler for a given hook supported by the module.
• They do this by changing the state of the entity passed as part of the hook.
• This entity is not the plain DTO that we're sending in the events right now. It's a use-case interaction object crafted specifically for the hooks purpose (example below).
• First-class TypeScript support will make the addHook method correctly documented for auto-completion.
• If the user doesn't provide their input (no hook is registered), the default behavior implemented by the module would be to accept.
• The example above showcases filter hooks–as you can see, the offerProposalFilter could be fully replaced by a hook glm.market.addHook("onInitialOffer", (receivedInitialOffer) => ...)

Module implementation example

Consider the following code:

/** Example data type, just like one of our DTOs -> Invoice, DebitNote, OfferProposal, etc ...*/
type SomeOffer = { amount: number; price: number };

/**
 * This is what we're deciding upon
 *
 * Each "interactive" hook defines its own "request/response" that can be influenced by the user.
 *
 * It's here where you define what interactions the user can make with the entity, and what are the rules of the interaction
 *
 * In fact, this seems to resemble a use-case object.
 */
class ReceivedInitialOffer {
  /** This tells what's the decision from the hook chain */
  status: "received" | "accepted" | "rejected" = "received";

  /** This conveys the details of the request - the context for the decision */
  public data: SomeOffer

  constructor(data: SomeOffer) {
    this.data = data;
  }

  accept() {
    this.status = "accepted";
  };

  reject() {
    this.status = "rejected";
  };
}


/**
 * Declaration of hook names and handler signatures for nice developer experience
 */
interface ExampleMarketModuleHooks extends HooksDictionary {
  onOffer: (request: ReceivedInitialOffer) => void;
}

/**
 * Example module that uses the hooks internally to implement logic which considers end-users input
 */
class ExampleMarketModule {
  // Use the built-in hooks engine
  private hooks = new HookEngine<ExampleMarketModuleHooks>();

  // Expose a single method allowing controlled access to the hooks
  addHook<T extends keyof ExampleMarketModuleHooks>(hookName: T, handler: ExampleMarketModuleHooks[T]) {
    this.hooks.addHook(hookName, handler)
  }

  // Focus on your business
  async work() {
    const offer: SomeOffer = {
      price: 100,
      amount: 2000,
    }

    // Ask for user input when needed, provide defaults when needed
    const decision = await this.dispatchOnOffer(offer);

    console.log('feedback', decision);

    if (decision.status === "accepted") {
      console.log('We will accept the offer');
    } else if (decision.status === "rejected") {
      console.log('We will reject the offer');
    } else {
      console.log('We will do nothing');
    }
  }

  private async dispatchOnOffer(offer: SomeOffer): Promise<ReceivedInitialOffer> {
    // Give users the choice
    const response = await this.hooks.dispatchHooks("onOffer", new ReceivedInitialOffer(offer));

    // If they didn't choose, fallback to the default
    if (response.status === "received") {
      console.log("DEFAULT HOOK > Offer still considered as received, I will accept it", offer);
      response.accept();
    }

    // And continue
    return response;
  }
}

Key points:

• The details about how hooks are implemented have been moved to a separate utility class HooksEngine; this way all modules that want to use hooks must implement the private hook property and public addHook method.
• Since we want to support asynchronous hooks, all hook-based methods will have to be async accordingly.
• The interaction objects like the presented ReceivedInitialOffer, which wraps the SomeOffer DTO, are not interactive objects that trigger side effects—they do not call APIs or services (like accepting an invoice, etc.). Their main purpose is to convey the end user's interest and decision so that the module can take respective actions using the internal services, based on user feedback.

Execution example

$ npx tsx tmp/hooks-engine.ts 

AMOUNT HOOK > Executing amount check hook ReceivedInitialOffer {
  status: 'received',
  data: { price: 100, amount: 2000 }
}
AMOUNT HOOK > The offer is big enough, continue

PRICE HOOK > Executing price check hook ReceivedInitialOffer {
  status: 'received',
  data: { price: 100, amount: 2000 }
}
PRICE HOOK > The offer is cheap enough, continue

DEFAULT HOOK > Offer still considered as received, I will accept it { price: 100, amount: 2000 }
feedback ReceivedInitialOffer {
  status: 'accepted',
  data: { price: 100, amount: 2000 }
}

We will accept the offer

[mgordel]

In the example, I noticed that hooks can be added multiple times. Does this mean they are queued and executed in the order they are added? (I assume this queuing mechanism is implemented by the HookEngine?)

So hooks provide entities to the user, in which they can define behaviors (at a specified moment in the module), and then the effect of this behavior directly affects the logic of the module. The effect of the hook's operation will be only a change in the state of the entity which will cause a side effect by the module during further execution.
Is such a change in the state of the same entity made by hooks queued one after another? And in that case, does the last hook make the final decision?

This seems clear and consistent with what we want to achieve. From the design perspective I do not see any obstacles, but it will require identifying key points in modules and entity objects whose behavior will enable the expansion of the logic of a given module. And it will probably be good to secure the execution of hooks (especially async), e.g. with timeouts and good error handling.

From my side I give the green light.

[grisha87]

Thanks @mgordel for your feedback!

Does this mean they are queued and executed in the order they are added?

They are lined up in an array of callbacks and are executed in the order they are added. When used by plugins, they order of adding plugins will also determine the order of adding the hooks.

So hooks provide entities to the user, in which they can define behaviors (at a specified moment in the module), and then the effect of this behavior directly affects the logic of the module. The effect of the hook's operation will be only a change in the state of the entity which will cause a side effect by the module during further execution.
Is such a change in the state of the same entity made by hooks queued one after another? And in that case, does the last hook make the final decision?

The proposed solution is based on the chain of responsibility pattern, where a request goes through a chain of hanlders and oonce the request reaches the end of the chain it's considered final.

The hooks provide an entity to the user, and the user can guide the algorighm in the module by making a devision by altering the state of the entity. If there are hooks A, B and C, when A will alter the state of the entity (example: reject a debit note), and then hook B will receive that modified version.

As for And in that case, does the last hook make the final decision? - yes, the last hook has the final word.

[...] And it will probably be good to secure the execution of hooks (especially async), e.g. with timeouts and good error handling.

Definitely!