Skip to content

Commit

Permalink
docs: add cardano to kafka example
Browse files Browse the repository at this point in the history
  • Loading branch information
scarmuega committed Dec 13, 2021
1 parent 3ac89e4 commit 2f560cb
Show file tree
Hide file tree
Showing 2 changed files with 69 additions and 0 deletions.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ Similar to the well-known db-sync tool provided by IOHK, _Oura_ can be used as a

Given its small memory / cpu footprint, _Oura_ can be deployed side-by-side with your Cardano node even in resource-constrained environments, such as Raspberry PIs.

For an example of how to use _Oura_ as a bridge, check the [Cardano => Kafka](docs/kafka.md) setup instructions.

### As a Trigger of Custom Actions

_Oura_ running in `daemon` mode can be configured to use custom filters to pinpoint particular transaction patterns and trigger actions whenever it finds a match. For example: send an email when a particular policy / asset combination appears in a transaction; call an AWS Lambda function when a wallet delegates to a particular pool; send a http-call to a webhook each time a metadata key appears in the TX payload;
Expand Down
67 changes: 67 additions & 0 deletions docs/kafka.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Cardano => Kafka Example

This example shows how to leverage _Oura_ to stream data from a Cardano node into a _Kafka_ topic.

## About Kafka

> Apache Kafka is a framework implementation of a software bus using stream-processing. It is an open-source software platform developed by the Apache Software Foundation written in Scala and Java.
Find [more info](https://en.wikipedia.org/wiki/Apache_Kafka) about _Kafka_ in wikipedia.

## Prerequisites

This examples assumes the following prerequisites:

- A running Cardano node locally accesible via a unix socket.
- A Kafka cluster accesible through the network.
- An already existing Kafka topic where to output events
- _Oura_ binary release installed in local system

## Instructions

### 1. Create an Oura configuration file `caradno2kafka.toml`

```toml
[source]
type = "N2C"
address = ["Unix", "/opt/cardano/cnode/sockets/node0.socket"]
magic = "testnet"

[sink]
type = "Kafka"
brokers = ["kafka-broker-0:9092"]
topic = "cardano-events"
```

Some comments regarding the above configuration:

- the `[source]` section indicates _Oura_ from where to pull chain data.
- the `N2C` source type value tells _Oura_ to get the data from a Cardano node using Node-to-Client mini-protocols (chain-sync instantiated to full blocks).
- the `address` field indicates that we should connect via `Unix` socket at the specified path. This value should match the location of your local node socket path.
- the `magic` field indicates that our node is running in the `testnet` network. Change this value to `mainnet` if appropriate.
- the `[sink]` section tells _Oura_ where to send the information it gathered.
- the `Kafka` sink type value indicates that _Oura_ should use a _Kafka_ producer client as the output
- the `brokers` field indicates the location of the _Kafka_ brokers within the network. Several hostname:port pairs can be added to the array for a "cluster" scenario.
- the `topic` fields indicates which _Kafka_ topic to used for the outbound messages.

### 2. Run _Oura_ in `daemon` mode

Run the following command from your terminal to start the daemon process:

```sh
RUST_LOG=info oura daemon --config cardano2kafka.toml
```

You should see an output similar to the following:

```
[2021-12-13T22:16:43Z INFO oura::sources::n2n::setup] handshake output: Accepted(7, VersionData { network_magic: 764824073, initiator_and_responder_diffusion_mode: false })
[2021-12-13T22:16:43Z INFO oura::sources::n2n::setup] chain point query output: Some(Tip(Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6"), 6624258))
[2021-12-13T22:16:43Z INFO oura::sources::n2n::setup] node tip: Point(47867448,"f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6")
[2021-12-13T22:16:44Z INFO oura::sources::n2n] rolling block to point Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6")
[2021-12-13T22:16:52Z INFO oura::sources::n2n] requesting block fetch for point Some(Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6"))
[2021-12-13T22:17:15Z INFO oura::sources::n2n] requesting block fetch for point Some(Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6"))
[2021-12-13T22:17:20Z INFO oura::sources::n2n] requesting block fetch for point Some(Point(47867448, "f170baa5702c91b23580291c3a184195df7c77d3e1a03b3d6424793aacc850d6"))
```


0 comments on commit 2f560cb

Please sign in to comment.