Impl Logger AddContext summand

hs-abci-examples/nameservice/README.md

@@ -1,23 +1,147 @@
 # nameservice
 
-## Metrics
+## Metrics via Prometheus
 
 Run `nameservice` via the `Makefile`:
 
 ```
 make deploy-nameservice-local
 ```
 
-Along with running `nameservice`, this also starts a Prometheus] metrics server.
+Along with running `nameservice`, this command also starts a Prometheus metrics server.
 By default, the metrics server runs on `localhost:9200`. To use a different port, 
 set the `STATS_PORT` environment variable to the desired port value.
 
 To see these metrics in Datadog, follow the 
 [Prometheus host config instructions](https://docs.datadoghq.com/getting_started/integrations/prometheus/?tab=host#pagetitle)
-to configure a local Datadog agent to scrape the endpoint.
+to configure a local Datadog agent to scrape the endpoint. At minimum, to scrape all 
+`nameservice` prometheus metrics, the appropriate `conf.yaml` (described above) should
+contain the following settings:
+
+```yaml
+init_config:
+
+instances:
+
+  - prometheus_url: http://localhost:9200/metrics
+    ## namespace option prefixes all metric names in datadog
+    namespace: prometheus
+    ## metrics names used in the nameservice app
+    metrics:
+      - count_buy
+      - count_set
+      - count_delete
+      - histogram_buy*
+      - histogram_set*
+      - histogram_delete*
+```
 
 Alternatively, use the `docker-compose` command:
 
 ```
 make deploy-nameservice-docker
 ```
+
+Once the `nameservice` server is running, start a Tendermint node:
+
+```bash
+> tendermint init
+> tendermint node
+```
+
+Then run the `nameservice` tests:
+
+```bash
+make test-nameservice
+```
+
+Once the test run is completed, you should now be able to view metrics
+on [Datadog's metrics explorer](https://app.datadoghq.com/metric/explorer).
+Firstly, ensure that the prometheus server is populated with metrics by
+visiting `localhost:9200/metrics` in a browser. It should look something like this:
+
+```
+# TYPE count_buy counter
+count_buy  4
+# TYPE count_delete counter
+count_delete  1
+# TYPE count_set counter
+count_set  2
+# TYPE histogram_buy histogram
+histogram_buy_bucket{le="1.0e-4"} 0.0
+...
+# TYPE histogram_delete histogram
+histogram_delete_bucket{le="1.0e-4"} 0.0
+...
+# TYPE histogram_set histogram
+histogram_set_bucket{le="1.0e-4"} 0.0
+...
+```
+
+Now, in Datadog's metrics explorer, we can search for metrics
+prefixed with the `namespace` value (i.e., `prometheus`) set above:
+
+<img src="https://raw.githubusercontent.com/f-o-a-m/hs-abci/master/hs-abci-examples/nameservice/images/prometheus-metrics.png" width="50%"/>
+
+## Alternative Logging
+
+In addition to Prometheus metrics, the `nameservice` app includes options for
+logging to Elasticsearch (ES) and Datadog metrics logging. By default, the app
+logs everything to the console:
+
+```
+[2019-12-20 16:19:27][nameservice.server][Info][local][PID 89617][ThreadId 21][type:info] Request Received
+[2019-12-20 16:19:27][nameservice.server][Info][local][PID 89617][ThreadId 21][message_type:info][response_time:2.6e-5][message_count:1]
+```
+
+These logs include request/response info and some event and metric logging.
+
+Alternatively, the app is set up to log the same information to ES
+and Datadog by setting the following environment variables:
+
+```bash
+DD_API_KEY ## Datadog API key
+ES_HOST    ## Elasticsearch server host
+ES_PORT    ## Elasticsearch server port
+```
+
+We recommend using the [ELK (v683) docker image](https://hub.docker.com/r/sebp/elk/tags)
+to run an ES server alongside a Kibana instance to search and filter incoming logs.
+
+```bash
+> docker pull sebp/elk:683
+> docker run -p 5601:5601 -p 9201:9200 -p 5044:5044 -it --name elk sebp/elk:683
+```
+
+**NOTE:** This command remaps the `ES_PORT` value from `9200` to `9201` to avoid collision with
+the Prometheus server that `nameservice` runs by default. The Kibana instance runs on port `5601`.
+
+After both Kibana and ES are running, start `nameservice` and a Tendermint node 
+via the commands:
+
+```bash
+> DD_API_KEY=<DD_API_KEY> ES_HOST=localhost ES_PORT=9201 make deploy-nameservice
+> tendermint init
+> tendermint node
+```
+
+At this point, there should be no logs printed on the console. If a valid
+Datadog API Key is provided, metrics logs are available on
+[Datadog's log explorer](https://app.datadoghq.com/logs).
+
+<img src="https://raw.githubusercontent.com/f-o-a-m/hs-abci/master/hs-abci-examples/nameservice/images/logs-tab.png" width="25%"/>
+
+To view ES logs on Kibana, go to `Management` on the sidebar and create a Kibana index pattern for `nameservice`:
+
+<img src="https://raw.githubusercontent.com/f-o-a-m/hs-abci/master/hs-abci-examples/nameservice/images/kibana-management.png" width="25%"/>
+
+<img src="https://raw.githubusercontent.com/f-o-a-m/hs-abci/master/hs-abci-examples/nameservice/images/create-index.png" width="75%"/>
+
+Finally, under `Discover`, we can add filters to view specific logs:
+
+<img src="https://raw.githubusercontent.com/f-o-a-m/hs-abci/master/hs-abci-examples/nameservice/images/kibana-discover.png" width="25%"/>
+
+For example, after running `nameservice` tests, we can add a filter to see
+event logs for `NameClaimed`, `NameRemapped` and `NameDeleted`:
+
+<img src="https://raw.githubusercontent.com/f-o-a-m/hs-abci/master/hs-abci-examples/nameservice/images/create-filter.png" width="75%"/>

hs-abci-examples/nameservice/src/Nameservice/Application.hs

@@ -14,7 +14,6 @@ import qualified Tendermint.SDK.BaseApp          as BaseApp
 import           Tendermint.SDK.Crypto           (Secp256k1)
 import qualified Tendermint.SDK.Modules.Auth     as A
 
-
 type EffR =
    N.NameserviceEffs :&
    T.TokenEffs :&

hs-abci-examples/nameservice/src/Nameservice/Config.hs

@@ -51,7 +51,7 @@ makeAppConfig = do
     ddApiKey <- cs <$> MaybeT (lookupEnv "DD_API_KEY")
     pure $ P.MetricsScrapingConfig prometheusPort ddApiKey
   metricsMap <- newMVar empty
-  c <- BaseApp.makeContext (KL.InitialLogNamespace "dev" "simple-storage") prometheusEnv
+  c <- BaseApp.makeContext (KL.InitialLogNamespace "dev" "nameservice") prometheusEnv
   prometheusServer <- newIORef Nothing
   addScribesToLogEnv $
     AppConfig { _baseAppContext = c

hs-abci-examples/nameservice/src/Nameservice/Modules/Nameservice/Keeper.hs

@@ -9,7 +9,7 @@ import           Nameservice.Modules.Nameservice.Messages
 import           Nameservice.Modules.Nameservice.Types
 import           Nameservice.Modules.Token                (Token, TokenEffs,
                                                            burn, mint, transfer)
-import           Polysemy                                 (Member, Members, Sem,
+import           Polysemy                                 (Members, Sem,
                                                            interpret, makeSem)
 import           Polysemy.Error                           (Error, mapError,
                                                            throw)
@@ -50,7 +50,7 @@ eval = mapError BaseApp.makeAppError . evalNameservice
 --------------------------------------------------------------------------------
 
 setName
-  :: Member (Output BaseApp.Event) r
+  :: Members [BaseApp.Logger, Output BaseApp.Event] r
   => Members NameserviceEffs r
   => SetName
   -> Sem r ()
@@ -63,14 +63,17 @@ setName SetName{..} = do
         then throw $ UnauthorizedSet "Setter must be the owner of the Name."
         else do
           putWhois setNameName currentWhois {whoisValue = setNameValue}
-          BaseApp.emit NameRemapped
-             { nameRemappedName = setNameName
-             , nameRemappedNewValue = setNameValue
-             , nameRemappedOldValue = whoisValue
-             }
+          let event = NameRemapped
+                { nameRemappedName = setNameName
+                , nameRemappedNewValue = setNameValue
+                , nameRemappedOldValue = whoisValue
+                }
+          BaseApp.emit event
+          BaseApp.addContext (BaseApp.ContextEvent event) $
+            BaseApp.log BaseApp.Debug (cs $ BaseApp.makeEventType (Proxy :: Proxy NameRemapped))
 
 deleteName
-  :: Members [Token, Output BaseApp.Event] r
+  :: Members [BaseApp.Logger, Token, Output BaseApp.Event] r
   => Members NameserviceEffs r
   => DeleteName
   -> Sem r ()
@@ -84,13 +87,16 @@ deleteName DeleteName{..} = do
         else do
           mint deleteNameOwner whoisPrice
           deleteWhois deleteNameName
-          BaseApp.emit NameDeleted
-            { nameDeletedName = deleteNameName
-            }
+          let event = NameDeleted
+                { nameDeletedName = deleteNameName
+                }
+          BaseApp.emit event
+          BaseApp.addContext (BaseApp.ContextEvent event) $
+            BaseApp.log BaseApp.Debug (cs $ BaseApp.makeEventType (Proxy :: Proxy NameDeleted))
 
 
 buyName
-  :: Member (Output BaseApp.Event) r
+  :: Members [BaseApp.Logger, Output BaseApp.Event] r
   => Members TokenEffs r
   => Members NameserviceEffs r
   => BuyName
@@ -108,7 +114,7 @@ buyName msg = do
     Just whois -> buyClaimedName msg whois
     where
       buyUnclaimedName
-        :: Member (Output BaseApp.Event) r
+        :: Members [BaseApp.Logger, Output BaseApp.Event] r
         => Members TokenEffs r
         => Members NameserviceEffs r
         => BuyName
@@ -121,17 +127,21 @@ buyName msg = do
               , whoisPrice = buyNameBid
               }
         putWhois buyNameName whois
-        BaseApp.emit NameClaimed
-          { nameClaimedOwner = buyNameBuyer
-          , nameClaimedName = buyNameName
-          , nameClaimedValue = buyNameValue
-          , nameClaimedBid = buyNameBid
-          }
+        let event = NameClaimed
+              { nameClaimedOwner = buyNameBuyer
+              , nameClaimedName = buyNameName
+              , nameClaimedValue = buyNameValue
+              , nameClaimedBid = buyNameBid
+              }
+        BaseApp.emit event
+        BaseApp.addContext (BaseApp.ContextEvent event) $
+          BaseApp.log BaseApp.Debug (cs $ BaseApp.makeEventType (Proxy :: Proxy NameClaimed))
+
 
       buyClaimedName
         :: Members NameserviceEffs r
         => Members TokenEffs r
-        => Member (Output BaseApp.Event) r
+        => Members [BaseApp.Logger, Output BaseApp.Event] r
         => BuyName
         -> Whois
         -> Sem r ()
@@ -145,11 +155,14 @@ buyName msg = do
                                                  , whoisPrice = buyNameBid
                                                  , whoisValue = buyNameValue
                                                  }
-               BaseApp.emit NameClaimed
-                 { nameClaimedOwner = buyNameBuyer
-                 , nameClaimedName = buyNameName
-                 , nameClaimedValue = buyNameValue
-                 , nameClaimedBid = buyNameBid
-                 }
+               let event = NameClaimed
+                     { nameClaimedOwner = buyNameBuyer
+                     , nameClaimedName = buyNameName
+                     , nameClaimedValue = buyNameValue
+                     , nameClaimedBid = buyNameBid
+                     }
+               BaseApp.emit event
+               BaseApp.addContext (BaseApp.ContextEvent event) $
+                 BaseApp.log BaseApp.Debug (cs $ BaseApp.makeEventType (Proxy :: Proxy NameClaimed))
              else throw (InsufficientBid "Bid must exceed the price.")
 

hs-abci-examples/nameservice/src/Nameservice/Modules/Nameservice/Types.hs

@@ -107,6 +107,7 @@ instance FromJSON NameClaimed where
 instance BaseApp.ToEvent NameClaimed where
   makeEventType _ = "NameClaimed"
 instance BaseApp.FromEvent NameClaimed
+instance BaseApp.Select NameClaimed
 
 data NameRemapped = NameRemapped
   { nameRemappedName     :: Name
@@ -124,6 +125,7 @@ instance FromJSON NameRemapped where
 instance BaseApp.ToEvent NameRemapped where
   makeEventType _ = "NameRemapped"
 instance BaseApp.FromEvent NameRemapped
+instance BaseApp.Select NameRemapped
 
 data NameDeleted = NameDeleted
   { nameDeletedName :: Name
@@ -139,3 +141,4 @@ instance FromJSON NameDeleted where
 instance BaseApp.ToEvent NameDeleted where
   makeEventType _ = "NameDeleted"
 instance BaseApp.FromEvent NameDeleted
+instance BaseApp.Select NameDeleted

hs-abci-examples/nameservice/src/Nameservice/Modules/Token/Keeper.hs

@@ -3,6 +3,8 @@
 module Nameservice.Modules.Token.Keeper where
 
 import           Data.Maybe                         (fromMaybe)
+import           Data.Proxy
+import           Data.String.Conversions            (cs)
 import           Nameservice.Modules.Token.Messages (FaucetAccount (..))
 import           Nameservice.Modules.Token.Types    (Amount (..),
                                                      Faucetted (..),
@@ -45,15 +47,19 @@ eval = mapError BaseApp.makeAppError . evalToken
 --------------------------------------------------------------------------------
 
 faucetAccount
-  :: Members [Error TokenError, Output BaseApp.Event, Token] r
+  :: Members [BaseApp.Logger, Output BaseApp.Event] r
+  => Members TokenEffs r
   => FaucetAccount
   -> Sem r ()
 faucetAccount FaucetAccount{..} = do
   mint faucetAccountTo faucetAccountAmount
-  BaseApp.emit Faucetted
-    { faucettedAccount = faucetAccountTo
-    , faucettedAmount = faucetAccountAmount
-    }
+  let event = Faucetted
+        { faucettedAccount = faucetAccountTo
+        , faucettedAmount = faucetAccountAmount
+        }
+  BaseApp.emit event
+  BaseApp.addContext (BaseApp.ContextEvent event) $
+    BaseApp.log BaseApp.Debug (cs $ BaseApp.makeEventType (Proxy :: Proxy Faucetted))
 
 getBalance
   :: Member Token r
@@ -63,7 +69,8 @@ getBalance address =
   fromMaybe (Amount 0) <$> getBalance' address
 
 transfer
-  :: Members [Error TokenError, Output BaseApp.Event, Token] r
+  :: Members [BaseApp.Logger, Output BaseApp.Event] r
+  => Members TokenEffs r
   => Address
   -> Amount
   -> Address
@@ -79,15 +86,18 @@ transfer addr1 amount addr2 = do
       -- update both balances
       putBalance addr1 newBalance1
       putBalance addr2 newBalance2
-      BaseApp.emit $ TransferEvent
-        { transferEventAmount = amount
-        , transferEventTo = addr2
-        , transferEventFrom = addr1
-        }
+      let event = TransferEvent
+            { transferEventAmount = amount
+            , transferEventTo = addr2
+            , transferEventFrom = addr1
+            }
+      BaseApp.emit event
+      BaseApp.addContext (BaseApp.ContextEvent event) $
+        BaseApp.log BaseApp.Debug (cs $ BaseApp.makeEventType (Proxy :: Proxy TransferEvent))
     else throw (InsufficientFunds "Insufficient funds for transfer.")
 
 burn
-  :: Members [Error TokenError, Token] r
+  :: Members TokenEffs r
   => Address
   -> Amount
   -> Sem r ()

hs-abci-examples/nameservice/src/Nameservice/Modules/Token/Router.hs

@@ -5,15 +5,14 @@ import           Nameservice.Modules.Token.Keeper   (TokenEffs, burn,
 import           Nameservice.Modules.Token.Messages (Burn (..),
                                                      TokenMessage (..),
                                                      Transfer (..))
-import           Polysemy                           (Member, Members, Sem)
-import           Polysemy.Output                    (Output)
-import           Tendermint.SDK.BaseApp             (Event)
+import           Polysemy                           (Members, Sem)
+import           Tendermint.SDK.BaseApp             (BaseAppEffs)
 import           Tendermint.SDK.Types.Message       (Msg (..))
 import           Tendermint.SDK.Types.Transaction   (RoutedTx (..), Tx (..))
 
 router
   :: Members TokenEffs r
-  => Member (Output Event) r
+  => Members BaseAppEffs r
   => RoutedTx TokenMessage
   -> Sem r ()
 router (RoutedTx Tx{txMsg}) =