Avoid p2p dependency (#160)

This PR replaces all p2p endpoint requests with service endpoint requests avoiding the need to have a p2p configuration. This simplifies `chainweb-data`'s command line interface, but more importantly, it avoids depending on CW-N's p2p endpoint, which is supposed to be for peer to peer communication only and is subject to backwards-incompatible changes.

In particular, this PR moves the following requests to the service endpoint:
* [`GET <node-endpoint>/chainweb/0.0/<network-id>/cut`](https://github.com/kadena-io/chainweb-data/pull/160/files#diff-a55128f6e70aa85de943e6bbb87a99cfcf9fba62b1d9852376fdcfc3c6556681R188)
* [`GET <node-endpoint>/chainweb/0.0/<network-id>/chain/<chain-id>/payload/<block-hash>/outputs`](https://github.com/kadena-io/chainweb-data/pull/160/files#diff-a55128f6e70aa85de943e6bbb87a99cfcf9fba62b1d9852376fdcfc3c6556681R171)

* Avoid depending on the CW-N p2p endpoint

* Fix typo

* Add a comment explaining ignoredP2pParser

* Remove mentions of --p2p-{host,port} from README

* Unrelated minor README fix

We had used backticks for verbatim even though our README is .org

* Bump the chainweb-api commit

This commit allows us to run a fill operation against a fast-development
node, facilitating the testing for this PR

README.org

@@ -78,24 +78,23 @@ configured user permissions for a user named ~joe~, the following would connect
 to a local Postgres database at port 5432:
 
 #+begin_example
-chainweb-data <command> --service-host=<node> --p2p-host=<node> --dbuser=joe --dbname=chainweb-data
+chainweb-data <command> --service-host=<node> --dbuser=joe --dbname=chainweb-data
 #+end_example
 
 *** via a Postgres Connection String
 
 #+begin_example
-chainweb-data <command> --service-host=<node> --p2p-host=<node> --dbstring="host=localhost port=5432..."
+chainweb-data <command> --service-host=<node> --dbstring="host=localhost port=5432..."
 #+end_example
 
 ** Connecting to a Node
 
 ~chainweb-data~ syncs its data from a running ~chainweb-node~. The node's
-P2P address is specified with the ~--p2p-host~ command. The node's
 Service address is specified with the ~--service-host~ command.
-If custom ports are used, you can specify them with ~--service-port~ and ~--p2p-port~
+If a custom service enpoint port is used, you can specify it with ~--service-port~
 
 #+begin_example
-chainweb-data <command> --service-host=foo.chainweb.com --p2p-host=foo.chainweb.com ...
+chainweb-data <command> --service-host=foo.chainweb.com ...
 #+end_example
 
 *** Configuring the Node
@@ -133,7 +132,7 @@ will automatically run fill once a day to populate the DB with missing blocks.
 configuration value is ~true~.
 
 #+begin_example
-  > chainweb-data listen --service-host=foo.chainweb.com --p2p-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
+  > chainweb-data listen --service-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
   DB Tables Initialized
   28911337084492566901513774
 #+end_example
@@ -146,11 +145,11 @@ As a new block comes in, its chain number is printed as a single digit.
 ~server~ is just like ~listen~, but also runs an HTTP server that serves a
 few endpoints for doing common queries.
 Additionally, it can serve an OpenAPI v3 spec of the API when the hidden
-`--serve-swagger-ui` option is enabled, offering a basic interface for interacting
+~--serve-swagger-ui~ option is enabled, offering a basic interface for interacting
 with the API. This feature, however, is kept unofficial for now due to
 its rudimentary documentation.
 
-By specifying the optional `--no-listen` argument, the server can be made read-only,
+By specifying the optional ~--no-listen~ argument, the server can be made read-only,
 allowing multiple servers to serve from the same database.
 
 **** Endpoints
@@ -199,7 +198,7 @@ so it's possible to keep searching by making successive calls with subsequent ~C
 been improved to encompass all block filling operations.
 
 #+begin_example
-  > chainweb-data fill --service-host=foo.chainweb.com --p2p-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
+  > chainweb-data fill --service-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
 #+end_example
 
 *** backfill
@@ -215,7 +214,7 @@ chain first via ~listen~ before doing ~backfill~! If ~backfill~ detects any
 empty chains, it won't proceed.
 
 #+begin_example
-  > chainweb-data backfill --service-host=foo.chainweb.com --p2p-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
+  > chainweb-data backfill --service-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
   DB Tables Initialized
   Backfilling...
   [INFO] Processed blocks: 1000. Progress sample: Chain 9, Height 361720
@@ -245,7 +244,7 @@ releases. Use the ~fill~ command instead.
 ~single~.
 
 #+begin_example
-  > chainweb-data gaps --service-host=foo.chainweb.com --p2p-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
+  > chainweb-data gaps --service-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
   DB Tables Initialized
   [INFO] Processed blocks: 1000. Progress sample: Chain 9, Height 361624
   [INFO] Processed blocks: 2000. Progress sample: Chain 9, Height 362938
@@ -257,7 +256,7 @@ releases. Use the ~fill~ command instead.
 ~single~ allows you to sync a block at any location in the blockchain.
 
 #+begin_example
-  > chainweb-data single --chain=0 --height=200 --service-host=foo.chainweb.com --p2p-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
+  > chainweb-data single --chain=0 --height=200 --service-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
   DB Tables Initialized
   [INFO] Filled in 1 blocks.
 #+end_example
@@ -271,5 +270,5 @@ when orphans/forks are present at that height.
 ~check-schema~ is used to perform a check of the ORM definitions against the DB schema.
 
 #+begin_example
-  > chainweb-data check-schema --service-host=foo.chainweb.com --p2p-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
+  > chainweb-data check-schema --service-host=foo.chainweb.com --dbuser=joe --dbname=chainweb-data
 #+end_example

cabal.project

@@ -18,8 +18,8 @@ source-repository-package
 source-repository-package
     type: git
     location: https://github.com/kadena-io/chainweb-api.git
-    tag: ef2bee367136e298dfb45ea118d122350e7b3bc7
-    --sha256: sha256-VQ/JhpZaINeTZHNt7TZ9WaVGIXJEf9bDQ51cSG+auVI=
+    tag: 7fb1e73d2f7a0f0dd5703873c7204287cd7e8d25
+    --sha256: sha256-KaXVlMBVy6hDrKL2z54+jH4CIVjoWbS4X2RwT/QUMNo=
 
 source-repository-package
     type: git

haskell-src/exec/Chainweb/Gaps.hs

@@ -32,7 +32,6 @@ import           Network.Connection (TLSSettings(TLSSettingsSimple))
 import           Network.HTTP.Client
 import           Network.HTTP.Client.TLS
 import           System.Logger hiding (logg)
-import qualified System.Logger as S
 import           System.Exit (exitFailure)
 import           Text.Printf
 
@@ -123,7 +122,6 @@ _test_headersBetween_and_payloadBatch = do
       manager <- newManager $ mkManagerSettings (TLSSettingsSimple True False False) Nothing
       let urlHost_ = "localhost"
           serviceUrlScheme = UrlScheme Http $ Url urlHost_ 1848
-          p2pUrl = Url urlHost_ 443
           nodeInfo = NodeInfo
             {
                 _nodeInfo_chainwebVer = "mainnet01"
@@ -137,7 +135,6 @@ _test_headersBetween_and_payloadBatch = do
           _env_httpManager = manager
         , _env_dbConnPool = undefined
         , _env_serviceUrlScheme = serviceUrlScheme
-        , _env_p2pUrl = p2pUrl
         , _env_nodeInfo = nodeInfo
         , _env_chainsAtHeight = undefined
         , _env_logger = undefined
@@ -185,7 +182,6 @@ _test_getBlockGaps dbHost dbPort dbUser password dbName = withHandleBackend defa
           _env_httpManager = m
         , _env_dbConnPool = pool
         , _env_serviceUrlScheme = UrlScheme Http $ Url "localhost" 1848
-        , _env_p2pUrl = Url "localhost" 443
         , _env_nodeInfo = ni
         , _env_chainsAtHeight = fromMaybe (error "chainsAtMinHeight missing") $ map (second (map (ChainId . fst))) <$> (_nodeInfo_graphs ni)
         , _env_logger = lr

haskell-src/exec/Chainweb/Lookups.hs

@@ -167,7 +167,7 @@ payloadWithOutputs env (T2 cid0 hsh0) = do
   pure res
   where
     v = _nodeInfo_chainwebVer $ _env_nodeInfo env
-    url = showUrlScheme (UrlScheme Https $ _env_p2pUrl env) <> T.unpack query
+    url = showUrlScheme (_env_serviceUrlScheme env) <> T.unpack query
     query = "/chainweb/0.0/" <> v <> "/chain/" <> cid <> "/payload/" <> hsh <> "/outputs"
     cid = T.pack $ show cid0
     hsh = unDbHash hsh0
@@ -184,8 +184,8 @@ queryCut :: Env -> IO (Either ApiError ByteString)
 queryCut e = do
   let v = _nodeInfo_chainwebVer $ _env_nodeInfo e
       m = _env_httpManager e
-      u = _env_p2pUrl e
-      url = printf "%s/chainweb/0.0/%s/cut" (showUrlScheme $ UrlScheme Https u) (T.unpack v)
+      u = _env_serviceUrlScheme e
+      url = printf "%s/chainweb/0.0/%s/cut" (showUrlScheme u) (T.unpack v)
   req <- parseRequest url
   res <- handleRequest req m
   pure $ responseBody <$> res
@@ -350,7 +350,7 @@ mkTxEvents :: Int64 -> ChainId -> DbHash BlockHash -> (CW.Transaction,Transactio
 mkTxEvents height cid blk (tx,txo) = zipWith (mkEvent cid height blk (Just rk)) (_toutEvents txo) [0..]
   where
     rk = DbHash $ hashB64U $ CW._transaction_hash tx
-    
+
 
 mkEvent :: ChainId -> Int64 -> DbHash BlockHash -> Maybe (DbHash TxHash) -> Value -> Int64 -> Event
 mkEvent (ChainId chainid) height block requestkey ev idx = Event

haskell-src/exec/Main.hs

@@ -67,10 +67,9 @@ main = do
               logg Info $ "Constructing rich list using given db-path: " <> fromString fp
               return fp
           richList logg fp version
-        Args c pgc us u _ ms mbMigFolder -> do
+        Args c pgc us _ ms mbMigFolder -> do
           logg Info $ "Using database: " <> fromString (show pgc)
           logg Info $ "Service API: " <> fromString (showUrlScheme us)
-          logg Info $ "P2P API: " <> fromString (showUrlScheme (UrlScheme Https u))
           withCWDPool pgc $ \pool -> do
             runMigrations pool logg ms mbMigFolder
             let mgrSettings = mkManagerSettings (TLSSettingsSimple True False False) Nothing
@@ -82,7 +81,7 @@ main = do
                 case mcids of
                   Nothing -> logg Error "Node did not have graph information" >> exitFailure
                   Just cids -> do
-                    let !env = Env m pool us u ni cids logg
+                    let !env = Env m pool us ni cids logg
                     case c of
                       Listen etlenv ->
                           case etlenv of
@@ -112,7 +111,7 @@ main = do
       & loggerConfigThreshold .~ level
     backendConfig = defaultHandleBackendConfig
     getLevel = \case
-      Args _ _ _ _ level _ _ -> level
+      Args _ _ _ level _ _ -> level
       RichListArgs _ level _ -> level
       MigrateOnly _ level _ -> level
       CheckSchema _ level -> level

haskell-src/lib/ChainwebData/Env.hs

@@ -69,7 +69,7 @@ import           Text.Printf
 type MigrationsFolder = FilePath
 
 data Args
-  = Args Command Connect UrlScheme Url LogLevel MigrationAction (Maybe MigrationsFolder)
+  = Args Command Connect UrlScheme LogLevel MigrationAction (Maybe MigrationsFolder)
     -- ^ arguments for all but the richlist command
   | RichListArgs NodeDbPath LogLevel ChainwebVersion
     -- ^ arguments for the Richlist command
@@ -81,7 +81,6 @@ data Env = Env
   { _env_httpManager :: Manager
   , _env_dbConnPool :: Pool Connection
   , _env_serviceUrlScheme :: UrlScheme
-  , _env_p2pUrl :: Url
   , _env_nodeInfo :: NodeInfo
   , _env_chainsAtHeight :: [(BlockHeight, [ChainId])]
   , _env_logger :: LogFunctionIO Text
@@ -260,10 +259,15 @@ envP = Args
   <$> commands
   <*> connectP
   <*> urlSchemeParser "service" 1848
-  <*> urlParser "p2p" 443
   <*> logLevelParser
   <*> migrationP
   <*> migrationsFolderParser
+  -- We keep the p2p options around for backwards compatibility, but they're unused
+  <* ignoredP2pParser
+  where
+    ignoredP2pParser = ()
+      <$ strOption (long "p2p-host" <> internal <> value ("unused" :: String))
+      <* strOption (long "p2p-port" <> internal <> value ("unused" :: String))
 
 migrationsFolderParser :: Parser (Maybe MigrationsFolder)
 migrationsFolderParser = optional $ strOption