Skip to content

Commit

Permalink
Optimize the recent transactions query (#119)
Browse files Browse the repository at this point in the history
This PR adds instrumentation around the connection pool used by the endpoint handlers to keep track of the number of requests currently pending or being served. It then uses this information in order to scale down the amount of work that's performed per request by the search endpoints in order to make it easier to maintain a fair resource allocation under load across multiple clients.

This PR still doesn't stop a single client from making a large number of search requests in parallel, but `chainweb-data` expects that this would be handled via rate limiting at the API gateway layer. In theory, it could be argued that it would be better for the gateway to handle such throttling as well, but it seems harder to implement there in practice.

This PR also increases the maximum `scanLimit` of the search endpoints from 20000 to 50000 (but subject to load throttling) in order to reduce network roundtrips. It used to take us around 50-100ms to scan 20000 rows, so now each request should take between 100-250ms, which is still low enough and now each request should scan around 10 hours worth of blockchain history.


* Optimize the recent transactions query

* Avoid the JOIN on the blocks table entirely

Now that we're reading the tx height from the transaction itself,
we don't have any reason left to JOIN on the blocks table
  • Loading branch information
enobayram authored Jan 4, 2023
1 parent d05b267 commit bddcf54
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions exec/Chainweb/Server.hs
Original file line number Diff line number Diff line change
Expand Up @@ -674,11 +674,9 @@ queryRecentTxs logger pool = do
runSelectReturningList $ select $ do
limit_ 20 $ orderBy_ (desc_ . getHeight) $ do
tx <- all_ (_cddb_transactions database)
blk <- all_ (_cddb_blocks database)
guard_ (_tx_block tx `references_` blk)
return
( (_tx_chainId tx)
, (_block_height blk)
, (_tx_height tx)
, (unBlockId $ _tx_block tx)
, (_tx_creationTime tx)
, (_tx_requestKey tx)
Expand Down

0 comments on commit bddcf54

Please sign in to comment.