Update to ILPv4 + LPI2 #401

justmoon · 2017-12-18T17:21:58Z

Depends on interledger-deprecated/ilp-compat-plugin#7

emschwartz

Overall LGTM, but I think @sentientwaffle should also review this because he's more familiar with a lot of the code this PR touches

emschwartz · 2017-12-18T17:30:40Z

package.json

    "five-bells-shared": "^25.1.0",
    "ilp": "~11.2.0",
-    "ilp-packet": "~1.3.0",
+    "ilp-packet": "^2.0.0",
    "ilp-plugin-bells": "^15.0.0",


Should we stop including plugin bells?

emschwartz · 2017-12-18T17:32:33Z

src/lib/executeSourceTransfer.js

-  const fulfillPromise = promiseRetry(retryOpts, function (retry, number) {
-    return Promise.resolve(plugin.fulfillCondition(sourceTransferID, fulfillment, fulfillmentData))
-      .catch(function (err) {
-        if (shouldRetry(err)) {


Should we add fulfillment retry logic to the ilp-compat-plugin?

I do think that fulfillment retries should be in the plugin because it depends on the ledger protocol if and how you want to retry.

Should it be in ilp-plugin-compat? I don't think it's worth the effort. Retry logic is an optimization, it's only to minimize unnecessary connector losses. I'd say we migrate to LPI2 and then add retry logic per plugin. Different plugins may want to handle retries quite differently, depending on whether they're TCP or UDP based, session or request based or to deal with some other ledger/transport-specific quirks.

emschwartz · 2017-12-18T17:37:47Z

src/lib/utils.js

@@ -22,40 +20,6 @@ function getPairs (arr) {
  }, [])
 }

-/**
- * Deterministically generate a UUID from a secret and a public input.


Right now Mojaloop depends on having the same transfer ID across all of the ledgers. Taking the transfer id out of the LPI will make it more difficult to upgrade the connector used in that project. Do we have any alternative way of attaching the transfer ID from the previous ledger and making the connector use the same one on the outgoing transfer, or do we need to just make Mojaloop not depend on having the same transfer IDs?

sentientwaffle

LGTM except for a couple minor notes

sentientwaffle · 2017-12-18T19:00:41Z

src/lib/config.js

@@ -149,6 +149,8 @@ function getLocalConfig () {
  features.debugReplyNotifications =
    Config.castBool(Config.getEnv(envPrefix, 'DEBUG_REPLY_NOTIFICATIONS'))

+  const account = Config.getEnv(envPrefix, 'ILP_ADDRESS') || 'unknown'


Add ILP_ADDRESS to the README.

sentientwaffle · 2017-12-18T19:10:18Z

src/lib/route-broadcaster.js

@@ -39,9 +39,10 @@ class RouteBroadcaster {

    this.autoloadPeers = config.autoloadPeers
    this.defaultPeers = config.peers
-    // peersByLedger is stored in the form { ledgerPrefix ⇒ { connectorAddress ⇒ true } }
+    // peersByLedger is stored in the form { ledgerPrefix ⇒ true }


s/peersByLedger/peers

michielbdejong · 2017-12-18T19:39:10Z

src/lib/route-broadcaster.js

-        // timeout the plugin.sendRequest Promise just so we don't have it hanging around forever
-        timeout: this.routeBroadcastInterval
-      })
+    const broadcastPromise = this.ledgers.getPlugin(adjacentLedger).sendRequest({


Does this.ledgers.getPlugin(adjacentLedger) return an LPI1 or an LPI2 interface? Because the sendRequest method, needed for broadcasting routes, seems to be absent in LPI2?

It's not in the spec but it is in the ilp-compat-plugin as a deprecated feature

sendRequest won't exist when compat() returns the original LPI2 plugin (https://github.com/interledgerjs/ilp-compat-plugin/blob/d8331126a92ad2b65a7e7c4513ce44c13d485340/index.js#L31-L33).

So then this PR will break route broadcasts?

LPI1 plugins will work thanks to compat; LPI2 plugins will be broken.

They should be switched to use either interledger payments or the internet

in a similar way to @justmoon 's DHCP on ILP proposal (for asking your peer to assign you an ILP address), it would be easy to send a payment to your peer for which the destination is something like peer.broadcast, and the condition is either trivially fulfillable (if you're paying to send broadcasts) or unfulfillable (if you're not paying to send broadcasts). I don't see why that's worse/harder than using messaging.

Sure, that's what plugin-xrp-escrow already does. And if a BTP connection is available between the connectors then they can just use a BTP Message packet. But that's what plugin.sendRequest already nicely abstracts, and we agree that ilp-connector needs it, so then instead of removing code from the stack, we have just split the ledger plugin into a separate transfer plugin and messaging plugin, which is just moving code around.

We don't actually need a messaging plugin anymore though, because transfers are expected to be just as fast as messages. And setting the amount to 1 means that the impact on your liquidity is negligible. This just lets us keep plugins as simple as possible, which should make them less buggy and more secure

Ben and I discussed this on Slack and reduced this question to interledger/rfcs#357

codecov-io · 2017-12-19T16:26:35Z

Codecov Report

Merging #401 into master will decrease coverage by 10.27%.
The diff coverage is 74.54%.

@@             Coverage Diff             @@
##           master     #401       +/-   ##
===========================================
- Coverage   84.82%   74.54%   -10.28%     
===========================================
  Files          22       51       +29     
  Lines         336     1756     +1420     
  Branches       49      278      +229     
===========================================
+ Hits          285     1309     +1024     
- Misses         51      447      +396

Impacted Files	Coverage Δ
src/errors/invalid-packet-error.ts	`100% <100%> (ø)`
src/services/routing-table.ts	`100% <100%> (ø)`
src/errors/invalid-amount-specified-error.ts	`100% <100%> (ø)`
src/errors/unacceptable-amount-error.ts	`100% <100%> (ø)`
src/errors/unreachable-error.ts	`100% <100%> (ø)`
src/errors/unacceptable-expiry-error.ts	`100% <100%> (ø)`
src/lib/ilp-errors.ts	`100% <100%> (ø)`
src/errors/no-route-found-error.ts	`100% <100%> (ø)`
src/common/log.ts	`100% <100%> (ø)`
src/errors/invalid-fulfillment-error.ts	`100% <100%> (ø)`
... and 92 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 25963d7...bb8f737. Read the comment docs.

emschwartz

Looks awesome to me. @sentientwaffle should definitely review this again

emschwartz · 2017-12-27T08:01:58Z

src/controllers/ildcp-host.js

+    writer.writeVarOctetString(Buffer.from(sourceAccount, 'ascii'))
+    writer.writeUInt8(info.currencyScale)
+    writer.writeVarOctetString(Buffer.from(info.currency, 'utf8'))
+    return writer.getBuffer()


This encoding should probably be added to the ilp-packet module

Yeah, I wasn't sure where to put it. It's not ilp-packet layer, so I don't think ilp-packet is actually the right place unless we turn that into ilp-formats. Plus - is it really worth modularizing four lines of code?

Fair enough

emschwartz · 2017-12-27T08:04:45Z

src/controllers/ilp-prepare.js

+      const { nextHop, nextHopPacket } = await this.routeBuilder.getNextHopPacket(sourceAccount, parsedPacket)
+
+      log.debug('sending outbound ilp prepare. destination=%s amount=%s', destination, nextHopPacket.amount)
+      const result = await this.sendDataToPeer(nextHop, IlpPacket.serializeIlpPrepare(nextHopPacket))


Shouldn't we modify the ILP packet in place rather than re-encoding? (That could also be a separate PR though)

Imho, that's premature optimization. Not long ago, we were ok going with a text-based format that would have been an order of magnitude slower. It's just not worth worrying about trying to get zero-copy performance at this stage. There are much lower hanging fruit.

emschwartz · 2017-12-27T08:10:29Z

src/errors/remote-quote-error.js

+
+    // TODO: Is this the right error code? Maybe we should pass on the error we
+    //   we received where possible?
+    this.ilpErrorCode = codes.T00_INTERNAL_ERROR


I'm not sure these errors will be temporary if other connectors just stop supporting ILQP

Good point - what's the right error code here? Or should we add one?

(I ran into a few cases where I was like: might be good to add a code for this - or make an existing code less specific.)

emschwartz · 2017-12-27T08:12:40Z

src/lib/plugin-store.js

+      logging: log.debug,
+      omitNull: true,
+      // All transactions should be done with isolation level SERIALIZABLE
+      // TOOD:


What's this TODO for?

This "TOOD" was going to say: So we still need SERIALIZABLE isolation level. (My guess is no.)

emschwartz · 2017-12-27T08:18:17Z

src/services/balances.js

+  }
+
+  handleMoney (amount) {
+    // TODO: Implement balance logic


This should be implemented ASAP after this PR is merged

Yeah. I think that's a job for @sharafian - he's written that code about 15 times already since it used to be in each plugin.

emschwartz · 2017-12-27T08:20:45Z

src/services/route-builder.js

+
+  // Make sure that neither amount exceeds 15 significant digits.
+  if (rate.gt(1)) {
+    return new LiquidityCurve([ [0, 0], [ PROBE_AMOUNT / rate, PROBE_AMOUNT ] ])


Shouldn't this be using bignumber.js' math functions?

Copied the existing code to make sure the results are exactly the same - I still like that idea so we don't have to update the tests as much. But we'll see once we get to updating the integration tests how it'll play out.

sentientwaffle · 2017-12-28T18:08:11Z

README.md

  }, {
    "targetPrefix": "cny.",
-    "connectorLedger": "ilpdemo.red."
-    "connectorAccount": "lpdemo.red.cny_connector"
+    "peerAddress": "ilpdemo.red."


I think the peerAddress for this route should be ilpdemo.blue. (or something other than ilpdemo.red.), otherwise it will be caught by the targetPrefix: "" route anyway.

sentientwaffle · 2017-12-28T18:29:41Z

src/services/route-broadcaster.js

+
+  async start () {
+    try {
+      await this.reloadLocalRoutes()


this.reloadLocalRoutes is sync, no need to await.

sentientwaffle · 2017-12-28T18:29:53Z

src/services/route-broadcaster.js

+    await new Promise(resolve => setTimeout(resolve, this.config.routeBroadcastInterval))
+
+    try {
+      await this.reloadLocalRoutes()


this.reloadLocalRoutes is sync, no need to await.

sentientwaffle · 2017-12-28T18:39:09Z

src/backends/fixerio/index.js

-   * @param {String} params.destination_ledger The URI of the destination ledger
-   * @param {String} params.source_currency The source currency
-   * @param {String} params.destination_currency The destination currency
+   * @param {String} sourceAccount The URI of the source ledger


These are ILP addresses, not ledger URIs (the code is correct; the comment is wrong).

sentientwaffle · 2017-12-28T18:48:44Z

src/backends/fixerio/index.js

-   * @param {String} params.destination_ledger The URI of the destination ledger
-   * @param {String} params.source_amount The amount of the source asset we want to send
-   * @param {String} params.destination_amount The amount of the destination asset we want to send
+   * @param {String} params.sourceAccount The URI of the source ledger


sourceAccount and destinationAccount are ILP addresses.

Not anymore after config refactor.

sentientwaffle · 2017-12-28T19:42:23Z

src/controllers/ilp-prepare.js

+          })
+      }
+
+      this.backend.submitPayment({


Should the submitPayment call be inside of the result[0] === TYPE_ILP_FULFILL block? Otherwise it might be submitted even if sendDataToPeer replied with a rejection.

sentientwaffle · 2017-12-28T21:19:41Z

src/lib/utils.js

+
+      if (nextSegmentEnd === -1) {
+        prefix = address
+        return false


No need to return false, just return.

sentientwaffle · 2017-12-28T21:30:17Z

src/routing/peer.js

+const PrefixMap = require('./prefix-map')
+const log = require('../common').log.create('routing-peer')
+
+const PEER_PROTOCOL_PREFIX = 'peer'


ilp-prepare.js declares this as 'peer.'.

sentientwaffle · 2017-12-28T21:35:51Z

src/routing/peer.js

+    const newRoutes = routes.filter(route => route.epoch > this.epoch && route.nextHop !== this.address).map(route => ({
+      source_ledger: this.address,
+      destination_ledger: route.prefix,
+      min_message_window: 1,


This should reference the configured minMessageWindow.

Removed for now.

sentientwaffle · 2017-12-28T21:42:39Z

src/routing/peer.js

+        data: {
+          new_routes: newRoutes,
+          hold_down_time: holdDownTime,
+          unreachable_through_me: unreachableAccounts,


unreachableThroughMe, not unreachableAccounts.

sentientwaffle · 2018-01-02T19:39:18Z

src/services/accounts.js

+
+    const store = this.store.getPluginStore(accountAddress)
+
+    creds.options.prefix = accountAddress


Is this line necessary (see line 93 prefix: accountAddress + '.',)? If it is, shouldn't the prefix be set with a + '.' suffix?

sentientwaffle · 2018-01-02T21:09:09Z

src/services/config.js

+    features.debugReplyNotifications =
+    Config.castBool(Config.getEnv(envPrefix, 'DEBUG_REPLY_NOTIFICATIONS'))
+
+    this.address = Config.getEnv(envPrefix, 'ILP_ADDRESS') || ''


This parameter should be documented in the README.

Yeah, README needs a major rewrite...

sentientwaffle · 2018-01-10T15:44:39Z

src/lib/route-builder.js

-    this._verifyLedgerIsConnected(nextHop.destinationLedger)
-
-    // As long as the fxSpread > slippage, the connector won't lose money.
-    const expectedSourceAmount = new BigNumber(nextHop.sourceAmount).times(1 - this.slippage)


This slippage computation wasn't added back in the refactor (the config option still exists, though).

sentientwaffle · 2018-01-16T17:19:48Z

README.md

+| -------------- | ------- | ------------------------------------------------------------------------ |
+| `[]`           | object  | Object describing middleware instance.                                   |
+| `[].type`      | string  | NPM module that should be `require`d to load the middleware constructor. |
+| `[].priority`  | integer | Priority at which this middleware should be inserted. Must be unique.    |


Is this field implemented? I don't see it used in the code.

Nope, it was from a previous version, I'll take it out!

Both of these changes are needed to get the integration tests passing.

sentientwaffle · 2018-01-18T19:08:34Z

README.md

+| `*.assetCode`                 | string  | Currency code or other asset identifier that will be passed to the backend to select the correct rate for this account.                                                                                                                                                                                     |
+| `*.assetScale`                | integer | Interledger amounts are integers, but most currencies are typically represented as fractional units, e.g. cents. This property defines how many Interledger units make up one regular units. For dollars, this would usually be set to 9, so that Interledger amounts are expressed in nanodollars.         |
+| `*.balance`                   | object  | _Optional_ Defines whether the connector should maintain and enforce a balance for this account. This setting is enforced by the built-in `balance` middleware.                                                                                                                                             |
+| `*.balance.maximumDebt`       | string  | How much money (in atomic units) this counterparty is allowed to owe us before we start rejecting their packets. Each incoming ILP prepare packet increases the amount owed, each incoming settlement lowers it. Provided as an integer in a string.                                                        |


should be balance.maximum (or the balance middleware needs to change).

emschwartz · 2018-01-19T19:26:06Z

Woohoo! What a monster PR. Nicely work @justmoon

emschwartz approved these changes Dec 18, 2017

View reviewed changes

sentientwaffle approved these changes Dec 18, 2017

View reviewed changes

michielbdejong reviewed Dec 18, 2017

View reviewed changes

michielbdejong mentioned this pull request Dec 18, 2017

Update to LPI2 interledgerjs/ilp#122

Merged

refactor: use lpi2

6279cac

justmoon force-pushed the refactor/st-lpi2 branch from 4cc5e35 to 6279cac Compare December 19, 2017 16:26

justmoon added 4 commits December 23, 2017 15:32

refactor: new routing/quoting engine

12b6a33

refactor: turn config into a class

21f9d3f

refactor: use reduct for dependency injection

9021056

refactor: use latest lpi2 version (ilp4)

b23fe4a

justmoon force-pushed the refactor/st-lpi2 branch from 9d17bb5 to b23fe4a Compare December 26, 2017 23:21

emschwartz approved these changes Dec 27, 2017

View reviewed changes

justmoon added 4 commits December 27, 2017 08:15

chore: update deps

910d77c

fix: avoid timeout preventing node from exiting

954fad9

fix: parse CONNECTOR_DB_URI correctly

c191939

chore: fix linting issues

3988c85

sentientwaffle requested changes Dec 28, 2017

View reviewed changes

justmoon added 7 commits December 28, 2017 17:39

feat: pluggable store, switch default from sqlite to leveldown

45421ff

feat: connector should log its address on startup

daeedaa

refactor: config system overhaul

427d785

refactor: routing fixes and simplified format

e7e6dfd

feat(store): make memdown its own store type.

f04711d

feat(config): add missing store config options

9b80835

refactor: reduce dependencies

49efb2a

sentientwaffle reviewed Jan 2, 2018

View reviewed changes

justmoon force-pushed the refactor/st-lpi2 branch from 4bc732c to 5ced0f2 Compare January 3, 2018 03:35

refactor: port to typescript

913ca83

justmoon force-pushed the refactor/st-lpi2 branch from 5ced0f2 to 913ca83 Compare January 3, 2018 03:58

sentientwaffle reviewed Jan 10, 2018

View reviewed changes

fix: correctly generate store config object

7b73f45

justmoon force-pushed the refactor/st-lpi2 branch 4 times, most recently from f7cf20e to 3a3cc48 Compare January 16, 2018 01:36

feat: add middleware framework

5d0867d

justmoon force-pushed the refactor/st-lpi2 branch from 3a3cc48 to 5d0867d Compare January 16, 2018 02:21

justmoon added 2 commits January 15, 2018 21:14

perf: lazily create plugin store

9b38350

style: fix linting issues

3597b98

sentientwaffle reviewed Jan 16, 2018

View reviewed changes

justmoon and others added 7 commits January 16, 2018 11:18

fix(types): plugins are event emitters

1aaa3cb

fix(middleware): always pass an object as the options

bed0e87

fix(config): remove unused config options

4367d43

chore(npm): rebuild schemas on npm run build

c36e4e6

feat: warn user about unrecognized configuration variables

bc8cea1

chore(docker): update docker file - use node 8

e09ec66

fix: handle "integer" config; relay broadcasts

38596f0

Both of these changes are needed to get the integration tests passing.

sentientwaffle reviewed Jan 18, 2018

View reviewed changes

justmoon added 2 commits January 18, 2018 23:41

chore: use nyc for code coverage

4cf3387

test: fix tests

29c1a8a

justmoon force-pushed the refactor/st-lpi2 branch from 9030e87 to 28c6873 Compare January 19, 2018 08:17

test: remove references to missing classes

b33434a

justmoon force-pushed the refactor/st-lpi2 branch from 28c6873 to b33434a Compare January 19, 2018 08:22

justmoon added 2 commits January 19, 2018 11:19

test: fix tests hanging

6ed7717

docs: update README with latest config schema

bb8f737

justmoon force-pushed the refactor/st-lpi2 branch from 8a8a992 to bb8f737 Compare January 19, 2018 19:19

justmoon merged commit 6c72764 into master Jan 19, 2018

justmoon mentioned this pull request Feb 21, 2018

Phase 0: Minimum Viable Network (Pong) interledger-deprecated/interledger#7

Open

9 tasks


		const store = this.store.getPluginStore(accountAddress)

		creds.options.prefix = accountAddress

Update to ILPv4 + LPI2 #401

Update to ILPv4 + LPI2 #401

Conversation

justmoon commented Dec 18, 2017

emschwartz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sentientwaffle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Dec 19, 2017 • edited Loading

Codecov Report

emschwartz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emschwartz commented Jan 19, 2018

codecov-io commented Dec 19, 2017 •

edited

Loading