-
Notifications
You must be signed in to change notification settings - Fork 20
Conversation
This PR doesn't pass tests yet, it's WIP but wanted to start the discussion. |
name: this.name, | ||
size: this.size, | ||
hash: this.multihash ? mh.toB58String(this.multihash) : undefined | ||
hash: this.multihash ? mh.toB58String(this.multihash) : null | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There's also couple of places where the code needs to handle callback vs. Promises, mainly the DAGNodeFactory methods like create(). Will update/fix that soon. |
return this._multihash ? mh.toB58String(this._multihash) : undefined | ||
} | ||
|
||
set hash() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/hash/multihash
throw ImmutableError | ||
} | ||
|
||
get hash() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/hash/multihash
Let me know if you want to hear the reasoning behind the changes. |
const DAGLink = require('./dag-link') | ||
|
||
class DAGNodeFactory { | ||
static create (data, dagLinks, hashAlg) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isn't this the same as exporting a function called create?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, I can see the value of having the create function in a separate file, having to do: dagPB.dagNodeFactory.create(..)
, seems a tad long, just to create an instance, we can scope the create
function withing dagPB.DAGNode even if it is in a separate file.
return l | ||
} | ||
|
||
// haadcode: are the .name vs .Name for backwards compatibility? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep (go-ipfs outputs json with that)
}) | ||
|
||
// haadcode: would be better to have: links = sort(links, util.linkSort) | ||
sort(links, util.linkSort) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the sort is a sort inplace, which works well with a immutable property
.then((serialized) => util.hash(hashAlg, serialized)) | ||
.then((multihash) => new DAGNode(data, links, serialized, multihash)) | ||
*/ | ||
}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't believe we are going to change this internal component to a Promised based one now. As we discussed in the past and again at the last gathering, so far the guideline is callbacks at the internals, promises+callbacks at the top level
.
This will only change if there is a huge discussion and decision here: ipfs/js-ipfs#557
} | ||
|
||
// haadcode: shouldn't we clone the serialized and multihash properties too? | ||
return DAGNodeFactory.create(data, links, dagNode.serialized, dagNode.multihash) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.create should not take the serialized and multihash, that is generated inside create (or it was in the main PR), note that if you add a link, then the serialized and multihash is different, this is probably why this PR is failing the tests.
} | ||
|
||
// haadcode: shouldn't we clone the serialized and multihash properties too? | ||
return DAGNodeFactory.create(data, links, dagNode.serialized, dagNode.multihash) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const data = DAGNodeFactory._cloneData(dagNode) | ||
const links = DAGNodeFactory._cloneLinks(dagNode) | ||
// haadcode: shouldn't we clone the serialized and multihash properties too? | ||
return DAGNodeFactory.create(data, links, dagNode.serialized, dagNode.multihash) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|
||
class DAGNode { | ||
constructor (data, links, serialized, size, multihash, json) { | ||
constructor (data, links, serialized, multihash) { | ||
// Should these be "throw new Error()" instead? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that is what assert do if it is missing
data: this.data, | ||
links: this.links.map((l) => l.json), | ||
hash: mh.toB58String(this.multihash), | ||
size: this.size |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this size is incorrect, size of a dagnode is the length of its serialized version + the cumulative length of all its links
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that the size comes from this.size
:
get size () {
return this.links.reduce((sum, l) => sum + l.size, this.serialized.length)
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since it is immutable, we don't need to do this ops everytime, just on creation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right. Same for the other toJSON() props. Will change it so that they all get calculated in the constructor. A small speed improvement is a big speed improvement :)
@@ -1,6 +1,6 @@ | |||
'use strict' | |||
|
|||
exports.DAGNode = require('./dag-node.js') | |||
exports.DAGNode = require('./dag-node-factory.js') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we need to expose the class too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can't expose the DAGNode class. If we do that, DAGNode will become mutable, ie. doing const node = new DAGNode(...)
becomes possible which is not what we want.
The general idea of the factory pattern is to expose only methods for creating the DAGNodes, not for users to directly use DAGNode as it was before. This is the first step to immutability.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can force the users to always use the 'create', but exposing the Class does not make any instance mutable though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we expose the DAGNode class, it'll be possible for users to use it, ie. use the constructor. The only way to force not to do that is to not expose it and that, like said, is the first step to an immutable data structure. This is a generally known and widely-used pattern: don't expose the data structure (for use) but rather provide an interface that returns the data structure (DAGNode in our case).
Is there a reason why we would need to expose it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good
Will address the feedback, push and come back to the comments |
Thank you for giving a thorough review and coming up with a new structure. I've made a fair amount of comments, it would be ideal if tests were passing so that we know the PR is correct (you will see comments showing on why it's not). Things I like and will certainly adopt for #6:
Things I'm not in favour and given our previous discussions, they won't be included in the main PR
Things that don't make sense to me to have throughout our codebase:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs review: #7 (comment)
The PR now passes tests. Linter is happy except one (too many nested callbacks in tests), which was there from the original PR. |
I removed the Promises from all the methods. I do think we should expose them with Promises, too, as we do in all our public facing APIs. I assume users will need to create DAGNodes themselves (ie. not used just internally), right?
This is matter of a) preference and b) code style. If we use classes, we should use them consistently. If we use functions and prototypical inheritance, then use that consistently across the project. As for preference, I personally think wrapping the functions to static methods in a class is much more clearer than exporting functions (looks cleaner and makes the namespacing intuitive). |
I also updated the README to reflect how the DAGNode would be used with the changes in this PR but in short: dagPB.DAGNode.create(..., cb) It's the same as in the original PR. |
Rad! We are converging! Made a couple of comments :) |
@diasdavid what do you think about changing dag-node-factory.js to dag-node-utils.js and DAGNodeFactory to DAGNodeUtils? I originally named it Fatory thinking it'll only create DAGNodes but since we have all the utils (addLink, etc.) in there, I think DAGNodeUtils would make more sense. |
const DAGNode = require('./dag-node') | ||
const DAGLink = require('./dag-link') | ||
|
||
class DAGNodeFactory { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's move this to
module.exports = create
function create ...
const DAGLink = require('./dag-link') | ||
|
||
class DAGNodeFactory { | ||
static create (data, dagLinks, hashAlg, callback) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wrap the create in promisify
, so that it supports both.
plus add a test for promises
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to ref: #7 (comment)
Note: It actualy might make sense to support also a promises API for the .create function since it is a top level consumed object (from ipfs.object.get), so that people don't have their promise chain broken, however, we are not dropping callbacks.
dagLinks = [] | ||
} | ||
if (typeof hashAlg === 'function') { | ||
// empty obj |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what does this comment mean?
data = undefined | ||
} | ||
if (typeof dagLinks === 'function') { | ||
// empty obj |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what does this comment mean?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know, thought you left them there originally? :)
} | ||
|
||
DAGNodeFactory.create(data, links, callback) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if we want this methods in a thing that is called factory.
Not only forces people to have to require the module if they want to add a link:
const node = <some dagnode I got from ipfs.object.get
// now I want to add a link
const dagNodeFactory = require('ipld-dag-pb').dagNodeFactory
dagNodeFactory.addLink(node...)
while it could just be (as it was):
const newNode = node.addLink(link)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost, it would be used like this:
const node = <some dagnode I got from ipfs.object.get
// now I want to add a link
const DAGNode = require('ipld-dag-pb').DAGNode
DAGNode.addLink(node...)
This is a general functional pattern whereas const newNode = node.addLink(link)
is really neither. If you have an instance node
and you call a method on it, that implies the state of the node changes (not that it returns a new node).
DAGNodeFactory.create(data, links, callback) | ||
} | ||
|
||
static removeLink (dagNode, nameOrMultihash, callback) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as addLink
data: this.data, | ||
links: this.links.map((l) => l.json), | ||
hash: mh.toB58String(this.multihash), | ||
size: this.size |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes it so that folks can change the values by doing node._data = something
while in the previous case that would not happen. Are we happy with that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While that is true, I think documenting what IS available and prefixing members with "_" is a strong-enough indication how to use it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If people want to shoot themselves in the foot they will find a way to do so ;)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good :)
|
||
stable.inplace(links, util.linkSort) | ||
get size () { | ||
return this.links.reduce((sum, l) => sum + l.size, this.serialized.length) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is still getting calculated every single time size is requested, if a node has 100 links, this starts getting expensive.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pushed a change for this.
@@ -1,6 +1,6 @@ | |||
'use strict' | |||
|
|||
exports.DAGNode = require('./dag-node.js') | |||
exports.DAGNode = require('./dag-node-factory.js') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good
@@ -29,10 +29,10 @@ module.exports = (repo) => { | |||
expect(Buffer.isBuffer(node.data)).to.be.true.mark() | |||
expect(node.size).to.be.above(0).mark() | |||
|
|||
util.serialize(node, (err, serialized) => { | |||
DAGNode._serialize(node, (err, serialized) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the serialize and deserialize need to remain at the util
package, it is the definition of the IPLD Format
https://github.com/ipld/interface-ipld-format#ipld-format-utils
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't make sense. Why would the serialization functionality be separate from all other functionality of a specific data structure (DAGNode)?
Also, not possible atm as deserialize uses .create() (two-way coupling between utils and dag-node-utils).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Take that back, I understand what you mean with the comment. However, the util API doesn't make sense to me. Why would they need to be namespaced under utils instead of haing them on the top level? Ie.
const DAGNode = require('dag-node')
const node = DAGNode.create(...)
DAGNode.serialize(node, ...)
DAGNode.deserialize(node, ...)
// etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can move the utils to one place, but my questions still stands: why do we need utils
to be part of the API?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ignore all this, I'm not gonna bikeshed this now. I'll make some changes and push.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@haadcode it does make sense because util
and resolver
is what an IPLD Resolver needs in order to process an IPLD Format, this is generic to any API resolver.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does IPLD need util
?
this is generic to any API resolver.
What does that mean?
@@ -61,6 +61,7 @@ | |||
"lodash": "^4.17.0", | |||
"ncp": "^2.0.0", | |||
"pre-commit": "^1.1.3", | |||
"rimraf": "^2.5.4" | |||
"rimraf": "^2.5.4", | |||
"webpack": "2.1.0-beta.25" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not needed anymore
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How so?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
latest aegir fixes the issues, and webpack shouldn't be explicitly depended on anyway as it is brought in by aegir.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as libp2p/js-libp2p-floodsub#9 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Installing with latest aegir still has the same problem as before: browser tests don't run. The error is
Uncaught Error: Module '/Users/samuli/code/js-ipld-dag-pb/node_modules/buffer/index.js' is not a loader (must have normal or pitch function)
at loadLoader (node_modules/loader-runner/lib/loadLoader.js:35:10)
at iteratePitchingLoaders (node_modules/loader-runner/lib/LoaderRunner.js:164:2)
at runLoaders (node_modules/loader-runner/lib/LoaderRunner.js:357:2)
at NormalModule.doBuild (node_modules/webpack/lib/NormalModule.js:131:2)
at NormalModule.build (node_modules/webpack/lib/NormalModule.js:179:15)
at Compilation.buildModule (node_modules/webpack/lib/Compilation.js:127:9)
at node_modules/webpack/lib/Compilation.js:306:10
at node_modules/webpack/lib/NormalModuleFactory.js:74:13
at NormalModuleFactory.applyPluginsAsyncWaterfall (node_modules/tapable/lib/Tapable.js:123:70)
at onDoneResolving (node_modules/webpack/lib/NormalModuleFactory.js:49:11)
at onDoneResolving (node_modules/webpack/lib/NormalModuleFactory.js:165:6)
at node_modules/webpack/lib/NormalModuleFactory.js:161:6
at node_modules/async/dist/async.js:3853:9
at node_modules/async/dist/async.js:484:16
at iteratorCallback (node_modules/async/dist/async.js:1084:13)
at node_modules/async/dist/async.js:988:16
@@ -2,28 +2,57 @@ | |||
|
|||
const mh = require('multihashes') | |||
|
|||
const ImmutableError = new Error('Immutable property') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would be nicer to extend Error
and use like this
class ImmutableError extends Error {}
throw new ImmutableError('Size is immutable')
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Refactored the errors to be specific now.
} | ||
|
||
set multihash (multihash) { | ||
throw ImmutableError |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is not good, if you don't use new
here it will have an incorrect stacktrace I think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dignifiedquire not sure if it applies to errors in the same way, but yeah, your suggestion above -- #7 (comment) -- makes sense.
exports.addLink = addLink | ||
exports.removeLink = removeLink | ||
exports.toDAGLink = toDAGLink | ||
exports.util = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why are these namespaced and the others aren't?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See #7 (review)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
because of IPLD Format definition looking for the util
} | ||
|
||
exports.create = create | ||
exports.clone = clone |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why did clone get separated from the class node? const copy = node.clone()
was nice
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as for node.addLink
. Let's either go with functional style, or OO style. Let's not mix the two.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good
|
||
exports.create = create | ||
exports.clone = clone | ||
exports.addLink = addLink |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still not convinced that these methods should not be at at the node level
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Of course they should. These are all functions that one can do to create and manipulate a DAGNode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, maybe I am, this answer was hidden #7 (comment)
@@ -50,9 +49,11 @@ dagPB.util | |||
Create a new DAGNode | |||
|
|||
```JavaScript | |||
var node = new dagPB.DAGNode([<data>, <[links]>]) | |||
dagPB.DAGNode.create([<data>, <[links]>, hashAlgorithm, callback]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
needs to be updated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still needs to be updated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What needs to be updated here? This is the function signature.
``` | ||
|
||
#### TODO: update the functions docs from here on out | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
needs to be done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still needs to be updated
@@ -61,6 +61,7 @@ | |||
"lodash": "^4.17.0", | |||
"ncp": "^2.0.0", | |||
"pre-commit": "^1.1.3", | |||
"rimraf": "^2.5.4" | |||
"rimraf": "^2.5.4", | |||
"webpack": "2.1.0-beta.25" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as libp2p/js-libp2p-floodsub#9 (comment)
name: this.name, | ||
size: this.size, | ||
hash: this.multihash ? mh.toB58String(this.multihash) : undefined | ||
hash: this.multihash ? mh.toB58String(this.multihash) : null |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it is the same thing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, they're not. But let's not get into that now, will change it back to undefined.
} | ||
|
||
set multihash (multihash) { | ||
throw new Error(`Can't set property: 'multihash' is immutable`) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we get that "ImmutableError" here too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wrapping them to ImmutableError doesn't make sense anymore. What we would need to do is:
class ImmutableError extends Error {
constructor(error) {
super(error)
}
}
So let's just throw an Error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
} | ||
|
||
function removeLink (dagNode, nameOrMultihash, callback) { | ||
let data = _cloneData(dagNode) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can be a const
|
||
function removeLink (dagNode, nameOrMultihash, callback) { | ||
let data = _cloneData(dagNode) | ||
let links = _cloneLinks(dagNode) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can be const
} else if (Buffer.isBuffer(nameOrMultihash)) { | ||
links = dagNode.links.filter((link) => !link.hash.equals(nameOrMultihash)) | ||
} else { | ||
throw new Error('second arg needs to be a name or multihash') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if the function uses callbacks, errors should go in callbacks
if (node.data && node.data.length > 0) { | ||
pbn.Data = node.data | ||
} else { | ||
pbn.Data = null // new Buffer(0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please don't change this, it needs to be a Buffer of size 0, we had a problem in the past because the serialized versions were different when data was empty (go vs js)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't change this, it was like that in the original PR. Will change it to Buffer(0)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, not gonna change this back. If I do, one of the tests starts failing:
1) Node.js Tests DAGNode create an empty node:
AssertionError: expected 2 to equal 0
+ expected - actual
-2
+0
at DAGNode.create (test/dag-node-test.js:93:33)
Which way should it be? Change the code AND test, or leave it as it is?
exports.addLink = addLink | ||
exports.removeLink = removeLink | ||
exports.toDAGLink = toDAGLink | ||
exports.util = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
because of IPLD Format definition looking for the util
const DAGNode = require('./dag-node') | ||
const DAGLink = require('./dag-link') | ||
|
||
const hash = (type, data, cb) => multihashing(data, type, cb) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this is the case, just switch the arguments where hash is called and make it be multihashing
@@ -50,9 +49,11 @@ dagPB.util | |||
Create a new DAGNode | |||
|
|||
```JavaScript | |||
var node = new dagPB.DAGNode([<data>, <[links]>]) | |||
dagPB.DAGNode.create([<data>, <[links]>, hashAlgorithm, callback]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still needs to be updated
``` | ||
|
||
#### TODO: update the functions docs from here on out | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still needs to be updated
} | ||
|
||
set multihash (multihash) { | ||
throw new Error(`Can't set property: 'multihash' is immutable`) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
} | ||
this._data = data || new Buffer(0) | ||
this._links = links || [] | ||
this._serialized = serialized || new Buffer(0) // TODO: default serialized object |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
needs to be done
this._data = data || new Buffer(0) | ||
this._links = links || [] | ||
this._serialized = serialized || new Buffer(0) // TODO: default serialized object | ||
this._multihash = multihash || new Buffer(0) // TODO: default multihash object |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is the default multihash? I don't believe this is correct
// It's a multihash | ||
newLink = new DAGLink(null, dagNode.size, nodeOrMultihash) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This piece is incorrect, (also already gave reviews to this before). Again see line 67 to 97 #6 (comment)
Don't delete code + comments just because, information gets lost
Also, the addLink function, will need the size
of link if just a name and multihash are passed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please provide details as to what is incorrect when reviewing. Just saying this is incorrect is not helpful feedback. My original comment was "I don't understand what is meant here with your comments, please review if this is correct". Provide possible solutions, not just "this is not ok".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are correct, apologies for not being clear.
Right now, the thing that is missing from what I see is:
- We need to also pass the size of the link as an argument, because when we pass a name+multihash, we also need to pass the size of that link, otherwise the size of this node will be miscalculated
- This method can only be considered complete if the tests for adding and removing links are passing, right now they are commented out and with an incomplete migration, which is why I had this part in a TODO too. So, we need the tests to be migrated to this new API and making sure it is all correct
|
||
if (newLink) { | ||
links.push(newLink) | ||
sort(links, linkSort) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
create does the sorting, we don't need to sort it twice
|
||
function removeLink (dagNode, nameOrMultihash, callback) { | ||
const data = _cloneData(dagNode) | ||
let links = _cloneLinks(dagNode) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it can be const
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It can't, see the code couple of lines below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doing push on array doesn't require it to be let
return new DAGLink(link.Name, link.Tsize, link.Hash) | ||
}) | ||
|
||
sort(links, linkSort) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
create does sorting
if (node.data && node.data.length > 0) { | ||
pbn.Data = node.data | ||
} else { | ||
pbn.Data = null//new Buffer(0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(also, already had reviewed) this is a bug that was already caught in the past, null is not the same as new Buffer(0)
for protobuf serialization
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes and my question was: "Which way should it be?". Should I change the test so that it passed when this is changed to new Buffer(0)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should be as it was, new Buffer(0)
, it is about how protobuf serialization libraries work, we know for a fact that our 'empty objects' are different from go ones if we don't have a empty buffer when we go and serialize
One more review. Thank you for enduring throughout this PR, it is really good to get you reviewing all of this as a consumer of all the API. Seems also that you are getting a speed up in IPLD from it too :) Another comment: @haadcode I've reviewed at least 3 times the same 4 or 5 parts of this PR, some of which were bugs and still are there. It would save a lot of time if you make changes in smaller commits, avoid renaming files (util.js -> utils.js) and review every piece of feedback before asking for more, I feel a lot of information is getting lost and a lot of energy spent as well. |
Perhaps something went wrong in Github, but the "same 3 times" was with questions from me to you to clarify your comments. I have NOT committed anything without reviewing your feedback. |
Ok, one more round. Addressed everything in the feedback except promises. Everything else is "final" as far as I'm concerned. |
After the CR and the required changes, I think the code is not as clean anymore as the original code in the PR was. Furthermore, the IPLD format api with the .resolver and .util (especially the util) doesn't make sense and should be reconsidered. Things are a) more coupled and b) don't provide clean namespacing/naming and structure and even makes the module internal implementation to adhere to those decisions. However, for the sake of urgency, I'll leave this as is. There's been way too much bikeshedding regarding code style as oppose to functionality, correctness and API design and that takes away from motivation and from getting stuff done in timely manner. We need to stop using the argument "because it's like that elsewhere" as a catch-all when it clearly produces lower quality code and design. Take what you want from this PR. |
Sounds good 👍
This sound like functionality to me, what made you drop it? This change will cause errors. ❤️ Other minor things:
|
…lder, make tests pass again
3c235fe
to
ea904a7
Compare
As discussed, I finished the last points of review in this PR, fixed tests, updated docs and also give a spin to a better addLink API |
const pbn = proto.PBNode.decode(data) | ||
|
||
const links = pbn.Links.map((link) => { | ||
return new DAGLink(link.Name, link.Tsize, link.Hash) | ||
}) | ||
|
||
stable.inplace(links, exports.linkSort) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Refactored #6