feat: pin

[AdamStone]

• core
  • add
  • rm
  • ls
  • isPinned
  • isPinnedWithType
  • directKeys / recursiveKeys / internalKeys
  • flush (persist keys to datastore)
  • load (restore keys from datastore)
• cli
  • add
  • rm
  • ls
• http
  • add
  • rm
  • ls

Updated core interface to align with interface-ipfs-core, still need to update cli and http.

[daviddias]

@AdamStone, js-ipfs has gone through a good amount of updates, including bug fixes and new features. Would you like to revisit pinning again? rebase master on your branch and continue the PR

[AdamStone]

Sure, I guess I should have been rebasing rather than merging to begin with. I think the core implementation is basically done, so I'll update the description. The failing test should pass once this issue is fixed.

[dignifiedquire]

Are we sure about the licence for this module, can't find info on npm nor the repo.

[AdamStone]

Ah, good point. I'll swap in something with MIT license.

[dignifiedquire]

thanks :)

[daviddias]

This is looking good @AdamStone :)

Would you also like to lead the ipfs-inactive/interface-js-ipfs-core#34, so that we have the same battery of tests for js-ipfs and js-ipfs-api for the pinning API.

Also, go-ipfs stores the root hash of the pinset inside level-db (datastore). Currently, js-ipfs doesn't use leveldb (datastore) because it is not implemented in ipfs-repo, would you like to tackle that as well -> ipfs/js-ipfs-repo#13

[AdamStone]

@diasdavid Sorry for the delayed response, I see you've already started on ipfs/interface-ipfs-core#34 but I can continue with that if you like. Am I understanding correctly that it basically just involves migrating the core pinning tests from js-ipfs to interface-ipfs-core?

For the datastore, I did notice the discrepancy between js-ipfs and go-ipfs when working out how to store the pin set and ended up just putting it in the block store because I didn't see a good alternative. I just read through ipfs/js-ipfs-repo#13 but found it pretty confusing at first pass. I think I'd better do a little more research before committing to take that on.

[daviddias]

@AdamStone sounds good. Let me know if you would like to go through it on IRC or a hangout, you can also join today's js-ipfs call https://github.com/ipfs/pm

[daviddias]

@AdamStone good news, we've finished the Pin interface in interface-ipfs-core, so now in order to validate this Pin implementation, you just need to make sure it passes those tests

• https://github.com/ipfs/interface-ipfs-core/tree/master/API/pin
• https://github.com/ipfs/interface-ipfs-core/blob/master/src/pin.js

[AdamStone]

Sounds good, I'll take a look tonight.

[AdamStone]

@diasdavid Can you clarify, when pin.add succeeds and returns "an array of objects that represent the files that were pinned", should this include indirectly-pinned objects when the add is recursive?

[daviddias]

Since there is the type "recursive pin", we only need to know that the head was recursively pin. So no, we don't need to return them all.

[AdamStone]

@diasdavid In that case, are there any circumstances where this should return more than just one object, since add expects a single hash? Or does that mean it should return the full pin set, not just the new addition?

[daviddias]

The best is to see how go-ipfs does it through the CLI and match the expectations as closely as possible.

[AdamStone]

@diasdavid The go-ifps cli seems pretty flexible. You can pass one or more b58 hashes or ipfs paths like /ipfs/b58hash/link. For both add and rm it prints only the hashes that were changed by that operation, along with the type. I had tried to make the js pin CLI work with an ipfs path and behave like the go CLI, but I didn't notice you could pass more than one item. So I've adjusted the pin core methods to be more consistent with the CLI and to accept one hash/path or an array of them. Not sure if we want to test for those scenarios as well.

I found I needed to make a few modifications to the tests, so I opened a PR in interface-ipfs-core.

[loong]

Any updates on pinning?

[AdamStone]

I rebased what I have so far against current master here and in these related PRs for interface-ipfs-core and js-ifps-api. With these repos linked locally and with the changes in these PRs, all the pin tests are locally passing for me. However, there are placeholder workarounds in the code for two main blocking issues:

1. The pin datastore key is /local/pins/js because using the key from go-ifps '/local/pins causes the tests to try to load pins from go-ipfs-repo, which doesn't work. Still unclear if the repo just needs to be migrated, since trying to do so causes version mismatch errors.
2. The datastore is closed during the CLI core tests when trying to save and load pins due to some timing problem - the tests only pass if I manually open the datastore within the pin code.

@diasdavid I hate to leave this PR hanging again after getting this far, I was hoping this would be the final push to get all this wrapped up while I had the time to spend on it, but sorry to say I'm basically out of time now and I don't expect I'll be able to take this any further. Hopefully someone else can find it a useful starting point.