Sprint Nov 2

[daviddias]

Sprint Nov 2

Sprint Goals

• Get go-libp2p out

Sprint Discussions

Schedule

Please take notes in a separate pad, if you can, and link it here.

Endeavour	Lead	Pad	Permanent Link
Apps on IPFS	Juan Benet and Friedel Ziegelmayer	https://public.etherpad-mozilla.org/p/nov-2-apps-on-ipfs
infrastructure	Lars Gierth	https://public.etherpad-mozilla.org/p/nov-2-infrastructure
libp2p	David Dias	https://public.etherpad-mozilla.org/p/nov-2-libp2p	#46 (comment)
js-ipfs	David Dias	https://public.etherpad-mozilla.org/p/nov-2-js-ipfs	#46 (comment)
go-ipfs	Jeromy Johnson	https://public.etherpad-mozilla.org/p/nov-2-go-ipfs	#46 (comment)
testing + ci	Jeromy Johnson	https://public.etherpad-mozilla.org/p/nov-2-testing-ci
protocol + specs	Juan Benet	https://public.etherpad-mozilla.org/p/nov-2-protocol-specs
bitswap ml	Juan Benet	https://public.etherpad-mozilla.org/p/nov-2-bitswap-ml
Data Structures	Juan Benet	https://public.etherpad-mozilla.org/p/nov-2-data-structures

Please add the Agenda to the Pad before the endeavour sprint starts.

Sprint Deliverables

• Add yourself below, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

[<% name %>](@<% your_GitHub_handle %)

David Dias

• libp2p
  • Continue making modules work in the browser
    • js-peer-id tests, browser tests, move to RSA based key (instead of ECDH), conform with go-ipfs libp2p/js-peer-id#3 (also make it use RSA instead of ECDH)
    • js-multiaddr lint, tests, browser tests, readme update multiformats/js-multiaddr#4
    • js-peer-info ongoing, needs multiaddr first libp2p/js-peer-info#3
  • Get webrtc-explorer as one of the transports (blocked by node-webrtc not compiling on the Mac OS X, problem introduced with XCode7 and El Capitan)
• npm's registry on IPFS
  • update ipfs-blob-store to latest js-ipfs-api
  • download the entire npm
  • make a read-only reginabox
  • patch reginabox so that it can use a blob-store instead of ipfs directly
  • install modules via IPFS using reginabox+registry-static
• js-ipfs-api
• js-ipfs
  • jsipfs module that offers same API as full node, but delegates all the work to a node in the network Getting jsipfs out the door js-ipfs#35
• extra
  • cli tool to add commit signature automatically - https://github.com/diasdavid/sign-commit

[daviddias]

Endeavour leads, please update your sprint meeting pads with a proposed agenda for tomorrow

@jbenet @whyrusleeping @dignifiedquire @lgierth

[dignifiedquire]

I won't make it tomorrow, could someone take over for me please? @jbenet @diasdavid ?

[daviddias]

@dignifiedquire absolutely, could you to add some notes to the pad with info you would like everyone to know (or to think/decide)? Thanks :)

[daviddias]

libp2p

Lead: David Dias

Agenda

• quick update on the webrtc-update
• quick update on go-libp2p
• can we invest time in this sprint on go-libp2p? (Y/N)
• we need to make everything browser compatible (things like multihash can't depend on Node.js crypto)
• WebCrypto

You have 30 minutes for this agenda, 5 minutes before the meeting ends, consider moving the remaining items to a github discussion thread so that all the other sprint meetings can start in time.

Participants

• David Dias
• Jeromy Johnson
• Johan Kiviniemi (Ion)
• Victor
• Aaron (from ethereum)

Notes

• tentatively:
• Jeromy finishes the utp merge Monday
• Juan CR by Tuesday
• We can start ripping go-libp2p Wednesday
• go through every module and make sure it runs in browser
• post an entry on captain log about this and see if someone wants to chip in

After sprint meeting is finished, create the respective action items on the Github sprint issue

[daviddias]

js-ipfs

Lead: David Dias

Agenda

• quick update
• jsipfs next

You have 30 minutes for this agenda, 5 minutes before the meeting ends, consider moving the remaining items to a github discussion thread so that all the other sprint meetings can start in time.

Participants

• David Dias
• Johan Kiviniemi (Ion)
• Victor
• Aaron (from ethereum)

Notes

• bottom up appoarch (libp2p first)
• top down approach (jsipfs with js-ipfs-api, then bitswap, then merge with the bottom up)

After sprint meeting is finished, create the respective action items on the Github sprint issue

[whyrusleeping]

go-ipfs

Agenda

• watch juan walk through new york
• ipfs-update
• transport
• utp
• go-log
• ipld

Action items

• refactor transports
• rebase utp on transport code when done
• ipfs-update to run migrations
• ipfs-update integrated into go-ipfs
• go-log fixes
• ipld review
• bssim graph integrations

Participants

• @whyrusleeping
• @jbenet
• @chriscool
• @ion1
• @lgierth

[whyrusleeping]

@whyrusleeping's todos:

• refactor transports
• rebase utp on transport code when done
• ipfs-update to run migrations
• ipfs-update integrated into go-ipfs
• go-log fixes
• ipld review
• gx repos
• discuss gx as a generalized extensible package manager
• ipfs add perf

[jbenet]

@jbenet todos:

• Code Review
  • CR go-ipfs utp
  • CR go-ipfs refactored transports
  • CR ipfs-update -- if others want to help here pls do
  • CR go-log -- if others want to help here pls do
  • CR Blog post detailing quick install of IPFS latest ipfs-inactive/blog#8
• consensys demo
• web of trust conference: https://github.com/WebOfTrustInfo/rebooting-the-web-of-trust
• archive visit -- discuss: https://github.com/ipfs/archives open web, and other things
• org things

[ghost]

Infrastructure

Lead: Lars Gierth

Agenda

• dmca / refs
• keeping track of storage
• openipfs / pin-federation
• writable gateway
• metrics

You have 30 minutes for this agenda, 5 minutes before the meeting ends, consider moving the remaining items to a github discussion thread so that all the other sprint meetings can start in time.

Participants

@whyrusleeping
@victorbjelkholm
@chriscool
@lgierth

Notes

• candidates for ipfs.io/refs apart from dmca keep popping up: pinned items, storage inventory, releases of other software projects, etc.
• What's the point of having writeable gateway? Content disappears after a while... Ask Juan about use case
  • Making ipfs.io gateway writable infra#105
• using the ansible scripts in infrastructure.git
  • edit hosts file
  • secrets.yml => secrets_plaintext/secrets.yml
• wanna open up metrics.i.ipfs.io to the public. if you'd like to see certain ipfs or host metrics, talk to @lgierth, it's not too hard

After sprint meeting is finished, create the respective action items on the Github sprint issue

[ghost]

@lgierth's todos:

• dmca / ipfs.io/refs
  • [~] deploy refs.ipfs.io https://github.com/ipfs/gateway-dmca-denylist
  • [~] point ipfs.io/refs to refs.ipfs.io gateway: add /refs/* infra#95
  • ping @victorbjelkholm re: refs/openipfs
• gateway
  • hope: writable gateway Making ipfs.io gateway writable infra#105
  • [~] open up cors on gateways (allow origin *) gateway: open up CORS to everything (allow-origin *) infra#109
• org
  • move pollux to protocol labs account Transfer pollux host to a Protocol Labs account infra#73
    • more and more back and forth
  • [~] periodic ovh renewal
• archives
  • hope: get archive inventory together Keeping track of storage contents infra#106
  • hope: add notes about cdnjs cdnjs ipfs-inactive/archives#35
  • hope: add openwrt 15.05 on castor
• cjdns
  • discover peers from cjdns WIP: discover peers from cjdns kubo#1316
  • invite to fc00 development (aka go-cjdns)

[RichardLitt]

@RichardLitt's todos:

• 15 hours of work (minimum) on IFPS HTTP docs.
  • Look into automatically generating
  • Manually do the rest if needed. curl everything to make sure it works.
• Help out with designing a community infrastructure for meetups
• Sprint (GitHub) management
• Make the documentation around Commit Sign-Offs more friendly
• Parse IPFS readme's automatically using a CLI tool for ipfs-readme-standard.

[masylum]

@diasdavid why don't we get karma runner (:wink: @dignifiedquire) in the js- repos to test against phantom? Checking manually repo by repo seems cumbersome. We have to setup a CI for most of those repos anyway.

This week I'm toying with:

• Ruby sdk
• Abstract connection test suite (its unlikely that I will have any time but...)

[daviddias]

@masylum would it be that simple? Also, the idea is not only to say 'it is broken', but actually have a proper docs of how to use it in the browser (for example, how to use Streams or Buffers), see https://github.com/jbenet/js-multihashing as an example.

[chriscool]

@chriscool's todo:

• Get started with Google App Engine: https://cloud.google.com/appengine/docs/go/gettingstarted/introduction
• Come up with a todo list for https://github.com/ipfs/golang-build
• Come up with a question list for https://github.com/ipfs/golang-build
• start working on items from the above lists
• Start installing ipfs/golang-build on Google App Engine
• help @whyrusleeping if he needs help about git command extension mechanism

[masylum]

@diasdavid may not be that simple but I think its a first step. We have to setup CI to those repos anyway, so at least we could ensure that the ones that can run in the browser can run the tests both in node and browser-land.

[daviddias]

@masylum my current strategy is going one by one, but would be awesome to have that 'CI test machine' you are proposing that just 'shots at every repo'! :D

[kyledrake]

@kyledrake:

• libp2p site work
• finish DMCA work
• changes to blog post

A bunch of other stuff is churning fast right now, expect changes. Two conferences and a lot going on this week.

@masylum I've made some initial ruby related code that you may be interested in. I was definitely hoping to sit down and bang out a ruby API for go-ipfs at least. Check out https://github.com/neocities/ruby-multihash

[masylum]

@kyledrake nice! I'm just looking to do the API sdk. Like the python and node ones.

[hackergrrl]

Hopping in late on this sprint!

apps

• design /w @whyrusleeping on gx package manager on ipfs
• wrote ipcat (https://github.com/noffle/ipcat)

go-ipfs

• design pondering on a common ipfs shell interface (common interface in front of IpfsNode or HTTP API node kubo#1945)
• fallback-ipfs-shell: uses local node and falls back to ephemeral (https://github.com/noffle/fallback-ipfs-shell)

[daviddias]

SPRINT CHECKIN

• libp2p
  • Continue making modules work in the browser
    • js-peer-id tests, browser tests, move to RSA based key (instead of ECDH), conform with go-ipfs libp2p/js-peer-id#3 (also make it use RSA instead of ECDH)
    • js-multiaddr lint, tests, browser tests, readme update multiformats/js-multiaddr#4
    • js-peer-info ongoing, needs multiaddr first libp2p/js-peer-info#3
  • Get webrtc-explorer as one of the transports - I'm currently "blocked" in this one, I can make it work in Linux, but node-webrtc doesn't compile for Mac OS X now because incompatibilities introduced with XCode7 and El Capitan. This is a known problem, both from the Chrome team and the node-webrtc people, fixing it requires comprehension of Node.js native bindings and how libwebrtc works, fortunately, there are several people involved, testing and working on it.
• npm's registry on IPFS
  • update ipfs-blob-store to latest js-ipfs-api Feat/mfs ipfs-shipyard/ipfs-blob-store#5
  • make a read-only reginabox https://github.com/diasdavid/reginabox
  • patch reginabox so that it can use a blob-store instead of ipfs directly https://github.com/diasdavid/reginabox - I also tried to make the st module to be ipfs-blob-store compatible, so that folks could serve static files from network storage, but that PR started to become way too long and I just changed reginabox instead (reginabox was using st to as a static server)
  • download the entire npm - Started downloading npm with a fs-blob-store (> 48 hours), but got cut a bunch of times, fortunately, there is an option to resume. However the goal is to download it into IPFS (I started downloading it with fs-blob-store so that then it would be faster to add through ipfs-blob-store). Currently, js-ipfs-api doesn't work 100% with 0.4.0 due to API changes introduced in 0.3.9, I really need for 0.3.9 to be rebased on 0.4.0, which is kind of blocked until this PR refactor net code to use transports, in rough accordance with libp2p kubo#1937 gets merged. I've been sync with @whyrusleeping on this.
  • install modules via IPFS using reginabox+registry-static
• js-ipfs
  • [~] jsipfs module that offers same API as full node, but delegates all the work to a node in the network Getting jsipfs out the door js-ipfs#35 - This is a big effort and PR, but actually good one because it layers a ton of foundations to start upgrading jsipfs gradually till it becomes a full IPFS node
• extra
  • cli tool to add commit signature automatically - https://github.com/diasdavid/sign-commit
  • organize things for IPFS-Lisbon
  • prepare my talk to IPFS-Lisbon
  • work towards organising an IPFS-Porto (there are some people really interested :D)
  • review and merge some PRs
  • calendaring stuff

[RichardLitt]

• Opened Redesign Media Page ipfs-inactive/website#78
• Updated dates on Calendar
• Make a security.md for IPFS
• Opened Issue Reporting Guide kubo#1946: "Issue Reporting Guide"
• Copy-edited news PR. News ipfs-inactive/website#71
• [~] Looked at Victor's PR. Add current work in progress of readme.md and api.md ipfs-inactive/js-ipfs-http-client#102. More to do here
• [~] 15 hours of work (minimum) on IFPS HTTP docs. Did roughly 8 hours.
  • Look into automatically generating
  • [~] Manually do the rest if needed. curl everything to make sure it works. Got stuck figuring out curl, wasn't very clear
• Help out with designing a community infrastructure for meetups.
• Sprint (GitHub) management
• Make the documentation around Commit Sign-Offs more friendly
• Parse IPFS readme's automatically using a CLI tool for ipfs-readme-standard.

[ghost]

• dmca / ipfs.io/refs
  • deploy refs.ipfs.io https://github.com/ipfs/refs https://ipfs.io/ipns/refs.ipfs.io
  • point ipfs.io/refs to refs.ipfs.io gateway: add /refs/* infra#95
  • actual usage blocked by directory listings bug gateway: needs to better support URL rewrites kubo#1955
  • ping @victorbjelkholm re: refs/openipfs
• gateway
  • gzip compression ipfs/infra@e80ea37
  • [~] open up cors on gateways (allow origin *) gateway: open up CORS to everything (allow-origin *) infra#109
  • hope: writable gateway Making ipfs.io gateway writable infra#105
• org
  • move pollux to protocol labs account Transfer pollux host to a Protocol Labs account infra#73
    • more and more back and forth
  • periodic ovh renewal
• cjdns
  • catch up with cjd's new cryptoauth dev
  • [~] discover peers from cjdns WIP: discover peers from cjdns kubo#1316
  • invite to fc00 development (aka go-cjdns)
• infrastructure
  • dashboard cleanup and style improvements
  • daemon dashboard http://metrics.i.ipfs.io/dashboard/db/ipfs-daemon
  • added jbenet's SSH tunnel script (metrics.sh)
• archives
  • hope: get archive inventory together Keeping track of storage contents infra#106
  • hope: add notes about cdnjs cdnjs ipfs-inactive/archives#35
  • hope: add openwrt 15.05 on castor

[ion1]

• [~] Worked further on https://github.com/ipfs/go-ipfs/issues/1818 “max-age, ETag for /ipns requests” and Tests for ipns-caching kubo#1934 “Tests for ipns-caching”, CR pending.

[kyledrake]

@kyledrake checkin

• libp2p site work
• finish DMCA work
• changes to blog post

Very busy week, as predicted.

[jbenet]

@jbenet checkin

This week I talked at futurismnyc.com/schedule/blockchain-demo-night/. Then I went to SF. It was super productive. went to http://www.weboftrust.info, met with people, and visited the Archive. Then went to London and talked at devcon1 -- http://devcon.ethereum.org.

• Code Review
  • CR go-ipfs utp
  • CR go-ipfs refactored transports
  • CR ipfs-update -- if others want to help here pls do
  • CR go-log -- if others want to help here pls do
  • CR Blog post detailing quick install of IPFS latest ipfs-inactive/blog#8
• org things
• latest IPLD spec draft WIP: IPLD spec specs#37
• merged @diasdavid's PRs for browser + republish etc.
• Visits:
  • archive visit -- discuss: https://github.com/ipfs/archives open web, and other things
  • consensys demo futurismnyc.com/schedule/blockchain-demo-night/
  • web of trust conference: https://github.com/WebOfTrustInfo/rebooting-the-web-of-trust + http://www.weboftrust.info/
    • DPKI and "updating master keys" protocols
    • keychain description
  • ethereum devcon1 -- http://devcon.ethereum.org
    • make devcon1 talk

[masylum]

@masylum checkin

Couldn't do anything last week :(

We are organizing a hack day in Barcelona though: ipfs/community#67 (comment)

[whyrusleeping]

planned from sprint start

• refactor transports
• rebase utp on transport code when done
• ipfs-update to run migrations
• ipfs-update integrated into go-ipfs
• go-log fixes
• ipld review
• gx repos
• discuss gx as a generalized extensible package manager
• ipfs add perf (#1954)

added along the way

• query-vis tool

[dignifiedquire]

• bisect go-ipfs [0.3.9] ipfs daemon times out on second start kubo#1950
• Upgrade dependencies for webui Upgrade dependencies and some cleanup ipfs-webui#92

[NeoTeo]

Hi guys. A quick note to let you know my first pass at the Swift IPFS API client/shell is now done and passing tests and is here. Comments welcome. :)

[RichardLitt]

@NeoTeo Nice! That's awesome!

Nice pictures Juan!

Closing this in favor of ipfs/kubo#51.