Dev0.4.0

[whyrusleeping]

This is the development branch for ipfs version 0.4.0

It will contain many changes that are incompatible with previous versions, including repo modifications and protocol changes.

Any change wishing to make it into the 0.4.0 release should PR against this branch.

[jbenet]

And to think i doubted you triceracop gitcop :')

[whyrusleeping]

I have some nodes on the net running dev0.4.0, if anyone wants to try out the new branch, you can use them as bootstrap nodes:

/ip4/192.241.194.199/tcp/4001/ipfs/QmY1uAk28LushvGhTc9dvWhzAgeULwPbBjH2dAJgpkSkVA
/ip4/45.55.196.28/tcp/4001/ipfs/QmVStL8ADgdk8H46mfgAgp6uMrzEBzXf7UC4SpbpEHo8cx

I'm beginning a 'soak' test. So feel free to jump onto dev0.4.0, but be aware that things will be changing fast and may break.

[whyrusleeping]

@jbenet I think that we should pick a point in the very near future to make all new work be on top of dev040. That last rebase wasnt fun

[jbenet]

@whyrusleeping agreed. i want 0.4.0 in as well.

[whyrusleeping]

@jbenet how about, after CCC camp, all dev work moves to be on top of dev0.4.0?

[whyrusleeping]

current ref 6948d30

[jbenet]

@whyrusleeping: merge button says there's conflicts.

[whyrusleeping]

yeah, you merged something after i started the rebase.

[whyrusleeping]

alright, my remaining requirements for 0.4.0:

• utp
• importer arguments in file root block
• directory sharding
• ensure handshake can be added to without breaking protocol (may already be)

[jbenet]

directory sharding would be awesome but may be hard. we should:

• spec it out asap (this week?)
• drop it if it doesnt merge in time

[jbenet]

we need to plan out the upgrade path a bit better-- people have been worried about the "handshake breaks problem". in particular:

• need the plan for infrastructure (how long to run the old version) cc @lgierth
• how to talk to people and have them ready to switch over (flag days / periods suck in general)
• people have asked for a way to connect to the old protocols meanwhile, at least for some time. we could do this, by making dev0.4.0 speak both (but maybe display a warning).
• (the -Wall thing will be useful)

[ghost]

need the plan for infrastructure (how long to run the old version) cc @lgierth

We could do something like this maybe?

1. add generic messaging to ipfs daemon, "you should update regularly, you're running v0.3.10 from 2015-12-09"
   • should be there right as the first line (not as a debug message)
   • we might have to rely on ldflags for the date part, which won't work with go get (but with a makefile build)
2. update half the gateway/bootstrap nodes with the 0.4.0 release
3. update one more gateway/bootstrap node every week or so
   • we can change this interval at any point

Letting both 0.4.0 and 0.3.x serve ipfs.io would be very hairy (ipfs/infra#115 (comment)), so it's likely not worth that.
We can have a 0.3.x-backed http://v03x.ipfs.io or so though.

people have asked for a way to connect to the old protocols meanwhile, at least for some time. we could do this, by making dev0.4.0 speak both (but maybe display a warning).

I think there should be a very good case for adding this kind of backwards compatibility.
It might seem straightforward but the hidden cost is getting rid of it again.
There'll be plenty of future occasions for introducing this kind of technical debt ;)

We'd also have to deal with 0.4.0 nodes talking to all kinds of 0.3.x nodes,
which might be a source of distracting issues.

Instead, we should make the migration to 0.4.0 as simple as possible.
I'd guess the people asking for backward compatiblity are concerned about downtime during the migration?
A readonly mode would help there (#2051).

(the -Wall thing will be useful)

this sounds gcc-ish to me, or do you just mean the motd-like wall we talked about?

[jbenet]

@lgierth yes, all of what you say makes perfect sense and i agree with it. same views here.

the concerns brought up to me were that if they have a 0.3.x node out there, suddenly the gateway wont serve their content anymore and now it wont work for them, which sort of breaks the service. this is hard if they have deployed 0.3.x as part of another application, and it is embedded there, where they have no control over the end-user's updating it. it's certainly tricky. alphaness means we're ok with this, but worth thinking through to see if there's a reasonable way out of this for the future.

[ghost]

suddenly the gateway wont serve their content anymore and now it wont work for them

Is it about the gateway or about the swarm? When you said handshake backwards compat I assumed swarm.

[jbenet]

Is it about the gateway or about the swarm? When you said handshake backwards compat I assumed swarm.

it is about the swarm, but that transitively means the global https://ipfs.io gateway wont work for them either.

[whyrusleeping]

perhaps we should first have a v040.ipfs.io to be the dev0.4.0 gateway, and keep that for a while, we can also add a v03x.ipfs.io as an alias to the current ipfs.io, and urge people to use that if they don't intend to upgrade.

[whyrusleeping]

we could also do something like "make requests to both 0.4.0 and 0.3.10 and return whichever returns non error first"

[dignifiedquire]

🎉

[whyrusleeping]

wooooooo!

[daviddias]