Co-host websites

[hacdias]

During this week's weekly call, we went over the co-hosting feature of IPFS Desktop. I decided to remove it for now and move that code to another branch so we can test it better.

So, we should introduce a section in Settings to add and remove websites from the co-hosting list. It can even be coordinated with IPFS Companion! What if:

1. On Web UI we can manage the websites that get co-hosted.
2. On Companion we can add and remove websites through the toggler.
3. The websites list gets stored to the MFS root file /.cohosting or something like that.

Or... another idea could be:

Instead of pinning, .cohosting could be a directory inside MFS where we would copy the latest hash of some website to.

Both Desktop and Companion could run scripts from time to time to check if the websites were up to date.

@lidel I'd love to hear your opinions on this. It'd be really cool if this got coordinated with IPFS Companion so we'd use the same structure for saving the co-hosting list.

Notice that this would be a list per repository and not per installation. That's the only way I can think about now to create this sync'ed feature between Desktop & Companion!

[autonome]

This is fantastic, can't wait to see this ship in some form.

It might be worth identifying users' primary motivations for co-hosting websites, and doing some UX flows based on those. It would give a frame of reference for what you build, and what the language should be to describe it.

[lidel]

I really like the idea of persisting the list in a configuration file on MFS.

Been thinking about doing that for other node-bound features in ipfs-companion as well, and it feels what we will decide here will establish practice for other apps to follow.

Some ideas below:

Introduce {MFS}/.config/ directory

A lot of Linux apps uses ~/.foo for storing configuration, however that pollutes the root directory over time and a lot of apps switched convention to ~/.config/foo/ (defined by XDG Base Directory Specification).

We could establish similar convention by using /.config/cohosting/ for this feature. As a side-effect it enables us to add additional UI for things in /.config/ (e.g. hide and/or make it harder to move/delete things via webui).

It would also enable us to store app-specific things in
/.config/ipfs-companion/
/.config/ipfs-desktop/

MFS-based pinning

This is a pretty cool idea, it enables:

• Shared Web UI for adding/removing websites that works the same in Desktop/Companion
• User can browse cohosted resources with webui or cli and manually drop older versions if they wish.

I imagine we have two ways to do it:

(A) config-based

If we want to have a list of cohosted websites in a file:

• /.config/cohosting/cohostingrc – configuration file in which each line is a hostname
• /.config/cohosting/sites/ – cohosted data
  (separate directory with cohosted data makes it easy to remove all in bulk without removing config)
  /.config/cohosting/sites/example.com/latest
/.config/cohosting/sites/example.com/YYYY-MM-DD--HHMMSS

Con: introduces concept of "cohosting config file"

(B) directory-based

This is an easier way, that does not require custom config file.
All it needs is Files API (files.mkdir|ls|rm), and it looks better on Files screen of Web UI:

• /cohosting/example.com – existence of folder enables cohosting for this DNSLink website
  /cohosting/example.com/latest
/cohosting/example.com/YYYY-MM-DD--HHMMSS

I ❤️ this approach because:

• it does not introduce any config files
  • ipfs files ls /cohosting/ returns the list of cohosted websites
  • ipfs files mkdir /cohosting/docs.ipfs.io adds cohosted website
• makes it very easy to write custom tools and UIs for managing cohosted websites
• it is explicit in what it does, if user clicked the "cohost" button on a website, they immediately see "cohosting/example.com" in their Files
• easy to implement and interate on

How to handle orchestration

I agree with @autonome that we need to understand user base and tweak the language.
That is why cohosting will be an userland feature for now, meaning it is up to ipfs-desktop or ipfs-companion (or some standalone script) to take care of copying website data to MFS and periodically refreshing it.

At some point we may decide to migrate the orchestration to go-ipfs and js-ipfs. If we go with (B) it should be an easier task, as nothing more than Files API was used in the first place. For the sake of simplicity I propose we agree on refresh interval to be equal to current IPNS republishing window (12h?) and don't expose it to the user.

That way, we won't have the problem of config migration when we move it to core implementation.

Thoughts?

[hacdias]

Heey! So, I agree with you and also think the option (B) is the best one to go to. The 12 hours refreshing interval looks good to me too. How'd we coordinate between Desktop & Companion? Or each one would check every 12 hours without coordinating to each other?

[lidel]

I think if we go with (B) and agree that instead of latest directory we use regular YYYY-MM-DD--HHMMSS, then we could:

1. sort timestamps and grab the latest to tell the time a website was refreshed last time
2. IF timestamp is older than 12h, an app will refresh it
   • IF no change (same CID), just rename latest snapshot to the current timestamp
   • IF there is a new snapshot, create it under current timestamp

That way multiple apps can do the refreshing without stepping on each other toes, and entire thing is very simple, no need for locks / signaling.

[hacdias]

Completely agreed!

[lidel]

@hacdias I took a stab at writing a spec for this, let's continue in ipfs-shipyard/cohosting#2

[SgtPooki]

https://news.ycombinator.com/item?id=39208673 & https://neimanslab.org/2024-01-31/why-i-moved-my-blog-ipfs-to-server.html are related here. We could make it much easier for people to self-host their websites