[research] Dat protocol

[BenGamma]

Interesting Dat protocol suggested by @RangerMauve :

With that in mind I was wondering if you'd be interested in integrating P2P technologies instead of a traditional centralized back-end in order to give users of the tool more power over their own data and to open up use cases for working offline or over local networks.

I would say yes we are interested to build a plateform that will not depends on big corporations such as google or amazon but since it's a lot of effort and work to avoid, and because we don't have the necessary work force (for now) to build and maintain our how backend, we choose this solution. I suggest you have a look at this wiki section about our view on big companies.

I'm coming from the Dat community and organizing people in decentralized systems is something that I've been thinking about a lot. Specifically, how do we build systems that are truly operated by their users, don't rely on large corporations like Google or Amazon for hosting, and don't require communities to set up their own hosting infrastructure (thus more centralization that can go down).

Totaly agree on building systems that rely on big corp.
Do using Dat require a special browser ?

With Dat your building block is a folder that people can store their data in, and share over the P2P network using a URL. The idea is that only the owner of the Dat Archive can update it, but anybody that's downloaded a copy of it can share it to others that might need it. This is a useful building block for decentralized and offline-first systems because a person can add things to their archive whenever they want, and sync it out to others once they're connected to the internet or through local WiFi hotspots.

Dat is then relying on P2P protocol right ? That looks like very clever way to handle offline data. My only concern is that since P2P is used mostly for sharing files, unfortunatly some companies network firewals filtered it by default. That could be a problem, as we want to create a plateform which is easy to access.

I think that using Dat for your data layer can make your software more versatile, and will help communities stay independent of large corporations that could impose arbitrary costs and restrictions on them.

Feel free to provide a little example of how you would use Dat for this project !

[RangerMauve]

Do using Dat require a special browser ?

Yes and no. Right now there's only one browser that supports Dat out of the box: http://beakerbrowser.com/. It gives the best experience and makes full use of the P2P tech.

However, we've been working on dat-js which gives you the ability to pull data off of the Dat network using WebRTC for P2P, paired with gateways to pull and push data to the rest of the network.

I'm currently working on the load-dat-page project which will make it easy to load websites off of the Dat network within a browser.

unfortunatly some companies network firewals filtered it by default.

Dat doesn't use existing BitTorrent protocols for sharing data so it's less likely to be caught by packet inspection. As well, the new networking layer makes use of of hole-punching techniques which should help a lot with getting through difficult firewall setups. Finally, users can use services like hashbase to have more reliable always-on nodes somewhere int he cloud. Hashbase is all open source so it should be easy for a community to run their own version if they so choose.

Feel free to provide a little example of how you would use Dat for this project !

Ideally, the Dat ecosystem would replace any data storage within Firebase. Users would create their content using the DatArchive API, they would share it with each other using a P2P discovery mechanism, and the rest of the application that you have now could be built using that as it's primitives.

I wrote a blog post a while ago about a hypothetical blogging software based on Dat called Face2Face

[chrismclarke]

Quite simply, we want to build something that will be easily accessible to people around the world and as simple as possible to maintain. Whilst I'm strongly in favour of the ethos behind decentralisation, I don't think such bleeding edge technologies would be right for our project at this stage.

I'm going to close this issue for housekeeping purposes, but still feel free to add any further comment and I've also added a direct link to the wiki so that it is not lost:
https://github.com/OneArmyWorld/onearmy/wiki/Big-Corporations-(evil!)

[chrismclarke]

Cool blog article also!

And @tx4x , yes there's plenty of html->pdf out there (usually build on top of pdf.js), they can be a little bit fiddly but still for the most part workable. Data export options is something I hadn't thought about at all but could see how they could be very useful for different contexts. Feel free to create a separate issue if you wish to talk about these further.

[thisismattia]

Thanks for everyone's inputs on this topic. As @chrismclarke mentioned it is a dear topic to us but we have to be pragmatic with our resources at the moment as our priority number one is to go live before the end of summer. Don't get us wrong, we kinda hate Google and Amazon but for the time being they offer the necessary ease of implementation and reliability that we need to get the project off the ground.
This issue is closed on git but fully open in our heads, so happy to keep on hearing ideas and options.
Thank you @RangerMauve and @tx4x

[RogerKemp]

@tx4x and @chrismclarke - have a look at this url to PDF api @SpaceK33z forked https://github.com/Volst/url-to-pdf-api

[chrismclarke]

Awesome @RogerKemp! I really like the approach to use headless chrome to generate, likely much more robust than pdf.js. Will note this and move discussion to a different thread (#395).