LAN Sync [Migrated from owncloud/core]

[LukasReschke]

Dear all,

I am trying owncloud and it looks really great, but there is one feature DropBox has and that I cannot find in owncloud: the Lan Sync, i.e. the possibility for clients under the same Lan to synchronize directly between themselves, which limit the bandwidth.

For organizations working in remote areas with slow connection, it really makes a difference and I was wondering if the feature does exist and if not, if it would be possible to implement it.

Thanks and best regards,
Pierre

Migrated from owncloud/core#304

[RandolfCarter]

Isn't this the same as #203 ?

[LukasReschke]

I don't think so. This issue seems to be about the LAN sync between clients like Dropbox has it.

https://www.dropbox.com/help/137/en

[RandolfCarter]

Ah yeah, this one is between two clients in the same LAN, while the other is for connecting differently between server and client according to current location of the client! Thanks for pointing it out!

[zatricky]

This feature's power is especially compelling when you have remote sites with poor connectivity. I've been using my laptop to get ease the bandwidth requirements with Dropbox - would give me less reason to keep Dropbox around if this was implemented in ownCloud. :)

I'm not sure how Dropbox implemented the feature. The KISS method I can think of is to report the private network and masks to the server - if there's a match, a simple cryptographic between clients, coordinated by the server, can confirm local connectivity.

[zatricky]

https://en.wikipedia.org/wiki/Local_Peer_Discovery (References refers to http://forum.utorrent.com/viewtopic.php?id=63567 which mentions some implementation detail used there)

The above is also a KISS approach - a simple cryptographic would "seal the deal".

[nloui]

FWIW - We setup an ownCloud server in-house and then a second one in the cloud where the data is sync'd via Bit Sync (a little convoluted) but achieves something similar. DNS within the network points to our in-house ownCloud and outside the network, it goes to the cloud-based server.

[etiess]

Hello,

I live and work in Africa, where bandwith is low and expensive: it would be a great point to have this "LAN sync". Is it part of the plans for owncloud?

Thanks!

Etienne

[danimo]

@etiess As you can see, this issue is not scheduled, so it's not on the near-term list.

[etiess]

Thank you @danimo for your feedback. We'll be patient!

[menelic]

+1 for LAN sync, this would really be an important feature in contexts of low bandwidth, high connection costs but not only there. In US, UK and GER, where internet is comparably cheap and fast, sync on LAN is still attractive for multiple device scenarios, eg a music or movies folder synced between laptops, tablets and phones.

As OwnCloud positions itself as a collaborative suite with teh documents app, sync on LAN becomes an important feature in that regard too...please consider giving it more priority

[Herve5]

+1 for LAN sync; for me it'd become a reason to switch an entire research lab here…
(indeed, it's currently the reason they refuse to at this moment…)

[jancborchardt]

Is this related to the 1.6 »Sync Performance« milestone? @MTRichards @dragotin @danimo

[MTRichards]

It is likely far more complicated than we can put into 1.6. Peer to peer sync is of course interesting, but for me a feature request much further out. And, interestingly, there are those who don't want this enabled, so it has to be configurable.

[willie68]

+1 for LAN sync, too;
I work with PC's, Netbook, Laptop, sometimes all within the same network. So a quick sync of a client over the LAN would be desirable. Actually i'm just waiting on my new laptop to sync all data. Just 7GB, but i think it will take 32hours... (The server has only a 384kbit upload rate) My netbook with all data in sync is just 20cm away.

[benedikt-buchert]

This seems to be an interesting solution to do this in case you are using your own server: http://adammatthews.co.uk/2013/05/owncloud-bit-torrent-sync-dropbox-clone/

[zatricky]

This brings across an interesting point: People have opted to bypass ownCloud's builtin "sync" entirely because it is inadequate? :-(

FWIW, I'm not experimenting with ownCloud any more. I realise ownCloud has uses other than file sync - but btSync has had me sorted for a while already.

[Asuquo12]

+1 LAN Sync, This is very key for my organisation and for Owncloud as well. LAN Sync is the single reason we are looking at Dropbox. Please lets get this pushed up the priority ladder.

[ghost]

+1 LAN Sync!!

[muetze-online]

Me too. Mostly my two synced Macs are at different locations, but often they are in the same network. If I sync 2 GB of new files, they will be uploaded to the internet - that must be - but then they will be downloaded for my second machine from the internet, allthough the data is present via a gigabit ethernet. When dropbox offered this feature, I was very happy. Now I'm on ownCloud - far beyond dropbox with some needed features (but better in an overall privacy relationship). You get the users with speed and simpleness.

[Herve5]

Muetze, I hope what we write here will be read (we have little else to ask since we are not developing ourselves indeed) but for regularly checking this feed, what I understand is, a guy name Deepdiver closed the action here referring to another thread, that he also closed the same day.
He seems to behave like he really don't like the issue.

Will Owncloud die by lack of simplicity I don't know. I definitely continue to use it since it is, well, the only open alternative… The world is not so large after all :-(

[muetze-online]

This issue is still open - see on the top. The other issue is closed, because it is a duplicate of this issue.

[tflidd]

There is even an open-source alternative to bittorrent-sync: http://syncthing.net/

[step4wd]

+1 for LAN Sync but is there any news/update on the roadmap?

[guruz]

Discussion now on https://central.owncloud.org/t/gsoc-2017-fast-lan-sync/6271

[PVince81]

We had a discussion with @ogoffart, @IljaN and @felixheidecke today about bittorrent sync. This was without reading the contents of this ticket.

Here is a dump of the topics we looked at for bittorrent download for clients:

• some research done, not exhaustive
  • seems possible to provide bittorrent tracker functionality over HTTP which could remove the need for a dedicated service
  • https://github.com/SuprDewd/simpletracker
  • bittorrent supports encryption
  • research needed: does bittorrent encryption prevents MITM between peers to prevent finding out the hashes
  • research needed: is there a way for hackers to download disallowed files by knowing their ids/hashes ?
• server
  • can provide HTTP endpoints to implement bittorrent tracker functionality
    • provides list of peers and a way for peers to register for a specific OC file
    • checks access permission
    • optionally: only let peers of a specific OC group sync between themselves (if that even makes sense security-wise)
  • provides Webdav property "oc:torrent-id" on every file/folder
  • possibly set a min file size limit for files to be torrented instead of any file
  • seeds original file:
    • should server also implement a torrent client to be able to seed ? (might need external service)
    • can server advertise itself as web link (with auth) through tracker instead ?
  • more research: authentication/security
• desktop client: implementation using Qt (@ogoffart please insert here)
  • read "oc:torrent-id" from PROPFIND
  • connect to tracker for download
  • else fallback to regular download
• LAN sync: likely offline without server/tracker ? might be another topic... if yes, need to separate bittorrent sync discussion to a separate ticket

Now maybe some questions were answered in the old thread:

• consolidate points list with discussion from the thread

[guruz]

• Look why this bittorrent makes more sense than extending on the rsync/zsync based way we planned for not-syncing-whole-file-from-or-to-server.

[individual-it]

What about synching protocol https://github.com/syncthing ?

[c0fe]

Anything new on this?

[volker-raschek]

Hello,
I can't understand why such an essential function, which was asked for 5 years ago, is not finally being implemented.

Instead, it implements every nifty gimmick, but the client's essential function is completely neglected.

In order to continue to stand up to other service providers, such a function should be implemented slowly.

[c0fe]

@volker-raschek i am not sure if the dev team will be working on this themselves. I can start to work on it but I am concerned that the work will be disregarded and it all will be for naught.

[michaelstingl]

@volker-raschek @c0fe did you check the Syncthing approach?

[c0fe]

@michaelstingl i have heard of it but haven't done all that much research into it yet.

[michaelstingl]

I'd love to hear feedback if Syncthing could be used here. Maybe basic integration isn't that hard…

[c0fe]

@michaelstingl I would assume this would require the installation of Syncthing in addition to Webserver PHP and Nextcloud and management would go through some sort of hooks.

The question is what interface does Syncthing supports and if it is possible to do this through straight PHP or would require something else in addition.

[michaelstingl]

I'm not sure if Syncthing on the server is really needed. Syncthing comes with a REST API which shouldn't be too hard too connect.

[c0fe]

@michaelstingl if they have a REST API then it shouldn't be difficult at all.

[EstebanSalpeter]

hello there, what about letting the client act as a proxy server, taking local copies as cached files?

[biva]

@MorrisJobke the development looks not active here. Do you still collaborate between owncloud and nextcloud? Would you still consider this development? In Africa, where internet is slow and data is expensive (illimited plans are very rare), it would be very helpful. Thanks a lot for your consideration!

[biva]

As I did not see any update about this LAN sync issue here, I created another issue on the Nextcloud side: nextcloud/desktop#2010

Is there any update on your side? Thanks @michaelstingl for your involvement :)

[Railsimulatornet]

Any news?

[michaelstingl]

Any news?

Sorry, not on the radar.

[RokeJulianLockhart]

Why, @michaelstingl? #230 (comment), and nextcloud/desktop#2010 (comment) especially, demonstrate why this really is needed. You have confirmation that this would notably improve digital resilience in Ukraine. Surely that's rationale enough? If not, at least state why.

[michaelstingl]

I fully understand the demand, but I don't see any capacity to integrate such features at least the next 1-3 years. Issue can be re-opened whenever new options show u…