Use git to version notes

[rlouf]

Say I installed hackcmd on my own server. I would like to communicate with some of my collaborators or students through the platform, but

• I like to able to edit offline and not go in my browser to edit notes
• Some of my students would like to do the same
• But some of them are fine with a web interface

The ideal world would thus to have hackmd save the files in plain text (cf issue #90 ) and version them using git. Also, when someone modifies the file online, commit the changes.

I don't know exactly how you could do that without it being too complicated, but it looks like you must already have a way to manage conflicting changes (?). Which I think is the trickiest part in this.

[jackycute]

Thanks for using HackMD.
The features that you mentioned was pretty hard but at same time very interesting to solve.

HackMD now using an algorithm called Operational Transformation for the text conflictions.
But save the note into files and version them using git might have performance overhead because the I/O and also git seems not quick enough for realtime collaboration.

But this idea lead me to think if our DB would act like another git repo clone and have it's own commits and authorships, then you can pull from/push to your own local/remote repo to keep them synced.

Before trying these things out, we must track the note revision and the authorships which both not implemented yet.
And we might be the very first one to make it true - realtime collaboration with real version control.

[rlouf]

Here is a bunch of random thoughts.

A first problem is how to associate git users to users of Hackmd, but this shouldn't be too difficult... In fact, it is more of an user interface problem than a technical one. I think you should not make hackmd harder to use for users who don't need version control---ease of use is one of the appeals of the app. I believe these should see the same interface as they do now.
Only penalise the annoying folks like me who want to use version control, by having to activate it in advanced settings :)

Once that is solved, I believe you need implement two different solutions. Here are my thoughts:

OT to git

I don't know how OT works, but I guess that what you have as a resut is a succession of non-conflicting changes made by different users on the document?
So as you say, if you track revisions and authorship you can set rules for when to push the changes to a git repo? For instance after X mins of inactivity from a given user?

Git to OT

Updating the DB with changes pushed to the git repo is more tricky. I would say have post-push hooks server-side that update the database. But what if there is a conflict? Can you treat them the way OT treats them? (I don't know anything about it)
If that's the case, I guess it becomes as annoying as editing a document with several people at the same time, and so the app should not suffer from this?

But yes, versioned real-time collaboration is an exciting thing. That would reconcile users of google docs (yikes), and people who prefer to stay in the comfort of their own text editor.

[jackycute]

Wow, you get the ideas! Brilliant!
You can read the details of OT from here and the OT repo that adapted to hackmd is right here

I totally agreed that hackmd should keep it easy-to-use and always has intuitive & clean UI.
We'll make the git related actions as advanced feature and will be hidden as default.

OT to git

yes, OT did solve the conflicting changes and we can know when and set the corresponding rules for that.
So, this is my thought: to track the changes of the Doc, we'll record the diff patches (word-level) once the Doc is changed and meet some kind of rules (TBD), then calculate the new authorship map and write them into the DB.
For the users who don't really need git will use the versioning system we provided as above.
For the users who use git will set and connect the note to a repo or file in a repo, then the above diff patch with authorship will become several git commits and let user choose when to sync(pull/push).

Git to OT

I feel it's ok to let users trigger the process from git to OT instead using any hooks.
So it always can merge or solve conflicting changes like we did before using git (the files will show up some conflicting tags and when user solve that then it can be merge).

It's always my pleasure to make something really useful!

[jackycute]

refer PR #126 for an example

[FranckVE]

@jackycute, I suggest you take a look at NWJS (http://nwjs.io) and Electron (http://electron.atom.io) for porting a web app to the desktop with little refactoring. You probably know both, but just in case. They are just excellent.

[ccoenen]

@FranckVE was this meant for #235?

[jackycute]

@FranckVE I think you've missed the #31 and also #235
I used Electron before, so the thing is we have to solve other issues before doing that.
Thanks for suggestion.

[almereyda]

#218 also tells us that this doesn't neccessarily need to involve the filesystem layer, as plenty of git-enabled systems also offer HTTP JSON APIs for interaction with a git repository, such as GitHub or GitLab.
Then syncing snapshots of HackMD revisions with git commits is just another linked list. The revisions are already available, so one could easily check if git lacks behind and needs to be updated, or can be used for a fast-forward instead.

[joneepenk]

@jackycute any progress on this pls?

[jackycute]

@joneepenk Hi, I'm sorry but we're in the middle of refactoring.

[joneepenk]

@jackycute thx for the reply. just noticed it's attached to milestone 0.6.1. this is a really important feature for adoption HackMD in my team.

[frankgerhardt]

Very important feature

[iakat]

Any news with this?

[SISheogorath]

Sadly not. It's not a simple problem and actually needs some serious development work. It'll take time and resources. If someone feels inspired to do it, we really welcome and review PRs. 👍

[ccoenen]

@chauffer there's #126 and #303 - which was work in progress, but I modified it slightly and have it running with a 0.5.1 instance. It is only the "dump database contents to files and check those into git" part, not a sync.

Oh, and this will most certainly be broken with current master, I believe.

[brlin-tw]

Just my two cents: It would be great if HackMD's "Book mode" can be synced with a selected Git repo with Markdown files organized like the book's structure with folder/file style, like /README synced with /README.{md,markdown} and /Intro/About synced with /Intro/About.{md,markdown}.

UPDATE: Moved to #748 .

[almereyda]

Yes, the book mode already looks a lot like GitBook.

…

On 28 February 2018 at 21:44, 林博仁(Buo-Ren Lin) ***@***.***> wrote: Just my two cents: It would be great if HackMD's "Book mode" can be synced with a selected Git repo with Markdown files organized like the book's structure with folder/file style, like /README synced with /README.{md,markdown}. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#114 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABka_NLULB-5_Mb6xE4x5gkA7d_5kvLrks5tZbrHgaJpZM4IX9dQ> .

[ccoenen]

I don't see us changing the storage to Git any time soon, sorry.