Sync is very happy to delete your notebooks

[Skrylar]

Scenario 1:

• Synchronize to a local file directory.
• Delete the files inside the directory.
• Sync again.

Joplin will happily wipe out all of the notes from your database, with no obvious undo (or even confirmation step.)

Scenario 2:

• Synchronize to one file directory.
• Decide to instead sync to one on a different server (fresh directory, no history.)

Joplin weirdly decided to start deleting notebooks (but only some) while pushing others to the new remote. It seems to have gotten a strange idea that some of the content from the previous sync was supposed to be deleted, but content made after changing to the new directory wasn't.

In both cases I had to resort to pulling databases out of daily backups (and partially unsynched copies from other installs), which is well... 💣

---

The simplest in-app way to avoid these kind of apocalypses I can think of is perhaps a special trash bin area, which is not synced but items that get overwritten/deleted by the sync process can be moved or flagged. Some kind of trashed_at field could handle it in the database, although the UX would have to be figured out.

The option for a visual sync interface (ex. My Life Organized, Unison, some Git clients) that shows what is happening to what note and which side to keep would be ideal but understandably difficult to design and include.

[tessus]

Sorry, I was in the wrong issue...

[bernd-wechner]

This sounds like a real concern. One thing I'd expect above all others for such syncing tools as Joplin is a failsafe against data loss to be honest. I'll have a look at ensuring regular backups if my Nextcloud doesn't already do them. A good heads up as I've been a tad laissez faire about this.

[Outi-s]

Sync has been pretty stable so far but this does raise some concern. A fail-safe, a backup or a thrash-bin would be convenient for if anything goes haywire.

[laurent22]

This is a point of syncing - if you update an item, it's updated everywhere, if you delete an item, it's delete everywhere. If you delete all items, they are all deleted everywhere.

Likewise, if you delete all your notes in Evernote, will you complain that they also got deleted in the web app?

I can see how this can happen as a result of a user error, so there've been a few solutions discussed, like prompting the user when many items are going to be updated or deleted.

A message has also be added warning the user if they switch the URI in the WebDAV options (same warning should probably be added to filesystem sync too).

In the meantime it's recommended to make regular backup just to be sure you don't lose anything (personally I have a cron script that backup and version everything in a github repo so I can easily go back to any previous version).

[Skrylar]

This is a point of syncing - if you update an item, it's updated everywhere, if you delete an item, it's delete everywhere. If you delete all items, they are all deleted everywhere.

Condescention aside, I didn't delete any notebooks in the 100% clean sync directory. It just decided to start deleting some notebooks but not all. And there was no history in the empty directory, so what told it it was okay to do that?

Likewise, if you delete all your notes in Evernote, will you complain that they also got deleted in the web app?

Never used Evernote, but:

• MLO pops a confirmation window if it's confused (ex. a task is complete or delete on point A but has been modified at a later date but before sync on point B; there is a conflict here, and it asks the user to resolve it.)
• QOwnNotes keeps multiple versions of notes around (albeit thanks to its dependence on {own,next}Cloud for performing synchronization)
• Mindforger sync via git repositories also won't eat data. (Although this isn't a built-in sync, but the native format are markdown files and the creator does recommend git for this purpose.)
• Anki also keeps numerous copies of its database around, so sync clobbers wrt. AnkiWeb are recoverable there too.

Joplin is literally the only note solution I've tested so far that offers sync and has lost my data happily. If that isn't the definition of a severe problem, I guess I just don't understand what is.

[laurent22]

Where I disagree is that Joplin in these cases didn't lose data - that would be like saying you delete a note in Evernote, it gets synced to the client, and Evernote "lost data". No it didn't, it synced your action.

If for example a user deletes all their data in a synced git repository or in Nextcloud, it will delete it in Mindforger or QOwnCloud too, wouldn't it? Likewise Joplin application doesn't have any control over user actions on the cloud services - if they delete data, that action will be synced. If you delete it from the app though there will be a prompt.

I agree that UI can be improved anyway to allow users to recover from their errors and that's the two points I mentioned above.

[laurent22]

Joplin weirdly decided to start deleting notebooks (but only some) while pushing others to the new remote. It seems to have gotten a strange idea that some of the content from the previous sync was supposed to be deleted, but content made after changing to the new directory wasn't.

There's an entry about this in the FAQ and I will add the same warning as for WebDAV in the config below filesystem sync.

[Jestre]

There's an entry about this in the FAQ and I will add the same warning as for WebDAV in the config below filesystem sync.

Please do, for when I decided to move from Seafile to Nextcloud, I changed the sync URL in DevonThink, and it saw the empty repository and sync'ed all of the data from my laptop to the new repo. I did the same in Joplin, and it saw the empty repository and deleted all my notes on my laptop. Clearly less than desirable behavior.

[Skrylar]

Please do, for when I decided to move from Seafile to Nextcloud, I changed the sync URL in DevonThink, and it saw the empty repository and sync'ed all of the data from my laptop to the new repo. I did the same in Joplin, and it saw the empty repository and deleted all my notes on my laptop. Clearly less than desirable behavior.

This also happens if you are using shares with file repositories, as I just found out last night.

If you are using a file directory for sync, that directory is an NFS/Samba share, it is possible for a network or server problem to result in the network share becoming unmounted. Which means your local filesystem sees an empty directory. Which Joplin assumes to mean you want your entire notebook deleted again. Then the NFS/Samba share comes back up :/

[Outi-s]

@laurent22 which one would you suggest is ideal for a regular backup, jex or raw? (On Windows GUI app.)

[laurent22]

If you are using a file directory for sync, that directory is an NFS/Samba share, it is possible for a network or server problem to result in the network share becoming unmounted. Which means your local filesystem sees an empty directory. Which Joplin assumes to mean you want your entire notebook deleted again. Then the NFS/Samba share comes back up :/

Isn't it a bug in the way the share is setup though? If it cannot be mounted shouldn't it make it so that the folder does not exist at all in the system instead of showing an empty one? This indeed causes an issue with Joplin but probably other apps too. Eg. if you want to backup a directory with rsync, the tool will see an empty directory and ends up deleting the source.

[laurent22]

@laurent22 which one would you suggest is ideal for a regular backup, jex or raw? (On Windows GUI app.)

Raw is best because it will create individual files for each item. JEX is basically just a tar of a RAW directory.

[Outi-s]

@laurent22 which one would you suggest is ideal for a regular backup, jex or raw? (On Windows GUI app.)

Raw is best because it will create individual files for each item. JEX is basically just a tar of a RAW directory.

Thank you.