Skip to content
This repository has been archived by the owner on Sep 17, 2024. It is now read-only.

TagStorage can get corrupted during writing #35

Closed
kirillt opened this issue Mar 15, 2023 · 5 comments
Closed

TagStorage can get corrupted during writing #35

kirillt opened this issue Mar 15, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@kirillt
Copy link
Member

kirillt commented Mar 15, 2023

Sometimes, the file serving as tag storage becomes corrupted:

...
117421-975209069:visa
92126-3414598226:hmm
60272-1305102029:music
34362-6340143
(END OF FILE)

In the example above, resource id isn't complete and no tags following.

In fact, there is even bigger problem: in such cases, half of the storage is lost. Thanks to backup mechanism the loss can be mitigated. However, this must not happen at all.

Atomic writing should be implemented.

@kirillt
Copy link
Member Author

kirillt commented Mar 22, 2023

We need to explore possibilities to prevent "dirty writing". Atomic writing could be implemented like this:

  1. Create temporary file with unique name (use standard Java/Kotlin methods to obtain it).
  2. When writing is done, copy the temporary file into the storage file (.ark/tags in case of tags storage).
    We also must ensure that no storage is written from several threads/coroutines simultaneously. Otherwise, only one of writes would succeed.

There could be other ways to implement this OR ready-made libraries.

@shubertm
Copy link
Member

@kirillt
Copy link
Member Author

kirillt commented Mar 31, 2023

@ShubertMunthali what can cause this? It seems to me that setTags finishes before persist is called..

@shubertm
Copy link
Member

shubertm commented Mar 31, 2023

We have two coroutines there, setTags is called in Dispatchers.IO scope while persist is called in launch which inherits the same scope. Is it really necessary to spawn another coroutine in this case?

@kirillt
Copy link
Member Author

kirillt commented Oct 15, 2023

Superseded by #69

@kirillt kirillt closed this as completed Oct 15, 2023
@github-project-automation github-project-automation bot moved this from Todo to Done in Development Oct 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
Archived in project
Development

No branches or pull requests

2 participants