Question about file overwriting and big files

[davidefer]

Hi
I have a question about overwriting big files in LittleFS.
My file system consists of 32KB flash blocks with the following dimension (after format):
Total space 2064384
Used space 131072
Free space 1933312

If I write a 1MB (1024*1024) bytes file everything works.
The state is as follow:
Total space 2064384
Used space 1212416
Free space 851968

Now if I re-write the same file (LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC) without deleting it, then I get an error "no more space" while writing.
The state is as follow:
Total space 2064384
Used space 2064384
Free space 0

Please note that if I delete the file before overwriting it, then everything works.

I wanted to know what is the reason for that and if there is a general rule for the restrictions one must cope with when dealing with big files.

Thanks in advance

[geky]

Hi @davidefer, sorry for the very late response.

In littlefs, all files are copy-on-write (COW). This means that during a write a copy of the data is made until you call either sync or close. This gives us a copy to fall back on if we lose power. Unfortunately this means writes to a file can cause the file to inflate up to 2x the original size.

Deleting the file before writing is a bit different. If we lose power after the delete call, there will be no file. So the filesystem doesn't need to worry about creating a copy since "no file" is a valid state if we lose power.

We can mimic the delete behaviour if we manually call lfs_file_sync after we open with LFS_O_TRUNC, since this also establishes that "no file" is a valid state.

What to do with O_TRUNC on a COW filesystem doesn't have an obvious answer. I went with keeping a copy because it's safer if a user expects power resilience and you can always call lfs_file_sync to avoid the space error.