Excessive Storage Utilisation

[peternewman]

Storage is still excessive for me, even after installing 36.1 and running the Delete Cache thing.

Map cache size: 250 MB

Before:
App: 30.98 MB
User data: 170 MB
Cache: 475 MB
Total: 676 MB

After:
App: 30.98 MB
User data: 6.95 MB
Cache: 475 MB
Total: 513 MB

So it's certainly cleared my user data a lot, which is the database presumably? I only got back from a trip on the 9th, so maybe there was still some excess data there which was less than 14 days old, but I think my overall usage has been around this figure for quite some time...

Indeed previously:
User data: 313MB
Cache: 399MB

Originally posted by @peternewman in #3133 (comment)

How to Reproduce
I'm not really sure, use the app, travel to lots of different places maybe.

Versions affected
SC 36.1
Android 9

[westnordost]

"Cache" is exclusively the HTTP file cache for the tiles.

[westnordost]

A quick debugging revealed that the DiskLruCache (included in okhttp3 dependency, a dependency of tangram-es) indeed correctly prunes anything above the set limit, but for some reason, the files themselves seem to remain in the directory. At least, looking at the cache in the app system settings, the cache grows and grows.

[westnordost]

The cache size limit is set to 50 MB.

The cached tiles are at /storage/emulated/0/Android/data/de.westnordost.streetcomplete.debug/cache/tile_cache/ (for the debug build). I downloaded all files in this directory onto my PC:

size: 50.023.046 bytes
size on disk: 60.973.056 bytes

So, the cache is working just fine and it is Android which seems to report the size of the cache wrong.

I'll look into which directories actually count as the "cache" in Android.

[peternewman]

after installing 36.1 and running the Delete Cache thing.

"Cache" is exclusively the HTTP file cache for the tiles.

As a bit of an aside, should "Delete Cache" in the settings actually say "Delete Quest Cache"? From the changelog it only touches the quest cache. I wasn't expecting it to clear the other one, but I could see how it could confuse others.

So, the cache is working just fine and it is Android which seems to report the size of the cache wrong.

I'll look into which directories actually count as the "cache" in Android.

Strange, I'm glad you can reproduce it on the emulator too though. I did notice there is a flush command, does that need calling or something occasionally?

[westnordost]

Android claims that StreetComplete Dev v37.0 beta1 needs:

• 42.41 MB for app
• 80.65 MB for data
• 85.92 MB for cache

---

The app itself apart, there are two directories assigned to the app:

/data/data/de.westnordost.streetcomplete.debug/ (on internal memory)

/sdcard/Android/data/de.westnordost.streetcomplete.debug/ (on "external" SD card. If there isn't one, it's emulated)

The following is definitely data:

• databases
• app_webview
• files (in internal)
• no_backup
• shared_prefs
• files (in external)

In total, it is 76 MB.

The following is definitely cache:

• cache (on internal)
• code_cache
• cache (on external)

In total, it is 48.9 MB.

---

So, it doesn't add up. Nevermind the few MB on the data. For some reason, Android claims there is almost double as much cache in use as there is in data in that directory. So far, I have no explanation for this.

[westnordost]

@peternewman so your Android claims that there is a huge chunk of cache (495 MB) used by the app. Maybe one thing you could do: Connect your phone to your computer and have a look at the reported free space on your phone in the explorer

or however it looks like in the OS you use. ( heh, it looks like this particular view hasn't really been changed since Windows 95)

Then, go to the system settings and clear the cache of Streetcomplete. What is the difference in reported free space?

[peternewman]

Total free disk before was 502.6MB free, after 1.1GB

App: 37.56 MB
User data: 3.51 MB
Cache: 32.77 kB
Total: 41.10 MB

This is before relaunching SC or using it at all.

[westnordost]

Okay, so indeed there is as much data associated with the app on the disk as claimed, only apparently not in the directories I have looked for it.

Next step:

1. download the complete content of the phone onto a directory on the PC
2. clear cache. Download it again into a different directory
3. compare the two directories

[westnordost]

I reported the issue to tangram-es. I am not 100% sure it is a tangram issue, but 90% since the duplicate tile cache data cannot be found anywhere in the directories the Android system assigned to the app. And on other directories, I have no access because I have not rooted my phone.

[westnordost]

Ok, turns out this is not a bug.

It looks like the physical size for the tiles on the sdcard is simply about double the size as the logical size because many tiles are so small that they are below the minimum block size for the file system.

[peternewman]

It looks like the physical size for the tiles on the sdcard is simply about double the size as the logical size because many tiles are so small that they are below the minimum block size for the file system.

Thanks @westnordost . I sort of wondered if it might be something like that from the start, but my phone wasn't mounting properly on my laptop to try and run some equivalent commands to compare the two.

Is it worth a note on that settings page like the old hard drive warnings (or wing mirrors), "size on disk may be bigger than configured"?

[westnordost]

I don't think so