Entire log directory fetched from GCS during each periodic reload

[chihuahua]

This issue relates to #158. During each periodic reload (which by default happens every 5 seconds), TensorBoard fetches the entire log directory from GCS. This incurs very high costs for large log directories.

[nfelt]

Hmm, is this really worth tracking separately from #158? That one already says this in the second comment: "it seems that every reload directly from GCP causes every single file to be accessed separately via the GCS api ."

[nfelt]

I guess if we consider #158 to be more about general slowness (as described in the OP) even on reading directly from a single GCS file, then it is a separate issue that we're doing too many file accesses overall, so maybe it does make sense to keep this one for that latter concern. Up to you.

When you say "entire log directory" what exactly do you mean? Are we actually attempting to fetch the entire contents of every event file on every reload? I would have thought that DirectoryWatcher would ensure that we were only attempting to read the most recent event file within each run.

[chihuahua]

Actually, yeah, lets track via #158. Stakeholders are already on that thread. Closing this issue.