Crash when opening a 35MB, 13.7MM lines txt file

[innerlee]

• VSCode Version: 1.5.3
• OS Version: Ubuntu

I have a text file with 13765201 lines and each line contains an int range from 0 to 20. It's about 35M. Code crashes when opening it. There are other reports on big files (like #6474), however, unlike in this case, Codes will warn that the file is too large there. The bad part is that you cannot click on the wrong file.

Steps to Reproduce:

1. open a large file like here.
2. then wait.

[alexdima]

Even with the improvements in #30180 this will crash with OOM. The size of the file is not the cause, but the number of lines, where we spend 40-60 bytes (2 objects) to represent a line.

[rebornix]

With our next text buffer implementation, the metadata memory usage of line breaks decrease significantly. Old text buffer used 800+ MB memory for 13765201 lines metadata, but the new one only uses around 20 MB. As we won't release new text buffer impl this iteration, let's revisit this issue next week.

[rebornix]

@innerlee we re-implemented our text buffer and the memory usage of loading a file is less. For the file you provided, the metadata for the line breaks is 30MB so we used 75MB in total. The OOM issue should be gone if the system has enough memory.

[alexdima]

This 13MM lines file now opens fine.