Too slow on large files

[seagle0128]

This mode is beautiful, but brings significant performance issue for large files, e.g. 60000+ lines javascript file with js2-mode.
I have to disable it to improve the user experience.

[machinekoder]

Same is true for Python files with ~10000 lines.

[ankurdave]

I agree, parsing should never block user input. This will require adding checks for input-pending-t to all parsing loops, including the mode-specific ones.

Ideally parsing should also be incremental so it would always make progress, but that would be much more difficult.

[Hi-Angel]

@seagle0128 (or anyone else) can you please post a reproducer? The slowness on large files is kinda known problem, but I never could make it reliably reproduce, so whenever I had time to try to dig into it, there was just nothing to work with :(

[seagle0128]

I've already deletes color-identifiers-mode due to the performance issue, so unable to provide examples.

Maybe @ankurdave @machinekoder could provide?

[Hi-Angel]

I've already deletes color-identifiers-mode due to the performance issue, so unable to provide examples.

I see. Well, for the record: from what I've seen the problem was caused by Emacs generating too many "changed" events, way more than actually needed. However, I haven't seen the problem for quite some time even though I routinely work with 5000+ C files at my job, so the problem may have been fixed. At least it is with latest pre-release version.

If it wasn't fixed, then this patch may help, which one may try as well.

[seagle0128]

@Hi-Angel I will try later if I have some time. In my memory, color-identifiers-mode works well in C files, but doesn't for large js, python or ruby files. Hope the clue is helpful.

[Hi-Angel]

So, I managed to reproduce problem with a 26k lines python file created artificially by repeatedly concatenating a usual sized one. In this case it is specific to the way python files are parsed, and so I created a separate report for this: #103 I also just committed a change that alleviates this problem, so it should not be really noticeable.

I didn't manage to reproduce problem in a similar way by creating a huge 31k lines js file. Js, ruby and C files are all parsed by the same code in the plugin, so presumably the problem on that front is indeed long gone.

On top of that, latest master has undergone quite a lot of optimizations and cleanups. Also, given the problem often is mode-specific (as it is the case for python), having a generic all-encompassing issue might not be useful.

So, I'm closing this report. If you find some slowness in modes there's no report for, please create one. Thanks everyone for participation!