cider-eval-region has catastrophic performance

[phillord]

I find that cider-eval-region has catastrophic performace, when evaling
largish areas. In worst case it locks up Emacs in a way that Ctrl-G cannot
recover from.

This is true with the current release and with master.

Reproduction. Eval this code (in Emacs!).

#+begin_src elisp
(defun aaa()
(interactive)
(loop for i from 1 to 1000 do
(insert
(format "(defn a%s [])\n\n" i))))
#+end_src

Open a new clojure file, start cider, run M-x aaa. Then M-x
mark-whole-buffer, M-x cider-eval-region. The eval messages start up quite
quickly, then get very slow and Emacs becomes unrecoverable.

Interestingly, this is a regression. I tried with this commit:

commit 21b5857
Author: Chaitanya Koparkar [email protected]
Date: Mon Apr 25 21:27:31 2016 +0530

Always fetch ns-vars-with-meta from nREPL middleware (#1715)

And everything worked swimmingly (evaling in a second or so). There is nothing
magic about this commit -- I just noticed that I got different performance on
different computers, and it turned out to be because one Emacs had an old
version of cider at this commit.

Environment & Version information

CIDER version information

;; CIDER 0.13.0snapshot (package: 0.14.0), nREPL 0.2.12

(Also cider-latest from MELPA-stable!)

Lein/Boot version

lein 2.7.1

Emacs version

Emacs-25 HEAD

Operating system

Ubuntu 16.04

[bbatsov]

@cskksc Can you look into this when you have some time?

[phillord]

It's the overlays. The same ones are being created continuous; I think you have creating a quadratic number of them.

Using my reproduction case this is a print out of all the overlays in buffer after evaluating the first ten forms.

"--make-overlay (127:140:result:(after-string  => #'user/a10 ))"

"total overlays: (#<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 113 to 125 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 99 to 111 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 85 to 97 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 71 to 83 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 57 to 69 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 43 to 55 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 29 to 41 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 15 to 27 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 13 in temp.clj> #<overlay from 1 to 141 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj> #<overlay from 127 to 140 in temp.clj>)"

Will pick further.

[dpsutton]

Overlays were broken for a bit because clojure core was missing from the ns cache. This was fixed clojure-emacs/cider-nrepl@fbc326d in cider-nrepl. It's possible that this wasn't font locking much in the old version that "worked" and now is correctly font locking so is taking a long time.

You could verify this by setting cider-font-lock-dynamically to nil and seeing if the speed is restored.

[phillord]

@dpsutton No, the bug was Emacs side -- top showed nothing for Java, 100% CPU for Emacs. Alas, overlays are rather easy to break Emacs with; they don't garbage collect cleanly and really slow the buffer down.

[dpsutton]

On the Java side, we return what to font lock. It wasn't returning clojure core stuff for font locking. My guess is that Emacs is locking up because it has so much to font lock after the fix.

…

On Mar 18, 2017 5:14 PM, "Phil Lord" ***@***.***> wrote: @dpsutton <https://github.com/dpsutton> No, the bug was Emacs side -- top showed nothing for Java, 100% CPU for Emacs. Alas, overlays are rather easy to break Emacs with; they don't garbage collect cleanly and really slow the buffer down. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1962 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGFPTbwnxJEWLkgm36SL09HjoUp0qUwvks5rnFdCgaJpZM4MhfvR> .

[dpsutton]

My suspicion was off base. I am totally wrong on this one.

[bbatsov]

This is already fixed.