Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cider-macroexpand-all in deps.edn project of definterface crashes #721

Closed
jellelicht opened this issue Aug 27, 2021 · 17 comments · Fixed by #731
Closed

Cider-macroexpand-all in deps.edn project of definterface crashes #721

jellelicht opened this issue Aug 27, 2021 · 17 comments · Fixed by #731

Comments

@jellelicht
Copy link

jellelicht commented Aug 27, 2021

Expected behavior

Being able to macro expand forms that happen to define interfaces, in either deps.edn or leiningen projects

Actual behavior

A crash pop-up when macro expanding forms that happen to define interfaces, but only in deps.edn projects.

Steps to reproduce the problem

I made a tiny example that seems to be reproducible for at least one other person:
https://github.com/jellelicht/cider-macro-repro

Steps to reproduce the problem (alt)

(macroexpand-1 '(definterface IFoo)) with cider-nrepl's machinery

Environment & Version information

CIDER version information

; Connected to nREPL server - nrepl://localhost:44805                                                                                 
;; CIDER 1.1.1 (Plovdiv), nREPL 0.8.3                                                                                                  
;; Clojure 1.10.3, Java 12

Emacs version

27.2

Operating system

GNU Guix (current master)

Stacktrace

  Show: Project-Only All 
  Hide: Clojure Java REPL Tooling Duplicates  (52 frames hidden)

  This is an unexpected CIDER middleware error.
  Please submit a bug report via `M-x cider-report-bug`.

  If these stacktraces are occurring frequently, consider using the
  button(s) below to suppress these types of errors for the duration of
  your current CIDER session. The stacktrace buffer will still be
  generated, but it will "pop under" your current buffer instead of
  "popping over". The button toggles this behavior.

 Suppress macroexpand-error 

2. Unhandled java.lang.ClassCastException
   class clojure.lang.Var$Unbound cannot be cast to class
   clojure.lang.DynamicClassLoader (clojure.lang.Var$Unbound and
   clojure.lang.DynamicClassLoader are in unnamed module of loader
   'app')

              genclass.clj:  722  clojure.core/gen-interface
              genclass.clj:  688  clojure.core/gen-interface
               RestFn.java:  142  clojure.lang.RestFn/applyTo
                  Var.java:  705  clojure.lang.Var/applyTo
             Compiler.java: 6997  clojure.lang.Compiler/macroexpand1
                  core.clj: 4012  clojure.core/macroexpand-1
                  core.clj: 4014  clojure.core/macroexpand
                  walk.clj:  130  clojure.walk/macroexpand-all/fn
                  walk.clj:   65  clojure.walk/prewalk
                  walk.clj:   61  clojure.walk/prewalk
                  core.clj: 2628  clojure.core/partial/fn
                  core.clj: 2759  clojure.core/map/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                 Cons.java:   39  clojure.lang.Cons/next
                   RT.java:  713  clojure.lang.RT/next
                  core.clj:   64  clojure.core/next
                  core.clj: 3130  clojure.core/dorun
                  core.clj: 3136  clojure.core/doall
                  walk.clj:   47  clojure.walk/walk
                  walk.clj:   65  clojure.walk/prewalk
                  walk.clj:  126  clojure.walk/macroexpand-all
                  walk.clj:  126  clojure.walk/macroexpand-all
           macroexpand.clj:  118  cider.nrepl.middleware.macroexpand/expand-clj/fn
           macroexpand.clj:  117  cider.nrepl.middleware.macroexpand/expand-clj
           macroexpand.clj:  111  cider.nrepl.middleware.macroexpand/expand-clj
           macroexpand.clj:  216  cider.nrepl.middleware.macroexpand/macroexpansion
           macroexpand.clj:  212  cider.nrepl.middleware.macroexpand/macroexpansion
           macroexpand.clj:  222  cider.nrepl.middleware.macroexpand/macroexpansion-reply
           macroexpand.clj:  221  cider.nrepl.middleware.macroexpand/macroexpansion-reply
        error_handling.clj:  160  cider.nrepl.middleware.util.error-handling/eval2258/fn
              MultiFn.java:  234  clojure.lang.MultiFn/invoke
           macroexpand.clj:  225  cider.nrepl.middleware.macroexpand/handle-macroexpand
           macroexpand.clj:  224  cider.nrepl.middleware.macroexpand/handle-macroexpand
                  Var.java:  388  clojure.lang.Var/invoke
                 nrepl.clj:  283  cider.nrepl/wrap-macroexpand/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  522  cider.nrepl/wrap-clojuredocs/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  428  cider.nrepl/wrap-stacktrace/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  204  cider.nrepl/wrap-info/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
    interruptible_eval.clj:  154  nrepl.middleware.interruptible-eval/interruptible-eval/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  223  cider.nrepl/wrap-inspect/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
               session.clj:  363  nrepl.middleware.session/add-stdin/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
             load_file.clj:   81  nrepl.middleware.load-file/wrap-load-file/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  105  cider.nrepl/wrap-content-type/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                caught.clj:   97  nrepl.middleware.caught/wrap-caught/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  330  cider.nrepl/wrap-out/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  159  cider.nrepl/wrap-debug/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 print.clj:  234  nrepl.middleware.print/wrap-print/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  478  cider.nrepl/wrap-tracker/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
            sideloader.clj:  104  nrepl.middleware.sideloader/wrap-sideloader/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                lookup.clj:   51  nrepl.middleware.lookup/wrap-lookup/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
               session.clj:  309  nrepl.middleware.session/session/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  186  cider.nrepl/wrap-enlighten/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                server.clj:  130  nrepl.server/default-handler/fn
                server.clj:   22  nrepl.server/handle*
                server.clj:   19  nrepl.server/handle*
                server.clj:   39  nrepl.server/handle/fn
                  core.clj: 2034  clojure.core/binding-conveyor-fn/fn
                  AFn.java:   18  clojure.lang.AFn/call
           FutureTask.java:  264  java.util.concurrent.FutureTask/run
   ThreadPoolExecutor.java: 1128  java.util.concurrent.ThreadPoolExecutor/runWorker
   ThreadPoolExecutor.java:  628  java.util.concurrent.ThreadPoolExecutor$Worker/run
               Thread.java:  835  java.lang.Thread/run

1. Caused by clojure.lang.Compiler$CompilerException
   Error compiling NO_SOURCE_PATH at (0:0)
   #:clojure.error{:phase :macroexpansion, :line 0, :column 0, :source "NO_SOURCE_PATH", :symbol clojure.core/gen-interface}
             Compiler.java: 7023  clojure.lang.Compiler/macroexpand1
                  core.clj: 4012  clojure.core/macroexpand-1
                  core.clj: 4014  clojure.core/macroexpand
                  walk.clj:  130  clojure.walk/macroexpand-all/fn
                  walk.clj:   65  clojure.walk/prewalk
                  walk.clj:   61  clojure.walk/prewalk
                  core.clj: 2628  clojure.core/partial/fn
                  core.clj: 2759  clojure.core/map/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                 Cons.java:   39  clojure.lang.Cons/next
                   RT.java:  713  clojure.lang.RT/next
                  core.clj:   64  clojure.core/next
                  core.clj: 3130  clojure.core/dorun
                  core.clj: 3136  clojure.core/doall
                  walk.clj:   47  clojure.walk/walk
                  walk.clj:   65  clojure.walk/prewalk
                  walk.clj:  126  clojure.walk/macroexpand-all
                  walk.clj:  126  clojure.walk/macroexpand-all
           macroexpand.clj:  118  cider.nrepl.middleware.macroexpand/expand-clj/fn
           macroexpand.clj:  117  cider.nrepl.middleware.macroexpand/expand-clj
           macroexpand.clj:  111  cider.nrepl.middleware.macroexpand/expand-clj
           macroexpand.clj:  216  cider.nrepl.middleware.macroexpand/macroexpansion
           macroexpand.clj:  212  cider.nrepl.middleware.macroexpand/macroexpansion
           macroexpand.clj:  222  cider.nrepl.middleware.macroexpand/macroexpansion-reply
           macroexpand.clj:  221  cider.nrepl.middleware.macroexpand/macroexpansion-reply
        error_handling.clj:  160  cider.nrepl.middleware.util.error-handling/eval2258/fn
              MultiFn.java:  234  clojure.lang.MultiFn/invoke
           macroexpand.clj:  225  cider.nrepl.middleware.macroexpand/handle-macroexpand
           macroexpand.clj:  224  cider.nrepl.middleware.macroexpand/handle-macroexpand
                  Var.java:  388  clojure.lang.Var/invoke
                 nrepl.clj:  283  cider.nrepl/wrap-macroexpand/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  522  cider.nrepl/wrap-clojuredocs/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  428  cider.nrepl/wrap-stacktrace/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  204  cider.nrepl/wrap-info/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
    interruptible_eval.clj:  154  nrepl.middleware.interruptible-eval/interruptible-eval/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  223  cider.nrepl/wrap-inspect/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
               session.clj:  363  nrepl.middleware.session/add-stdin/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
             load_file.clj:   81  nrepl.middleware.load-file/wrap-load-file/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  105  cider.nrepl/wrap-content-type/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                caught.clj:   97  nrepl.middleware.caught/wrap-caught/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  330  cider.nrepl/wrap-out/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  159  cider.nrepl/wrap-debug/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 print.clj:  234  nrepl.middleware.print/wrap-print/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  478  cider.nrepl/wrap-tracker/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
            sideloader.clj:  104  nrepl.middleware.sideloader/wrap-sideloader/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                lookup.clj:   51  nrepl.middleware.lookup/wrap-lookup/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
               session.clj:  309  nrepl.middleware.session/session/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                 nrepl.clj:  186  cider.nrepl/wrap-enlighten/fn
            middleware.clj:   16  nrepl.middleware/wrap-conj-descriptor/fn
                server.clj:  130  nrepl.server/default-handler/fn
                server.clj:   22  nrepl.server/handle*
                server.clj:   19  nrepl.server/handle*
                server.clj:   39  nrepl.server/handle/fn
                  core.clj: 2034  clojure.core/binding-conveyor-fn/fn
                  AFn.java:   18  clojure.lang.AFn/call
           FutureTask.java:  264  java.util.concurrent.FutureTask/run
   ThreadPoolExecutor.java: 1128  java.util.concurrent.ThreadPoolExecutor/runWorker
   ThreadPoolExecutor.java:  628  java.util.concurrent.ThreadPoolExecutor$Worker/run
               Thread.java:  835  java.lang.Thread/run
@vemv vemv transferred this issue from clojure-emacs/cider Oct 5, 2021
@vemv
Copy link
Member

vemv commented Oct 5, 2021

Thanks!

Issue transfered to cider-nrepl which does the macroexpanding.

Might give this a shot soon enough.

@vemv
Copy link
Member

vemv commented Oct 5, 2021

I failed to reproduce the issue with this commit 1dfeef8 so probably I will be out of ideas.

If you have the spare time, maybe you can try the same over a couple other JDKs; 12 is bit of an odd one. The LTS JDKs are 8, 11 and 17.

8 is the less likely one to give odd errors (such as the one you shared: Unhandled java.lang.ClassCastException class clojure.lang.Var$Unbound cannot be cast to class clojure.lang.DynamicClassLoader (clojure.lang.Var$Unbound and clojure.lang.DynamicClassLoader are in unnamed module of loader 'app'))

@vemv
Copy link
Member

vemv commented Oct 5, 2021

There's also a low but real chance that setting :jvm-opts ["-Dorchard.use-dynapath=false"] in deps.edn per https://github.com/clojure-emacs/orchard/tree/fa5bbb670bd502e59dfa61cfa8e65def38897239#configuration-options would leave the classloaders cleaner (unaltered), making the issue less likely.

@jellelicht
Copy link
Author

jellelicht commented Oct 5, 2021

@vemv thanks, I will be trying out your suggestions, and report back here. Were you able to at least reproduce the issue as I did (via cider)?

@jellelicht
Copy link
Author

I can reproduce my issue using the linked repo with the following java versions, both with and without the use-dynapath settings.

  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16

I’ve not yet been able to test 17, as that has not been packaged for my distro yet. @vemv can I bother you to try to my reproducer in the repo I linked in my original issue?

I had one other person on the cider discord confirm my issue, but it would be nice if you can verify that you don’t see the issue at all (classic case of PEBKAC then on my end 😉).

@vemv
Copy link
Member

vemv commented Nov 29, 2021

I don't run a modern cider.el myself.

One thing you can try is renaming user.clj to something else - user has a special semantic and is loaded automatically / early.

@jellelicht
Copy link
Author

jellelicht commented Nov 29, 2021

I made sure to try with something that is not user.clj as well; same results. Would you be so kind as to move this back to the cider project in that case? TBH, I don’t really think the problem is here; since the macro-expanding the exact same code works when used in a lein project /w the same versions of pretty much everything else. Thanks for your help here either way!

EDIT: Since this issue only seems to occur for deps.edn projects, and not lein projects, I doubt we’ll be able to find + fix the problem here either way.

@vemv
Copy link
Member

vemv commented Nov 30, 2021

Would you be so kind as to move this back to the cider project in that case?

It's in bit of a grey zone, as we don't really know what's going on. We can default to avoid churn, I'm not sure you'd get any more help over the cider.el tracker.

n.b. as reported, so far this seems a rather low-impact corner case (which I'm willing to work through regardless).

One thing that would help is comparing the java processes. Try ps aux | grep java and please paste here the java processes that are ultimately running your code (e.g. the final JVM spawned by Lein, and the one JVM spawned by tools.deps).

It also would help to hit the macroexpand ns directly from a repl. This way we bypass a good chunk of the cider.el / nrepl machinery and can discard at least one hypothesis.

Cheers - V

@vemv
Copy link
Member

vemv commented Nov 30, 2021

btw an awesome repro might look like this:

clojure --eval "(require 'cider.nrepl.middleware.macroexpand), (cider.nrepl.middleware.macroexpand/,,, ,,,)"

@jellelicht
Copy link
Author

jellelicht commented Nov 30, 2021

Everything works as one would expect when directly calling into cider.nrepl.middleware.macroexpand, but I found a more fundamental issue (which might already give you some ideas on what the actual cause of the problem might be):

Given the following macro:

(defmacro dummy []                                                                                                                     
  (println  (deref clojure.lang.Compiler/LOADER))                                                                                      
  42)

And a deps.edn-based cider-repl will (correctly) M-x cider-macroexpand-all a call to (dummy) to 42. Kicker is, that the printen output will be:

#object[clojure.lang.Var$Unbound 0x3daadf30 Unbound: #<Var: --unnamed-->]

If one were to actually run (dummy) on the repl, one would see:

#object[clojure.lang.DynamicClassLoader 0x335cbf1b clojure.lang.DynamicClassLoader@335cbf1b]                                           
42

In a lein-based cider-repl, both will display a clojure.lang.DynamicCLassLoader!
@vemv Does this help in any way?

@vemv
Copy link
Member

vemv commented Dec 1, 2021

Yes, sure! That's much useful.

I reckon that cider.nrepl.middleware.macroexpand could bind LOADER to a new DynamicClassLoader instance if it's currently unbound.

n.b., being unbound is its default state, so there's possibly more than one reason why it might remain unbound when invoking cider.nrepl.middleware.macroexpand. Which is a good thing here: we'd fix this issue, and possibly other unforeseen code paths.

Does this sound good to you?

@vemv
Copy link
Member

vemv commented Dec 1, 2021

(thanks for the 👍; I repro'd + fixed this in the meantime. PR soon)

@jellelicht
Copy link
Author

Please Let me know if there’s something I can do to help test

@jellelicht
Copy link
Author

@vemv I can confirm that it fixes my use-case! Thanks again!

@plexus
Copy link
Contributor

plexus commented Dec 6, 2021

#object[clojure.lang.Var$Unbound 0x3daadf30 Unbound: #<Var: --unnamed-->]

This is misleading, this means the Var has no root binding, but it does usually have a thread-local binding during eval:

In a plain clj REPL:

user> [clojure.lang.Compiler/LOADER @clojure.lang.Compiler/LOADER]
;;=>
[#<Var: --unnamed--> #object[clojure.lang.DynamicClassLoader 0x7726e185 "clojure.lang.DynamicClassLoader@7726e185"]]

@vemv
Copy link
Member

vemv commented Dec 6, 2021

Got it, very interesting thanks!

Still, the open PR seems to make sense since its accompanying test would fail without the fix itself.

It can make sense if we consider during eval to not include the macroexpand phase?

i.e. the cider-macroexpand-all macroexpands, does not eval and therefore LOADER is truly unbound.

@vemv vemv closed this as completed in #731 Dec 10, 2021
@vemv
Copy link
Member

vemv commented Dec 22, 2021

Released in CIDER 1.2 / cider-nrepl 0.27.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants