MethodError: no method matching iterate(::Core.CodeInfo)

[pat-alt]

I'm trying to use Zygote on Julia v1.6.3, because unfortunately that's all that's currently available on the HPC I'm working on. It fails to precompile throwing the following error:

julia> using Zygote
@Info (Base): Precompiling Zygote [e88e6eb3-aa80-5325-afca-941959d7151f]                        
  │                                                                                             
  ╰──────────────────────────────────────────────── 
                       Fri, 18 Aug 2023 08:51:10 
 ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Core.CodeInfo)
Closest candidates are:
  iterate(::Union{LinRange, StepRangeLen}) at range.jl:664
  iterate(::Union{LinRange, StepRangeLen}, ::Int64) at range.jl:664
  iterate(::T) where T<:Union{Base.KeySet{var"#s77", var"#s76"} where {var"#s77", var"#s76"<:Dict}, Base.ValueIterator{var"#s75"} where var"#s75"<:Dict} at dict.jl:693
  ...
Stacktrace:
  [1] indexed_iterate(I::Core.CodeInfo, i::Int64)
    @ Base ./tuple.jl:89
  [2] pullback(::Function, ::Zygote.Context{false}, ::Int64, ::Vararg{Int64, N} where N)
    @ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:44
  [3] pullback(::Function, ::Int64, ::Int64)
    @ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:42
  [4] gradient(::Function, ::Int64, ::Vararg{Int64, N} where N)
    @ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:96
  [5] top-level scope
    @ ~/.julia/packages/Zygote/4rucm/src/precompile.jl:17
  [6] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [7] include
    @ ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:1 [inlined]
  [8] macro expansion
    @ ~/.julia/packages/Requires/Z8rfN/src/Requires.jl:40 [inlined]
  [9] precompile()
    @ Zygote ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:63
 [10] macro expansion
    @ ~/.julia/packages/PrecompileTools/0yi7r/src/workloads.jl:81 [inlined]
 [11] top-level scope
    @ ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:84
 [12] include
    @ ./Base.jl:386 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1235
 [14] top-level scope
    @ none:1
 [15] eval
    @ ./boot.jl:360 [inlined]
 [16] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [17] top-level scope
    @ none:1
in expression starting at /Users/FA31DU/.julia/packages/Zygote/4rucm/src/precompile.jl:17
in expression starting at /Users/FA31DU/.julia/packages/Zygote/4rucm/src/Zygote.jl:1
ERROR: 
──────────────────────────────────────── ErrorException ────────────────────────────────────────
╭──── Error Stack ─────────────────────────────────────────────────────────────────────────────╮
│                                                                                              │
│  ──────────────────────────────────── In module Base ────────────────────────────────────    │
│                                                                                              │
│         ╭───────────────────────────────────────────────────────────────────────────╮        │
│     (1) │  require(into::Module, mod::Symbol)                                       │        │
│         │  ./loading.jl:923                                                         │        │
│         ╰───────────────────────────────────────────────────────────── TOP LEVEL ───╯        │
│                                                                                              │
│  ────────────────────────────────────────────────────────────────────────────────────────    │
│                                 Skipped 4 frames in Base                                     │
│  ────────────────────────────────────────────────────────────────────────────────────────    │
│                                                                                              │
│         ╭───────────────────────────────────────────────────────────────────────────╮        │
│     (6) │  error(s::String)                                                         │        │
│         │  ./error.jl:33                                                            │        │
│         ╰──────────────────────────────────────────────────────────── ERROR LINE ───╯        │
│                                                                                              │
╰──── Error Stack ─────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────── ErrorException ───────────────────────────────────────╮
│                                                                                              │
│  Failed to precompile Zygote [e88e6eb3-aa80-5325-afca-941959d7151f] to /Users/FA31DU/.jul    │
│  ia/compiled/v1.6/Zygote/jl_chkkNu.                                                          │
│                                                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────╯

Is there anything obvious going on here? Is this salvageable or should I just wait for a newer Julia version to be operational on the HPC?

Thanks!

[ToucheSir]

Nothing obvious, I can't think if any scenario where we'd get back a chunk of IR instead of an actual function here. Can you paste the output of Zygote._pullback(Zygote.Context(), pow, 2, 3), where pow can be copied from https://github.com/FluxML/Zygote.jl/blob/master/src/precompile.jl?

[pat-alt]

I don't think I can properly test this when Zygote fails to precompile? Sorry if I'm missing something

julia> function pow(x, n)
         r = 1
         while n > 0
           n -= 1
           r *= x
         end
         return r
       end
pow (generic function with 1 method)

julia> Zygote._pullback(Zygote.Context(), pow, 2, 3)
ERROR: 
─────────────────────────────────────────────────────────────── UndefVarError ──────────────────────────────────────────────────────────────
╭──── Error Stack ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                                                                          │
│      ╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮           │
│  (1) │  top-level scope                                                                                                      │           │
│      │  REPL[6]:1                                                                                                            │           │
│      ╰───────────────────────────────────────────────────────────────────────────────────────────────────────── TOP LEVEL ───╯           │
│                                                                                                                                          │
╰──── Error Stack ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────── UndefVarError ─────────────────────────────────────────────────────────────╮
│                                                                                                                                          │
│  UndefVarError: Zygote not defined                                                                                                       │
│                                                                                                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯


julia> 

[pat-alt]

I've just moved that into the precompile.jl. Here's the output (where the stack trace is the same as in the OP):

julia> using Zygote
@Info (Base): Precompiling Zygote [e88e6eb3-aa80-5325-afca-941959d7151f]                                                                                      
  │                                                                                                                                                           
  ╰──────────────────────────────────────────────── 
                       Mon, 21 Aug 2023 14:33:39 
 [ Info: Testing pullback method:
CodeInfo(
1 ─       $(Expr(:meta, :inline))
│   %2  = Base.getfield(args, 1)
│   %3  = Base.getfield(args, 2)
│   %4  = (Vector{Any})()
│   %5  = (Vector{Any})()
│   %6  = (Vector{Any})()
│   %7  = (Vector{Any})()
│         phi_2_1 = %3
│         phi_2_2 = 1
│         phi_2_3 = 0x01
└──       goto #2
    @ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:3 within `pow'
2 ┄ %12 = Zygote._pullback(ctx, Zygote.:>, phi_2_1, 0)
│   %13 = Base.getindex(%12, 1)
│   %14 = Base.getindex(%12, 2)
│         Zygote._push!(%7, phi_2_3)
│         Zygote._push!(%6, %14)
└──       goto #5 if not %13
3 ─       goto #4
    @ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:4 within `pow'
4 ─ %19 = Zygote._pullback(ctx, Zygote.:-, phi_2_1, 1)
│   %20 = Base.getindex(%19, 1)
│   %21 = Base.getindex(%19, 2)
│   @ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:5 within `pow'
│   %22 = Zygote._pullback(ctx, Zygote.:*, phi_2_2, %2)
│   %23 = Base.getindex(%22, 1)
│   %24 = Base.getindex(%22, 2)
│         Zygote._push!(%5, %24)
│         Zygote._push!(%4, %21)
│         phi_2_1 = %20
│         phi_2_2 = %23
│         phi_2_3 = 0x02
└──       goto #2
5 ─ %31 = Base.tuple(%7, %6, %5, %4)
│   %32 = (Zygote.Pullback{Tuple{typeof(Zygote.pow), Int64, Int64}, Any})(%31)
│   %33 = Base.tuple(phi_2_2, %32)
└──       return %33
)
ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Core.CodeInfo)
Closest candidates are:
  iterate(::Union{LinRange, StepRangeLen}) at range.jl:664
  iterate(::Union{LinRange, StepRangeLen}, ::Int64) at range.jl:664
  iterate(::T) where T<:Union{Base.KeySet{var"#s77", var"#s76"} where {var"#s77", var"#s76"<:Dict}, Base.ValueIterator{var"#s75"} where var"#s75"<:Dict} at dict.jl:693
  ...
Stacktrace:
[...]

[ToucheSir]

Do you mind testing on a later 1.6 version? It seems to me that our CI should've caught this if LTS is failing, so perhaps some 1.6 patch release fixed internals so that Zygote works again.

[pat-alt]

I think the CI runs tests on v1.6.7, which works just fine for me locally:

julia> versioninfo()
Julia Version 1.6.7
Commit 3b76b25b64 (2022-07-19 15:11 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin21.4.0)
  CPU: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = code

julia> using Zygote

julia> 

[ToucheSir]

That's good. I think it'd probably be easier and more robust to see if you can get a newer patch/minor release of Julia than for us to add a workaround for issues which appear to have been fixed in a patch release. I'll leave this open for a bit though in case anyone does discover the root cause and it happens to be a straightforward fix.

[pat-alt]

Sounds good, thank you!