Precompilation affects function call in @generated function

[maleadt]

Running into some #17057 complexity on latest master, where depending on the position in the source file / precompilation is on or off, a generated function selects a different method to call.

Foo/src/Foo.jl:

__precompile__()

module Foo

type Original end

end

Bar/src/Bar.jl:

__precompile__()

module Bar

using Foo

transform{T}(::Type{T}) = T

@generated function test()
    return transform(Foo.Original)
end

immutable Transformed end
transform(::Type{Foo.Original}) = Transformed

end

test.jl:

using Bar
@show Bar.test()

If I run this code as-is, I get the expected (to me) behaviour of the Original type getting transformed to Transformed:

$ JULIA_LOAD_PATH=. JULIA_PKGDIR=. julia test.jl
Bar.test() = Bar.Transformed

However, if I disable precompilation, it matches the first definition of transform:

$ JULIA_LOAD_PATH=. JULIA_PKGDIR=. julia --compilecache=no test.jl
Bar.test() = Foo.Original

This is further exemplified by removing the first transform definition:

$ JULIA_LOAD_PATH=. JULIA_PKGDIR=. julia --compilecache=no test.jl
ERROR: LoadError: MethodError: no method matching transform(::Type{Foo.Original})
The applicable method may be too new: running in world age 20564, while current world is 20566.
Closest candidates are:
  transform(::Type{Foo.Original}) at Bar/src/Bar.jl:14 (method too new to be called from this world context.)
Stacktrace:
 [1] test(...) at Bar/src/Bar.jl:10
 [2] include_from_node1(::String) at ./loading.jl:532
 [3] include(::String) at ./sysimg.jl:14
 [4] process_options(::Base.JLOptions) at ./client.jl:308
 [5] _start() at ./client.jl:374
while loading test.jl, in expression starting on line 229

This error does not show if I run with precompilation enabled. Furthermore, if I move the @generated function test below the additional definition of transform the issue disappears.
Also, on 0.5 everything works 'as expected' (ie. Transformed is returned in all cases, without errors)

[yuyichao]

This appears to be documented.

Some operations that should not be attempted include:
....
Calling any function that is defined after the body of the generated function. This condition is relaxed for incrementally-loaded precompiled modules to allow calling any function in the module.

[maleadt]

Ah, I was only looking at methods.md...
In my original use-case the transform function was called indirectly:

# package infrastructure

transform{T}(::Type{T}) = T
function transform_arguments(argtyp)
    return transform(argtyp)
end

@generated function entrypoint(args)
    transform_arguments(...)
end


# somewhere else (possibly user code)

type Something     end
type SomethingElse end
transform(::Type{Something}) = SomethingElse

I guess this falls under the same restriction?

[vtjnash]

Yes. As you can see, generated function are now prohibited from masquerading as memorization memoization routines.

[simonster]

Does @pure (which does not appear to be documented at all) have the same restriction?

[vtjnash]

Officially undefined (in that it has been observed the behavior in this case is fairly unpredictable)

[cstjean]

As you can see, generated function are now prohibited from masquerading as memorization memoization routines.

Unfortunately, this restriction also makes it hairy to write an extensible compiler for a DSL implemented with parametric types. I can't define new types and say compiler_treat_type_as_constant(::MyNewType) = true. The only way of passing information back is through inheritance 🙁