fatal error in type inference (type bound) on CategoricalArrays.jl

[bkamins]

Steps to reproduce:

julia> versioninfo()
Julia Version 1.6.0-DEV.584
Commit 371bfa89d4 (2020-08-05 00:49 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

(the problem does seem not to be present on Windows)

then install DataFrames.jl 0.21.5 and run the following code:

julia> Random.seed!(1);

julia> df = DataFrame(a = rand([1:5;missing], 20), x1 = rand(1:100, 20),
                             x2 = rand(1:100, 20) +im*rand(1:100, 20));

julia> df.x3 = CategoricalVector{Union{Missing, Int}}(df.x1);

julia> gd = groupby(df, :a);

julia> combine(gd, :x3 => maximum∘skipmissing => :y)
ERROR: fatal error in type inference (type bound)
Stacktrace:
 [1] groupreduce(f::Function, op::Function, condf::Base.var"#66#67"{typeof(ismissing)}, adjust::Nothing, checkempty::Bool, incol::CategoricalVector{Union{Missing, Int64},UInt32,Int64,CategoricalValue{Int64,UInt32},Missing}, gd::GroupedDataFrame{DataFrame})
   @ DataFrames ~/.julia/packages/DataFrames/lfoTw/src/groupeddataframe/splitapplycombine.jl:1028
 [2] (::DataFrames.Reduce{typeof(max),Base.var"#66#67"{typeof(ismissing)},Nothing})(incol::CategoricalVector{Union{Missing, Int64},UInt32,Int64,CategoricalValue{Int64,UInt32},Missing}, gd::GroupedDataFrame{DataFrame})
   @ DataFrames ~/.julia/packages/DataFrames/lfoTw/src/groupeddataframe/splitapplycombine.jl:1032
 [3] _combine(f::Vector{Pair}, gd::GroupedDataFrame{DataFrame}, nms::Vector{Symbol}, copycols::Bool, keeprows::Bool)
   @ DataFrames ~/.julia/packages/DataFrames/lfoTw/src/groupeddataframe/splitapplycombine.jl:1150
 [4] combine_helper(f::Vector{Pair}, gd::GroupedDataFrame{DataFrame}, nms::Vector{Symbol}; keepkeys::Bool, ungroup::Bool, copycols::Bool, keeprows::Bool)
   @ DataFrames ~/.julia/packages/DataFrames/lfoTw/src/groupeddataframe/splitapplycombine.jl:589
 [5] _combine_prepare(gd::GroupedDataFrame{DataFrame}, cs::Union{Colon, typeof(nrow), Regex, AbstractString, Signed, Symbol, Unsigned, Pair, AbstractVector{T} where T, All, Between, InvertedIndex, StridedArray{T,1} where T, BitVector}; keepkeys::Bool, ungroup::Bool, copycols::Bool, keeprows::Bool)
   @ DataFrames ~/.julia/packages/DataFrames/lfoTw/src/groupeddataframe/splitapplycombine.jl:553
 [6] #combine#378
   @ ~/.julia/packages/DataFrames/lfoTw/src/groupeddataframe/splitapplycombine.jl:474 [inlined]
 [7] combine(gd::GroupedDataFrame{DataFrame}, cs::Pair{Symbol,Pair{Base.ComposedFunction{typeof(maximum),typeof(skipmissing)},Symbol}})
   @ DataFrames ~/.julia/packages/DataFrames/lfoTw/src/groupeddataframe/splitapplycombine.jl:474
 [8] top-level scope
   @ REPL[11]:1

[bkamins]

bump (I am not sure to whom this should be reported to directly, but this is a major problem for DataFrames.jl on Julia 1.6; CC @JeffBezanson)

We earlier had a similar issue with SentinelArrays.jl, reported here, but fortunately @quinnj was able to find a workaround. However, for this bug we have not found one yet.

Still I hope also JuliaData/DataFrames.jl#2326 can be resolved in Base also.

[JeffBezanson]

Only on master, so it must be fairly recent.

[bkamins]

Yes - Julia 1.5 is OK.

[bkamins]

Another case from today in a very similar part of DataFrames.jl code: JuliaData/DataFrames.jl#2382 (this time with SentinelArrays.jl and "Unreachable reached" and failing both on Julia 1.5 and Julia 1.6). I can open a separate issue if you feel it would be better to separate it (but I did not want to pollute issue tracker unnecessarily).

EDIT: I split it to #37179

[vtjnash]

Looks like the inference lookup was total garbage on master (was okay in v1.5), so we were inferring the wrong method.

Fixed by 102c9ac