Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SentinelVector breaks var aggregation #2382

Closed
bkamins opened this issue Aug 24, 2020 · 2 comments · Fixed by #2383
Closed

SentinelVector breaks var aggregation #2382

bkamins opened this issue Aug 24, 2020 · 2 comments · Fixed by #2383
Milestone

Comments

@bkamins
Copy link
Member

bkamins commented Aug 24, 2020

SentinelVector causes problems with fast aggregation path for var function:

julia> using DataFrames, CSV, Statistics

julia> df = CSV.read(IOBuffer("x,y\n1,1.0\n1,\n1,1.5\n2,2.0\n2,\n2,3.0\n"), DataFrame)
6×2 DataFrame
│ Row │ x     │ y        │
│     │ Int64 │ Float64? │
├─────┼───────┼──────────┤
│ 1   │ 1     │ 1.0      │
│ 2   │ 1     │ missing  │
│ 3   │ 1     │ 1.5      │
│ 4   │ 2     │ 2.0      │
│ 5   │ 2     │ missing  │
│ 6   │ 2     │ 3.0      │

julia> typeof.(eachcol(df))
2-element Array{DataType,1}:
 Array{Int64,1}
 SentinelArrays.SentinelArray{Float64,1,Float64,Missing,Array{Float64,1}}

julia> gdf = groupby(df, :x)
GroupedDataFrame with 2 groups based on key: x
First Group (3 rows): x = 1
│ Row │ x     │ y        │
│     │ Int64 │ Float64? │
├─────┼───────┼──────────┤
│ 1   │ 1     │ 1.0      │
│ 2   │ 1     │ missing  │
│ 3   │ 1     │ 1.5      │
⋮
Last Group (3 rows): x = 2
│ Row │ x     │ y        │
│     │ Int64 │ Float64? │
├─────┼───────┼──────────┤
│ 1   │ 2     │ 2.0      │
│ 2   │ 2     │ missing  │
│ 3   │ 2     │ 3.0      │

julia> combine(gdf, :y => var)
Unreachable reached at 0x7fad5ee5da96

signal (4): Illegal instruction
in expression starting at REPL[8]:1
Aggregate at /home/bkamins/.julia/packages/DataFrames/4ov0U/src/groupeddataframe/splitapplycombine.jl:1044
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2231 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
_combine at /home/bkamins/.julia/packages/DataFrames/4ov0U/src/groupeddataframe/splitapplycombine.jl:1150
#combine_helper#391 at /home/bkamins/.julia/packages/DataFrames/4ov0U/src/groupeddataframe/splitapplycombine.jl:589
combine_helper##kw at /home/bkamins/.julia/packages/DataFrames/4ov0U/src/groupeddataframe/splitapplycombine.jl:585
unknown function (ip: 0x7fad5ee5d161)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2231 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
#_combine_prepare#379 at /home/bkamins/.julia/packages/DataFrames/4ov0U/src/groupeddataframe/splitapplycombine.jl:553
unknown function (ip: 0x7fad5ee506ff)
_combine_prepare##kw at /home/bkamins/.julia/packages/DataFrames/4ov0U/src/groupeddataframe/splitapplycombine.jl:482 [inlined]
#combine#378 at /home/bkamins/.julia/packages/DataFrames/4ov0U/src/groupeddataframe/splitapplycombine.jl:474 [inlined]
combine at /home/bkamins/.julia/packages/DataFrames/4ov0U/src/groupeddataframe/splitapplycombine.jl:474
unknown function (ip: 0x7fad5ee4e225)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2231 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:117
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:206
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:157 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:566
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:660
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:840
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:790
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:883
eval at ./boot.jl:331
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
eval_user_input at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:134
repl_backend_loop at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:195
start_repl_backend at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:180
#run_repl#37 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:292
run_repl at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2231 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
#806 at ./client.jl:399
jfptr_YY.806_52378.clone_1 at /home/bkamins/julia/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
do_apply at /buildworker/worker/package_linux64/build/src/builtins.c:655
jl_f__apply_latest at /buildworker/worker/package_linux64/build/src/builtins.c:705
#invokelatest#1 at ./essentials.jl:710 [inlined]
invokelatest at ./essentials.jl:709 [inlined]
run_main_repl at ./client.jl:383
exec_options at ./client.jl:313
_start at ./client.jl:506
jfptr__start_43701.clone_1 at /home/bkamins/julia/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
unknown function (ip: 0x401931)
unknown function (ip: 0x401533)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4015d4)
Allocations: 39121077 (Pool: 39109470; Big: 11607); GC: 41
Illegal instruction (core dumped)
@bkamins
Copy link
Member Author

bkamins commented Aug 24, 2020

CC @quinnj @nalimilan

@bkamins bkamins added this to the 1.0 milestone Aug 24, 2020
@bkamins
Copy link
Member Author

bkamins commented Aug 24, 2020

Reported by @alvaroaguirre

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant