[BUG] `test_hash_groupby_collect_set_on_nested_type` and `test_hash_reduction_collect_set_on_nested_type` failed #8716

thirtiseven · 2023-07-14T03:32:46Z

Describe the bug
test_hash_groupby_collect_set_on_nested_type and test_hash_reduction_collect_set_on_nested_type will failed after this commit: 615156a, which changed the DataGen in integration tests. It also marked the failed cases as XFAIL.

This failure should not be caused by this commit. Before this commit, simply changing the seed (like 4) of the data generator in the test can make them fail.

Steps/Code to reproduce bug

@ignore_order(local=True)
@pytest.mark.parametrize('data_gen', _gen_data_for_collect_set_op, ids=idfn)
def test_hash_groupby_collect_set_on_nested_type(data_gen):
    assert_gpu_and_cpu_are_equal_collect(
        lambda spark: gen_df(spark, data_gen, length=100)
            .groupby('a')
            .agg(f.sort_array(f.collect_set('b'))))

and

@ignore_order(local=True)
@pytest.mark.parametrize('data_gen', _gen_data_for_collect_set_op, ids=idfn)
def test_hash_reduction_collect_set_on_nested_type(data_gen):
    assert_gpu_and_cpu_are_equal_collect(
        lambda spark: gen_df(spark, data_gen, length=100)
            .agg(f.sort_array(f.collect_set('b'))))

got:

FAILED ../../src/main/python/hash_aggregate_test.py::test_hash_groupby_collect_set_on_nested_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]][INJECT_OOM, IGNORE_ORDER({'local': True})]
FAILED ../../src/main/python/hash_aggregate_test.py::test_hash_reduction_collect_set_on_nested_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]][INJECT_OOM, IGNORE_ORDER({'local': True})]

log:

================================================================= FAILURES ==================================================================
_ test_hash_groupby_collect_set_on_nested_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]] _
[gw0] linux -- Python 3.8.3 /home/haoyangl/.pyenv/versions/3.8.3/bin/python

data_gen = [('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long...,['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]

    @ignore_order(local=True)
    @pytest.mark.parametrize('data_gen', _gen_data_for_collect_set_op, ids=idfn)
    # @pytest.mark.xfail(condition=is_before_spark_330(), reason='https://github.com/NVIDIA/spark-rapids/issues/8716')
    def test_hash_groupby_collect_set_on_nested_type(data_gen):
>       assert_gpu_and_cpu_are_equal_collect(
            lambda spark: gen_df(spark, data_gen, length=100)
                .groupby('a')
                .agg(f.sort_array(f.collect_set('b'))))

../../src/main/python/hash_aggregate_test.py:693:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../src/main/python/asserts.py:561: in assert_gpu_and_cpu_are_equal_collect
    _assert_gpu_and_cpu_are_equal(func, 'COLLECT', conf=conf, is_cpu_first=is_cpu_first)
../../src/main/python/asserts.py:492: in _assert_gpu_and_cpu_are_equal
    assert_equal(from_cpu, from_gpu)
../../src/main/python/asserts.py:105: in assert_equal
    _assert_equal(cpu, gpu, float_check=get_float_check(), path=[])
../../src/main/python/asserts.py:41: in _assert_equal
    _assert_equal(cpu[index], gpu[index], float_check, path + [index])
../../src/main/python/asserts.py:34: in _assert_equal
    _assert_equal(cpu[field], gpu[field], float_check, path + [field])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cpu = [Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=...hild7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None)]
gpu = [Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=...hild7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None)]
float_check = <function get_float_check.<locals>.<lambda> at 0x7fcebc7fb280>, path = [3, 'sort_array(collect_set(b), true)']

    def _assert_equal(cpu, gpu, float_check, path):
        t = type(cpu)
        if (t is Row):
            assert len(cpu) == len(gpu), "CPU and GPU row have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))
            if hasattr(cpu, "__fields__") and hasattr(gpu, "__fields__"):
                assert cpu.__fields__ == gpu.__fields__, "CPU and GPU row have different fields at {} CPU: {} GPU: {}".format(path, cpu.__fields__, gpu.__fields__)
                for field in cpu.__fields__:
                    _assert_equal(cpu[field], gpu[field], float_check, path + [field])
            else:
                for index in range(len(cpu)):
                    _assert_equal(cpu[index], gpu[index], float_check, path + [index])
        elif (t is list):
>           assert len(cpu) == len(gpu), "CPU and GPU list have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))
E           AssertionError: CPU and GPU list have different lengths at [3, 'sort_array(collect_set(b), true)'] CPU: 3 GPU: 4

../../src/main/python/asserts.py:39: AssertionError
----------------------------------------------------------- Captured stdout call ------------------------------------------------------------
### CPU RUN ###
### GPU RUN ###
### COLLECT: GPU TOOK 1.3745741844177246 CPU TOOK 3.6840076446533203 ###
CPU OUTPUT: [Row(a=-7540734677356764604, sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None)]), Row(a=-5831592707909023540, sort_array(collect_set(b), true)=[Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None)]), Row(a=-5133656973475552689, sort_array(collect_set(b), true)=[Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None)]), Row(a=-4426181692283497353, sort_array(collect_set(b), true)=[Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None)]), Row(a=-3917032101531217289, sort_array(collect_set(b), true)=[Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None)]), Row(a=-3502159106106506455, sort_array(collect_set(b), true)=[Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None)]), Row(a=-2697073954890740236, sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None)]), Row(a=-2123199122092230623, sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None)]), Row(a=-1, sort_array(collect_set(b), true)=[Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None)]), Row(a=207981845540287738, sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None)]), Row(a=393905103838704542, sort_array(collect_set(b), true)=[Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)]), Row(a=875130347651831881, sort_array(collect_set(b), true)=[Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)]), Row(a=4751953708995107450, sort_array(collect_set(b), true)=[Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)]), Row(a=6084712057446794809, sort_array(collect_set(b), true)=[Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None)]), Row(a=7198729688045931692, sort_array(collect_set(b), true)=[Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None)]), Row(a=7528354001793048440, sort_array(collect_set(b), true)=[Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)]), Row(a=7618709293599214015, sort_array(collect_set(b), true)=[Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None)]), Row(a=7984374766242566542, sort_array(collect_set(b), true)=[Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None)]), Row(a=9223372036854775807, sort_array(collect_set(b), true)=[Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None)])]
GPU OUTPUT: [Row(a=-7540734677356764604, sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None)]), Row(a=-5831592707909023540, sort_array(collect_set(b), true)=[Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None)]), Row(a=-5133656973475552689, sort_array(collect_set(b), true)=[Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None)]), Row(a=-4426181692283497353, sort_array(collect_set(b), true)=[Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None)]), Row(a=-3917032101531217289, sort_array(collect_set(b), true)=[Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None)]), Row(a=-3502159106106506455, sort_array(collect_set(b), true)=[Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None)]), Row(a=-2697073954890740236, sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None)]), Row(a=-2123199122092230623, sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None)]), Row(a=-1, sort_array(collect_set(b), true)=[Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None)]), Row(a=207981845540287738, sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None)]), Row(a=393905103838704542, sort_array(collect_set(b), true)=[Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)]), Row(a=875130347651831881, sort_array(collect_set(b), true)=[Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)]), Row(a=4751953708995107450, sort_array(collect_set(b), true)=[Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)]), Row(a=6084712057446794809, sort_array(collect_set(b), true)=[Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None)]), Row(a=7198729688045931692, sort_array(collect_set(b), true)=[Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None)]), Row(a=7528354001793048440, sort_array(collect_set(b), true)=[Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)]), Row(a=7618709293599214015, sort_array(collect_set(b), true)=[Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None)]), Row(a=7984374766242566542, sort_array(collect_set(b), true)=[Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None)]), Row(a=9223372036854775807, sort_array(collect_set(b), true)=[Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None)])]
_ test_hash_reduction_collect_set_on_nested_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]] _
[gw2] linux -- Python 3.8.3 /home/haoyangl/.pyenv/versions/3.8.3/bin/python

data_gen = [('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long...,['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]

    @ignore_order(local=True)
    @pytest.mark.parametrize('data_gen', _gen_data_for_collect_set_op, ids=idfn)
    # @pytest.mark.xfail(condition=is_before_spark_330(), reason='https://github.com/NVIDIA/spark-rapids/issues/8716')
    def test_hash_reduction_collect_set_on_nested_type(data_gen):
>       assert_gpu_and_cpu_are_equal_collect(
            lambda spark: gen_df(spark, data_gen, length=100)
                .agg(f.sort_array(f.collect_set('b'))))

../../src/main/python/hash_aggregate_test.py:736:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../src/main/python/asserts.py:561: in assert_gpu_and_cpu_are_equal_collect
    _assert_gpu_and_cpu_are_equal(func, 'COLLECT', conf=conf, is_cpu_first=is_cpu_first)
../../src/main/python/asserts.py:492: in _assert_gpu_and_cpu_are_equal
    assert_equal(from_cpu, from_gpu)
../../src/main/python/asserts.py:105: in assert_equal
    _assert_equal(cpu, gpu, float_check=get_float_check(), path=[])
../../src/main/python/asserts.py:41: in _assert_equal
    _assert_equal(cpu[index], gpu[index], float_check, path + [index])
../../src/main/python/asserts.py:34: in _assert_equal
    _assert_equal(cpu[field], gpu[field], float_check, path + [field])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cpu = [Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1...True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None), ...]
gpu = [Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1...True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None), ...]
float_check = <function get_float_check.<locals>.<lambda> at 0x7f617169f3a0>, path = [0, 'sort_array(collect_set(b), true)']

    def _assert_equal(cpu, gpu, float_check, path):
        t = type(cpu)
        if (t is Row):
            assert len(cpu) == len(gpu), "CPU and GPU row have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))
            if hasattr(cpu, "__fields__") and hasattr(gpu, "__fields__"):
                assert cpu.__fields__ == gpu.__fields__, "CPU and GPU row have different fields at {} CPU: {} GPU: {}".format(path, cpu.__fields__, gpu.__fields__)
                for field in cpu.__fields__:
                    _assert_equal(cpu[field], gpu[field], float_check, path + [field])
            else:
                for index in range(len(cpu)):
                    _assert_equal(cpu[index], gpu[index], float_check, path + [index])
        elif (t is list):
>           assert len(cpu) == len(gpu), "CPU and GPU list have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))
E           AssertionError: CPU and GPU list have different lengths at [0, 'sort_array(collect_set(b), true)'] CPU: 14 GPU: 20

../../src/main/python/asserts.py:39: AssertionError
----------------------------------------------------------- Captured stdout call ------------------------------------------------------------
### CPU RUN ###
### GPU RUN ###
### COLLECT: GPU TOOK 1.2127246856689453 CPU TOOK 4.060694456100464 ###
CPU OUTPUT: [Row(sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None), Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None), Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)])]
GPU OUTPUT: [Row(sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None), Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None), Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)])]
- generated xml file: /home/haoyangl/spark-rapids/integration_tests/target/run_dir-20230717100401-K55a/TEST-pytest-1689559441610286928.xml --
========================================================== short test summary info ==========================================================
FAILED ../../src/main/python/hash_aggregate_test.py::test_hash_groupby_collect_set_on_nested_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]][INJECT_OOM, IGNORE_ORDER({'local': True})]
FAILED ../../src/main/python/hash_aggregate_test.py::test_hash_reduction_collect_set_on_nested_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]][INJECT_OOM, IGNORE_ORDER({'local': True})]

Expected behavior
These cases should pass.

Environment details (please complete the following information)
Latest code with spark 311. (I also tested them with spark 330 and they can pass).

The text was updated successfully, but these errors were encountered:

pxLi · 2023-07-18T05:31:14Z

another failure got seen in internal CI JDK17-nightly ID: 62 (spark 330) looks like this is an intermittent error

[2023-07-18T05:16:06.967Z] FAILED ../../src/main/python/hash_aggregate_test.py::test_hash_groupby_collect_set_on_nested_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]][INJECT_OOM, IGNORE_ORDER({'local': True})] - AssertionError: CPU and GPU list have different lengths at [3, 'sort_array(collect_set(b), true)'] CPU: 3 GPU: 4

[2023-07-18T05:16:06.967Z] FAILED ../../src/main/python/hash_aggregate_test.py::test_hash_reduction_collect_set_on_nested_type[[('a', RepeatSeq(Long)), ('b', RepeatSeq(Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])))]][IGNORE_ORDER({'local': True})] - AssertionError: CPU and GPU list have different lengths at [0, 'sort_array(collect_set(b), true)'] CPU: 14 GPU: 20

mismatched output,

[2023-07-18T05:16:06.966Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

[2023-07-18T05:16:06.966Z] 

[2023-07-18T05:16:06.966Z] cpu = [Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1...True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None), ...]

[2023-07-18T05:16:06.966Z] gpu = [Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1...True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None), ...]

[2023-07-18T05:16:06.966Z] float_check = <function get_float_check.<locals>.<lambda> at 0x7f72f587bca0>

[2023-07-18T05:16:06.966Z] path = [0, 'sort_array(collect_set(b), true)']

[2023-07-18T05:16:06.966Z] 

[2023-07-18T05:16:06.966Z]     def _assert_equal(cpu, gpu, float_check, path):

[2023-07-18T05:16:06.966Z]         t = type(cpu)

[2023-07-18T05:16:06.966Z]         if (t is Row):

[2023-07-18T05:16:06.966Z]             assert len(cpu) == len(gpu), "CPU and GPU row have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))

[2023-07-18T05:16:06.966Z]             if hasattr(cpu, "__fields__") and hasattr(gpu, "__fields__"):

[2023-07-18T05:16:06.966Z]                 assert cpu.__fields__ == gpu.__fields__, "CPU and GPU row have different fields at {} CPU: {} GPU: {}".format(path, cpu.__fields__, gpu.__fields__)

[2023-07-18T05:16:06.966Z]                 for field in cpu.__fields__:

[2023-07-18T05:16:06.966Z]                     _assert_equal(cpu[field], gpu[field], float_check, path + [field])

[2023-07-18T05:16:06.966Z]             else:

[2023-07-18T05:16:06.966Z]                 for index in range(len(cpu)):

[2023-07-18T05:16:06.966Z]                     _assert_equal(cpu[index], gpu[index], float_check, path + [index])

[2023-07-18T05:16:06.966Z]         elif (t is list):

[2023-07-18T05:16:06.966Z] >           assert len(cpu) == len(gpu), "CPU and GPU list have different lengths at {} CPU: {} GPU: {}".format(path, len(cpu), len(gpu))

[2023-07-18T05:16:06.966Z] E           AssertionError: CPU and GPU list have different lengths at [0, 'sort_array(collect_set(b), true)'] CPU: 14 GPU: 20

[2023-07-18T05:16:06.966Z] 

[2023-07-18T05:16:06.966Z] ../../src/main/python/asserts.py:39: AssertionError

[2023-07-18T05:16:06.966Z] ----------------------------- Captured stdout call -----------------------------

[2023-07-18T05:16:06.966Z] ### CPU RUN ###

[2023-07-18T05:16:06.966Z] ### GPU RUN ###

[2023-07-18T05:16:06.966Z] ### COLLECT: GPU TOOK 0.19055843353271484 CPU TOOK 0.1242377758026123 ###

[2023-07-18T05:16:06.966Z] CPU OUTPUT: [Row(sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None), Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None), Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)])]

[2023-07-18T05:16:06.967Z] GPU OUTPUT: [Row(sort_array(collect_set(b), true)=[Row(child0=-110, child1=-11291, child2=358420087, child3=894377461322806935, child4=-1.6834864679694056e-31, child5=1.0303888663388996e-26, child6='¯óÏôÅkùê,dÌA~|×AÖ\x08VJé\x08yX^i{.\x1b\x83', child7=False, child8=datetime.date(4899, 3, 2), child9=datetime.datetime(6861, 9, 7, 19, 16, 50, 365146), child10=None), Row(child0=-108, child1=1267, child2=1917713880, child3=None, child4=None, child5=4.3214483342777574e-117, child6='[H\x89µ³hß\x14Æ\x12_XÕµoy\x08Y£àK;uk\x06\x15ç\x18\x02#', child7=True, child8=datetime.date(1590, 12, 27), child9=datetime.datetime(6166, 6, 5, 15, 21, 41, 824645), child10=None), Row(child0=-87, child1=18519, child2=105544272, child3=6446878210185090927, child4=None, child5=-4.2132496663975606e+247, child6='¦\r\x15\x16ep[|p\x9aË\x17ZZû\x82\x86\rê¼¨Ð³AúÍÿ\tÀò', child7=False, child8=datetime.date(3851, 2, 13), child9=datetime.datetime(297, 2, 6, 14, 13, 25, 35309), child10=None), Row(child0=-61, child1=-18968, child2=639916889, child3=2251193521986836658, child4=1.7728260685861134e+25, child5=-1.6653918844116466e+240, child6=' 7jgv#\x1aÑÚªï\x8d\x9fð\x15\x83\x1fîìAæöl\x0bå\\\x90æ9â', child7=True, child8=datetime.date(353, 7, 23), child9=datetime.datetime(551, 6, 24, 14, 12, 58, 187381), child10=None), Row(child0=-50, child1=-31138, child2=-1807864316, child3=-4041992126494898742, child4=4.065499056633676e+31, child5=7.554028173936504e-95, child6='ÆªOo©\x85·2Ôo!áP\x94\x9eþädÚ\x90õ4¢=L\x9däVi}', child7=True, child8=datetime.date(18, 7, 4), child9=datetime.datetime(6109, 8, 17, 22, 8, 3, 694767), child10=None), Row(child0=-41, child1=-24617, child2=848988934, child3=-7858370133784586516, child4=-1.740257146253725e-20, child5=3.5575370470599927e+284, child6='õ^\x1f·\x97ú·ÖF{/ZR*ø\x7fCýö\x05%A1Ð¶d\x04\x89øx', child7=True, child8=datetime.date(7802, 5, 13), child9=datetime.datetime(1279, 10, 18, 15, 49, 5, 515554), child10=None), Row(child0=-36, child1=28459, child2=2096953985, child3=1287002480498025462, child4=963.7640991210938, child5=9.376528689861087e+117, child6='\x05Â,ÀøYªe$«q;~\x04ëâ\x13h\x98ÇR\x04\x1e\x00©4@-\x0b°', child7=False, child8=datetime.date(8000, 2, 29), child9=datetime.datetime(5690, 4, 15, 13, 26, 32, 4190), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=-27, child1=-24753, child2=-1726248225, child3=3307548583936611830, child4=-3.656386885506302e+18, child5=nan, child6='5\x92%w|ð\x9f\x98!\x887DÚdÌ$\x95Xò\xad\x98_ÿ>\x0b¡\tÇ(µ', child7=False, child8=datetime.date(9497, 10, 18), child9=datetime.datetime(8913, 8, 27, 22, 48, 29, 877849), child10=None), Row(child0=1, child1=-23550, child2=823668003, child3=-6580116972850632313, child4=3.816635312018178e+23, child5=8.600719030180411e+134, child6='\x0eM!R\x02ÇB\x1a¡^õ\x8cCÎµ\x1dpØX-Ùrz\x07\x9c¨\x1cÅ¨¡', child7=None, child8=datetime.date(5437, 1, 2), child9=None, child10=None), Row(child0=6, child1=None, child2=-394822741, child3=-6874717541235784212, child4=4243308150784.0, child5=-3.638751059169508e+285, child6='M\x80²ÝÄ\x81¬m]ø\x14å\x05L·\x99®\x8e\x9a\x1a}oÝ\x01à\nãÔ\x84\x08', child7=False, child8=datetime.date(7026, 5, 5), child9=datetime.datetime(860, 3, 3, 14, 32, 46, 946330), child10=None), Row(child0=39, child1=-22177, child2=347245469, child3=5615010007311415123, child4=-0.16667649149894714, child5=-2.1002678349193272e+151, child6="%{³ÁaÃ\x13ò£|dÐ'Y\x04B½u\x99èÈ(!Ú\x13,ÜhÔþ", child7=False, child8=datetime.date(8508, 12, 19), child9=datetime.datetime(7713, 4, 6, 2, 18, 9, 495678), child10=None), Row(child0=61, child1=-25185, child2=-2111287335, child3=-9223372036854775808, child4=None, child5=4.813757617752221e+57, child6=':\x9aLno»7þöµ\x00ý¦\x98vFWÊ\x9eeslr÷t±²ñ\x1eú', child7=True, child8=datetime.date(6996, 5, 13), child9=datetime.datetime(1691, 9, 13, 20, 49, 32, 239246), child10=None), Row(child0=84, child1=22049, child2=897674100, child3=7081246721334085708, child4=-5.445810266389943e+33, child5=-6.542279108216022e+206, child6='çìéô\x1cÀFS¥`\x04\x97ª¶\x14Ó\r¼\xa0¬ôÉe\x8dá~ì:£\x15', child7=False, child8=None, child9=datetime.datetime(2103, 9, 30, 11, 10, 53, 134710), child10=None), Row(child0=125, child1=9497, child2=2139679874, child3=7956328026836954420, child4=-243864059904.0, child5=2.1874890847202007e-206, child6='w\x96\x96\x1d\x8dÏ\x9b\x80\x86Úcy@5øäP\x86Ê\x81\x9coØr)\x8c{rY\x0c', child7=True, child8=datetime.date(3, 10, 7), child9=datetime.datetime(9259, 12, 8, 20, 39, 28, 751490), child10=None)])]

[2023-07-18T05:16:06.967Z] =============================== warnings summary ===============================

[2023-07-18T05:16:06.967Z] src/main/python/hash_aggregate_test.py: 32 warnings

[2023-07-18T05:16:06.967Z]   /spark-3.3.0-bin-hadoop3/python/pyspark/sql/functions.py:315: FutureWarning: Deprecated in 3.2, use sum_distinct instead.

[2023-07-18T05:16:06.967Z]     warnings.warn("Deprecated in 3.2, use sum_distinct instead.", FutureWarning)

[2023-07-18T05:16:06.967Z] 

[2023-07-18T05:16:06.967Z] src/main/python/hive_delimited_text_test.py: 56 warnings

[2023-07-18T05:16:06.967Z]   /spark-3.3.0-bin-hadoop3/python/pyspark/sql/catalog.py:309: FutureWarning: createExternalTable is deprecated since Spark 2.2, please use createTable instead.

[2023-07-18T05:16:06.967Z]     warnings.warn(

[2023-07-18T05:16:06.967Z] 

[2023-07-18T05:16:06.967Z] src/main/python/window_function_test.py::test_lru_cache_datagen[INJECT_OOM]

[2023-07-18T05:16:06.967Z]   /home/jenkins/agent/workspace/jenkins-JDK17-nightly-62/integration_tests/src/main/python/window_function_test.py:1452: UserWarning: Cache info: CacheInfo(hits=2580, misses=792, maxsize=128, currsize=128)

[2023-07-18T05:16:06.967Z]     warnings.warn("Cache info: {}".format(info))

thirtiseven · 2023-07-19T05:49:59Z

I will try to fix this later if no one is working on it.

thirtiseven added bug Something isn't working ? - Needs Triage Need team to review and classify test Only impacts tests labels Jul 14, 2023

thirtiseven mentioned this issue Jul 14, 2023

String generation from complex regex in integration tests #8594

Merged

thirtiseven changed the title ~~[BUG] Two _collect_set_on_nested_type test failed when choosing different seeds.~~ [BUG] test_hash_groupby_collect_set_on_nested_type and test_hash_reduction_collect_set_on_nested_type failed Jul 17, 2023

mattahrens removed the ? - Needs Triage Need team to review and classify label Jul 18, 2023

razajafri mentioned this issue Jul 19, 2023

Fix parquet_write_test.py::test_hive_timestamp_value failure for Databricks [databricks] #8737

Merged

pxLi mentioned this issue Jul 19, 2023

xfail hash_groupby_collect_set and hash_reduction_collect_set on nested type cases [databricks] #8747

Merged

thirtiseven self-assigned this Jul 20, 2023

thirtiseven mentioned this issue Jul 24, 2023

Fix collect_set_on_nested_type tests failed #8783

Merged

jlowe closed this as completed in #8783 Jul 28, 2023

thirtiseven mentioned this issue Sep 18, 2023

[FEA] Enhance integration test to avoid test failures with different seeds #9241

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] `test_hash_groupby_collect_set_on_nested_type` and `test_hash_reduction_collect_set_on_nested_type` failed #8716

[BUG] `test_hash_groupby_collect_set_on_nested_type` and `test_hash_reduction_collect_set_on_nested_type` failed #8716

thirtiseven commented Jul 14, 2023 •

edited

Loading

pxLi commented Jul 18, 2023

thirtiseven commented Jul 19, 2023

[BUG] test_hash_groupby_collect_set_on_nested_type and test_hash_reduction_collect_set_on_nested_type failed #8716

[BUG] test_hash_groupby_collect_set_on_nested_type and test_hash_reduction_collect_set_on_nested_type failed #8716

Comments

thirtiseven commented Jul 14, 2023 • edited Loading

pxLi commented Jul 18, 2023

thirtiseven commented Jul 19, 2023

[BUG] `test_hash_groupby_collect_set_on_nested_type` and `test_hash_reduction_collect_set_on_nested_type` failed #8716

[BUG] `test_hash_groupby_collect_set_on_nested_type` and `test_hash_reduction_collect_set_on_nested_type` failed #8716

thirtiseven commented Jul 14, 2023 •

edited

Loading