Nightly feedstock build failed #151

github-actions · 2024-11-16T11:06:06Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

https://app.shortcut.com/tiledb-inc/story/59851

github-actions · 2024-11-17T11:06:22Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-18T11:06:40Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-11-18T16:40:25Z

Starting with the nightly build on Friday, Nov 15th, there is a test failure only for the linux-aarch64 builds:

=========================== short test summary info ============================
FAILED tiledb/tests/test_schema_evolution.py::test_schema_evolution_drop_fixed_attribute_and_add_back_as_var_sized - AssertionError: 
Arrays are not equal

Mismatched elements: 10 / 10 (100%)
 ACTUAL: array([b'', b'', b'', b'', b'', b'', b'', b'', b'', b''], dtype=object)
 DESIRED: array([b'\x80', b'\x80', b'\x80', b'\x80', b'\x80', b'\x80', b'\x80',
       b'\x80', b'\x80', b'\x80'], dtype='|S1')
== 1 failed, 526 passed, 2 skipped, 7 xfailed, 1 xpassed in 766.48s (0:12:46) ==
WARNING: Tests failed for tiledb-py-0.32.5.2024_11_15-py39h697c32b_0.conda - moving package to /home/conda/feedstock_root/build_artifacts/broken

jdblischak · 2024-11-18T16:42:34Z

The failing test was added on Friday in TileDB-Inc/TileDB-Py#2083. @kounelisagis could you please investigate why this new test only fails on linux-aarch64?

kounelisagis · 2024-11-18T16:54:49Z

Strange that this check is OS-dependent. Does this mean that the data in the TileDB array differ depending on the OS, or is it just a representation issue? I believe it’s the latter. In any case, I don’t think this is a deeper issue; it seems to be a matter of modifying the assertion. I will investigate further tomorrow and make a fix in a PR.

cc: @bekadavis9

github-actions · 2024-11-19T11:06:26Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

kounelisagis · 2024-11-19T11:59:16Z

All builds fail because of this: TileDB-Inc/TileDB-Py#2111 (comment)
Expected - working on this.

teo-tsirpanis · 2024-11-19T13:32:20Z

This test gets skipped in TileDB-Py CI. From a quick look the behavior exhibited in linux-aarch64 seems to be the correct one. 🤔

github-actions · 2024-11-20T11:06:34Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-11-20T15:18:30Z

Now we're back to only the linux-aarch64 test failures that originally opened this Issue. Though unclear if the deprecated functions problem has been fixed or not, since we haven't yet merged #152 to always remove the deprecated functions.

github-actions · 2024-11-21T11:19:59Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-22T11:19:49Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-23T11:18:16Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-24T11:17:37Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-25T11:19:42Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-11-25T16:02:03Z

@kounelisagis the linux-aarch64 tiledb-py-feedstock builds continue to fail the test test_schema_evolution.py::test_schema_evolution_drop_fixed_attribute_and_add_back_as_var_sized. Is there an associated story and/or PR for fixing this?

github-actions · 2024-11-26T11:19:49Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-27T11:20:15Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-28T11:06:31Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-29T11:06:30Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-11-30T11:06:11Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-01T11:06:15Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-02T11:06:39Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-03T11:06:32Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-12-03T14:40:30Z

Well last night's tiledb-py-feedstock builds were a complete mess. I'm guessing that my recent PR #154 to explicitly limit the build variants to py39 and py312 played a role. I'll investigate now

jdblischak · 2024-12-03T14:47:14Z

There appears to be multiple issues:

passed:
- linux-64 py39
└─ python 3.13.* *_cpython does not exist (perhaps a typo or a missing channel).
- linux-64 py313
- linux-aarch64 py313
- osx-arm64 py313
FAILED tiledb/tests/test_schema_evolution.py::test_schema_evolution_drop_fixed_attribute_and_add_back_as_var_sized - AssertionError: Arrays are not equal
- linux-aarch64 py39
libmambapy.bindings.specs.ParseError: Found invalid version predicate in "*.2024_12_02"
- osx-64 py313
- osx-64 py39
- osx-arm64 py39
- win-64 py313
- win-64 py39

jdblischak · 2024-12-03T15:12:12Z

The py313-specific solver errors were indeed due to #154. I didn't realize that the official name doesn't end in cpython (like all the previous versions) but instead it is 3.13.* *_cp313. Good news is that this is easy to fix

https://github.com/TileDB-Inc/tiledb-py-feedstock/blob/4af4729bab7a80e46332e5bfbe24a6d4b2d52f98/.ci_support/migrations/python313.yaml#L37

jdblischak · 2024-12-03T15:29:35Z

Also figured out the source of the invalid version predicate error. The passing linux-64 py39 build installed mamba 1.5.1, but all the other builds installed the latest mamba 2.0.4, and mamba 2 doesn't support regex in version strings (mamba-org/mamba#3601 (comment)). And unfortunately I don't see any conda-forge.yml configuration option that controls the version of mamba that is installed for the feedstock builds.

github-actions · 2024-12-04T11:06:43Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-12-04T16:28:04Z

Nightlies are still a mess:

passed:
- linux-64 py39 (must be some solver reason why mamba 2 can't be installed in linux-64 with py39)
nothing provides _numpy_rc needed by numpy-2.0.0rc1-py310...
- linux-64 py313
- linux-aarch64 py313
- osx-arm64 py313
FAILED tiledb/tests/test_schema_evolution.py::test_schema_evolution_drop_fixed_attribute_and_add_back_as_var_sized - AssertionError: Arrays are not equal
- linux-aarch64 py39
libmambapy.bindings.specs.ParseError: Found invalid version predicate in "*.2024_12_02"
- osx-64 py313
- osx-64 py39
- osx-arm64 py39
- win-64 py313
- win-64 py39

github-actions · 2024-12-05T11:06:45Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-12-05T14:14:29Z

After adding back the py312 build (#156), there are at least a reduced number of problems. Current build status:

linux-64: all passing because mamba 1.5 is installed
linux-aarch64: all failing with the known test failure, which should be fixed by Fix/Generalize test added in #2083 TileDB-Py#2116
osx-64, osx-arm64, win-64: all failing because mamba 2 is installed, which doesn't allow leading globs in the version requirement, eg tiledb *.YYYY_MM_DD (Inconsistencies with globs in MatchSpec parsing mamba-org/mamba#3601 (comment)). I need to come up with a workaround

github-actions · 2024-12-06T11:07:00Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-07T11:06:15Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-08T11:06:11Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-09T11:06:47Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-10T11:06:39Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-11T11:06:38Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

github-actions · 2024-12-12T11:06:34Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-12-12T21:07:35Z

Frustratingly the new failures from last night are because conda-build is now using the pin from .ci_support (which is 2.26) because I removed the recipe pin in #157

- package tiledb-2.26.2-hedb9d39_0 is excluded by strict repo priority

github-actions · 2024-12-13T11:06:42Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-12-13T16:59:56Z

Frustratingly the new failures from last night are because conda-build is now using the pin from .ci_support (which is 2.26) because I removed the recipe pin in #157

I fixed this problem this morning with #158

To test it, I manually triggered a tiledb-py job from main, which then pushed to tiledb-py-feedstock (TileDB-Inc/tiledb-py-feedstock@afa8182). The builds are installing the nightly tiledb conda binary 2.27.0.2024_12_12.

github-actions · 2024-12-18T11:06:28Z

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

github-actions · 2024-12-18T11:06:40Z

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak · 2024-12-18T15:28:47Z

Strange. The new tiledb-feedstock failures for osx-64 and win-64 were both download failures. And the centralized nightlies also suffered from download failures (TileDB-Inc/centralized-tiledb-nightlies#30). Is some central service experience a degradation?

jdblischak · 2024-12-18T15:34:06Z

I'm closing this Issue since it has gotten too long and covered too many different errors.

Note that we've started getting passing linux-aarch64 builds only because I skipped the failing test in conda-forge/tiledb-py-feedstock@b820944 in order to release TileDB-Py 0.33.0 (conda-forge/tiledb-py-feedstock#249). The hope is to fix this in a future 0.33.1 release.

github-actions bot added the nightly-failure label Nov 16, 2024

github-actions bot assigned shaunrd0, ihnorton, jdblischak and kounelisagis Nov 16, 2024

jdblischak mentioned this issue Nov 22, 2024

Never remove deprecations when building tiledb-py #153

Merged

kounelisagis mentioned this issue Nov 29, 2024

Fix/Generalize test added in #2083 TileDB-Inc/TileDB-Py#2116

Open

This was referenced Dec 3, 2024

Fix name of conda-forge py313 #155

Merged

Inconsistencies with globs in MatchSpec parsing mamba-org/mamba#3601

Open

jdblischak mentioned this issue Dec 4, 2024

Allow py313 to be a third Python build #156

Merged

jdblischak mentioned this issue Dec 11, 2024

Install latest tiledb from tiledb/label/nightlies (support mamba 2) #157

Merged

This was referenced Dec 13, 2024

ENH: update package version to 0.33.0 conda-forge/tiledb-py-feedstock#249

Merged

Specify latest tiledb using the pin 2.* #158

Merged

jdblischak closed this as completed Dec 18, 2024

Nightly feedstock build failed #151

Nightly feedstock build failed #151

Comments

github-actions bot commented Nov 16, 2024 • edited by ihnorton Loading

github-actions bot commented Nov 17, 2024

github-actions bot commented Nov 18, 2024

jdblischak commented Nov 18, 2024

jdblischak commented Nov 18, 2024

kounelisagis commented Nov 18, 2024

github-actions bot commented Nov 19, 2024

kounelisagis commented Nov 19, 2024

teo-tsirpanis commented Nov 19, 2024

github-actions bot commented Nov 20, 2024

jdblischak commented Nov 20, 2024

github-actions bot commented Nov 21, 2024

github-actions bot commented Nov 22, 2024

github-actions bot commented Nov 23, 2024

github-actions bot commented Nov 24, 2024

github-actions bot commented Nov 25, 2024

jdblischak commented Nov 25, 2024

github-actions bot commented Nov 26, 2024

github-actions bot commented Nov 27, 2024

github-actions bot commented Nov 28, 2024

github-actions bot commented Nov 29, 2024

github-actions bot commented Nov 30, 2024

github-actions bot commented Dec 1, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 3, 2024

jdblischak commented Dec 3, 2024

jdblischak commented Dec 3, 2024

jdblischak commented Dec 3, 2024

jdblischak commented Dec 3, 2024

github-actions bot commented Dec 4, 2024

jdblischak commented Dec 4, 2024

github-actions bot commented Dec 5, 2024

jdblischak commented Dec 5, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 7, 2024

github-actions bot commented Dec 8, 2024

github-actions bot commented Dec 9, 2024

github-actions bot commented Dec 10, 2024

github-actions bot commented Dec 11, 2024

github-actions bot commented Dec 12, 2024

jdblischak commented Dec 12, 2024

github-actions bot commented Dec 13, 2024

jdblischak commented Dec 13, 2024

github-actions bot commented Dec 18, 2024

github-actions bot commented Dec 18, 2024

jdblischak commented Dec 18, 2024

jdblischak commented Dec 18, 2024

github-actions bot commented Nov 16, 2024 •

edited by ihnorton

Loading