Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test(snowflake): fix snapshots #8919

Merged
merged 4 commits into from
Apr 10, 2024
Merged

test(snowflake): fix snapshots #8919

merged 4 commits into from
Apr 10, 2024

Conversation

cpcloud
Copy link
Member

@cpcloud cpcloud commented Apr 9, 2024 •
edited
Loading

Fixes snapshots.

I also disabled dependency caching except in cases where we're installing all extras: benchmarking and doctests. It doesn't seem like we're getting much from it in other cases except bizarre failures.

@cpcloud cpcloud added this to the 9.0 milestone Apr 9, 2024
@cpcloud cpcloud added tests Issues or PRs related to tests ci Continuous Integration issues or PRs sql Backends that generate SQL snowflake The Snowflake backend labels Apr 9, 2024
@cpcloud cpcloud force-pushed the fix-snowflake-snapshots branch from d50b63c to 7f08bbc Compare April 10, 2024 11:43
@cpcloud cpcloud requested a review from gforsyth April 10, 2024 14:39
@cpcloud cpcloud force-pushed the fix-snowflake-snapshots branch 2 times, most recently from 4db0fbd to da1e5e6 Compare April 10, 2024 18:34
@cpcloud
Copy link
Member Author

cpcloud commented Apr 10, 2024

Something is very odd:

Python 3.11.8 build passes: https://github.com/ibis-project/ibis/actions/runs/8589831494/job/23536526662
Python 3.11.9 build does not pass: https://github.com/ibis-project/ibis/actions/runs/8622089822/job/23632458256

@gforsyth
Copy link
Member

dask/dask#11038

so I'm not sure what to do about this -- we can't upgrade dask at the moment because of the dask-expr rewrite...

@cpcloud
Copy link
Member Author

cpcloud commented Apr 10, 2024 •
edited
Loading

I don't think there's anything to do in this PR, but perhaps we should explore whether dask-expr works, or just do nothing for now?

I can't remember if someone already tried upgrading to the latest dask.

@gforsyth
Copy link
Member

Yeah, I think for now we do nothing on the dask side -- I'll poke at dask-expr and confirm whether it breaks things for us

@gforsyth
Copy link
Member

do you want to clean up the commits and rebase this so we keep the snowflake snapshot fixes separate from the dask and CI workarounds? Or just squash?

@cpcloud
Copy link
Member Author

cpcloud commented Apr 10, 2024

Let's just squash :)

@cpcloud
Copy link
Member Author

cpcloud commented Apr 10, 2024

Lemme run the snowflake tests real quick first though.

@cpcloud
Copy link
Member Author

cpcloud commented Apr 10, 2024

Clouds are good:

…/ibis on  fix-snowflake-snapshots is 📦 v8.0.0 via 🐍 v3.12.2 via ❄️   impure (ibis-3.12.2-env) took 8s
❯ pytest -m snowflake --snapshot-update -n 8 --dist loadgroup -q
bringing up nodes...
xx..xxxxx.xxxxxxxxx.xxxx.....xxx.xxxxxxx.xxxxx..x.x.xx.x..x....x.x.x.x...xxx.xxxxx.x.....xx.xx.x.x..xx.x.....x..xx...x............................x.x..................................x.. [ 11%]
.........x........................................................x.......................x.....x.....x.....................x......xx......................xx.x.x..............x...x...... [ 22%]
x.....x.xxx...x......x.......x...........x.....x........x.....x..........x....xx...........xx.x........x...x..................x........xxx......x..x...x....x.x...x...x.x..x........x...x. [ 33%]
...........x...........x...................xx.....x....x.....x...x........x..x..xxxx..x............x..x...x......xxx.............x...x..xxx.............x.......x........x.x.x.....x...... [ 45%]
.........x..........xxx.x.x................x.....x............................xx.............x...x......x........x......x.......xxxxx.x.xxxx..xx.x..xxxxxxxxxxx..s.x.........x............ [ 56%]
..x..............x...........x.......x..x.....x....x............x..x....x..........xx..............x......x.....xx..........x.....x..................x...................x........x.....x. [ 67%]
..x.......................x.....................................................x.x............x.....x........x.x.........xx.........x...x........x.s.......x.............................. [ 79%]
.........x...............................x...sx.....x..........................ss............................................s............................................................ [ 90%]
.............................................s..............................................................................................Exception ignored in: <socket.socket fd=25, family=2, type=1, proto=6, laddr=('192.168.50.72', 51562), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=25, family=2, type=1, proto=6, laddr=('192.168.50.72', 51562), raddr=('34.102.189.10', 443)>
..Exception ignored in: <socket.socket fd=25, family=2, type=1, proto=6, laddr=('192.168.50.72', 51572), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=25, family=2, type=1, proto=6, laddr=('192.168.50.72', 51572), raddr=('34.102.189.10', 443)>
.Exception ignored in: <socket.socket fd=25, family=2, type=1, proto=6, laddr=('192.168.50.72', 51580), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=25, family=2, type=1, proto=6, laddr=('192.168.50.72', 51580), raddr=('34.102.189.10', 443)>
....Exception ignored in: <socket.socket fd=22, family=2, type=1, proto=6, laddr=('192.168.50.72', 51602), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=22, family=2, type=1, proto=6, laddr=('192.168.50.72', 51602), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=22, family=2, type=1, proto=6, laddr=('192.168.50.72', 51620), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=22, family=2, type=1, proto=6, laddr=('192.168.50.72', 51620), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=21, family=2, type=1, proto=6, laddr=('192.168.50.72', 51614), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=21, family=2, type=1, proto=6, laddr=('192.168.50.72', 51614), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=21, family=2, type=1, proto=6, laddr=('192.168.50.72', 51634), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=21, family=2, type=1, proto=6, laddr=('192.168.50.72', 51634), raddr=('34.102.189.10', 443)>
.Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 51660), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 51660), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 51664), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 51664), raddr=('34.102.189.10', 443)>
..Exception ignored in: <socket.socket fd=24, family=2, type=1, proto=6, laddr=('192.168.50.72', 43650), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=24, family=2, type=1, proto=6, laddr=('192.168.50.72', 43650), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=24, family=2, type=1, proto=6, laddr=('192.168.50.72', 43664), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=24, family=2, type=1, proto=6, laddr=('192.168.50.72', 43664), raddr=('34.102.189.10', 443)>
.Exception ignored in: <socket.socket fd=30, family=2, type=1, proto=6, laddr=('192.168.50.72', 43670), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=30, family=2, type=1, proto=6, laddr=('192.168.50.72', 43670), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=26, family=2, type=1, proto=6, laddr=('192.168.50.72', 43678), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=26, family=2, type=1, proto=6, laddr=('192.168.50.72', 43678), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=26, family=2, type=1, proto=6, laddr=('192.168.50.72', 43686), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=26, family=2, type=1, proto=6, laddr=('192.168.50.72', 43686), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43696), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43696), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43704), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43704), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43718), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43718), raddr=('34.102.189.10', 443)>
..Exception ignored in: <socket.socket fd=27, family=2, type=1, proto=6, laddr=('192.168.50.72', 43734), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=27, family=2, type=1, proto=6, laddr=('192.168.50.72', 43734), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43748), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43748), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43752), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43752), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43762), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43762), raddr=('34.102.189.10', 443)>
Exception ignored in: <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43770), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=6, laddr=('192.168.50.72', 43770), raddr=('34.102.189.10', 443)>
.Exception ignored in: <socket.socket fd=26, family=2, type=1, proto=6, laddr=('192.168.50.72', 43776), raddr=('34.102.189.10', 443)>
Traceback (most recent call last):
  File "/nix/store/3q8rlrnc9p0rlfzpw5zhswfpnyjrbrc5-python3-3.12.2-env/lib/python3.12/site-packages/snowflake/connector/vendored/urllib3/poolmanager.py", line 223, in clear
    self.pools.clear()
ResourceWarning: unclosed <socket.socket fd=26, family=2, type=1, proto=6, laddr=('192.168.50.72', 43776), raddr=('34.102.189.10', 443)>
.                                [100%]
1405 passed, 7 skipped, 232 xfailed in 168.49s (0:02:48)

@cpcloud cpcloud merged commit ab0eb57 into ibis-project:main Apr 10, 2024
86 checks passed
@cpcloud cpcloud deleted the fix-snowflake-snapshots branch April 10, 2024 19:52
@cpcloud cpcloud mentioned this pull request Apr 11, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gforsyth gforsyth gforsyth approved these changes

Assignees
No one assigned
Labels
ci Continuous Integration issues or PRs snowflake The Snowflake backend sql Backends that generate SQL tests Issues or PRs related to tests
Projects
None yet
Milestone
9.0
Development

Successfully merging this pull request may close these issues.
2 participants
@cpcloud @gforsyth