Pylance slows down when using the vulkan library

[Jordan-Cottle]

Environment data

• Language Server version: 2024.8.2
• OS and version: win32 x64
• Python version 3.9.7:
• python.analysis.indexing: true
• python.analysis.typeCheckingMode: basic

Code Snippet

from dataclasses import dataclass


@dataclass
class Foo:
    a: int


import vulkan

vulkan.VkValidationCacheCreateInfoEXT()

foo = Foo(a=47)

Libraries installed:

$ .venv/Scripts/python -m pip freeze
cffi==1.17.1
pycparser==2.22  
vulkan==1.3.275.1

https://pypi.org/project/vulkan/

Repro Steps

1. Create a new blank virtual environment: python3 -m venv .venv
2. Install the vulkan library: .venv/Scripts/python.exe -m pip install vulkan
3. Open a new file: code main.py
4. Import vulkan
5. Use auto complete for one of the functions like vulkan.VkAccelerationStructureCreateInfoNV()
6. Try to use other pylance features and observe how slow it is, even for code and completions not related to vulkan

After using this vulkan library, it feels like pylance is chewing on something very large and takes forever to provide assistance that used to happen instantly.

I noticed that this vulkan library has only a few source code files, but the ones that are there are pretty obnoxious. One has 22k lines of code, and another is just 12 lines of code, but one of those lines is a GIANT byte string (on disk, it's almost as big as the 22k line file). And to top it all off, the entire library is imported in the package's __init__.py as from vulkan._vulkan import * # noqa

Expected behavior

I'd expect that pylance could provide analysis promptly like it usually does, perhaps with an initial delay to process the large file once.

Actual behavior

Pylance takes a very long time to provide any intellisense, or to update problems it saw previously. "Go to definition" also takes a very long time.

It takes a long time for previously reported errors to resolve after being fixed for example. I was able to very casually open the snipping app and select this error and still have plenty of time remaining after the screenshot was taken where the message was reported. Re-introducing the error similarly takes a long time to re-recognize the problem.

For a small toy example like I reproduced here, the delay can be something like 30 seconds or so. But in my real project where I have a lot more files and code in my project, and many of those files import vulkan the time between updates and pylance assistance feels significantly longer.

I did notice that if I let things calm down and don't touch any of the code that touches vulkan, I can get normal pylance performance back, but if I touch any of the vulkan code and trigger pylance to re-check it again things slow down again.

If I use the TYPE_CHECKING variable to trick pylance into ignoring things I can mitigate the issue too, but then pylance is obviously upset about everywhere I use the vulkan library and obviously won't help me with it (and honestly, I'd really love to have some intelligence for these functions and constants)

from typing import TYPE_CHECKING

if not TYPE_CHECKING:
    import vulkan

Logs

No trace logs (I can capture them if needed), but I did run a cpu profile to hopefully capture what specifically might be taking a long time

2024-09-04 23:11:15.561 [info] [Info  - 11:11:15 PM] (2184) [fg_id_0] Profiling started
2024-09-04 23:11:15.762 [info] [Info  - 11:11:15 PM] (2184) [bg_1_id_0] Profiling started
2024-09-04 23:11:15.804 [info] [Info  - 11:11:15 PM] (2184) [bg_3_id_0] Profiling started
2024-09-04 23:12:11.268 [info] [Info  - 11:12:11 PM] (2184) [BG(1)] Long operation: checking: file:///c%3A/Users/Jordan/Documents/Programming/Python%20Graphics/SDL2_Vulkan/vulkan_engine/test.py (15750ms)
2024-09-04 23:12:11.268 [info] [Info  - 11:12:11 PM] (2184) [BG(1)] Long operation: analyzing: file:///c%3A/Users/Jordan/Documents/Programming/Python%20Graphics/SDL2_Vulkan/vulkan_engine/test.py (15750ms)
2024-09-04 23:12:36.007 [info] [Info  - 11:12:36 PM] (2184) [BG(1)] Long operation: checking: file:///c%3A/Users/Jordan/Documents/Programming/Python%20Graphics/SDL2_Vulkan/vulkan_engine/test.py (20742ms)
2024-09-04 23:12:36.008 [info] [Info  - 11:12:36 PM] (2184) [BG(1)] Long operation: analyzing: file:///c%3A/Users/Jordan/Documents/Programming/Python%20Graphics/SDL2_Vulkan/vulkan_engine/test.py (20742ms)
2024-09-04 23:12:58.257 [info] [Info  - 11:12:58 PM] (2184) [fg_id_0}] Profiling stopped at c:\Users\Jordan\AppData\Local\Temp\pyright-2184-X0IqoqVQBmiO\pylance_2024.8.2_fg_id_0.cpuprofile
2024-09-04 23:12:58.325 [info] [Info  - 11:12:58 PM] (2184) [bg_1_id_0}] Profiling stopped at c:\Users\Jordan\AppData\Local\Temp\pyright-2184-X0IqoqVQBmiO\pylance_2024.8.2_bg_1_id_0.cpuprofile
2024-09-04 23:12:58.346 [info] [Info  - 11:12:58 PM] (2184) [bg_3_id_0}] Profiling stopped at c:\Users\Jordan\AppData\Local\Temp\pyright-2184-X0IqoqVQBmiO\pylance_2024.8.2_bg_3_id_0.cpuprofile

pyright-2184-X0IqoqVQBmiO.zip

[erictraut]

Thanks for the bug report and the clear repro steps.

It looks like the vulkan library contains a bunch auto-generated code including several very large tuple expressions. One of them includes nearly 10K entries! Pyright (the type checker upon which pylance is built) has some O(n^2) logic when inferring tuple types. That explains why we're seeing such poor performance in this case.

I've added an optimization to pyright to mitigate this issue. It now checks for very large tuples and falls back on a lighter-weight type analysis technique when it encounters very large tuple expressions.

This change will be included in the next release of pyright and in some future release of pylance.

[Jordan-Cottle]

Awesome, thank you for the quick fix! This will take a lot of the pain out of what was supposed to be a fun side project 😄

[StellaHuang95]

This issue has been fixed in prerelease version 2024.9.100, which we've just released. You can find the changelog here: CHANGELOG.md