List size is capped by browser/memory

[sveta-slepner]

Describe the bug

When trying to create a list with a large count of rows, one of two things happens:

A. If the count is over an arbitrary number, capped by each browser individually, the scroll won't go over that row.
For instance, for Chrome that number is 479348, and you can't scroll to the remaining items if there are any.

B. If the number is too large (for instance 150 mil), the tab will simply crash.
I believe this is caused by the getMeasurements function from the virtual-core library, as it tries to create an array of the count size.

Your minimal, reproducible example

https://codesandbox.io/p/sandbox/silent-wildflower-594ycm

Steps to reproduce

1. In the sandbox click on one of the example buttons that will set a larger number, or edit the count property manually.

Expected behavior

A. I expect the scroll to allow the user to reach the exact number specified in the count.
B. I expect the browser not to crash .

How often does this bug happen?

Every time

Screenshots or Videos

No response

Platform

• Any but specifically I'm using:
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36

tanstack-virtual version

v3.0.0-beta.54

TypeScript version

No response

Additional context

No response

Terms & Code of Conduct

• I agree to follow this project's Code of Conduct
• I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

[ddolcimascolo]

Hi all,

@tannerlinsley I just started using the library (the React binding, specifically) to implement an infinite list component and it works pretty well. I really like the approach of providing the basic feature of virtualizing a large list and letting us implement the rendering, fetching, etc.

I experienced this issue, and enabling debug: true show that all the time is spent is getMeasurements as @sveta-slepner mentioned, I have timings around 100ms in this method, and the scrolling is laggy at best :) I worked around this problem by providing a smaller count to the virtualizer, increasing it as more pages are fetched. Scrolling is fast even with thousands of items loaded.

Anything I can do to help fix that?

As a side note, when are you planning to release a stable v3?

Thanks,
David

[tannerlinsley]

I'll take a look at this asap.

[knownasilya]

Cross posting from discord.

I think the virtualizer holds on to dom nodes. Example app: https://stackblitz.com/edit/sb1-41eywcye?file=index.html
I changed pages from home to project a few times and this is the result, I'd expect the number of nodes to go down. In my own app I see this but it keeps growing even after 1gb of memory.

My app crashes pretty fast in Safari v17 (v18 is more resilient it seems)

[piecyk]

I think the virtualizer holds on to dom nodes.

that is true, we store it via measureElement https://github.com/TanStack/virtual/blob/main/packages/virtual-core/src/index.ts#L753

and this part clears it via isConnected https://github.com/TanStack/virtual/blob/main/packages/virtual-core/src/index.ts#L793-L801

[knownasilya]

@piecyk hum, I don't see any cleanup on unmount though. Am I missing it or does it not happen?
I tried manually clearing the elements cache on unmount, but it was already empty, so probably not the culprit.

This line https://github.com/TanStack/virtual/blob/main/packages/virtual-core/src/index.ts#L101 could probably be observer.disconnect()

[piecyk]

@knownasilya yeah there is any, the idea was that they will be cleared by measureElement, but let's also clear it in cleanup.

This line https://github.com/TanStack/virtual/blob/main/packages/virtual-core/src/index.ts#L101 could probably be observer.disconnect()

could be, but there should not be any difference because we always observe one rect element

[piecyk]

I think the virtualizer holds on to dom nodes. Example app: https://stackblitz.com/edit/sb1-41eywcye?file=index.html

@knownasilya example not found?

[knownasilya]

@piecyk I made it public, should be good now

[piecyk]

@piecyk I made it public, should be good now

Thanks, from quick look noticed that there is pretty old version of @tanstack/react-virtual, please update to latest, also clicked around and didn't noticed memory leaking.

[knownasilya]

I updated the version and now I can't reproduce the leak, so maybe we're good? Bad news for me, probably means there is another issue in my app.

[knownasilya]

Might be a react 17 vs 18.. facebook/react#17666

[geferon]

I updated the version and now I can't reproduce the leak, so maybe we're good? Bad news for me, probably means there is another issue in my app.

I'm having this exact same issue in Vue as well, so I'm not so sure if it's a framework specific issue?

[geferon]

The spikes going down is me changing pages, removing and re-adding the component, and this is in Vue.