Can't start manticore after update from 5.0.2 to 6.3.8, OOM error

[dzek69]

Bug Description:

I use docker images, I used 5.0.2 and decided to upgrade. After upgrading the image to 6.3.8 Manticore crashes with OOM error:
FATAL: out of memory (unable to allocate 14401991088 bytes)

I have 8GB of RAM, Manticore tries to allocate ~14GB in memory (during loading not-so-big table, I have much bigger tables that used to work just fine).

I reported this issue first on Telegram: https://t.me/manticoresearch_en/1678 and apparently if you have enough RAM Manticore won't crash, but won't finish starting anyway.

I uploaded my tables with issue-20241201 name.

Sergey on Telegram suggested this seems to be a data backward compatibility issue.

Manticore Search Version:

daemon version '6.3.8 d17bd2b6b@24112202 (columnar 2.3.0 88a01c3@24052206) (secondary 2.3.0 88a01c3@24052206) (knn 2.3.0 88a01c3@24052206)'

Operating System Version:

Debian GNU/Linux 12 (bookworm)

Have you tried the latest development version?

No

Internal Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

• Implementation completed
• Tests developed
• Documentation updated
• Documentation reviewed
• Changelog updated