Microsoft SQL - Maximum call stack size exceeded

[gehal4]

Describe the bug
There is an exception when using Microsoft SQL node and executing the query (Execute query - operation), when extracting the table with more then 120000 records.
The error message is : Maximum call stack size exceeded

To Reproduce
Steps to reproduce the behavior:

1. Create a workflow
2. Create a node with execute query and extract a table with more then 120000 records ie. 150000

Expected behavior
A clear and concise description of what you expected to happen.

Environment (please complete the following information):

• OS: Windows
• n8n Version: 1.22.6
• Node.js Version: 18.16.0
• Database system
• Operation mode: main

Additional context

[Joffcom]

Hey @gehal4,

Thanks for raising this, I have created NODE-1052 as the internal ticket to get this resolved. For now as a workaround it would be worth seeing if you can load in smaller sets of data.

[netroy]

@gehal4
We just built a docker image n8nio/n8n:fix-microsoft-SQL-max-call-stack-issue with a potential fix.
Unfortunately we can't release custom builds to NPM. Would it be possible for you to test a fix for this on docker?

[gehal4]

Hi,

After making the changes, it works but it gets slow (and sometimes stops working) for 2 reasons :

• The output with larger amount of data >150k (sometimes around >200k) gets stuck (the UI) fails, maybe it would be smarter to limit the display of the output to a certain amount like between 1k-50k max and add a button to press and load more when we open the node, if you want to display more.
• The spinning in the main button "Excecute Workflow" never ends when running datasets with more then 150k records and we never get the Workflow finished message, this is happening when working on editing mode.

In general I think it works great up to 150k, but with larger amount of data I think there are a lot of improvements to work on.

PS: increased --max-old-space-size=8192

[netroy]

it works but it gets slow

We are aware of this issue, and slowly working towards reducing the resource usage of n8n on large amount of data. Unfortunately that won't be fixed by a quick PR. But, hopefully we'll have some improvements around that in this quarter. Until then I'd recommend trying different techniques to split the data up.
PS: This operation is likely a lot slower in a manual execution, and it's possible that production executions are already performing a lot better.