Why use json-stream-stringify instead of JSONStream for stringification?

[ackinc]

I'm seeing a huge difference in execution time for stringifying a large, nested JSON array (length 50k, 2GB in memory, 700MB on disk)

The following JSONStream implementation takes ~20secs

function jsonStreamWrite(filepath, records) {
  return new Promise(resolve => {
    const writeStream = fs.createWriteStream(filepath);

    const jsonStringifyStream = JSONStream.stringify();
    jsonStringifyStream.on('end', resolve);

    stream.Readable.from(records).pipe(jsonStringifyStream).pipe(writeStream)
  })
}

... while big-json takes ~3.5 hours (!)

function bigJsonWrite(filepath, records) {
  return new Promise(resolve => {
    const writeStream = fs.createWriteStream(filepath);

    const stringifyStream = bigJson.createStringifyStream({body: records});
    stringifyStream.on('end', resolve);

    stringifyStream.pipe(writeStream);
  })
}

[DonutEspresso]

IIRC - JSONStream still uses JSON.stringify under the hood for key/val pairs, so you could conceivably still run into the V8 limit.

If you are fairly confident about your object sizes, then I would recommend folks use JSONStream directly. This module exists to do the thing which isn't otherwise possible via existing solutions, though I agree the performance penalty is quite egregious here. If you are aware of other more performant stringification libraries that don't depend on JSON.stringify, happy to explore further.