Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ion 1.1 managed writer should flush and reset buffers when a symbol table is manually written #873

Open
tgregg opened this issue Jun 5, 2024 · 1 comment

Comments

@tgregg
Copy link
Contributor

tgregg commented Jun 5, 2024

When the user (or, more likely, a system reader via writeValues) writes a symbol table or IVM, the writer should flush any previously buffered data to the output. This is a performance optimization, and should allow parity with Ion 1.0. Here's some data to support that, produced using the ion-java-benchmark-cli, which writes data using a system reader provided to IonWriter.writeValues.

Ion 1.0:
194 ms / op

Ion 1.1 - all symbols interned
710 ms / op

Ion 1.1 - all symbols inline
234 ms / op

Ion 1.1 - all symbols interned - forced flush every 500 values
175 ms / op

Ion 1.1 - all symbols inline - forced flush every 500 values
229 ms / op

@tgregg
Copy link
Contributor Author

tgregg commented Jun 5, 2024

Update: we should still make sure this is being done, but it may not be the cause of the performance discrepancies noted. The benchmark CLI isn't currently doing a system write during the timed portion of the write benchmark; only during the one-time conversion performed before generating the write instructions or for read benchmarks that provide a different format as input. More investigation needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant