Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug which caused VCLLVM to get stuck #1149

Merged
merged 1 commit into from
Feb 7, 2024

Conversation

superaxander
Copy link
Member

When processing a large LLVM-IR file we get stuck because VCLLVM is waiting for someone to read its output stream because the buffer has filled up. Instead of first waiting for the program to finish we can read the output into a ByteArrayOutputStream while its running.

@pieter-bos
Copy link
Member

Ah, nice catch! Shall we use a similar approach to this? That is: do the transfer action in a different thread started prior to the process wait. That way the data is not duplicated in memory first.

@superaxander
Copy link
Member Author

I don't think it's easy to avoid duplicating in memory here. In this case we need to end up with a buffer containing all the bytes which will be the array underlying the ByteArrayOutputStream. An InputStream#transferTo call would use a similar buffer and actually this code can be simplified a bit more by calling input.transferTo(baos) in the loop. The CodedInputStream does use another buffer because it will be reading from the baos but that's relatively small.

Of course we can run just call Program.parseFrom(process.getOutputStream) and catch any exceptions that throws to turn that into an LLVMParseError and read the error stream instead of waiting for the process to finish ourselves. In that case we don't even need another thread.

@pieter-bos pieter-bos merged commit 4304099 into utwente-fmt:dev Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants