IO Design

NGS Input/Output Design - WIP

Requirements (WIP)

• echo(), log(), warn(), debug(), etc functions should continue working as is (without any changes to code)
  • It is OK to change the implementation and have some underlying complexity to allow the required flexibility
• Support being the backend of Web UI
• Support being the backend (aka kernel) of Jupyter
• Very convenient work with
  • stdin/stdout/stderr
  • files
• Support blocking IO (at least as interface) because it's very convenient in many cases
• Support messages routing considering the following use case: in console, some messages will go to stdout and some to stderr while when functioning as a Jupyter or Web backend the messages will go through same channel/socket.
• Each message should carry rich metadata which can be lost during serialization, when the destination does not support the provided metadata.

Design

• Modeled after Netty, which uses intercepting filters pattern.
  • Types
    • Channel (and appropriate subtypes)
    • ChannelPipeline
    • (TBD)
• Probably: Global IO variable, instance of (TBD, probably Channel), encapsulating both input and output.

TODO

Resources

• Netty Channels
• Java Streams
• ReactiveX
• Reactive Streams
• Go Channels

Open Issues / Maybe TODO

• Have a Buf type?