Add batch command-context manager on top of run-context manager #532
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
christian-monch
force-pushed
the
minimal-run-batch-context
branch
from
f0f5f50
to
0c7da95
Compare
Co-authored-by: Michael Hanke <[email protected]> This commit adds a `batchcommand`-context-manager that keeps track of a subprocess and supports batch-style communication. That means, it allows to send a request to the subprocess and will return the response from the subprocess. How the output of the subprocess has to be interpreted to yield a response is defined by the protocol that is provided to the batchcommand-context-manager. The batchcommand-context-manager uses the run-context-manager to ensure that the exit-handler waits for process termination. The commit also contains three convenience functions that provide context-manager for common interaction patterns (common in `datalad` and `datalad_next` anyways): 1. `stdout_batchcommand`: this context-manager writes to `stdin` of the subprocess and returns the next, yet unread, chunk of bytes that was written to `stdout` of the subprocess. 2. `stdoutline_batchcommand`: this context-manager writes to `stdin` of the subprocess and returns the next, yet unread, line that was written to `stdout` of the subprocess. Line-separators and whether they are included in the output can be chosen when creating the context-manager. 3. `annexjson_batchcommand`: this context-mananger writes to `stdin` of the subprocess and returns a JSON-object, that is created from the next, yet unread, line that is written to `stdout` of the subprocess. This context-mamanger is geared towards git-annex batch commands with JSON-result output, i.e. git-annex invocations with the parameters `--batch`, `--json`, `--json-error-messages`. The tests provide an example for protocol that handles output that varies in size, i.e. output of python-programs. The class that implements the protocol is called `PythonProtocol`.
christian-monch
force-pushed
the
minimal-batch
branch
from
5aea03b
to
98ed48f
Compare
|
With #538 merged, we are betting on a new horse for subprocess execution. I believe this PR can be closed and/or reposted in rejuvenated form. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is a Draft-PR because it is against another PR.
This PR adds a
batchcommand-context manager. Abatchcommand-context manager starts a subprocess and supports batch-style communication with it. That means, the user can send a request to the subprocess and receive the response from the subprocess.How bytes that are read from the subprocess are assembled into a response is determined by the protocol that is used when the
batchcommand-context manager is created. Typical responses would be single lines of text or JSON-encoded data.The PR contains three convenience functions that provide context-manager for common interaction patterns (common in
dataladanddatalad_nextanyways):stdout_batchcommand: this context-manager writes tostdinof the subprocess and returns the next, yet unread, chunk of bytes that was written tostdoutof the subprocess.stdoutline_batchcommand: this context-manager writes tostdinof the subprocess and returns the next, yet unread, line that was written tostdoutof the subprocess. Line-separators and whether they are included in the output can be chosen when creating the context-manager.annexjson_batchcommand: this context-mananger writes tostdinof the subprocess and returns a JSON-object, that is created from the next, yet unread, line that is written tostdoutof the subprocess. This context-mananger is geared towards git-annex batch commands with JSON-result output, i.e. git-annex invocations with the parameters--batch,--json, or--json-error-messages.The tests contain an example for a protocol,
PythonProtocolthat handles output from an interactive python interpreter, which varies in size.