Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Request] Long-running API operations should return Accepted response and notify by WebSocket #286

Closed
Tracked by #550
andrewazores opened this issue Feb 14, 2024 · 1 comment · Fixed by #698
Closed
Tracked by #550
Assignees
Labels
feat New feature or request

Comments

@andrewazores
Copy link
Member

Describe the feature

API operations like archiving active recordings, generating reports (from active or archived recordings), "View in Grafana" (transfer file to jfr-datasource) etc. can take a long time to complete depending on external factors such as the size of a JFR file, the speed of network connections between Cryostat and target applications or report generator or jfr-datasources, etc. Currently these API actions are implemented such that the client sends a request to Cryostat, then Cryostat performs the action completely, and then Cryostat sends a response to the client based on the result of the performed action. Instead, Cryostat should receive the client's request, perform some initial validations, and send either a 4xx response if the client's request failed the validations, or a 202 response if it will proceed. Then, the final result of the action should be emitted as a WebSocket notification.

Taking active recording archiving as an example, currently the flow is like this:

  1. Client sends a PATCH request to Cryostat
  2. Cryostat receives the PATCH and creates an S3 file upload request
  3. Cryostat opens a connection (JMX or Agent HTTP) to the target JVM and begins to receive JFR bytes
  4. Cryostat pipes the target JFR bytes to the S3 upload
  5. Cryostat finishes reading the JFR bytes from the target and finalizes the S3 upload
  6. Cryostat responds to the client's PATCH with some information about the uploaded file

Instead, it should go like this:

  1. Client sends a PATCH request to Cryostat
  2. Cryostat receives the PATCH and checks that the specified active recording exists and that the target JVM is reachable (ex. try to open a connection and do something relatively lightweight like compute its JVM ID). If this check succeeds respond to the PATCH with 202, if it fails respond with a 404 (recording not found) or 502 (target not reachable) etc.
  3. In the background, Cryostat creates the S3 file upload request, opens a target connection, pipes the bytes, etc. - same as steps 2-5 above
  4. Cryostat emits a WebSocket notification, either indicating task successful completion or task failure.

Anything other information?

No response

@andrewazores andrewazores added feat New feature or request needs-triage Needs thorough attention from code reviewers labels Feb 14, 2024
@andrewazores andrewazores moved this to Todo in 3.0.0 release Feb 14, 2024
@andrewazores andrewazores moved this from Todo to Stretch Goals in 3.0.0 release Feb 20, 2024
@Josh-Matsuoka Josh-Matsuoka self-assigned this Jun 5, 2024
@andrewazores andrewazores moved this to Backlog in 4.0.0 release Jun 7, 2024
@andrewazores andrewazores removed the needs-triage Needs thorough attention from code reviewers label Jul 5, 2024
@andrewazores andrewazores mentioned this issue Jul 5, 2024
6 tasks
@andrewazores
Copy link
Member Author

I had some thoughts swimming in my brain last night while trying to fall asleep.

  1. Client sends a PATCH request to Cryostat
  2. Cryostat receives the PATCH and checks that the specified active recording exists and that the target JVM is reachable (ex. try to open a connection and do something relatively lightweight like compute its JVM ID). If this check succeeds respond to the PATCH with 202, if it fails respond with a 404 (recording not found) or 502 (target not reachable) etc. An OK response (ie. HTTP 202) should include some job identifier, for example a UUID generated in this request handler.
  3. In the background, Cryostat creates the S3 file upload request, opens a target connection, pipes the bytes, etc. - same as steps 2-5 above. This background request should not be kicked off (job added to work queue, or worker thread kicked off, etc.) until the HTTP response from the previous step has been fully written out to and received by the client. This ensures that the client knows the ID of the job before the job can possibly have been completed.
  4. Cryostat emits a WebSocket notification, either indicating task successful completion or task failure, including the job ID in the message.
  5. The client has been listening for a message of this type with this ID, receives it, and acts upon it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat New feature or request
Projects
Status: Done
Status: Stretch Goals
Development

Successfully merging a pull request may close this issue.

2 participants