perf(session): Improve list response time #2160
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This makes it so jobs will all be naturally grouped together in file listing.
This will delete sessions that have been in the termianted state for longer than the given threshold.
There is a new periodic job that will delete terminated sessions that are older than one hour. However, since until now terminated sessions were not deleted, the first time the job runs there could be a large number of sessions that need deleted. This could have a negative impact on a running cluster. To mitigate the operational risk, this adds a migration to perform an initial delete while the cluster is offline.
Including the nested connection information in the session list response had a negative impact on performance as it created a significant multiplier on the number of rows that needed to be fetched from the database. It was also determined that this information was used by neither the admin ui nor the desktop client. Note that nested connection information can still be retrieved via a read request for a specific session.
It was determined that it is a more common use case for users to request non-terminated sessions (i.e. sessions in `pending`, `active`, or `canceling` when requesting sessions. Notably, end users would need non-terminated sessions to: - Establish a new connection to a session. - Terminate a session So now by default a list request to `/v1/sessions` will *not* include terminated sessions. To still enable use cases where terminated sessions are needed, this adds a new query parameter to include the terminated session: `/v1/sessions?include_terminated=true`. Note that this query parameter is considered experimental and subject to change.
github-actions
bot
added
api
api/sessions
core
core/db
core/gen
core/proto
core/session
core/sql
core/ui
labels
johanbrandhorst
previously approved these changes
Jun 9, 2022
johanbrandhorst
requested changes
Jun 9, 2022
johanbrandhorst
approved these changes
Jun 9, 2022
tmessi
added a commit
that referenced
this pull request
Jun 5, 2023
As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941
tmessi
added a commit
that referenced
this pull request
Jun 5, 2023
As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941
tmessi
added a commit
that referenced
this pull request
Jun 5, 2023
As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941
tmessi
added a commit
that referenced
this pull request
Jun 6, 2023
As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941
tmessi
added a commit
that referenced
this pull request
Jun 6, 2023
As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941
tmessi
added a commit
that referenced
this pull request
Jun 6, 2023
As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941
tmessi
added a commit
that referenced
this pull request
Jun 6, 2023
…ns (#3288) As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941
tmessi
added a commit
that referenced
this pull request
Jun 6, 2023
…ns (#3288) As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941 (cherry picked from commit 7aef341)
tmessi
added a commit
that referenced
this pull request
Jun 6, 2023
As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941
tmessi
added a commit
that referenced
this pull request
Jun 6, 2023
…ns (#3288) As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: #2160 Ref: 3207067 Fixes: 1e3c941 (cherry picked from commit 7aef341)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Notable changes include:
include_terminated=truequery parameter to return terminatedsessions.
returned for a read of an individual session.