Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching for improvements in kernel startup #6702

Closed
DonJayamanne opened this issue Jul 16, 2021 · 4 comments
Closed

Caching for improvements in kernel startup #6702

DonJayamanne opened this issue Jul 16, 2021 · 4 comments
Assignees
Labels
bug Issue identified by VS Code Team member as probable bug *duplicate Issue identified as a duplicate of another issue(s) notebook-execution Kernels issues (start/restart/switch/execution, install ipykernel) perf Performance issues verified Verification succeeded
Milestone

Comments

@DonJayamanne
Copy link
Contributor

Today Jupyter extension calls API in VS Code to determine:

  • If IPyKernel is installed
  • Determine if the version of IPYKernel satisfies some requirements (>=6.0)
    • Used in debugging
  • etc

Suggestion: Cache in Python extension

  • When getting the version, we use the code import xyz; print(xyz.__version__)
    • I'm suggesting we also get the __file__
  • From the file we can determine the directory where this module is installed, get the last modified date/time of that & cache it
  • When we calling this code in the same VS Code session, we can check if that folder has been modified, if not, then return the cached version (no we avoid running a whole process all over, just a simple disc IO)

Optionally, we can ignore the cache when restarting VS Code (i.e. the cache is only per-session).
This way, if something doesn't work properly, all we need to do is ask the user to restart VS Code (this is a worst case scenario).

Suggestion: Cache in Python extension

  • Optionally we could perform this same algorithm in the Jupyter extension. Instead of checking the last modified date of the individual package folder, we could check the last modified date of the Python environment folder.
  • If user installs/updates/uninstalls packages in a python environment, then the python environment folders modified date changes, hence we can invalidate the cache easily at our end.
  • Ie.. cache key = Interpreter Path + Last Modifeid date of interpreter folder

Caching in Jupyter extension is super easy & safe, with great benefits for us.

@rchiodo @DavidKutu @joyceerhl /cc

@greazer greazer added the notebook-execution Kernels issues (start/restart/switch/execution, install ipykernel) label Aug 4, 2021
@greazer greazer added perf Performance issues bug Issue identified by VS Code Team member as probable bug labels Oct 9, 2021
@DonJayamanne DonJayamanne added this to the October 2021 milestone Oct 12, 2021
@rchiodo rchiodo modified the milestones: October 2021, November 2021 Oct 27, 2021
@DonJayamanne
Copy link
Contributor Author

Solution Implemented:

  • If we have detected ipykernel for an interprerter once before, avoid looking for this before we start kernels.
  • If user subsequently uninstalls ipykernel, then we'll detect this when kernel fails to start.

@DonJayamanne DonJayamanne self-assigned this Nov 24, 2021
@DonJayamanne
Copy link
Contributor Author

Duplicate of #8196

@DonJayamanne DonJayamanne marked this as a duplicate of #8196 Nov 24, 2021
@DonJayamanne
Copy link
Contributor Author

DonJayamanne commented Dec 1, 2021

Testing

  • Open VS Code stable, create a notebook, run a cell with print statement
  • Keep the notebook opened.
  • Reload VS Code and run the cell immediately.
  • Check the logs in Jupyter output panel to determine the time from activation to time take to run the cell.

Try the same in VS Code insiders, (remember to reload)

  • The times should be faster in VS Code insiders.

Note:

  • Things are faster only when running a cell for an existing notebook that you ran previously.
  • You might want to try a conda environment (on windows) as they are slower (Mac/Linux is faster & the time difference isn't as noticeable)
  • Optionally use a conda enviornment used for the Golden Scneraio (https://github.com/microsoft/vscode-jupyter-internal/issues/200)

@rchiodo
Copy link
Contributor

rchiodo commented Dec 1, 2021

/verified

@rchiodo rchiodo added the verified Verification succeeded label Dec 1, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2023
@DonJayamanne DonJayamanne added the *duplicate Issue identified as a duplicate of another issue(s) label Nov 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Issue identified by VS Code Team member as probable bug *duplicate Issue identified as a duplicate of another issue(s) notebook-execution Kernels issues (start/restart/switch/execution, install ipykernel) perf Performance issues verified Verification succeeded
Projects
None yet
Development

No branches or pull requests

3 participants