Add the ability to save Git login temporarily (#1099)

* Adjust git command handlers

- Adjust git push, pull, fetch, clone handlers, allowing `auth` and
`cache_credentials` to be passed in in requests.
- Allow `credential.helper` to pass through `git config` handler's filters.

* Add `credential_helper` to server settings

Allow users to configure the value of `credential_helper` to be used in
the credentials caching logic.

* Implement logic for setting git credential helper

Implemennt methods that ensure `credential.helper` in `git config` to be
used for credentials caching.

* Extend git commands to support credentials caching

Extend git command logics (push, pull, fetch, clone) to support
credentials caching when `auth` is provided.

* Add tests for the new credentials caching logic

Add tests to test the newly-implemented credentials caching logic for
`git push`, `git pull`, `git fetch` and `git clone`.

* Extend `Git.IAuth` and `CredentialsBox`

Extend `Git.IAuth` and `CredentialsBox` for credentials caching. There
is a checkbox for saving Git credentials temporarily.

Stylings are also made for `CredentialsBox`. `jp-RedirectForm` does not
work well with `input` checkboxes.

* Extend `clone()`, `push()`, `pull()` in model

Let each of these methods also include `auth?.cacheCredentials` to POST
to the backend.

* Extend model for `_fetchRemotes()`

Add extra class members to take care of the fetch poll when credentials
are required.

* Implement `fetch()`

Add method `fetch()` that is similar to how `push()` or `pull()` works.

`_fetchRemotes()` now calls `fetch()`, and if credentials are required,
blocks further calls and signals a credentials requirement status change.

* Remove a redundant member

`_fetchBlocked` is not needed.

* Extend `showGitOperationDialog()`

This dialog now supports `fetch` operation, and upon successful
authentication, unblocks the fetch poll by changing the credentials
requirement state to `false`.

* Match the `IGitExtension` with `push()` in model

Also let `showGitOperationDialog()` accept `IGitExtension` `model`
instead of the concrete class `GitExtension`.

* Extend `StatusWidget`

The component now has a boolean for Git credentials requirement.
Also add a callback connected to the model to update that boolean.

* Reuse `showGitOperationDialog()` in `StatusWidget`

Inject the model to the component and reuse `showGitOperationDialog()`
for the `fetch` operation.

* Add a notification dot to `StatusWidget`

Add a notification dot to the component so that it looks like the `push`
(when the local branch is ahead) button and `pull` button (when the
local branch is behind) on the top toolbar in the Git panel.

* Implement `ensure_git_credential_cache_daemon()`

This method spawns a Git credential cache daemon so that the caching
mechanism can work.

* Extend `check/ensure_credential_helper()`

Let these methods check if a cache daemon is required, and invoke
`ensure_git_credential_cache_daemon()` if on Linux, or else respond with
an error message.

* Refactor `push/pull/fetch/clone`

Let them return an error response if the credential helper cannot be
ensured.

* Refactor and adjust tests to adopt new changes

- Use `assert_awaited_` instead of `assert_called_` in some places.
- Use `assert_has_awaits` instead of `assert_has_calls` in some places.
- Use name `mock_execute` instead of `mock_authentication`.
- Mock `sys.platform` as `"linux"`.
- Make sure the `ensure_git_credential_cache_daemon()` is called
accordingly.

* Include `"message"` in `fetch()`'s response

This is so that the front-end can recognize this response with
`IResultWithMessage`.

* Lint files with ESLint

* Adjust `test_handlers.py` to adopt new changes

Add argument `cache_credentials` to assertions.

* Revert `await` back to `call`

This is for compatibility with Python 3.7's `unittest`.

* Adopt PR review suggestions

/pull/1099#pullrequestreview-919970442

* Fix CI

- /pull/1099#pullrequestreview-926143691
- Remove redundant tests
- Adjust tests to match changed method signatures
- Add a missing `self.config()` call when ensuring Git credential
helper

* Reformat `model.ts` with Prettier

Also fix a minor typo.

Co-authored-by: Dat Quach <[email protected]>
Co-authored-by: Zeshan Fayyaz <[email protected]>

jupyterlab_git/__init__.py

@@ -4,7 +4,7 @@
 import json
 from pathlib import Path
 
-from traitlets import List, Dict, Unicode
+from traitlets import List, Dict, Unicode, default
 from traitlets.config import Configurable
 
 from ._version import __version__
@@ -37,6 +37,18 @@ class JupyterLabGit(Configurable):
         # TODO Validate
     )
 
+    credential_helper = Unicode(
+        help="""
+            The value of Git credential helper will be set to this value when the Git credential caching mechanism is activated by this extension.
+            By default it is an in-memory cache of 3600 seconds (1 hour); `cache --timeout=3600`.
+        """,
+        config=True,
+    )
+
+    @default("credential_helper")
+    def _credential_helper_default(self):
+        return "cache --timeout=3600"
+
 
 def _jupyter_server_extension_points():
     return [{"module": "jupyterlab_git"}]

jupyterlab_git/git.py

@@ -8,6 +8,8 @@
 import shlex
 import subprocess
 import traceback
+from typing import Dict, List, Optional
+from unittest.mock import NonCallableMock
 from urllib.parse import unquote
 
 import nbformat
@@ -37,6 +39,8 @@
 GIT_BRANCH_STATUS = re.compile(
     r"^## (?P<branch>([\w\-/]+|HEAD \(no branch\)|No commits yet on \w+))(\.\.\.(?P<remote>[\w\-/]+)( \[(ahead (?P<ahead>\d+))?(, )?(behind (?P<behind>\d+))?\])?)?$"
 )
+# Git cache as a credential helper
+GIT_CREDENTIAL_HELPER_CACHE = re.compile(r"cache\b")
 
 execution_lock = tornado.locks.Lock()
 
@@ -174,9 +178,15 @@ class Git:
     A single parent class containing all of the individual git methods in it.
     """
 
+    _GIT_CREDENTIAL_CACHE_DAEMON_PROCESS: subprocess.Popen = None
+
     def __init__(self, config=None):
         self._config = config
 
+    def __del__(self):
+        if self._GIT_CREDENTIAL_CACHE_DAEMON_PROCESS:
+            self._GIT_CREDENTIAL_CACHE_DAEMON_PROCESS.terminate()
+
     async def config(self, path, **kwargs):
         """Get or set Git options.
 
@@ -273,6 +283,8 @@ async def clone(self, path, repo_url, auth=None):
         """
         env = os.environ.copy()
         if auth:
+            if auth.get("cache_credentials"):
+                await self.ensure_credential_helper(path)
             env["GIT_TERMINAL_PROMPT"] = "1"
             code, output, error = await execute(
                 ["git", "clone", unquote(repo_url), "-q"],
@@ -296,7 +308,7 @@ async def clone(self, path, repo_url, auth=None):
 
         return response
 
-    async def fetch(self, path):
+    async def fetch(self, path, auth=None):
         """
         Execute git fetch command
         """
@@ -308,15 +320,29 @@ async def fetch(self, path):
             "--all",
             "--prune",
         ]  # Run prune by default to help beginners
-
-        code, _, fetch_error = await execute(cmd, cwd=cwd)
+        env = os.environ.copy()
+        if auth:
+            if auth.get("cache_credentials"):
+                await self.ensure_credential_helper(path)
+            env["GIT_TERMINAL_PROMPT"] = "1"
+            code, _, fetch_error = await execute(
+                cmd,
+                cwd=cwd,
+                username=auth["username"],
+                password=auth["password"],
+                env=env,
+            )
+        else:
+            env["GIT_TERMINAL_PROMPT"] = "0"
+            code, _, fetch_error = await execute(cmd, cwd=cwd, env=env)
 
         result = {
             "code": code,
         }
         if code != 0:
             result["command"] = " ".join(cmd)
             result["error"] = fetch_error
+            result["message"] = fetch_error
 
         return result
 
@@ -1005,6 +1031,8 @@ async def pull(self, path, auth=None, cancel_on_conflict=False):
         """
         env = os.environ.copy()
         if auth:
+            if auth.get("cache_credentials"):
+                await self.ensure_credential_helper(path)
             env["GIT_TERMINAL_PROMPT"] = "1"
             code, output, error = await execute(
                 ["git", "pull", "--no-commit"],
@@ -1048,7 +1076,13 @@ async def pull(self, path, auth=None, cancel_on_conflict=False):
         return response
 
     async def push(
-        self, remote, branch, path, auth=None, set_upstream=False, force=False
+        self,
+        remote,
+        branch,
+        path,
+        auth=None,
+        set_upstream=False,
+        force=False,
     ):
         """
         Execute `git push $UPSTREAM $BRANCH`. The choice of upstream and branch is up to the caller.
@@ -1062,6 +1096,8 @@ async def push(
 
         env = os.environ.copy()
         if auth:
+            if auth.get("cache_credentials"):
+                await self.ensure_credential_helper(path)
             env["GIT_TERMINAL_PROMPT"] = "1"
             code, output, error = await execute(
                 command,
@@ -1543,3 +1579,127 @@ async def tag_checkout(self, path, tag):
                 "command": " ".join(command),
                 "message": error,
             }
+
+    async def check_credential_helper(self, path: str) -> Optional[bool]:
+        """
+        Check if the credential helper exists, and whether we need to setup a Git credential cache daemon in case the credential helper is Git credential cache.
+
+        path: str
+            Git path repository
+
+        Return None if the credential helper is not set.
+        Otherwise, return True if we need to setup a Git credential cache daemon, else False.
+
+        Raise an exception if `git config` errored.
+        """
+
+        git_config_response: Dict[str, str] = await self.config(path)
+        if git_config_response["code"] != 0:
+            raise RuntimeError(git_config_response["message"])
+
+        git_config_kv_pairs = git_config_response["options"]
+        has_credential_helper = "credential.helper" in git_config_kv_pairs
+
+        if not has_credential_helper:
+            return None
+
+        if has_credential_helper and GIT_CREDENTIAL_HELPER_CACHE.match(
+            git_config_kv_pairs["credential.helper"].strip()
+        ):
+            return True
+
+        return False
+
+    async def ensure_credential_helper(
+        self, path: str, env: Dict[str, str] = None
+    ) -> None:
+        """
+        Check whether `git config --list` contains `credential.helper`.
+        If it is not set, then it will be set to the value string for `credential.helper`
+        defined in the server settings.
+
+        path: str
+            Git path repository
+        env: Dict[str, str]
+            Environment variables
+        """
+
+        try:
+            has_credential_helper = await self.check_credential_helper(path)
+            if has_credential_helper == False:
+                return
+        except RuntimeError as e:
+            get_logger().error("Error checking credential helper: %s", e, exc_info=True)
+            return
+
+        cache_daemon_required = has_credential_helper == True
+
+        if has_credential_helper is None:
+            credential_helper: str = self._config.credential_helper
+            await self.config(path, **{"credential.helper": credential_helper})
+            if GIT_CREDENTIAL_HELPER_CACHE.match(credential_helper.strip()):
+                cache_daemon_required = True
+
+        # special case: Git credential cache
+        if cache_daemon_required:
+            try:
+                self.ensure_git_credential_cache_daemon(cwd=path, env=env)
+            except Exception as e:
+                get_logger().error(
+                    "Error setting up Git credential cache daemon: %s", e, exc_info=True
+                )
+
+    def ensure_git_credential_cache_daemon(
+        self,
+        socket: Optional[pathlib.Path] = None,
+        debug: bool = False,
+        force: bool = False,
+        cwd: Optional[str] = None,
+        env: Dict[str, str] = None,
+    ) -> None:
+        """
+        Spawn a Git credential cache daemon with the socket file being `socket` if it does not exist.
+        If `debug` is `True`, the daemon will be spawned with `--debug` flag.
+        If `socket` is empty, it is set to `~/.git-credential-cache-daemon`.
+        If `force` is `True`, a daemon will be spawned, and if the daemon process is accessible,
+        the existing daemon process will be terminated before spawning a new one.
+        Otherwise, if `force` is `False`, the PID of the existing daemon process is returned.
+        If the daemon process is not accessible, `-1` is returned.
+        `cwd` and `env` are passed to the process that spawns the daemon.
+        """
+
+        if not socket:
+            socket = pathlib.Path.home() / ".git-credential-cache" / "socket"
+
+        if socket.exists():
+            return
+
+        if self._GIT_CREDENTIAL_CACHE_DAEMON_PROCESS is None or force:
+
+            if force and self._GIT_CREDENTIAL_CACHE_DAEMON_PROCESS:
+                self._GIT_CREDENTIAL_CACHE_DAEMON_PROCESS.terminate()
+
+            if not socket.parent.exists():
+                socket.parent.mkdir(parents=True, exist_ok=True)
+                socket.parent.chmod(0o700)
+
+            args: List[str] = ["git", "credential-cache--daemon"]
+
+            if debug:
+                args.append("--debug")
+
+            args.append(socket)
+
+            self._GIT_CREDENTIAL_CACHE_DAEMON_PROCESS = subprocess.Popen(
+                args,
+                cwd=cwd,
+                env=env,
+            )
+
+            get_logger().debug(
+                "A credential cache daemon has been spawned with PID %d",
+                self._GIT_CREDENTIAL_CACHE_DAEMON_PROCESS.pid,
+            )
+
+        elif self._GIT_CREDENTIAL_CACHE_DAEMON_PROCESS.poll():
+            self.ensure_git_credential_cache_daemon(socket, debug, True, cwd, env)

jupyterlab_git/handlers.py

@@ -67,13 +67,16 @@ async def post(self, path: str = ""):
             {
               'repo_url': 'https://github.com/path/to/myrepo',
               OPTIONAL 'auth': '{ 'username': '<username>',
-                                  'password': '<password>'
+                                  'password': '<password>',
+                                  'cache_credentials': true/false
                                 }'
             }
         """
         data = self.get_json_body()
         response = await self.git.clone(
-            self.url2localpath(path), data["clone_url"], data.get("auth", None)
+            self.url2localpath(path),
+            data["clone_url"],
+            data.get("auth", None),
         )
 
         if response["code"] != 0:
@@ -170,7 +173,11 @@ async def post(self, path: str = ""):
         """
         POST request handler, fetch from remotes.
         """
-        result = await self.git.fetch(self.url2localpath(path))
+        data = self.get_json_body()
+        result = await self.git.fetch(
+            self.url2localpath(path),
+            data.get("auth", None),
+        )
 
         if result["code"] != 0:
             self.set_status(500)