GitHub PAT error with https protocol: detects user GH password instead of the token

[lcolladotor]

Hi,

Louise @lahuuki was setting up a new R package with usethis and we encountered a confusing error message that is really a gh error derived from gitcreds output.

## Code
usethis::use_github()

## Error + traceback
Error in validate_gh_pat(new_gh_pat(x)) : 
  A GitHub PAT must consist of 40 hexadecimal digits
> traceback()
10: stop(cond)
9: throw(new_error("A GitHub PAT must consist of 40 hexadecimal digits"))
8: validate_gh_pat(new_gh_pat(x))
7: gh_pat(token$password %||% "")
6: gh_token(.api_url)
5: .token %||% gh_token(.api_url)
4: gh::gh_whoami(.api_url = host)
3: withCallingHandlers(expr, message = function(c) if (inherits(c, 
       classes)) tryInvokeRestart("muffleMessage"))
2: suppressMessages(gh::gh_whoami(.api_url = host))
1: usethis::use_github()

This issue gets resolved if you run (or edit your ~/.Rprofile) with:

## For usethis::use_git()
options(usethis.protocol = "ssh")

which I had in my macOS but Louise didn't in her macOS setup.

Without those lines of code, Louise gets her GitHub password which is not 40 characters long.

## Working through the internal code step by step in Louise's setup
> gh::gh_token(NULL)
Error in validate_gh_pat(new_gh_pat(x)) : 
  A GitHub PAT must consist of 40 hexadecimal digits
> gh:::default_api_url()
[1] "https://api.github.com"
> gh:::get_hosturl(gh:::default_api_url())
[1] "https://github.com"
> gitcreds::gitcreds_get(gh:::get_hosturl(gh:::default_api_url()))
<gitcreds>
  protocol: https
  host    : github.com
  username: lahuuki
  password: <-- hidden -->
> token <- gitcreds::gitcreds_get(gh:::get_hosturl(gh:::default_api_url()))
> token$password

while I indeed get my 40 characters long GitHub personal access token (PAT).

## Working through the internal code step by step in my setup
> gh::gh_token(NULL)
(a truncated token, manually edited by me)
> gh:::default_api_url()
[1] "https://api.github.com"
> gh:::get_hosturl(gh:::default_api_url())
[1] "https://github.com"
> gitcreds::gitcreds_get(gh:::get_hosturl(gh:::default_api_url()))
<gitcreds>
  protocol: NA
  host    : NA
  username: NA
  password: <-- hidden -->
> token <- gitcreds::gitcreds_get(gh:::get_hosturl(gh:::default_api_url()))
> token$password
[1] "(I get a 40 character token here)"
> nchar(token$password)
[1] 40

The help file at https://usethis.r-lib.org/reference/git_protocol.html doesn't seem to indicate that using a https protocol would be a problem. Looking at our ~/.Rprofile and ~/.gitconfig nothing seems to jump at me. Both of us have a GITHUB_PAT="40 characters long token" entry in our ~/.Renviron.

So, from the docs it seems to me that https is supported. I'm not sure if some specific configuration in Louise's setup is leading to the error, or if it's a change in the expected output from gitcreds inside the gh code. In any case, I'm reporting the issue here such that other usethis users can locate the error and solution more quickly that we did.

Best,
Leo

Session info:

> packageVersion("gh")
[1] ‘1.2.0’
> packageVersion("gitcreds")
[1] ‘0.1.1’
> packageVersion("usethis")
[1] ‘2.0.0’

[jennybc]

This is the best set of instructions re: GitHub setup:

https://usethis.r-lib.org/articles/articles/git-credentials.html

There's a lot to read above, but upon skimming, it sound like at some point maybe someone was prompted for PAT = "personal access token", but provided a password instead? Hard for me to guess.

GitHub is shutting down HTTPS access with a username+password, so anyone using HTTPS, which I recommend, absolutely needs to get a PAT and start using it anyway.

[lcolladotor]

Hi,

Looking at this again, I think that r-lib/gitcreds#16 might have been related. There might be a git credential saved on her macOS Keychain Access with the incorrect info (password instead of PAT), which makes the issue hard to reproduce for me. Or maybe gitcreds is using some info Louise had already saved in her Keychain Access (I don't use it myself)???

Ultimately, I was able to get the same error though, which I think can happen because the message by gitcreds of Enter password or token that can lead you to type your GitHub password instead of your GitHub Personal Access Token (PAT); just like Jenny anticipated.

Once that has happened, the error message by gh::gh_token(NULL) (which is called by several usethis functions) doesn't really point users to a path for solving the issue. One possible solution would be to add a tryCatch() call and then prompt the user to run gitcreds::gitcreds_set() again to replace their current credentials with their PAT instead of their GitHub password.

If you want, I can send a PR. If you consider that it's out of scope for usethis feel free to close the issue.

Best,
Leo

Details

I don't think that I've run gitcreds::gitcreds_set(url = "https://github.com") myself, but if I do, I see that it prompts for the password or token, so maybe that's the prompt Jenny referred to.

In any case, I get on both macOS and Windows:

> gitcreds::gitcreds_delete(url = "https://github.com")
> gitcreds::gitcreds_get(url = "https://github.com")
<gitcreds>
  protocol: NA
  host    : NA
  username: NA
  password: <-- hidden -->
> nchar(gitcreds::gitcreds_get(url = "https://github.com")$password)
[1] 40

as in, I never see my GitHub username listed by gitcreds() unlike Louise.

If I run create_github_token() I do see the prompt to run gitcreds::gitcreds_set().

> options(usethis.protocol = "https")
> usethis::create_github_token()
* Call `gitcreds::gitcreds_set()` to register this token in the local Git credential store
  It is also a great idea to store this token in any password-management software that you use
√ Opening URL 'https://github.com/settings/tokens/new?scopes=repo,user,gist,workflow&description=R:GITHUB_PAT'

If I then follow the recommendation, I get

> gitcreds::gitcreds_set()


? Enter password or token: 5efcc09fcc37da4f1935fdbb16e0fbf5c33f1d5d
-> Adding new credentials...
-> Removing credetials from cache...
-> Done.

## no worries, I've deleted that PAT from my GH account

If I then check to see if my username will pop up, it doesn't. But that's because gitcreds_get() is finding my GITHUB_PAT from my ~/.Renviron file.

> gitcreds::gitcreds_get(url = "https://github.com")
<gitcreds>
  protocol: NA
  host    : NA
  username: NA
  password: <-- hidden -->
> gitcreds::gitcreds_get(url = "https://github.com")$password
[1] "my_real_token (censored)"

Then if I comment out my GITHUB_PAT in my ~/.Renviron and restart R I do see more info and the get the matching token.

> gitcreds::gitcreds_get(url = "https://github.com")
<gitcreds>
  protocol: https
  host    : github.com
  path    : 
  username: PersonalAccessToken
  password: <-- hidden -->
> gitcreds::gitcreds_get(url = "https://github.com")$password
[1] "5efcc09fcc37da4f1935fdbb16e0fbf5c33f1d5d"

I can then remove this information.

> gitcreds::gitcreds_delete(url = "https://github.com")

-> Your current credentials for 'https://github.com':

  protocol: https
  host    : github.com
  path    : 
  username: PersonalAccessToken
  password: <-- hidden -->

-> What would you like to do? 

1: Keep these credentials
2: Delete these credentials
3: See the password / token

Selection: 2
-> Removing current credentials...
-> Removing credetials from cache...
-> Done.

If I then run the setter again with my real GitHub password, it goes through. Yet it thinks that I'm giving it a Personal Access Token (PAT).

> gitcreds::gitcreds_set(url = "https://github.com")


? Enter password or token: my_real_github_password
-> Adding new credentials...
-> Removing credetials from cache...
-> Done.

> gitcreds::gitcreds_get(url = "https://github.com")
<gitcreds>
  protocol: https
  host    : github.com
  path    : 
  username: PersonalAccessToken
  password: <-- hidden -->
> gitcreds::gitcreds_get(url = "https://github.com")$password
[1] "my_real_github_password"

At this point, I can then reproduce the error Louise was getting, although our gitcreds::gitcreds_get(url = "https://github.com") outputs still differ (her shows her username, mine says PersonalAccessToken).

> gh::gh_token(NULL)
Error in validate_gh_pat(new_gh_pat(x)) : 
  A GitHub PAT must consist of 40 hexadecimal digits

So, like Jenny said earlier, maybe Louise typed her GitHub password instead of pasting the GitHub PAT. Yet once that happened, the error messages were very confusing.

To restore my setup, I ran gitcreds::gitcreds_delete(url = "https://github.com"), then I re-enabled my true GitHub PAT on my ~/.Renviron (you can edit it with usethis::edit_r_environ()), and well, I have options(usethis.protocol = "ssh") on my ~/.Rprofile (usethis::edit_r_profile()).

Hopefully this issue will help anyone running into this problem in the future.

[lcolladotor]

Ahh, now I see r-lib/gh#133. Interesting!

[jennybc]

I feel like this issue is muddying the waters.

As I've tried to explain in this vignette:

https://usethis.r-lib.org/articles/articles/git-credentials.html

These are the recommendations:

1. Adopt HTTPS as your Git transport protocol
2. Turn on two-factor authentication for your GitHub account
3. Use a personal access token (PAT) for all Git remote operations from the command line or from R
4. Allow tools to store and retrieve your credentials from the Git credential store

If you follow those instructions, I don't think you encounter a problem. I specifically recommend against storing the PAT in a startup file. I think this was just a case of something incorrect being entered into the cache (a password instead of a PAT), then predictable downstream errors due to that.

[lcolladotor]

Thanks again Jenny! I just updated biocthis's setup steps based on your feedback.