Document How To Install Python Package from a Private Repo #104
Comments
|
I want to work on #104 |
nikita-bykov
added
documentation
|
Is there a canonical way to do this now in 2022? |
|
I would also love to know if there was a canonical way to do this in 2022. |
|
So far the best way I've found to do this is by creating a Machine User and inviting it to the organization. Then, in requirements.txt you can add lines like: Generate and add an SSH key for the machine user, then use something like to set the key in your workflow from a secret. There is more information at Using organization Python package in Github actions without Python repository although this only seems to work for one package and so that's why I ended up having to use the machine user approach as usually if you depend on one dependency in an organization you will depend on others. Hope somebody finds this useful. |
|
Not really canonical but this workaround might be a little better: Just add |
|
I tackled the same issue a long time ago. My conclusion at the time was much the same as the one described by croth. Then I learned that GitHub recommends authentication via HTTPS, and I felt the need to reconsider this issue, but I neglected it. Recently, I had to do some maintenance on an old project, so I thought about my own best practices at this point. It is a combination of the methods described by croth and victorsevero. In other words, do the following.
I checked and found that for GitHub Actions, this can be easily accomplished by using webfactory/ssh-agent. |
|
I have the following up and running successful: steps:
# https://github.com/actions/checkout
- name: checkout
uses: actions/[email protected]
# https://github.com/marketplace/actions/setup-python
- name: Setup Python
uses: actions/[email protected]
with:
python-version: "3.10"
- name: replace requirements
run: sed -i "s/ssh:\/\/git@github\.com/https:\/\/${{ secrets.GH_USERNAME }}:${{ secrets.GH_PAT_WITH_ACCESS_TO_OTHER_REPOS }}@github\.com/g" requirements.txt
- name: install
run: pip install -e . && pip install -r requirements.txt
- name: test
run: python test.pywill try out another variant with a gh app token. |
|
Same issue, and had to resort to @victorsevero (and others') solution |
|
There's another option now that I believe is a bit more secure than managing a separate machine user and its associated credentials: Create a new Github App with "Contents" permissions, add it to the private repositories that the Github Action needs to access, and use something like https://github.com/marketplace/actions/action-github-app-token to get an access token that can be used during git checkout. Unfortunately this still requires the app's private key to be saved as a secret for the Action. |
|
@aripollak this solution works Pretty Good as of today, and has a nice restrictive access element to it. Thank you 🙇🏽♂️ |
|
@tyriis Thank you for this elegant dependency-free solution! |
|
@aripollak How did you manage to successfully use a Github App token with pip? I can use the private_key as a way of configuring the checkout action, but unsure how to re-use the private key to let pip authenticate and clone dependencies. |
|
So I got it all wrong on my previous comment. Then, once your app has been created and added to the required repos (or if you add it to the top level of your organization, you can select which repos the app has access to), you can use create-github-app-token action to generate an installation token that can access multiple repositories. The only issue is this token can be used to sign-in using https. We can override global urls with In my case I was using ssh in my requirements.txt file, so these also get overriden by the global git configuration. - name: Get our app token from Github App
uses: actions/create-github-app-token@v1
id: app_token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
# owner is required, otherwise the creds will fail the checkout step
owner: ${{ github.repository_owner }}
- name: Checkout from GitHub
uses: actions/[email protected]
with:
submodules: true # In my case I'm also using this token to clone sub-modules
token: ${{ steps.app_token.outputs.token }}
# This step is necessary to allow pip install private packages hosted in github.
- name: Setup token for Python installation
run: git config --global url."https://oauth2:${GH_TOKEN}@github.com".insteadOf ssh://[email protected] # replace with whatever authentication method you're currently using
env:
GH_TOKEN: ${{ steps.app_token.outputs.token }}Thanks to this person for suggesting this approach: more info on github app authentication: Remember also to create your APP_ID and PRIVATE_KEY secret variables from your App configuration: |
Does this method work in an organization setting? I have two private repositories, one is a python package and other one is where I want to install. I did the things mentioned, but it fails. Could it be because the repositories are in organization? I get the following error: |
|
@mohit2512sharma . This works in an organization and it is what we're currently using. |
Thanks for your comment. @pmabres . Your comment does work, unfortunately I had a typo in my |
I have a number of Python Packages in private (company) repos and I am using GitHub Actions to run pytest on commits. One of the repos depends on packages from other repos. When pip runs from the Action, I see the following error:
Please document how the user can grant access to private repos to the Action. For example, I solved the problem using the following:
While this works, it is a little tedious. Tech Support suggested I use HTTPS with a username and password to check out the packages from the private repositories. I would prefer to not use this method. It would likely require me to create and maintain a "fake" user account just for checking repositories in GitHub Actions. I would much prefer to use a personal access token (like I did above), but in a more simplified manner.
The text was updated successfully, but these errors were encountered: