How to add github secrets to env variable

[machineCYC]

Below is the yml file, I hope to add github secrets to env variable.

name: Publish Docker image
on:
  push:
    branches: [ main ]

jobs:
  push_to_registry:
    name: Push Docker image to Docker Hub
    runs-on: ubuntu-latest

    steps:
      - name: Check out the repo
        uses: actions/checkout@v2

      - name: Log in to Docker Hub
        uses: docker/login-action@v1
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Push to Docker Hub
        uses: docker/build-push-action@v2
        with:
          push: true
          tags: docker_hub_name/image_name:latest
          secrets: |
            "MYSQL_HOST=${{ secrets.MYSQL_HOST}}"
            "MYSQL_USER=${{ secrets.MYSQL_USER}}"
            "MYSQL_PASSWORD=${{ secrets.MYSQL_PASSWORD}}"
            "MYSQL_PORT=${{ secrets.MYSQL_PORT }}"

Then use syntax os.environ.get("MYSQL_HOST", "") to get my database information when build image.

But use the below method still not work, have something syntax wrong? or other methods can do it?

secrets: |
  "MYSQL_HOST=${{ secrets.MYSQL_HOST}}"
  "MYSQL_USER=${{ secrets.MYSQL_USER}}"
  "MYSQL_PASSWORD=${{ secrets.MYSQL_PASSWORD}}"
  "MYSQL_PORT=${{ secrets.MYSQL_PORT }}"

[crazy-max]

@machineCYC

to get my database information when build image.

Wonder why you would include/use database credentials while building a Docker image?

But use the below method still not work, have something syntax wrong? or other methods can do it?

Can't tell without a Dockerfile. Do you have a link to your repo?

[machineCYC]

@crazy-max It's a private repo for crawler application, so need to save the data to MySQL db.

below is the Dockerfile

FROM continuumio/miniconda3:4.3.27

RUN apt-get update

RUN mkdir /workspace
COPY . /workspace/
WORKDIR /workspace/

# install package
RUN pip install pipenv && pipenv sync

# genenv
RUN python genenv.py

# time
RUN echo "Asia/Taipei" > /etc/timezone
RUN dpkg-reconfigure -f noninteractive tzdata

CMD ["/bin/bash"]

Below is the python script call genenv.py. It can get the MySQL db information from the environment, and create the .env file for config to get the MySQL db information.

import os


def get_config_from_environment(env_content):
    # get config from env
    env_content += "MYSQL_HOST={}\n".format(os.environ.get("MYSQL_HOST", ""))
    env_content += "MYSQL_USER={}\n".format(os.environ.get("MYSQL_USER", ""))
    env_content += "MYSQL_PASSWORD={}\n".format(os.environ.get("MYSQL_PASSWORD", ""))
    env_content += "MYSQL_PORT={}\n".format(os.environ.get("MYSQL_PORT", ""))
    return env_content


env_content = ""
env_content = get_config_from_environment(env_content)

with open(".env", "w", encoding="utf8") as env:
    env.write(env_content)

[crazy-max]

@machineCYC Thanks for your feedback. So in your case your workflow is right but the Dockerfile doesn't use those secrets through the RUN --mount=type=secret. Something like this should work:

# syntax=docker/dockerfile:1.2
FROM continuumio/miniconda3:4.3.27

RUN apt-get update

RUN mkdir /workspace
COPY . /workspace/
WORKDIR /workspace/

# install package
RUN pip install pipenv && pipenv sync

# genenv
RUN --mount=type=secret,id=MYSQL_HOST \
  --mount=type=secret,id=MYSQL_USER \
  --mount=type=secret,id=MYSQL_PASSWORD \
  --mount=type=secret,id=MYSQL_PORT \
  export MYSQL_HOST=$(cat /run/secrets/MYSQL_HOST) && \
  export MYSQL_USER=$(cat /run/secrets/MYSQL_USER) && \
  export MYSQL_PASSWORD=$(cat /run/secrets/MYSQL_PASSWORD) && \
  export MYSQL_PORT=$(cat /run/secrets/MYSQL_PORT) && \
  python genenv.py

# time
RUN echo "Asia/Taipei" > /etc/timezone
RUN dpkg-reconfigure -f noninteractive tzdata

CMD ["/bin/bash"]

Fyi, we can't use the secret id directly as an env var atm. That's why we have to cat. (see moby/buildkit#2122)

[machineCYC]

@crazy-max it's a great method, but this method can not build the image locally right?

[crazy-max]

@machineCYC Everything this action does, you can do the same thing locally. You just have to copy the generated build command displayed in the action log and that's it. This one for example:

For secrets you have to create them first.

[machineCYC]

@crazy-max Create the secrets locally. before executing the blue color comment?
How to create the secrets locally?

[crazy-max]

@machineCYC

Create the secrets locally. before executing the blue color comment?

Yes

How to create the secrets locally?

https://docs.docker.com/engine/reference/commandline/secret_create/#create-a-secret

You can also provide secrets through env vars. So let's say you already have the env var MYSQL_HOST set locally, you can then set secret in buildx command: --secret id=mysql_host,env=MYSQL_HOST and use it like this in your Dockerfile:

RUN --mount=type=secret,id=mysql_host \
  export MYSQL_HOST=$(cat /run/secrets/mysql_host) && \
  echo $MYSQL_HOST

[AndreiCalazans]

This worked for me. Thanks, I documented everything here https://andrei-calazans.com/posts/2021-06-23/passing-secrets-github-actions-docker

[machineCYC]

@crazy-max Thanks

[nidalhaddad]

Hi, i tried to reproduce the same code as @AndreiCalazans but when i enter my docker container the secrets folder is empty
here are my 2 scripts.
github actions :

  - name: Build & Push image
      uses: docker/build-push-action@v2
      with:
        file: /server/Dockerfile
        secrets: | 
          "NODE_ENV=${{ secrets.NODE_ENV }}"
        push: true
        tags: ${{ steps.ecr-login.outputs.registry }}/lcm-backend:latest

Docker file

#Origin, Node version 16.13.2 with npm version 8.1
FROM node:lts-bullseye-slim

RUN --mount=type=secret,id=NODE_ENV export NODE_ENV=$(cat /run/secrets/NODE_ENV) 

CMD [ "/bin/bash"]

[urfolomeus]

I am getting a similar result. It seems that the secret comes through as ***. This seems to be expected if we look at the table at the bottom of the docs. Is there a way to get to the value held in the secret?

Workflow

...
- name: Build and push Docker image
        uses: docker/[email protected]
        with:
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          secrets: |
            "ssh_private_key=${{ secrets.SSH_PRIVATE_KEY }}"

Dockerfile

FROM ruby:3.1.2

RUN mkdir -p /root/.ssh
RUN --mount=type=secret,id=ssh_private_key \
    cat /run/secrets/ssh_private_key >> /root/.ssh/id_rsa && \
    echo "Secret is $(cat /run/secrets/ssh_private_key)" && \
    echo "File contains $(cat /root/.ssh/id_rsa)"

Output

#6 [3/4] RUN --mount=type=secret,id=ssh_private_key     cat /run/secrets/ssh_private_key >> /root/.ssh/id_rsa &&     echo "Secret is $(cat /run/secrets/ssh_private_key)" &&     echo "File contains $(cat /root/.ssh/id_rsa)"
#6 0.355 Secret is ***
#6 0.357 File contains ***
#6 DONE 0.4s

[crazy-max]

enter my docker container the secrets folder is empty

It's only at build time, they are not persisted.

I am getting a similar result. It seems that the secret comes through as ***

Log output of GitHub Actions hide sensitive data like secrets with ***.

[urfolomeus]

Thanks, @crazy-max. It hadn't occurred to me to check that! Everything was working fine, it turned out that the reason I was seeing an error was that my private key was not PEM-formatted. 😅

[alejoGT1202]

Hello I'm trying to do this but when I tried to access to my secret key in python is set to None.

Workflow

      - name: Build and push Docker image
        id: build-and-push
        uses: docker/build-push-action@v2
        with:
          context: .
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          secrets: |
           "CASSANDRA_CLIENT_ID=${{ secrets.CASSANDRA_CLIENT_ID }}"
           "CASSANDRA_CLIENT_SECRET=${{ secrets.CASSANDRA_CLIENT_SECRET }}"
          labels: ${{ steps.meta.outputs.labels }}

Dockerfile

RUN --mount=type=secret,id=CASSANDRA_CLIENT_ID \
  --mount=type=secret,id=CASSANDRA_CLIENT_SECRET \
   export CASSANDRA_CLIENT_ID=$(cat /run/secrets/CASSANDRA_CLIENT_ID) && \
   export CASSANDRA_CLIENT_SECRET=$(cat /run/secrets/CASSANDRA_CLIENT_SECRET) && \
   echo $CASSANDRA_CLIENT_ID

[nidalhaddad]

@amigohuhu can you provide a code sample.