Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running EAS Build with --local can't access EAS Secrets but also can't access .env.local #2392

Closed
melyux opened this issue May 21, 2024 · 21 comments
Assignees
Labels
eas build needs review Issue is ready to be reviewed by a maintainer

Comments

@melyux
Copy link

melyux commented May 21, 2024

Build/Submit details page URL

No response

Summary

Cloud EAS Builds have access to EAS Secrets, but local EAS Builds don't. The documentation says the solution is to use environment variables for local EAS Builds instead. However, local EAS Builds only seem to have access to the .env file, but not .env.production nor .env.local nor any other standard .env file resolution that Expo uses.

I tried un-gitignore-ing the .env.local files, and it still didn't work. The logs seem to indicate that it's being loaded:

[PREBUILD] env: load .env.local .env.production .env

But in the end, I still get the Sentry error that the SENTRY_AUTH_TOKEN in .env.local was not loaded: ❌ error: Auth token is required for this request.

If I put SENTRY_AUTH_TOKEN in the plain old .env., the build succeeds.

(P.S. Even if .env.local was in .gitignore, this should still be working, since local builds have no access to EAS Secrets and I don't want to commit my .env.local secrets.)

Managed or bare?

Managed

Environment

expo-env-info 1.2.0 environment info:
System:
OS: macOS 14.4.1
Shell: 5.9 - /bin/zsh
Binaries:
Node: 22.1.0 - /opt/homebrew/bin/node
npm: 10.7.0 - /opt/homebrew/bin/npm
Watchman: 2024.05.06.00 - /opt/homebrew/bin/watchman
Managers:
CocoaPods: 1.15.2 - /opt/homebrew/bin/pod
SDKs:
iOS SDK:
Platforms: DriverKit 23.5, iOS 17.5, macOS 14.5, tvOS 17.5, visionOS 1.2, watchOS 10.5
IDEs:
Xcode: 15.4/15F31d - /usr/bin/xcodebuild
npmPackages:
expo: ^50.0.0 => 50.0.8
react: 18.2.0 => 18.2.0
react-native: 0.73.4 => 0.73.4
npmGlobalPackages:
eas-cli: 9.0.7
Expo Workflow: bare

✔ Check Expo config for common issues
✔ Check package.json for common issues
✔ Check dependencies for packages that should not be installed directly
✔ Check npm/ yarn versions
✔ Check for issues with metro config
✔ Check for common project setup issues
✔ Check for legacy global CLI installed locally
✔ Check that native modules do not use incompatible support packages
✔ Check Expo config (app.json/ app.config.js) schema
✔ Check native tooling versions
✖ Check that packages match versions required by installed Expo SDK
✔ Check that native modules use compatible support package versions for installed Expo SDK

Error output

No response

Reproducible demo or steps to reproduce from a blank project

  1. Create an Expo project with something that requires an environment variable during build, e.g. Sentry
  2. Use an .env.local file in the root with an environment variable needed for the build, e.g. SENTRY_AUTH_TOKEN
  3. Don't .gitignore the .env.local file
  4. Run eas build --platform ios --local and see that it claims to load .env.local variables but they are not made available to parts of the build, e.g. Sentry
@melyux melyux added the needs review Issue is ready to be reviewed by a maintainer label May 21, 2024
@curtisgibeaut
Copy link

We are dealing with a similar issue for some reason local builds randomly stopped picking up environment variables.

@mgscreativa
Copy link

mgscreativa commented Jun 18, 2024

The same here, put .env and .env.local on project root and using npx eas build --profile production-preview --platform android --clear-cache --local got

[RUN_GRADLEW] env: load .env

But I expected to be .env.local as stated
here, here and here

@brentvatne, @dsokal, @szdziedzic can you guys take a look at this?, because it would be great to be able to use .env.local in local eas builds

Environment

expo-env-info 1.2.0 environment info:
System:
OS: Linux 5.15 Linux Mint 21.3 (Virginia)
Shell: 5.1.16 - /bin/bash
Binaries:
Node: 18.19.0 - ~/.nvm/versions/node/v18.19.0/bin/node
Yarn: 1.22.21 - ~/.nvm/versions/node/v18.19.0/bin/yarn
npm: 10.2.3 - ~/.nvm/versions/node/v18.19.0/bin/npm
Watchman: 4.9.0 - /usr/bin/watchman
npmPackages:
expo: ~51.0.14 => 51.0.14
react: 18.2.0 => 18.2.0
react-native: 0.74.2 => 0.74.2
npmGlobalPackages:
eas-cli: 10.0.2
Expo Workflow: managed

@wongk
Copy link

wongk commented Jun 19, 2024

+1

@kevinhylant
Copy link

Just to confirm, when running eas build with the local flag, it should be able to access EAS project secrets & they should be made available in eas.json & app.config.ts via process.env.[variableName], correct? And this behavior would be the same for remote builds too, correct?

I’ve saved my googles-services.json / GoogleService-Info.plist in EAS Secrets for my project & gitingored the files in my repo when I run the build script locally it cannot find the secret files.

I just want to make sure I’m understanding this properly.

@thozh
Copy link

thozh commented Jul 9, 2024

I’m encountering a similar issue where updated values in eas.json aren’t being recognized during local builds.

@enigosi
Copy link

enigosi commented Jul 31, 2024

Just to confirm, when running eas build with the local flag, it should be able to access EAS project secrets & they should be made available in eas.json & app.config.ts via process.env.[variableName], correct? And this behavior would be the same for remote builds too, correct?

I’ve saved my googles-services.json / GoogleService-Info.plist in EAS Secrets for my project & gitingored the files in my repo when I run the build script locally it cannot find the secret files.

I just want to make sure I’m understanding this properly.

eas secrets are not supported in local builds
https://docs.expo.dev/build-reference/local-builds/#limitations

@brentvatne
Copy link
Member

The same here, put .env and .env.local on project root and using npx eas build --profile production-preview --platform android --clear-cache --local got

[RUN_GRADLEW] env: load .env

we load env vars specified in your build profile. if you want to also load .env, see more info in this thread: https://expo.canny.io/feature-requests/p/flag-to-load-env-file-during-local-builds

@kevinhylant
Copy link

@brentvatne thanks for that article/reference, I’m thinking specifically about environment variables like a GOOGLE_SERVICES_JSON which it is recommended be stored as a secret on EAS servers & made available to app.config.ts via process.env.GOOGLE_SERVICES_JSON.

If we save this file to remote EAS Servers & then want to create a local build, how should we access this file in our app.config.ts properly? I’d love to just write .env.process.GOOGLE_SERVICES_JSON like in your docs but I’m not seeing how to do this for local eas builds since it ignores any google_services files I might have in my directory as they’re git ignored.

@melyux
Copy link
Author

melyux commented Aug 1, 2024

@brentvatne Can you reopen this? As stated in the issue, we can't use the ENV variables in EAS config because they're secrets and the config is committed to git. We can't use .env because that's committed too. We can't use .env.locsl because EAS Build --local doesn't recognize .local env files, unlike Expo itself, which does.

EAS Build --local should recognize .local env files since it has no access to secrets. Right now, the only way to provide secrets to local builds is to literally prefix the secrets to the command like "VAR=something eas build --local ...", which is crazy

@szdziedzic
Copy link
Member

@kevinhylant The solution that can work here for you is:

  1. Trigger local builds as GOOGLE_SERVICES_BASE64=$(base64 -i ./path/to/google-services.json) eas build --local ... - when triggering the local build the base64 -i ./path/to/google-services.json call will be executed correctly even if the file itself is gitignored.
  2. Add a preinstall hook that will decode GOOGLE_SERVICES_BASE64 to the google-services.json file during the build process:
{
  "name": "my-app",
  "scripts": {
    "eas-build-pre-install": "echo $GOOGLE_SERVICES_BASE64 | base64 --decode > ./path/to/google-services.json",
    // ...
  },
  "dependencies": {
    "expo": "^51.0.0"
    // ...
  }
}

@szdziedzic
Copy link
Member

@melyux I used your repro steps and I was able to reproduce the issue.

However this weird behavior allowing SENTRY_AUTH_TOKEN to be loaded from .env but not from .env.local|development|production|... doesn't come from a bug in EAS build process / @expo/env package (used to load .env files inside of Expo CLI). The values from .env files inside of EAS Build process as of today are only supported inside of actions performed through Expo CLI (npx expo) like prebuild, bundling your JS code and resolving Expo config.

Uploading source maps to Sentry servers (this is where your build fails) is not an action performed through Expo CLI therefore it doesn't have access to .env files through Expo CLI. It turns out that Sentry CLI tries to load the env vars from .env file (and only .env file) by default. If you use .env the Sentry CLI can find the SENTRY_AUTH_TOKEN there, so everything works fine. If you use .env.local it can't, because by default it only checks for .env. That explains the failures that you are observing.

It seems like Sentry CLI supports SENTRY_DOTENV_PATH env variable to choose the .env file to load the env vars from. If you do SENTRY_DOTENV_PATH=../.env.local eas build --local ... your build should work well.

When it comes to making env vars from .env.local accessible throughout the whole build process (in a way that env vars from buildProfile.env are) as of today I would suggest using this simple script inside of the pre-install hook to set environment variables dynamically during the build process using set-env binary.

./source_env_local.sh:

#!/bin/bash

# Path to the .env file
ENV_FILE=".env.local"

# Check if .env file exists
if [ ! -f "$ENV_FILE" ]; then
    echo "Error: $ENV_FILE file does not exist."
    exit 1
fi

# Read .env file line by line
while IFS= read -r line; do
    # Skip empty lines and lines starting with # (comments)
    if [[ -z "$line" || "$line" =~ ^# ]]; then
        continue
    fi

    # Use regex to extract env variable KEY and VALUE
    if [[ "$line" =~ ^([^=]+)=(.*) ]]; then
        key="${BASH_REMATCH[1]}"
        value="${BASH_REMATCH[2]}"

        # Use set-env binary to set the environment variable
        set-env "$key" "$value"
        if [ $? -ne 0 ]; then
            echo "Failed to set $key"
        else
            echo "Set $key"
        fi
    fi
done < "$ENV_FILE"

package.json:

{
  "name": "my-app",
  "scripts": {
    "eas-build-pre-install": "./source_env_local.sh",
    // ...
  },
  "dependencies": {
    "expo": "^51.0.0"
    // ...
  }
}

@szdziedzic
Copy link
Member

@melyux It's even better to just use direnv to automatically export the env vars form .env when you enter the project directory and then you don't need to worry about running any scripts when doing eas build --local, because env vars are already in your environment

@fernandatoledo
Copy link

fernandatoledo commented Aug 7, 2024

@szdziedzic I tried the option with the script and once again, if I have the envfile tracked and is not in gitignore then it works perfectly but the second I add my envfile to gitignore the script can't find the file 😬

Screenshot 2024-08-07 at 3 20 25 PM

@melyux have you find any solution? Thanks!

@szdziedzic
Copy link
Member

@fernandatoledo

I tried the option with the script and once again, if I have the envfile tracked and is not in gitignore then it works perfectly but the second I add my envfile to gitignore the script can't find the file 😬

Yeah, that's true eas build --local respects .gitignore by default (if something is gitignored it won't be copied to the eas build --local working directory with the rest of the project, to see how the contents of project archive available during the build process look like use eas build:inspect -p platform --stage archive --output outputPath --profile profileName command). If you want to override the .gitignore behavior and have different ignore rules for eas build --local you can use .easignore.

https://github.com/expo/fyi/blob/main/eas-build-archive.md

I also think that using direnv to export the env vars form .env.local automatically can be a viable option to solve this problem.

@melyux
Copy link
Author

melyux commented Oct 7, 2024

@szdziedzic The .easignore method didn't work for me. It doesn't un-ignore anything, it can only add to the gitignore list.

For direnv, it's not easy to have the whole team all install a non-npm tool just to have this work. I wish there was an npm-native to do what direnv does so I could add it to package.json as a dev dependency and call it a day.

@melyux
Copy link
Author

melyux commented Oct 7, 2024

Even with the direnv solution, unignoring entire files required by the build doesn't work (e.g. google-services.json) since it's not provided as a value but as a path to app.config.js. EAS Secrets gets around this by uploading the entire file and making that file available to the build, but there's no way to do it locally.

@brentvatne Consider re-opening this because of these shortcomings. Life would be much simpler if EAS Build --local behaved the same as EAS Build.

@anhtuan219
Copy link

I exported the path of google-services.json to the environment variable to replace eas secret and it works
Detail here

@melyux
Copy link
Author

melyux commented Oct 8, 2024

I found a simpler workaround, for both gitignored files that you need in the build process (e.g. GoogleService-Info.plist) and for gitignored .env files with environment files that you need in the build process (SENTRY_AUTH_TOKEN). These will both work by uploading them to EAS Secrets when using the cloud EAS Builds (of course), but Expo doesn't seem to care much about the local builds because it doesn't make them money.

📂 Files

For files that you need to include in the build process, like GoogleService-Info.plist and google-services.json, but that you rightfully added to .gitignore, EAS Build --local will fail to access them because they are in .gitignore. To solve this, we have to use .easignore.

Create .easignore in your repo's root directory (even if your Expo directory is not the root), otherwise it doesn't work for some reason. I had my .gitignore in my Expo sub-directory, but putting the .easignore in the sub-directory didn't work, only in the root directory. In your .easignore file, which will override your .gitignore completely, add things you don't want to upload to EAS. This will be similar to your .gitignore but doesn't have to be as exact, just put the big directories like this, but don't add anything you want the EAS Build to have available to it, like .env.local or google-services.json:

**/ios/
**/android/
**/expo/

You can check which files will be uploaded to EAS Build (and used in EAS Build --local) by running eas build:inspect -p <ios|android> --stage archive --output outputPath --profile <your-profile-here> from your Expo directory. This will create an outputPath directory that will have inside of it everything it will copy from your project into the EAS Build sandbox to build on. Delete this outputPath folder after you checked it, it's supposed to be a temp folder.

Play with the .easignore until only the files you want in your EAS Build are left in your outputPath temp folder, like your source code, Google services files, .env files, etc.

☁️ Environment variables from .env files

Unlike the cloud EAS Build, the EAS Build --local command doesn't have access to EAS Secrets. And even if you take the step above to force-include your .gitignored .env files (or .env.local, .env.development, ..env.development.local, etc.) files in the EAS Build sandbox, it still won't work with things like SENTRY_AUTH_TOKEN and your build will fail, even if that variable was in your .env files. The .env parsing is broken in EAS Build, so the only thing that works (and why EAS Secrets works in the cloud) is literally providing any environment variables as environment variables to the eas build --local command.

Now, you could just mimic this for your EAS Build --local runs by inlining your environment variables like this: SENTRY_AUTH_TOKEN="asdfasdf..." eas build --local ... and it will work. But instead, we will do this automatically:

Go to your Expo directory and run npm install --save-dev dotenv-cli. This will install dotenv-cli to your project's development dependencies, which is a project that will read your .env files' variables and use them as environment variables for whatever command you run after it! After installing, you can run eas build --local with your .env files like this:

npx dotenv -c -- eas build --platform android --profile production --local

This will read your .env and .env.local files and use their variables as environment variables when running eas build --platform android ....

If you want to include .env.production and .env.production.local, you can use this to include those in addition to .env and .env.local in the usual dotenv resolution order:

npx dotenv -c production -- eas build --platform android --profile production --local

I added this to my package.json as a script so I can easily run this with a shortcut (npm run build-prod-android-eas):

{
    ...
    "scripts": {
        "build-prod-android-eas": "npx dotenv -c production -- eas build --platform android --profile production --local",
    },
    ...
}

@gregfenton
Copy link
Contributor

but Expo doesn't seem to care much about the local builds because it doesn't make them money.

@melyux I find this to be an unfair and easily debunked comment. There is so much that you get "for free" (at not cost) from Expo that your comment is insulting.

The fact that a company has established a revenue model on their freely available open sourced technology does not make them an evil-only-cares-about-the-money entity.

Your writeup shows two things:

  • people can do a lot of powerful things using Expo & EAS tools without spending a dime
  • you, @melyux, are starting to understand how Expo & EAS actually work....just like they are documented....that you can read.....at no cost.

@szdziedzic
Copy link
Member

hey,

check it out https://expo.dev/blog/environment-variables I believe that using a new system with sensitive or plain text visibility should be a good fit here 👀

@melyux
Copy link
Author

melyux commented Nov 23, 2024

@gregfenton My main problem is that it took us many, many hours to find a workaround for this, and then Expo releases the solution (haven't tried it yet) out of nowhere without any sign they were going to do anything about it, on this issue or any of the other similar issues.

I'm super happy a possible solution came quickly. At the time I wrote that, I hadn't yet realized that Expo in general was such a stopgap solution, filling the gap between other corporations and projects and subject to their whims, and not a huge team to begin with. We pay for EAS services by the way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
eas build needs review Issue is ready to be reviewed by a maintainer
Projects
None yet
Development

No branches or pull requests