FIPS-compliant builds on dockerhub #23353

tabacco · 2022-10-03T23:01:09Z

Title: Include FIPS-compliant builds on dockerhub

Description:

I'd love to see pre-built images with FIPS-compliant boringssl available on dockerhub and other image hosting platforms.

Relevant Links

FIPS compliant version of docker image in docker repository #8107 - This was requested years ago and referred to the now-defunct getenvoy project.

github-actions · 2022-11-03T20:01:29Z

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

phlax · 2022-11-03T20:58:31Z

cc @lizan @plallin

not sure if this is something we prioritize - im aware the getenvoy tetrate builds provided this and that is no longer the case

as i have said elsewhere my main concern is that we multiply the number of binaries/builds that we publish but perhaps this is something we should look at

tomscarlato · 2022-11-07T15:53:58Z

Is there currently a way for users to enable FIPS on envoy without bazel? I'd be glad to change the code on a local fork if that's an option but I don't know where that change would need to happen.

plallin · 2022-11-07T16:02:48Z

@tomscarlato In order to be FIPS-compliant, Envoy needs to be compiled with a FIPS-compliant version of BoringSSL. The build tool for Envoy is Bazel and afaik there is no other supported build tool, which means you will have to use Bazel to build Envoy with FIPS compliance. I would be delighted to be wrong.

I have not yet manage to build Envoy offline either, so far

plallin · 2022-11-10T18:00:50Z

So, I have been experimenting a little building Envoy with BoringSSL-FIPS.

I think I might have hijacked this thread with 2 different concerns:
(1) obtaining a fips-compliant binary of Envoy (whether it's made available and ready to download and use or by compiling it with Bazel)
(2) Build Envoy offline

My setup is as follows on AWS - warning, the Ec2 instance used cost big money:

Image: AmazonLinux2 with FIPS enabled (ref)
Instance type: c5d.24xlarge (⚠️ Big bucks $$$ ⚠️ )
100GB SSD (I was trying with 50GB and ran out of disk space 😓 )

Startup script:

yum groupinstall -y "Development Tools"
yum install -y wget tree mlocate vim ccache sudo xz tar \
  python3-devel python-devel python-pip python3-pip \
  java-1.8.0-openjdk-headless \
  llvm-devel llvm llvm-libs llvm llvm-static libtool \
  kernel rsync ninja-build clang cmake3 \
  curl-devel xz-devel expat-devel libarchive libarchive-devel bzip2-devel jsoncpp-devel \
  gcc10-binutils-devel.x86_64 gcc10-c++.x86_64 gcc10-plugin-devel.x86_64 gcc10-gdb-plugin.x86_64 \
  dracut-fips
wget -O /usr/local/bin/bazel https://github.com/bazelbuild/bazelisk/releases/download/v1.12.0/bazelisk-linux-amd64 &&     chmod +x /usr/local/bin/bazel
pip install --upgrade pip
yum update -y
ln -s /usr/bin/ninja-build /usr/bin/ninja
ln -sf /usr/bin/cmake3 /usr/bin/cmake
ln -s /usr/bin/ld.gold /usr/bin/lld
ln -s /usr/bin/ld.gold /usr/bin/ld.lld
sudo dracut -f
sudo /sbin/grubby --update-kernel=ALL --args="fips=1"
sudo reboot

Case 1: Building FIPS Envoy (All traffic enabled)

In this case you can just build Envoy and grab the binary after the build completes. With 96 cores, it takes 15 minutes:

export PATH=$PATH:/usr/local/bin
cd /home/ && git clone https://github.com/envoyproxy/envoy.git && cd envoy && git checkout v.24.0
cd /home/envoy && CC=clang /usr/local/bin/bazel build --config=clang --verbose_failures --define boringssl=fips -c opt envoy

Case 2: Build Envoy without FIPS compliance, (almost) offline

Cloning the repo alone isn't enough, you will need to grab the dependency (requires Internet access) first.

export PATH=$PATH:/usr/local/bin
cd /home/ && git clone https://github.com/envoyproxy/envoy.git && cd envoy && git checkout v.24.0
bazel sync --repository_cache=~/home/envoy-sync-deps

Then, from that same machine with Internet access disabled:

export USE_BAZEL_VERSION=5.1.1  # https://github.com/bazelbuild/bazelisk/issues/88
cd /home/envoy && CC=clang /usr/local/bin/bazel build --config=clang --verbose_failures --distdir=/home/envoy-sync-deps/ -c opt envoy

Note: I'm not sure exactly what bazel sync exacgly does. I assumed it downloaded the dependencies to the repository_cache you pass it. It does download resources to that folder (it ends up being almost 9GB). However, if you upload that repository somewhere and download it on another instance, bazel build won't work. It would still attempt to use the network.

Case 3: Build Envoy with FIPS compliance, (kind of) offline

This does not work. The issue with bazel sync is that it doesn't seem to be downloading the FIPS-specific dependencies. I don't think there is a flag to instruct bazel sync to download them. This results in a failure later when you need to build Envoy with no Internet access as it tries and fails to fetch dependencies. Unfortunately I have not managed to move past that which makes FIPS offline builds not supported at the moment. I can provide the error for this in a later comment as this post is getting long.

Additionally, bazel sync itself does not seem to be very robust. When I run it, it fails systematically while also managing to actually fetch the required dependencies. This makes triaging of "red herring" vs legitimate errors difficult. One of the error, for example, that that some npm dependencies depend on other platforms than linux (npm ERR! notsup Unsupported platform for [email protected]: wanted {"os":"win32","arch":"x64"} (current: {"os":"linux","arch":"x64"})). This will fail 100% on any platform. There are similar dependencies for linux which would make it fail on other platforms.

So, in summary:, bazel sync may be used to fetch dependencies but it's not entirely reliable. It has to be run on the same machine where the offline build will be run from which limits its use. It won't fetch FIPS-specific dependencies which make offline FIPS builds unsupported.

plallin · 2022-11-10T18:02:53Z

Error message on trying to build FIPS-Envoy "offline"

  exec env - \
    BAZEL_COMPILER=clang \
    BAZEL_LINKLIBS=-l%:libstdc++.a \
    BAZEL_LINKOPTS=-lm \
    CC=clang \
    CXX=clang++ \
    PATH=/root/.cache/bazelisk/downloads/bazelbuild/bazel-5.1.1-linux-x86_64/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; #!/bin/bash

set -e

# BoringSSL build as described in the Security Policy for BoringCrypto module (2020-07-02):
# https://csrc.nist.gov/CSRC/media/projects/cryptographic-module-validation-program/documents/security-policies/140sp3678.pdf

# This works only on Linux-x86_64.
if [[ `uname` != "Linux" || `uname -m` != "x86_64" ]]; then
  echo "ERROR: BoringSSL FIPS is currently supported only on Linux-x86_64."
  exit 1
fi

# Bazel magic.
ROOT=$(dirname external/boringssl_fips/boringssl/BUILDING.md)/..
pushd $ROOT

# Build tools requirements:
# - Clang compiler version 7.0.1 (https://releases.llvm.org/download.html)
# - Go programming language version 1.12.7 (https://golang.org/dl/)
# - Ninja build system version 1.9.0 (https://github.com/ninja-build/ninja/releases)

# Override $PATH for build tools, to avoid picking up anything else.
export PATH="$(dirname `which cmake`):/usr/bin:/bin"

# Clang 7.0.1
VERSION=7.0.1
SHA256=02ad925add5b2b934d64c3dd5cbd1b2002258059f7d962993ba7f16524c3089c
PLATFORM="x86_64-linux-gnu-ubuntu-16.04"

curl -sLO https://releases.llvm.org/"$VERSION"/clang+llvm-"$VERSION"-"$PLATFORM".tar.xz \
  && echo "$SHA256" clang+llvm-"$VERSION"-"$PLATFORM".tar.xz | sha256sum --check
tar xf clang+llvm-"$VERSION"-"$PLATFORM".tar.xz

export HOME="$PWD"
printf "set(CMAKE_C_COMPILER \"clang\")\nset(CMAKE_CXX_COMPILER \"clang++\")\n" > ${HOME}/toolchain
export PATH="$PWD/clang+llvm-$VERSION-$PLATFORM/bin:$PATH"

if [[ `clang --version | head -1 | awk '\''{print $3}'\''` != "$VERSION" ]]; then
  echo "ERROR: Clang version doesn'\''t match."
  exit 1
fi

# Go 1.12.7
VERSION=1.12.7
SHA256=66d83bfb5a9ede000e33c6579a91a29e6b101829ad41fffb5c5bb6c900e109d9
PLATFORM="linux-amd64"

curl -sLO https://dl.google.com/go/go"$VERSION"."$PLATFORM".tar.gz \
  && echo "$SHA256" go"$VERSION"."$PLATFORM".tar.gz | sha256sum --check
tar xf go"$VERSION"."$PLATFORM".tar.gz

export GOPATH="$PWD/gopath"
export GOROOT="$PWD/go"
export PATH="$GOPATH/bin:$GOROOT/bin:$PATH"

if [[ `go version | awk '\''{print $3}'\''` != "go$VERSION" ]]; then
  echo "ERROR: Go version doesn'\''t match."
  exit 1
fi

# Ninja 1.9.0
VERSION=1.9.0
SHA256=1b1235f2b0b4df55ac6d80bbe681ea3639c9d2c505c7ff2159a3daf63d196305
PLATFORM="linux"

curl -sLO https://github.com/ninja-build/ninja/releases/download/v"$VERSION"/ninja-"$PLATFORM".zip \
  && echo "$SHA256" ninja-"$PLATFORM".zip | sha256sum --check
unzip -o ninja-"$PLATFORM".zip

export PATH="$PWD:$PATH"

if [[ `ninja --version` != "$VERSION" ]]; then
  echo "ERROR: Ninja version doesn'\''t match."
  exit 1
fi

# Clean after previous build.
rm -rf boringssl/build

# Build BoringSSL.
cd boringssl
mkdir build && cd build && cmake -GNinja -DCMAKE_TOOLCHAIN_FILE=${HOME}/toolchain -DFIPS=1 -DCMAKE_BUILD_TYPE=Release ..
ninja
ninja run_tests

# Verify correctness of the FIPS build.
if [[ `tool/bssl isfips` != "1" ]]; then
  echo "ERROR: BoringSSL tool didn'\''t report FIPS build."
  exit 1
fi

# Move compiled libraries to the expected destinations.
popd
mv $ROOT/boringssl/build/crypto/libcrypto.a bazel-out/k8-opt/bin/external/boringssl_fips/crypto/libcrypto.a
mv $ROOT/boringssl/build/ssl/libssl.a bazel-out/k8-opt/bin/external/boringssl_fips/ssl/libssl.a

bazel sync did not fetch all dependencies for it. I think it just fetch the usual dependencies but the not FIPS-specific ones. I don't believe there is a flag I could pass for it.

plallin · 2022-11-10T18:08:11Z

bazel sync logs (this always results in a failure even though offline build is later possible)

[root@ip-172-31-34-226 envoy]# CC=clang /usr/local/bin/bazel sync
ERROR: /home/envoy/WORKSPACE:13:19: fetching _clang_tools rule //external:clang_tools: java.io.IOException: _clang_tools rule //external:clang_tools must create a directory
DEBUG: Rule 'com_github_bufbuild_buf' indicated that a canonical reproducible form can be obtained by dropping arguments ["tags"]
DEBUG: Repository com_github_bufbuild_buf instantiated at:
/home/envoy/WORKSPACE:9:23: in <toplevel>
/home/envoy/bazel/api_repositories.bzl:4:21: in envoy_api_dependencies
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:47:26: in api_dependencies
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:9:23: in external_http_archive
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/envoy_http_archive.bzl:16:17: in envoy_http_archive
Repository rule http_archive defined at:
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
DEBUG: Rule 'wasmtime__wasmtime_c_api_macros__0_19_0' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1648756221 -0700"
DEBUG: Repository wasmtime__wasmtime_c_api_macros__0_19_0 instantiated at:
/home/envoy/WORKSPACE:17:25: in <toplevel>
/home/envoy/bazel/repositories_extra.bzl:31:33: in envoy_dependencies_extra
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/proxy_wasm_cpp_host/bazel/cargo/wasmtime/crates.bzl:744:10: in wasmtime_fetch_remote_crates
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule new_git_repository defined at:
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/git.bzl:186:37: in <toplevel>
ERROR: /home/envoy/WORKSPACE:13:19: fetching _default_envoy_dev rule //external:envoy_dev: java.io.IOException: _default_envoy_dev rule //external:envoy_dev must create a directory
INFO: Repository emscripten_npm_win instantiated at:
/home/envoy/WORKSPACE:21:25: in <toplevel>
/home/envoy/bazel/dependency_imports.bzl:48:20: in envoy_dependency_imports
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/emsdk/emscripten_deps.bzl:95:20: in emscripten_deps
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/build_bazel_rules_nodejs/index.bzl:78:17: in npm_install
Repository rule npm_install defined at:
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl:775:30: in <toplevel>
ERROR: An error occurred during the fetch of repository 'emscripten_npm_win':
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 752, column 13, in _npm_install_impl
fail("npm_install failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: npm_install failed: (npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile
npm notice
npm notice New major version of npm available! 7.20.3 -> 9.1.1
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.1.1>
npm notice Run npm install -g [email protected] to update!
npm notice
npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for [email protected]: wanted {"os":"win32","arch":"x64"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS: win32
npm ERR! notsup Valid Arch: x64
npm ERR! notsup Actual OS: linux
npm ERR! notsup Actual Arch: x64

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2022-11-10T14_53_33_780Z-debug.log
)
ERROR: /home/envoy/WORKSPACE:21:25: fetching npm_install rule //external:emscripten_npm_win: Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 752, column 13, in _npm_install_impl
fail("npm_install failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: npm_install failed: (npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile
npm notice
npm notice New major version of npm available! 7.20.3 -> 9.1.1
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.1.1>
npm notice Run npm install -g [email protected] to update!
npm notice
npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for [email protected]: wanted {"os":"win32","arch":"x64"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS: win32
npm ERR! notsup Valid Arch: x64
npm ERR! notsup Actual OS: linux
npm ERR! notsup Actual Arch: x64

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2022-11-10T14_53_33_780Z-debug.log
)
DEBUG: /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle/internal/go_repository.bzl:209:18: com_github_golang_mock: gazelle: finding module path for import a: go get: malformed module path "a": missing dot in first path element
gazelle: finding module path for import a: go get: malformed module path "a": missing dot in first path element
DEBUG: /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle/internal/go_repository.bzl:209:18: org_golang_x_crypto: gazelle: finding module path for import golang.org/x/sys/windows: finding module path for import golang.org/x/sys/windows: package golang.org/x/sys/windows: build constraints exclude all Go files in /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle_go_repository_cache/pkg/mod/golang.org/x/[email protected]/windows
DEBUG: /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle/internal/go_repository.bzl:209:18: org_golang_x_net: gazelle: finding module path for import golang.org/x/sys/windows: finding module path for import golang.org/x/sys/windows: package golang.org/x/sys/windows: build constraints exclude all Go files in /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle_go_repository_cache/pkg/mod/golang.org/x/[email protected]/windows
INFO: Repository emscripten_npm_mac instantiated at:
/home/envoy/WORKSPACE:21:25: in <toplevel>
/home/envoy/bazel/dependency_imports.bzl:48:20: in envoy_dependency_imports
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/emsdk/emscripten_deps.bzl:88:20: in emscripten_deps
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/build_bazel_rules_nodejs/index.bzl:78:17: in npm_install
Repository rule npm_install defined at:
/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl:775:30: in <toplevel>
ERROR: An error occurred during the fetch of repository 'emscripten_npm_mac':
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 752, column 13, in _npm_install_impl
fail("npm_install failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: npm_install failed: (npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile
npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"x64,x86,arm64"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS: darwin
npm ERR! notsup Valid Arch: x64,x86,arm64
npm ERR! notsup Actual OS: linux
npm ERR! notsup Actual Arch: x64

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2022-11-10T14_53_52_272Z-debug.log
)
ERROR: /home/envoy/WORKSPACE:21:25: fetching npm_install rule //external:emscripten_npm_mac: Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 752, column 13, in _npm_install_impl
fail("npm_install failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: npm_install failed: (npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile
npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"x64,x86,arm64"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS: darwin
npm ERR! notsup Valid Arch: x64,x86,arm64
npm ERR! notsup Actual OS: linux
npm ERR! notsup Actual Arch: x64

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2022-11-10T14_53_52_272Z-debug.log
)
DEBUG: /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle/internal/go_repository.bzl:209:18: co_honnef_go_tools: gazelle: finding module path for import CheckDeprecatedassist: go get: malformed module path "CheckDeprecatedassist": missing dot in first path element
DEBUG: /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle/internal/go_repository.bzl:209:18: org_golang_x_exp: gazelle: finding module path for import golang.org/x/sys/windows: finding module path for import golang.org/x/sys/windows: package golang.org/x/sys/windows: build constraints exclude all Go files in /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle_go_repository_cache/pkg/mod/golang.org/x/[email protected]/windows
gazelle: finding module path for import golang.org/x/sys/windows: finding module path for import golang.org/x/sys/windows: package golang.org/x/sys/windows: build constraints exclude all Go files in /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle_go_repository_cache/pkg/mod/golang.org/x/[email protected]/windows
DEBUG: /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle/internal/go_repository.bzl:209:18: com_github_census_instrumentation_opencensus_proto: gazelle: rule //gen-go/agent/metrics/v1:metrics imports "github.com/census-instrumentation/opencensus-proto/gen-go/agent/common/v1" which matches multiple rules: //gen-go/agent/common/v1:common and //src/opencensus/proto/agent/common/v1:common. # gazelle:resolve may be used to disambiguate
gazelle: rule //gen-go/agent/metrics/v1:metrics imports "github.com/census-instrumentation/opencensus-proto/gen-go/metrics/v1" which matches multiple rules: //gen-go/metrics/v1:metrics and //src/opencensus/proto/metrics/v1:metrics. # gazelle:resolve may be used to disambiguate
gazelle: rule //gen-go/agent/metrics/v1:metrics imports "github.com/census-instrumentation/opencensus-proto/gen-go/resource/v1" which matches multiple rules: //gen-go/resource/v1:resource and //src/opencensus/proto/resource/v1:resource. # gazelle:resolve may be used to disambiguate
gazelle: rule //gen-go/agent/trace/v1:trace imports "github.com/census-instrumentation/opencensus-proto/gen-go/agent/common/v1" which matches multiple rules: //gen-go/agent/common/v1:common and //src/opencensus/proto/agent/common/v1:common. # gazelle:resolve may be used to disambiguate
gazelle: rule //gen-go/agent/trace/v1:trace imports "github.com/census-instrumentation/opencensus-proto/gen-go/resource/v1" which matches multiple rules: //gen-go/resource/v1:resource and //src/opencensus/proto/resource/v1:resource. # gazelle:resolve may be used to disambiguate
gazelle: rule //gen-go/agent/trace/v1:trace imports "github.com/census-instrumentation/opencensus-proto/gen-go/trace/v1" which matches multiple rules: //gen-go/trace/v1:trace and //src/opencensus/proto/trace/v1:trace. # gazelle:resolve may be used to disambiguate
gazelle: rule //gen-go/metrics/v1:metrics imports "github.com/census-instrumentation/opencensus-proto/gen-go/resource/v1" which matches multiple rules: //gen-go/resource/v1:resource and //src/opencensus/proto/resource/v1:resource. # gazelle:resolve may be used to disambiguate
gazelle: rule //gen-go/trace/v1:trace imports "github.com/census-instrumentation/opencensus-proto/gen-go/resource/v1" which matches multiple rules: //gen-go/resource/v1:resource and //src/opencensus/proto/resource/v1:resource. # gazelle:resolve may be used to disambiguate
DEBUG: /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_gazelle/internal/go_repository.bzl:209:18: com_google_cloud_go: gazelle: finding module path for import google.golang.org/api/gensupport: go get: module google.golang.org/api@upgrade found (v0.103.0), but does not contain package google.golang.org/api/gensupport
gazelle: finding module path for import google.golang.org/api/gensupport: go get: module google.golang.org/api@upgrade found (v0.103.0), but does not contain package google.golang.org/api/gensupport
gazelle: finding module path for import google.golang.org/genproto/googleapis/cloud/videointelligence/v1beta1: go: downloading google.golang.org/genproto v0.0.0-20221109142239-94d6d90a7d66
go get: module google.golang.org/genproto@upgrade found (v0.0.0-20221109142239-94d6d90a7d66), but does not contain package google.golang.org/genproto/googleapis/cloud/videointelligence/v1beta1
gazelle: finding module path for import google.golang.org/genproto/googleapis/cloud/videointelligence/v1beta1: go: downloading google.golang.org/genproto v0.0.0-20221109142239-94d6d90a7d66
go get: module google.golang.org/genproto@upgrade found (v0.0.0-20221109142239-94d6d90a7d66), but does not contain package google.golang.org/genproto/googleapis/cloud/videointelligence/v1beta1
ERROR: npm_install failed: (npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile
npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"x64,x86,arm64"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS: darwin
npm ERR! notsup Valid Arch: x64,x86,arm64
npm ERR! notsup Actual OS: linux
npm ERR! notsup Actual Arch: x64

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2022-11-10T14_53_52_272Z-debug.log
)
Loading: loading...

plallin · 2022-11-10T18:09:37Z

Logs on trying to use the the repository_cache from bazel sync as Envoy's distdir:

[root@ip-172-31-39-46 envoy]# cd /home/envoy && CC=clang /usr/local/bin/bazel build --config=clang --verbose_failures --distdir=/home/envoy-sync-deps -c opt envoy
INFO: Repository com_google_googleapis instantiated at:
  /home/envoy/WORKSPACE:9:23: in <toplevel>
  /home/envoy/bazel/api_repositories.bzl:4:21: in envoy_api_dependencies
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:22:26: in api_dependencies
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:9:23: in external_http_archive
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/envoy_http_archive.bzl:16:17: in envoy_http_archive
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
WARNING: Download from https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz failed: class java.io.IOException connect timed out
ERROR: An error occurred during the fetch of repository 'com_google_googleapis':
   Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 100, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp15480540142575535956/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out
ERROR: /home/envoy/WORKSPACE:9:23: fetching http_archive rule //external:com_google_googleapis: Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 100, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp15480540142575535956/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out
INFO: Reading rc options for 'build' from /home/envoy/.bazelrc:
  'build' options: --color=yes --workspace_status_command=bash bazel/get_workspace_status --incompatible_strict_action_env --host_force_python=PY3 --java_runtime_version=remotejdk_11 --tool_java_runtime_version=remotejdk_11 --platform_mappings=bazel/platform_mappings --enable_platform_specific_config --define absl=1 --action_env=CC --action_env=CXX --action_env=LLVM_CONFIG --action_env=PATH --@com_googlesource_googleurl//build_config:system_icu=0 --test_env=HEAPCHECK=normal --test_env=PPROF_PATH
ERROR: @com_googlesource_googleurl//build_config:system_icu :: Error loading option @com_googlesource_googleurl//build_config:system_icu: no such package '@com_google_googleapis//': java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp15480540142575535956/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out

phlax · 2022-11-10T18:11:40Z

re the errors syncing it might be worth bumping that to its own issue - not sure of the problem - but looks like something with our setup - not sure

phlax · 2022-11-10T18:12:27Z

re issues with using distdir - putting above errors aside - im wondering if its because the api depdendencies are in their own (bazel) namespace

lizan · 2022-11-10T23:57:44Z

The key part for building FIPS mode in offline is https://github.com/envoyproxy/envoy/blob/main/bazel/external/boringssl_fips.genrule_cmd

Those curl command are not managed by Bazel, so bazel sync doesn't know anything about those dependencies. Without patching this file it won't work.

plallin · 2022-11-11T00:30:47Z

That modifies the output of envoy --version ; if you build from a clean branch then the output of envoy --version will state that (documented here)

plallin · 2022-11-11T11:44:58Z

And additionally even for non-FIPS I am not managing to make offline builds work unless I am fetching the dependencies on the same node as I run the build on. Fetching the dependencies and storing them in some file system (such as mounted EBS) for reuse by offline nodes does not work.

On advice from Phlax I was using this issue which might help for offline builds. This thread mentions the use of --override_repository to achieve offline builds; this also doesn't work:

Attempt using --distdir from an offline node where the dependencies from bazel sync where mounted:

[root@ip-172-31-36-47 envoy]# CC=clang /usr/local/bin/bazel build --config=clang --verbose_failures --distdir=/home/envoy-sync-deps/ -c opt envoy
INFO: Repository com_google_googleapis instantiated at:
  /home/envoy/WORKSPACE:9:23: in <toplevel>
  /home/envoy/bazel/api_repositories.bzl:4:21: in envoy_api_dependencies
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:22:26: in api_dependencies
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:9:23: in external_http_archive
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/envoy_http_archive.bzl:16:17: in envoy_http_archive
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
WARNING: Download from https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz failed: class java.io.IOException connect timed out
ERROR: An error occurred during the fetch of repository 'com_google_googleapis':
   Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 100, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp2439276146851912679/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out
ERROR: /home/envoy/WORKSPACE:9:23: fetching http_archive rule //external:com_google_googleapis: Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 100, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp2439276146851912679/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out
INFO: Reading rc options for 'build' from /home/envoy/.bazelrc:
  'build' options: --color=yes --workspace_status_command=bash bazel/get_workspace_status --incompatible_strict_action_env --host_force_python=PY3 --java_runtime_version=remotejdk_11 --tool_java_runtime_version=remotejdk_11 --platform_mappings=bazel/platform_mappings --enable_platform_specific_config --define absl=1 --action_env=CC --action_env=CXX --action_env=LLVM_CONFIG --action_env=PATH --@com_googlesource_googleurl//build_config:system_icu=0 --test_env=HEAPCHECK=normal --test_env=PPROF_PATH
ERROR: @com_googlesource_googleurl//build_config:system_icu :: Error loading option @com_googlesource_googleurl//build_config:system_icu: no such package '@com_google_googleapis//': java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp2439276146851912679/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out

Attempt using both --distdir and --override_repository from an offline node where the dependencies from bazel sync where mounted:

[root@ip-172-31-36-47 envoy]# CC=clang /usr/local/bin/bazel build --config=clang --verbose_failures --distdir=/home/envoy-sync-deps/ --override_repository=repository-name=/home/envoy-sync-deps/ -c opt envoy
INFO: Repository com_google_googleapis instantiated at:
  /home/envoy/WORKSPACE:9:23: in <toplevel>
  /home/envoy/bazel/api_repositories.bzl:4:21: in envoy_api_dependencies
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:22:26: in api_dependencies
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:9:23: in external_http_archive
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/envoy_http_archive.bzl:16:17: in envoy_http_archive
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
WARNING: Download from https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz failed: class java.io.IOException connect timed out
ERROR: An error occurred during the fetch of repository 'com_google_googleapis':
   Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 100, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp16523393607368485334/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out
ERROR: /home/envoy/WORKSPACE:9:23: fetching http_archive rule //external:com_google_googleapis: Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 100, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp16523393607368485334/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out
INFO: Reading rc options for 'build' from /home/envoy/.bazelrc:
  'build' options: --color=yes --workspace_status_command=bash bazel/get_workspace_status --incompatible_strict_action_env --host_force_python=PY3 --java_runtime_version=remotejdk_11 --tool_java_runtime_version=remotejdk_11 --platform_mappings=bazel/platform_mappings --enable_platform_specific_config --define absl=1 --action_env=CC --action_env=CXX --action_env=LLVM_CONFIG --action_env=PATH --@com_googlesource_googleurl//build_config:system_icu=0 --test_env=HEAPCHECK=normal --test_env=PPROF_PATH
ERROR: @com_googlesource_googleurl//build_config:system_icu :: Error loading option @com_googlesource_googleurl//build_config:system_icu: no such package '@com_google_googleapis//': java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp16523393607368485334/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out

Attempt using just --override_repository:

[root@ip-172-31-36-47 envoy]# CC=clang /usr/local/bin/bazel build --config=clang --verbose_failures --override_repository=repository-name=/home/envoy-sync-deps/ -c opt envoy
INFO: Repository com_google_googleapis instantiated at:
  /home/envoy/WORKSPACE:9:23: in <toplevel>
  /home/envoy/bazel/api_repositories.bzl:4:21: in envoy_api_dependencies
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:22:26: in api_dependencies
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/repositories.bzl:9:23: in external_http_archive
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/envoy_api/bazel/envoy_http_archive.bzl:16:17: in envoy_http_archive
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
WARNING: Download from https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz failed: class java.io.IOException connect timed out
ERROR: An error occurred during the fetch of repository 'com_google_googleapis':
   Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 100, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp1594140329204263211/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out
ERROR: /home/envoy/WORKSPACE:9:23: fetching http_archive rule //external:com_google_googleapis: Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 100, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp1594140329204263211/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out
INFO: Reading rc options for 'build' from /home/envoy/.bazelrc:
  'build' options: --color=yes --workspace_status_command=bash bazel/get_workspace_status --incompatible_strict_action_env --host_force_python=PY3 --java_runtime_version=remotejdk_11 --tool_java_runtime_version=remotejdk_11 --platform_mappings=bazel/platform_mappings --enable_platform_specific_config --define absl=1 --action_env=CC --action_env=CXX --action_env=LLVM_CONFIG --action_env=PATH --@com_googlesource_googleurl//build_config:system_icu=0 --test_env=HEAPCHECK=normal --test_env=PPROF_PATH
ERROR: @com_googlesource_googleurl//build_config:system_icu :: Error loading option @com_googlesource_googleurl//build_config:system_icu: no such package '@com_google_googleapis//': java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /root/.cache/bazel/_bazel_root/fc68cbdaae275f76493dda1f31bdf65a/external/com_google_googleapis/temp1594140329204263211/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: connect timed out

plallin · 2022-11-11T15:36:42Z

On reading the documentation more, bazel sync downloads the dependencies to the output directory ; I'm still not quite sure what --distdir is used for but it's unsufficient for offline builds.

If the output directory is moved or shared with an offline instance, then it's possible to (1) use a middlebox to run bazel sync, (2) save the output directory, and (3) mount the output directory to the offline instance and run bazeul build successfully.

Obviously this is only valid for non-FIPS Envoy, as FIPS-Envoy would need to be patched as indicated by @lizan

cshah04 · 2022-11-21T16:48:31Z

Facing a similar issue when building for FIPS mode
@plallin were you able to build it successfully ?

Repository rule http_archive defined at: /build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel> WARNING: Download from https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz failed: class java.net.SocketException Unexpected end of file from server ERROR: An error occurred during the fetch of repository 'com_google_googleapis': Traceback (most recent call last): File "/build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 45, in _http_archive_impl download_info = ctx.download_and_extract( Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/com_google_googleapis/temp600748501653445832/82944da21578a53b74e547774cf62ed31a05b[841](https://code.pan.run/pandb/pandb-infra/mica/infra/hyrule-envoy/-/jobs/18623666#L841).tar.gz: Unexpected end of file from server ERROR: Error fetching repository: Traceback (most recent call last): File "/build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 45, in _http_archive_impl download_info = ctx.download_and_extract( Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/com_google_googleapis/temp600748501653445832/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: Unexpected end of file from server ERROR: no such package '@com_google_googleapis//': java.io.IOException: Error downloading [https://github.com/googleapis/googleapis/archive/82944da21578a53b74e547774cf62ed31a05b841.tar.gz] to /build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/com_google_googleapis/temp600748501653445832/82944da21578a53b74e547774cf62ed31a05b841.tar.gz: Unexpected end of file from server INFO: Elapsed time: 256.326s INFO: 0 processes. error building image: error building stage: failed to execute command: waiting for process to exit: exit status 1 Running after_script 00:00 Running after script... $ echo "END" END

plallin · 2022-11-21T17:01:31Z

So I'm not sure what exactly is your set up so I have to take assumptions...

So my guess here is you are behind a proxy and you've edited the repository_locations.bzl to point to your proxy. This does not work, because of recursive dependencies. You get over the first layer of dependencies (the ones from repository_locations.bzl) and then those dependencies download more dependencies and those aren't set up to use your proxy, they go straight to the Internet, and since this is blocked, it fails.

Alternatively you nay have done none no changes to repository_locations.bzl and so Bazel tries to download dependencies from then Internet from a server with no Internet access and it fails.

Either way I don't think your issue is with FIPS; if you try to build Envoy without the FIPS flag it's likely to fail as well.

If you are using proxies / offline nodes don't forget to patch https://github.com/envoyproxy/envoy/blob/main/bazel/external/boringssl_fips.genrule_cmd as otherwise this goes stright to the Internet

cshah04 · 2022-11-21T17:29:06Z

I tried both ways with proxy and without proxy, in both ways I had the same issue. And I even tried with the non-FIPS mode and you were right, I got similar issue. Anyway, I tried with your the above patch and still having issues, Do I need to pass any arguments ?

root@f610cee9e396:/source# ./ci/do_ci.sh bazel.sizeopt.server_only
No remote cache is set, skipping setup remote cache.
ENVOY_SRCDIR=/source
ENVOY_BUILD_TARGET=//source/exe:envoy-static
ENVOY_BUILD_ARCH=x86_64
2022/11/21 17:17:38 Downloading https://releases.bazel.build/4.1.0/release/bazel-4.1.0-linux-x86_64...
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
Cloning into '/build/envoy-filter-example'...
remote: Enumerating objects: 21580, done.
remote: Counting objects: 100% (188/188), done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 21580 (delta 186), reused 124 (delta 122), pack-reused 21392
Receiving objects: 100% (21580/21580), 1.99 MiB | 7.22 MiB/s, done.
Resolving deltas: 100% (21366/21366), done.
Note: checking out 'ac6a66a4a1c08138ccc03c23aafc9637b2df55a1'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at ac6a66a4 Changes to make example buildable with new Envoy include paths (#149)
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
building using 6 CPUs
building for x86_64
clang toolchain with libc++ configured
bazel size optimized build...
Building (type=sizeopt target=//source/exe:envoy-static debug=//source/exe:envoy-static.dwp name=envoy)...
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
INFO: SHA256 (https://golang.org/dl/?mode=json&include=all) = fd2453d8584506176b8e5bdc858eaabf8bb4ae778d98f4ca2d1cfa22a65c9ecf
ERROR: /build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/boringssl_fips/BUILD.bazel:28:8: in cmd attribute of genrule rule @boringssl_fips//:build: $(dirname) not defined
INFO: Repository com_github_mirror_tclap instantiated at:
  /source/WORKSPACE:13:19: in <toplevel>
  /source/bazel/repositories.bzl:182:29: in envoy_dependencies
  /source/bazel/repositories.bzl:327:26: in _com_github_mirror_tclap
  /source/bazel/repositories.bzl:29:23: in external_http_archive
  /build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/envoy_api/bazel/envoy_http_archive.bzl:16:17: in envoy_http_archive
Repository rule http_archive defined at:
  /build/tmp/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
ERROR: Analysis of target '//source/exe:envoy-static' failed; build aborted: Analysis of target '@boringssl_fips//:build' failed
INFO: Elapsed time: 60.383s
INFO: 0 processes.

plallin · 2022-11-21T18:21:45Z

So it appears the server you are running this from does not have Internet access. You could try the following curl to confirm:

curl -vvv https://github.com/googleapis/googleapis/

As far as I'm aware this is not doable. I have tried for a while to build envoy "offline" or behind proxies and this always fail due to recursive dependencies.

I don't know how to fix it - and if I'm honest I don't think this can be fixed.

I would recommend that you work towards being allowed to use servers with access to the Internet, else I don't think you will be able to build Envoy.

I would be absolutely delighted to be wrong :)

phlax · 2022-11-21T18:27:24Z

i might be wrong but i think the issue is that you are only fetching the envoy deps not the api ones

not sure exactly how bazel sync works - but i would try running it on/from the api dir if poss

cshah04 · 2022-11-22T16:01:54Z

The server I am running this does have internet access & I tried the above command and was able to get the content.
But somehow when I run the bazel command, something goes wrong and I end up with the above errors

github-actions · 2022-12-22T20:01:14Z

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

github-actions · 2022-12-29T20:01:30Z

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

tabacco added enhancement Feature requests. Not bugs or questions. triage Issue requires triage labels Oct 3, 2022

ggreenway added area/ci and removed triage Issue requires triage labels Oct 4, 2022

github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Nov 3, 2022

phlax removed the stale stalebot believes this issue/PR has not been touched recently label Nov 3, 2022

github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Dec 22, 2022

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Dec 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FIPS-compliant builds on dockerhub #23353

FIPS-compliant builds on dockerhub #23353

tabacco commented Oct 3, 2022

github-actions bot commented Nov 3, 2022

phlax commented Nov 3, 2022

tomscarlato commented Nov 7, 2022

plallin commented Nov 7, 2022

plallin commented Nov 10, 2022

plallin commented Nov 10, 2022

plallin commented Nov 10, 2022

plallin commented Nov 10, 2022

phlax commented Nov 10, 2022

phlax commented Nov 10, 2022

lizan commented Nov 10, 2022

plallin commented Nov 11, 2022

plallin commented Nov 11, 2022

plallin commented Nov 11, 2022

cshah04 commented Nov 21, 2022

plallin commented Nov 21, 2022

cshah04 commented Nov 21, 2022 •

edited

Loading

plallin commented Nov 21, 2022

phlax commented Nov 21, 2022

cshah04 commented Nov 22, 2022 •

edited

Loading

github-actions bot commented Dec 22, 2022

github-actions bot commented Dec 29, 2022

FIPS-compliant builds on dockerhub #23353

FIPS-compliant builds on dockerhub #23353

Comments

tabacco commented Oct 3, 2022

github-actions bot commented Nov 3, 2022

phlax commented Nov 3, 2022

tomscarlato commented Nov 7, 2022

plallin commented Nov 7, 2022

plallin commented Nov 10, 2022

Case 1: Building FIPS Envoy (All traffic enabled)

Case 2: Build Envoy without FIPS compliance, (almost) offline

Case 3: Build Envoy with FIPS compliance, (kind of) offline

plallin commented Nov 10, 2022

plallin commented Nov 10, 2022

plallin commented Nov 10, 2022

phlax commented Nov 10, 2022

phlax commented Nov 10, 2022

lizan commented Nov 10, 2022

plallin commented Nov 11, 2022

plallin commented Nov 11, 2022

plallin commented Nov 11, 2022

cshah04 commented Nov 21, 2022

plallin commented Nov 21, 2022

cshah04 commented Nov 21, 2022 • edited Loading

plallin commented Nov 21, 2022

phlax commented Nov 21, 2022

cshah04 commented Nov 22, 2022 • edited Loading

github-actions bot commented Dec 22, 2022

github-actions bot commented Dec 29, 2022

cshah04 commented Nov 21, 2022 •

edited

Loading

cshah04 commented Nov 22, 2022 •

edited

Loading