Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove relay extension from AWS layer #2068

Merged
merged 3 commits into from
May 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 0 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,18 +68,6 @@ jobs:
pip install virtualenv
# This will also trigger "make dist" that creates the Python packages
make aws-lambda-layer

echo "Saving SDK_VERSION for later"
export SDK_VERSION=$(grep "VERSION = " sentry_sdk/consts.py | cut -f3 -d' ' | tr -d '"')
echo "SDK_VERSION=$SDK_VERSION"
echo "SDK_VERSION=$SDK_VERSION" >> $GITHUB_ENV
- name: Upload Python AWS Lambda Layer
uses: getsentry/action-build-aws-lambda-extension@v1
sl0thentr0py marked this conversation as resolved.
Show resolved Hide resolved
with:
artifact_name: ${{ github.sha }}
zip_file_name: sentry-python-serverless-${{ env.SDK_VERSION }}.zip
build_cache_paths: ${{ env.CACHED_BUILD_PATHS }}
build_cache_key: ${{ env.BUILD_CACHE_KEY }}
- name: Upload Python Packages
uses: actions/upload-artifact@v3
with:
Expand Down
78 changes: 78 additions & 0 deletions .github/workflows/test-integration-loguru.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Test loguru

on:
push:
branches:
- master
- release/**

pull_request:

# Cancel in progress workflows on pull_requests.
# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

permissions:
contents: read

env:
BUILD_CACHE_KEY: ${{ github.sha }}
CACHED_BUILD_PATHS: |
${{ github.workspace }}/dist-serverless

jobs:
test:
name: loguru, python ${{ matrix.python-version }}, ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 45

strategy:
fail-fast: false
matrix:
python-version: ["3.5","3.6","3.7","3.8","3.9","3.10","3.11"]
# python3.6 reached EOL and is no longer being supported on
# new versions of hosted runners on Github Actions
# ubuntu-20.04 is the last version that supported python3.6
# see https://github.com/actions/setup-python/issues/544#issuecomment-1332535877
os: [ubuntu-20.04]

steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Setup Test Env
run: |
pip install coverage "tox>=3,<4"

- name: Test loguru
timeout-minutes: 45
shell: bash
run: |
set -x # print commands that are executed
coverage erase

# Run tests
./scripts/runtox.sh "py${{ matrix.python-version }}-loguru" --cov=tests --cov=sentry_sdk --cov-report= --cov-branch
coverage combine .coverage*
coverage xml -i

- uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.xml

check_required_tests:
name: All loguru tests passed or skipped
needs: test
# Always run this, even if a dependent job failed
if: always()
runs-on: ubuntu-20.04
steps:
- name: Check for failures
if: contains(needs.test.result, 'failure')
run: |
echo "One of the dependent jobs have failed. You may need to re-run it." && exit 1
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ help:

dist: .venv
rm -rf dist dist-serverless build
sl0thentr0py marked this conversation as resolved.
Show resolved Hide resolved
$(VENV_PATH)/bin/pip install wheel
$(VENV_PATH)/bin/python setup.py sdist bdist_wheel
.PHONY: dist

Expand Down
1 change: 1 addition & 0 deletions linter-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ types-certifi
types-redis
types-setuptools
pymongo # There is no separate types module.
loguru # There is no separate types module.
flake8-bugbear==22.12.6
pep8-naming==0.13.2
pre-commit # local linting
2 changes: 1 addition & 1 deletion scripts/aws-delete-lamba-layer-versions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ set -euo pipefail
# override default AWS region
export AWS_REGION=eu-central-1

LAYER_NAME=SentryPythonServerlessSDKLocalDev
LAYER_NAME=SentryPythonServerlessSDK-local-dev
sl0thentr0py marked this conversation as resolved.
Show resolved Hide resolved
VERSION="0"

while [[ $VERSION != "1" ]]
Expand Down
47 changes: 6 additions & 41 deletions scripts/aws-deploy-local-layer.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,55 +9,20 @@
set -euo pipefail

# Creating Lambda layer
echo "Creating Lambda layer in ./dist-serverless ..."
echo "Creating Lambda layer in ./dist ..."
make aws-lambda-layer
echo "Done creating Lambda layer in ./dist-serverless."

# IMPORTANT:
# Please make sure that this part does the same as the GitHub action that
# is building the Lambda layer in production!
# see: https://github.com/getsentry/action-build-aws-lambda-extension/blob/main/action.yml#L23-L40

echo "Downloading relay..."
mkdir -p dist-serverless/relay
curl -0 --silent \
--output dist-serverless/relay/relay \
"$(curl -s https://release-registry.services.sentry.io/apps/relay/latest | jq -r .files.\"relay-Linux-x86_64\".url)"
chmod +x dist-serverless/relay/relay
echo "Done downloading relay."

echo "Creating start script..."
mkdir -p dist-serverless/extensions
cat > dist-serverless/extensions/sentry-lambda-extension << EOT
#!/bin/bash
set -euo pipefail
exec /opt/relay/relay run \
--mode=proxy \
--shutdown-timeout=2 \
--upstream-dsn="\$SENTRY_DSN" \
--aws-runtime-api="\$AWS_LAMBDA_RUNTIME_API"
EOT
chmod +x dist-serverless/extensions/sentry-lambda-extension
echo "Done creating start script."

# Zip Lambda layer and included Lambda extension
echo "Zipping Lambda layer and included Lambda extension..."
cd dist-serverless/
zip -r ../sentry-python-serverless-x.x.x-dev.zip \
. \
--exclude \*__pycache__\* --exclude \*.yml
cd ..
echo "Done Zipping Lambda layer and included Lambda extension to ./sentry-python-serverless-x.x.x-dev.zip."

echo "Done creating Lambda layer in ./dist"

# Deploying zipped Lambda layer to AWS
echo "Deploying zipped Lambda layer to AWS..."
ZIP=$(ls dist | grep serverless | head -n 1)
echo "Deploying zipped Lambda layer $ZIP to AWS..."

aws lambda publish-layer-version \
--layer-name "SentryPythonServerlessSDK-local-dev" \
--region "eu-central-1" \
--zip-file "fileb://sentry-python-serverless-x.x.x-dev.zip" \
--zip-file "fileb://dist/$ZIP" \
--description "Local test build of SentryPythonServerlessSDK (can be deleted)" \
--compatible-runtimes python3.6 python3.7 python3.8 python3.9
--no-cli-pager

echo "Done deploying zipped Lambda layer to AWS as 'SentryPythonServerlessSDK-local-dev'."
Expand Down
28 changes: 24 additions & 4 deletions scripts/build_aws_lambda_layer.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ def __init__(
# type: (...) -> None
self.base_dir = base_dir
self.python_site_packages = os.path.join(self.base_dir, PYTHON_SITE_PACKAGES)
self.out_zip_filename = f"sentry-python-serverless-{SDK_VERSION}.zip"

def make_directories(self):
# type: (...) -> None
Expand Down Expand Up @@ -57,16 +58,35 @@ def create_init_serverless_sdk_package(self):
"scripts/init_serverless_sdk.py", f"{serverless_sdk_path}/__init__.py"
)

def zip(self):
# type: (...) -> None
subprocess.run(
[
"zip",
"-q", # Quiet
"-x", # Exclude files
"**/__pycache__/*", # Files to be excluded
"-r", # Recurse paths
self.out_zip_filename, # Output filename
PYTHON_SITE_PACKAGES, # Files to be zipped
],
cwd=self.base_dir,
check=True, # Raises CalledProcessError if exit status is non-zero
)

def build_layer_dir():
shutil.copy(
os.path.join(self.base_dir, self.out_zip_filename),
os.path.abspath(DIST_PATH)
)

def build_packaged_zip():
with tempfile.TemporaryDirectory() as base_dir:
layer_builder = LayerBuilder(base_dir)
layer_builder.make_directories()
layer_builder.install_python_packages()
layer_builder.create_init_serverless_sdk_package()

shutil.copytree(base_dir, "dist-serverless")
layer_builder.zip()


if __name__ == "__main__":
build_layer_dir()
build_packaged_zip()
10 changes: 1 addition & 9 deletions scripts/init_serverless_sdk.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,9 @@
from typing import Any


def extension_relay_dsn(original_dsn):
dsn = Dsn(original_dsn)
dsn.host = "localhost"
dsn.port = 5333
dsn.scheme = "http"
return str(dsn)


# Configure Sentry SDK
sentry_sdk.init(
dsn=extension_relay_dsn(os.environ["SENTRY_DSN"]),
dsn=os.environ["SENTRY_DSN"],
integrations=[AwsLambdaIntegration(timeout_warning=True)],
traces_sample_rate=float(os.environ["SENTRY_TRACES_SAMPLE_RATE"]),
)
Expand Down
Loading