s3 zip includes files that should be matched by the exclude setting

[waynr]

Context

I am attempting to deploy a Flask app as an AWS Lambda + API Gateway but pretty am consistently seeing way too many files I don't want in the zip produced by the package command. The specific files I want to include are all those found in the .tox directory of my git repo.

My working directory has a file named zappa_settings.json with the following contents:

{
    "pytest": {
        "app_function": "autoapp.app",
        "aws_region": "us-west-2",
        "exclude": [".tox/*", "tmp/*"],
        "profile_name": "default",
        "s3_bucket": "zappa-wnwh277zd",
        "slim_handler": "true",
    }
}

I happen to be using ZappaCLI programmatically for some pytest fixtures I am working on, but the problem is also reproducible when running zappa package pytest

I chose to use .tox/* because that is how I see it done in zappa itself with .git/*:
https://github.com/Miserlou/Zappa/blob/master/zappa/core.py#L174

And I can confirm that .git/* does the trick since I don't see any paths in the resulting archive starting with .git/

Expected Behavior

I shouldn't see any paths starting with .tox/ in the archive that zappa produces.

Actual Behavior

I see a metric shit ton of paths starting with .tox/ in the archive that zappa produces.

Possible Fix

For my specific use case it will probably make sense to add .tox/* to zappa.core.ZIP_EXCLUDES.

Steps to Reproduce

• make a bunch of tox environments in a project
• add .tox/* to the exclude list in your zappa settings
• run zappa package <envname>

Your Environment

• Zappa version used: o.43.2
• Operating System and Python version: Debian 9 and Python 3.6.2
• The output of pip freeze:

argcomplete==1.8.2
base58==0.2.4
beautifulsoup4==4.6.0
boto3==1.4.5
botocore==1.5.40
bumpversion==0.5.3
cached-property==1.3.0
certifi==2017.7.27.1
chardet==3.0.4
click==6.7
colorama==0.3.9
decorator==4.1.2
docker==2.5.1
docker-compose==1.15.0
docker-pycreds==0.2.1
dockerpty==0.4.1
docopt==0.6.2
docutils==0.14
durationpy==0.5
flake8==2.6.0
Flask==0.12.2
flask-dynamo==0.1.1
future==0.16.0
futures==3.1.1
hjson==3.0.0
idna==2.6
ipython==6.1.0
ipython-genutils==0.2.0
itsdangerous==0.24
jedi==0.10.2
Jinja2==2.9.6
jmespath==0.9.3
jsonschema==2.6.0
kappa==0.6.0
lambda-packages==0.16.1
MarkupSafe==1.0
mccabe==0.5.3
pexpect==4.2.1
pickleshare==0.7.4
pkg-resources==0.0.0
placebo==0.8.1
pluggy==0.3.1
prompt-toolkit==1.0.15
ptyprocess==0.5.2
py==1.4.34
pycodestyle==2.0.0
pyflakes==1.2.3
Pygments==2.2.0
pytest==3.1.3
pytest-runner==2.11.1
python-dateutil==2.6.1
python-slugify==1.2.4
PyYAML==3.12
requests==2.18.4
s3transfer==0.1.10
simplegeneric==0.8.1
six==1.10.0
texttable==0.8.8
toml==0.9.2
tox==2.3.1
tqdm==4.15.0
traitlets==4.3.2
troposphere==1.9.5
Unidecode==0.4.21
urllib3==1.22
virtualenv==15.1.0
waitress==1.0.2
wcwidth==0.1.7
WebOb==1.7.3
websocket-client==0.44.0
WebTest==2.0.28
Werkzeug==0.12
wsgi-request-logger==0.4.6
zappa==0.43.2

• Link to your project (optional):
• Your zappa_settings.py:

{
    "pytest": {
        "app_function": "autoapp.app",
        "aws_region": "us-west-2",
        "exclude": [".tox/*", "tmp/*"],
        "profile_name": "default",
        "s3_bucket": "zappa-wnwh277zd",
        "slim_handler": "true",
    }
}

[waynr]

So it looks like the correct way to exclude a directory is just to give its name without any glob character, as in .tox vs .tox/*. My problem is that i had been editing the wrong staging env in my zappa settings file early on and ended up attempting to follow the example shown in zappa.core.ZIP_EXCLUDES. I would submit a patch to remove the misleading example but it's hard to say with the jumble of code in this project and relatively slim testing if that would cause problems for someone else. Gonna close this issue.

[Bartvds]

The glob syntax does nothing for me either, and the docs mention exclude uses regex patterns (not glob), but I couldn't get that to work as expected. Using the plain directory names worked though.

[Miserlou]

Reopening, seems like this is a documentation bug?

[Bartvds]

@Miserlou I guess it depends on what is intended. The texts says regex, the example shows globs, and the code does it's own thing. (personally I'd say globs would be ideal).

[lamiru]

Same issue here