Merge pull request #1583 from awslabs/develop

Release v0.33.0

.pylintrc

@@ -59,7 +59,7 @@ confidence=
 # --enable=similarities". If you want to run only the classes checker, but have
 # no Warning level messages displayed, use"--disable=all --enable=classes
 # --disable=W"
-disable=R0201,W0613,I0021,I0020,W1618,W1619,R0902,R0903,W0231,W0611,R0913,W0703,C0330,R0204,I0011,R0904,C0301
+disable=R0201,W0613,W0640,I0021,I0020,W1618,W1619,R0902,R0903,W0231,W0611,R0913,W0703,C0330,R0204,I0011,R0904,R0914,C0301
 
 
 [REPORTS]

appveyor.yml

@@ -61,18 +61,8 @@ for:
       - "pylint --rcfile .pylintrc samcli"
       # There are some functional tests that are currently broken due to not being updated with changed code or still running with node4.3 runtimes
       # We need to update those but this allows us to at least runs the ones we currently have working
-      - "pytest tests/functional/commands/validate tests/functional/commands/cli/test_global_config.py"
+      - "pytest -n 4 tests/functional/commands/validate tests/functional/commands/cli/test_global_config.py"
 
-      # Runs only in Linux
-      - sh: "pytest -vv tests/integration"
-      - sh: "/tmp/black --check setup.py tests samcli scripts"
-      - sh: "python scripts/check-isolated-needs-update.py"
-
-      # Smoke tests run in parallel - it runs on both Linux & Windows
-      # Presence of the RUN_SMOKE envvar will run the smoke tests
-      # Note: temporarily removing as with current dependencies we require syslog on windows
-      # which is not present on stdlib.
-      # - ps: "If ($env:RUN_SMOKE) {pytest -n 4 -vv tests/smoke}"
   - 
     matrix:
       only:
@@ -128,11 +118,11 @@ for:
       - "pylint --rcfile .pylintrc samcli"
       # There are some functional tests that are currently broken due to not being updated with changed code or still running with node4.3 runtimes
       # We need to update those but this allows us to at least runs the ones we currently have working
-      - "pytest tests/functional/commands/validate tests/functional/commands/cli/test_global_config.py"
+      - "pytest -n 4 tests/functional/commands/validate tests/functional/commands/cli/test_global_config.py"
 
       # Runs only in Linux
       - sh: "pytest -vv tests/integration"
-      - sh: "pytest -vv -n 4 tests/regression"
+      - sh: "pytest -vv tests/regression"
       - sh: "/tmp/black --check setup.py tests samcli scripts"
 
       # Set JAVA_HOME to java11
@@ -142,13 +132,3 @@ for:
       # Smoke tests run in parallel - it runs on both Linux & Windows
       # Presence of the RUN_SMOKE envvar will run the smoke tests
       - ps: "If ($env:RUN_SMOKE) {pytest -n 4 -vv tests/smoke}"
-
-  - 
-    matrix:
-      only:
-        - ONLY_SMOKE: 1
-
-    test_script:
-      # Smoke tests run in parallel
-      - sh: "venv/Scripts/activate"
-      - sh: "pytest -n 4 -vv tests/smoke"

designs/package_during_deploy.md

@@ -0,0 +1,178 @@
+Package during `sam deploy`
+====================================
+
+
+What is the problem?
+--------------------
+
+Today while using `sam deploy` the specified `--template-file` or `--template` is expected to have packaged artifacts references in the given template file.
+
+This is driven by the following workflow.
+
+`sam build` -> `sam package --s3-bucket ... --s3-prefix ... --output-template-file packaged.yaml`.
+
+This workflow builds the requisite code and packages those built artifacts into an s3 bucket, optionally under a given s3 prefix.
+
+If a developer can optionally cut through this process without requiring an explicit `package` command, but rather have `sam deploy` package to a given s3 bucket, it cuts the number of steps before needing to deploy and test in the cloud.
+
+This also reduces friction in the `author` and `test` loop.
+
+Ideal end state.
+
+`sam build` -> `sam deploy ..... --s3-bucket ....`
+
+
+What will be changed?
+---------------------
+
+Addition of extra parameters that are currently supported by `sam package` over to `sam deploy` with all of them being optional.
+
+Additional parameters that need to be on `sam deploy` that are not on `sam package`.
+
+* `--metadata`
+* `--use-json`
+
+Parameters that dont need to be added.
+
+* `--output-template-file`
+ * 	An explicit `output-template-file` is created on the fly during packaging in the deploy phase.
+
+If the expectation is to package and deploy in one command, One can now do.
+
+`sam deploy --stack-name sam-package-on-deploy --capabilities CAPABILITY_IAM --s3-bucket sam-package-bucket`
+
+There is no explicit need to pass in a `--template-file` or `--template` parameter, if one is not passed in it to defaults to trying to find the template.yaml that was generated by `sam build` which is located at `.aws-sam/build/template.yaml`
+
+The old method of deploying pre-packaged artifacts will continue to work as before.
+
+* `sam deploy --template-file packaged.yaml --stack-name sam-package-on-deploy --capabilities  CAPABILITY_IAM`
+
+If a deployment is done without running `sam build` prior we still go ahead and deploy with the given `template.yaml` in the project root. This might still result in a successful deploy, but not a deploy with the correct build artifacts.
+
+
+Future of `sam package`?
+---------------------
+
+* `sam package` will continue to exist in the state it is today and will continue to be improved upon separately.
+
+Success criteria for the change
+-------------------------------
+
+User do not require to run `sam package` as part of their author test loop, except for CI/CD purposes, where `sam package` can be run and the packaged template file can be passed to cloudformation deploy actions.
+
+
+Out-of-Scope
+------------
+
+The s3 bucket where the packaged artifacts go is not abstracted in this design. In the future, the s3 bucket could be specified via a configuration file.
+
+This is currently in design in : https://github.com/awslabs/aws-sam-cli/pull/1503
+
+User Experience Walkthrough
+---------------------------
+
+`sam build` -> `sam deploy`
+
+`sam build` -> `sam package` -> `sam deploy`
+
+Provide identical experiences in terms of a deploying the same stack, with exactly same artifacts.
+
+
+Implementation
+==============
+
+CLI Changes
+-----------
+
+* Add new arguments `--metadata`, `--use-json` and modify existing `--template-file` or `--template` to look for a default `template.yaml`  that exists under `.aws-sam/build/`
+
+### Breaking Change
+
+* Not a breaking change , but there are optional behavorial changes that a user can subscribe into by supplying a non-packaged template file and an s3 bucket.
+
+Design
+------
+
+* Changes to Deploy command's click options
+* Attempt to package on every deploy if an appropriate s3 bucket is specified and deploy using the output template file during package. 
+* If a pre-packaged template is specified, an attempt to package does not change the template and the same template is used for deploy.
+* The parameters that share the same name across package and deploy are collapsed together. eg: `--kms-key-id` , if a kms-key-id is specified that same key is used across both packaging and deploy purposes.
+
+`.samrc` Changes
+----------------
+
+None
+
+Security
+--------
+
+**What new dependencies (libraries/cli) does this change require?**
+N/A
+
+**What other Docker container images are you using?**
+N/A
+
+**Are you creating a new HTTP endpoint? If so explain how it will be
+created & used**
+N/A
+
+**Are you connecting to a remote API? If so explain how is this
+connection secured**
+N/A
+
+**Are you reading/writing to a temporary folder? If so, what is this
+used for and when do you clean up?**
+
+Possibly reading from a configuration file in the future.
+
+**How do you validate new .samrc configuration?**
+
+N/A
+
+What is your Testing Plan (QA)?
+===============================
+
+Goal
+----
+* Regression tests on previous functionality of `sam deploy`
+* Integration tests on automatic packaging on `sam deploy`
+
+Pre-requesites
+--------------
+N/A
+
+Test Scenarios/Cases
+--------------------
+* Re-deploy a stack that was deployed with a packaged template before hand using the new sam deploy menthod.
+
+`sam deploy --template-file packaged.yaml --stack-name sam-stack --capabilities CAPABILITY_IAM`
+
+`sam deploy --stack-name sam-stack --capabilities CAPABILITY_IAM`
+
+The new stack should not have any changes.
+
+
+Expected Results
+----------------
+
+* Regresssion and Integration tests pass.
+
+Documentation Changes
+=====================
+* Required nature of `--template-file`, `--template` parameter has a series of defaults that are looked at during `sam deploy` similair to `sam package`. 
+* If `--template-file` or `--template` points to a non-packaged template-file, `--s3-bucket` becomes required to be able to effectively package and deploy in one command using `sam deploy`.
+
+Open Issues
+============
+
+Task Breakdown
+==============
+
+-   \[x\] Send a Pull Request with this design document
+-   \[ \] Build the command line interface
+-   \[ \] Build the underlying library
+-   \[ \] Unit tests
+-   \[ \] Functional Tests
+-   \[ \] Integration tests
+-   \[ \] Run all tests on Windows
+-   \[ \] Update documentation