Prepare docs for May 2022 provider's release

This documentation update also (following the rule agreed in https://github.com/apache/airflow/blob/main/README.md#support-for-providers) bumps mininimum supported version of Airflow for all providers to 2.2 and it constitutes a breaking change and major version bump for all providers.
apache · Jun 7, 2022 · 7b356e4 · 7b356e4
1 parent 3e51d80
commit 7b356e4
Show file tree

Hide file tree

Showing 303 changed files with 7,156 additions and 4,204 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -801,20 +801,20 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
         run: >
           breeze verify-provider-packages --use-airflow-version wheel --use-packages-from-dist
           --package-format wheel
-      - name: "Remove airflow package and replace providers with 2.1-compliant versions"
+      - name: "Remove airflow package and replace providers with 2.2-compliant versions"
         run: |
           rm -vf dist/apache_airflow-*.whl \
              dist/apache_airflow_providers_cncf_kubernetes*.whl \
              dist/apache_airflow_providers_celery*.whl
           pip download --no-deps --dest dist \
               apache-airflow-providers-cncf-kubernetes==3.0.0 \
               apache-airflow-providers-celery==2.1.3
-      - name: "Install and test provider packages and airflow on Airflow 2.1 files"
+      - name: "Install and test provider packages and airflow on Airflow 2.2 files"
         run: >
-          breeze verify-provider-packages --use-airflow-version 2.1.0
-          --use-packages-from-dist --package-format wheel --airflow-constraints-reference constraints-2.1.0
+          breeze verify-provider-packages --use-airflow-version 2.2.0
+          --use-packages-from-dist --package-format wheel --airflow-constraints-reference constraints-2.2.0
         env:
-          # The extras below are all extras that should be installed with Airflow 2.1.0
+          # The extras below are all extras that should be installed with Airflow 2.2.0
           AIRFLOW_EXTRAS: "airbyte,alibaba,amazon,apache.atlas.apache.beam,apache.cassandra,apache.drill,\
             apache.druid,apache.hdfs,apache.hive,apache.kylin,apache.livy,apache.pig,apache.pinot,\
             apache.spark,apache.sqoop,apache.webhdfs,arangodb,asana,async,\

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -336,7 +336,7 @@ repos:
         # This check might be removed when min-airflow-version in providers is 2.2
       - id: check-airflow-2-1-compatibility
         name: Check that providers are 2.1 compatible.
-        entry: ./scripts/ci/pre_commit/pre_commit_check_2_1_compatibility.py
+        entry: ./scripts/ci/pre_commit/pre_commit_check_2_2_compatibility.py
         language: python
         pass_filenames: true
         files: ^airflow/providers/.*\.py$

diff --git a/README.md b/README.md
@@ -399,14 +399,14 @@ might decide to add additional limits (and justify them with comment)
 ## Support for providers
 
 Providers released by the community have limitation of a minimum supported version of Airflow. The minimum
-version of Airflow is the `MINOR` version (2.1, 2.2 etc.) indicating that the providers might use features
+version of Airflow is the `MINOR` version (2.2, 2.3 etc.) indicating that the providers might use features
 that appeared in this release. The default support timespan for the minimum version of Airflow
 (there could be justified exceptions) is that we increase the minimum Airflow version, when 12 months passed
 since the first release for the MINOR version of Airflow.
 
 For example this means that by default we upgrade the minimum version of Airflow supported by providers
-to 2.2.0 in the first Provider's release after 21st of May 2022 (21st of May 2021 is the date when the
-first `PATCHLEVEL` of 2.1 (2.1.0) has been released.
+to 2.3.0 in the first Provider's release after 11th of October 2022 (11th of October 2021 is the date when the
+first `PATCHLEVEL` of 2.2 (2.2.0) has been released.
 
 ## Contributing
 

diff --git a/airflow/providers/airbyte/CHANGELOG.rst b/airflow/providers/airbyte/CHANGELOG.rst
@@ -24,6 +24,20 @@
 Changelog
 ---------
 
+3.0.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+* This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
+  providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Add explanatory note for contributors about updating Changelog (#24229)``
+
+
 2.1.4
 .....
 

diff --git a/airflow/providers/airbyte/provider.yaml b/airflow/providers/airbyte/provider.yaml
@@ -22,6 +22,7 @@ description: |
   `Airbyte <https://airbyte.io/>`__
 
 versions:
+  - 3.0.0
   - 2.1.4
   - 2.1.3
   - 2.1.2
@@ -31,7 +32,7 @@ versions:
   - 1.0.0
 
 additional-dependencies:
-  - apache-airflow>=2.1.0
+  - apache-airflow>=2.2.0
 
 integrations:
   - integration-name: Airbyte

diff --git a/airflow/providers/alibaba/CHANGELOG.rst b/airflow/providers/alibaba/CHANGELOG.rst
@@ -24,6 +24,31 @@
 Changelog
 ---------
 
+2.0.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+* This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
+  providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
+
+Features
+~~~~~~~~
+
+   * ``SSL Bucket, Light Logic Refactor and Docstring Update for Alibaba Provider (#23891)``
+
+Misc
+~~~~
+
+   * ``Apply per-run log templates to log handlers (#24153)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Migrate Alibaba example DAGs to new design #22437 (#24130)``
+   * ``Add explanatory note for contributors about updating Changelog (#24229)``
+
+
 1.1.1
 .....
 

diff --git a/airflow/providers/alibaba/provider.yaml b/airflow/providers/alibaba/provider.yaml
@@ -22,13 +22,14 @@ description: |
     Alibaba Cloud integration (including `Alibaba Cloud <https://www.alibabacloud.com//>`__).
 
 versions:
+  - 2.0.0
   - 1.1.1
   - 1.1.0
   - 1.0.1
   - 1.0.0
 
 additional-dependencies:
-  - apache-airflow>=2.1.0
+  - apache-airflow>=2.2.0
 
 integrations:
   - integration-name: Alibaba Cloud OSS

diff --git a/airflow/providers/amazon/CHANGELOG.rst b/airflow/providers/amazon/CHANGELOG.rst
@@ -24,6 +24,51 @@
 Changelog
 ---------
 
+4.0.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+* This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
+  providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
+
+Features
+~~~~~~~~
+
+* ``Add partition related methods to GlueCatalogHook: (#23857)``
+* ``Add support for associating  custom tags to job runs submitted via EmrContainerOperator (#23769)``
+* ``Add number of node params only for single-node cluster in RedshiftCreateClusterOperator (#23839)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``fix: StepFunctionHook ignores explicit set 'region_name' (#23976)``
+* ``Fix Amazon EKS example DAG raises warning during Imports (#23849)``
+* ``Move string arg evals to 'execute()' in 'EksCreateClusterOperator' (#23877)``
+
+Misc
+~~~~
+
+* ``Light Refactor and Clean-up AWS Provider (#23907)``
+* ``Update sample dag and doc for RDS (#23651)``
+* ``Reformat the whole AWS documentation (#23810)``
+* ``Replace "absolute()" with "resolve()" in pathlib objects (#23675)``
+* ``Apply per-run log templates to log handlers (#24153)``
+* ``Refactor GlueJobHook get_or_create_glue_job method. (#24215)``
+* ``Update the DMS Sample DAG and Docs (#23681)``
+* ``Update doc and sample dag for Quicksight (#23653)``
+* ``Update doc and sample dag for EMR Containers (#24087)``
+* ``Add AWS project structure tests (re: AIP-47) (#23630)``
+* ``Add doc and sample dag for GCSToS3Operator (#23730)``
+* ``Remove old Athena Sample DAG (#24170)``
+* ``Clean up f-strings in logging calls (#23597)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Add explanatory note for contributors about updating Changelog (#24229)``
+   * ``Introduce 'flake8-implicit-str-concat' plugin to static checks (#23873)``
+
 3.4.0
 .....
 

diff --git a/airflow/providers/amazon/provider.yaml b/airflow/providers/amazon/provider.yaml
@@ -22,6 +22,7 @@ description: |
     Amazon integration (including `Amazon Web Services (AWS) <https://aws.amazon.com/>`__).
 
 versions:
+  - 4.0.0
   - 3.4.0
   - 3.3.0
   - 3.2.0
@@ -41,7 +42,7 @@ versions:
   - 1.0.0
 
 additional-dependencies:
-  - apache-airflow>=2.1.0
+  - apache-airflow>=2.2.0
 
 integrations:
   - integration-name: Amazon Athena

diff --git a/airflow/providers/apache/beam/CHANGELOG.rst b/airflow/providers/apache/beam/CHANGELOG.rst
@@ -24,6 +24,31 @@
 Changelog
 ---------
 
+4.0.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+* This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
+  providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
+
+Features
+~~~~~~~~
+
+* ``Added missing project_id to the wait_for_job (#24020)``
+* ``Support impersonation service account parameter for Dataflow runner (#23961)``
+
+Misc
+~~~~
+
+* ``chore: Refactoring and Cleaning Apache Providers (#24219)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Add explanatory note for contributors about updating Changelog (#24229)``
+   * ``AIP-47 - Migrate beam DAGs to new design #22439 (#24211)``
+
 3.4.0
 .....
 

diff --git a/airflow/providers/apache/beam/provider.yaml b/airflow/providers/apache/beam/provider.yaml
@@ -22,6 +22,7 @@ description: |
     `Apache Beam <https://beam.apache.org/>`__.
 
 versions:
+  - 4.0.0
   - 3.4.0
   - 3.3.0
   - 3.2.1
@@ -34,7 +35,7 @@ versions:
   - 1.0.0
 
 additional-dependencies:
-  - apache-airflow>=2.1.0
+  - apache-airflow>=2.2.0
 
 integrations:
   - integration-name: Apache Beam

diff --git a/airflow/providers/apache/cassandra/CHANGELOG.rst b/airflow/providers/apache/cassandra/CHANGELOG.rst
@@ -24,6 +24,25 @@
 Changelog
 ---------
 
+3.0.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+* This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
+  providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
+
+Misc
+~~~~
+   * ``chore: Refactoring and Cleaning Apache Providers (#24219)``
+
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Add explanatory note for contributors about updating Changelog (#24229)``
+   * ``AIP-47 - Migrate cassandra DAGs to new design #22439 (#24209)``
+
 2.1.3
 .....
 

diff --git a/airflow/providers/apache/cassandra/provider.yaml b/airflow/providers/apache/cassandra/provider.yaml
@@ -22,6 +22,7 @@ description: |
     `Apache Cassandra <http://cassandra.apache.org/>`__.
 
 versions:
+  - 3.0.0
   - 2.1.3
   - 2.1.2
   - 2.1.1
@@ -32,7 +33,7 @@ versions:
   - 1.0.0
 
 additional-dependencies:
-  - apache-airflow>=2.1.0
+  - apache-airflow>=2.2.0
 
 integrations:
   - integration-name: Apache Cassandra

diff --git a/airflow/providers/apache/drill/CHANGELOG.rst b/airflow/providers/apache/drill/CHANGELOG.rst
@@ -24,6 +24,25 @@
 Changelog
 ---------
 
+2.0.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+* This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
+  providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Add explanatory note for contributors about updating Changelog (#24229)``
+   * ``AIP-47 - Migrate drill DAGs to new design #22439 (#24206)``
+   * ``Prepare provider documentation 2022.05.11 (#23631)``
+   * ``Clean up in-line f-string concatenation (#23591)``
+
+.. Review and move the new changes to one of the sections above:
+   * ``chore: Refactoring and Cleaning Apache Providers (#24219)``
+
 1.0.4
 .....
 

diff --git a/airflow/providers/apache/drill/provider.yaml b/airflow/providers/apache/drill/provider.yaml
@@ -22,14 +22,15 @@ description: |
     `Apache Drill <https://drill.apache.org/>`__.
 
 versions:
+  - 2.0.0
   - 1.0.4
   - 1.0.3
   - 1.0.2
   - 1.0.1
   - 1.0.0
 
 additional-dependencies:
-  - apache-airflow>=2.1.0
+  - apache-airflow>=2.2.0
 
 integrations:
   - integration-name: Apache Drill

diff --git a/airflow/providers/apache/druid/CHANGELOG.rst b/airflow/providers/apache/druid/CHANGELOG.rst
@@ -24,6 +24,26 @@
 Changelog
 ---------
 
+3.0.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+* This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow
+  providers support policy https://github.com/apache/airflow/blob/main/README.md#support-for-providers
+
+Misc
+~~~~
+
+* ``chore: Refactoring and Cleaning Apache Providers (#24219)``
+
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``AIP-47 - Migrate druid DAGs to new design #22439 (#24207)``
+   * ``Add explanatory note for contributors about updating Changelog (#24229)``
+
 2.3.3
 .....
 

diff --git a/airflow/providers/apache/druid/provider.yaml b/airflow/providers/apache/druid/provider.yaml
@@ -22,6 +22,7 @@ description: |
   `Apache Druid <https://druid.apache.org/>`__.
 
 versions:
+  - 3.0.0
   - 2.3.3
   - 2.3.2
   - 2.3.1
@@ -36,7 +37,7 @@ versions:
   - 1.0.0
 
 additional-dependencies:
-  - apache-airflow>=2.1.0
+  - apache-airflow>=2.2.0
 
 integrations:
   - integration-name: Apache Druid