diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst index 2b763f7ce703b..ed9ea874af846 100644 --- a/RELEASE_NOTES.rst +++ b/RELEASE_NOTES.rst @@ -21,6 +21,124 @@ .. towncrier release notes start + +Airflow 2.3.3 (2022-07-05) +-------------------------- + +Significant Changes +^^^^^^^^^^^^^^^^^^^ + +We've upgraded Flask App Builder to a major version 4.* (#24399) +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +Flask App Builder is one of the important components of Airflow Webserver, as +it uses a lot of dependencies that are essential to run the webserver and integrate it +in enterprise environments - especially authentication. + +The FAB 4.* upgrades a number of dependencies to major releases, which upgrades them to versions +that have a number of security issues fixed. A lot of tests were performed to bring the dependencies +in a backwards-compatible way, however the dependencies themselves implement breaking changes in their +internals so it might be that some of those changes might impact the users in case they are using the +libraries for their own purposes. + +One important change that you likely will need to apply to Oauth configuration is to add +``server_metadata_url`` or ``jwks_uri`` and you can read about it more +in `this issue `_. + +Here is the list of breaking changes in dependencies that comes together with FAB 4: + + * ``Flask`` from 1.X to 2.X `breaking changes `__ + + * ``flask-jwt-extended`` 3.X to 4.X `breaking changes: `__ + + * ``Jinja2`` 2.X to 3.X `breaking changes: `__ + + * ``Werkzeug`` 1.X to 2.X `breaking changes `__ + + * ``pyJWT`` 1.X to 2.X `breaking changes: `__ + + * ``Click`` 7.X to 8.X `breaking changes: `__ + + * ``itsdangerous`` 1.X to 2.X `breaking changes `__ + +Bug Fixes +^^^^^^^^^ + +- Allow for ``LOGGING_LEVEL=DEBUG`` (#23360) +- Fix grid date ticks (#24738) +- Debounce status highlighting in Grid view (#24710) +- Fix Grid vertical scrolling (#24684) +- don't try to render child rows for closed groups (#24637) +- Do not calculate grid root instances (#24528) +- Maintain grid view selection on filtering upstream (#23779) +- Speed up ``grid_data`` endpoint by 10x (#24284) +- Apply per-run log templates to log handlers (#24153) +- Don't crash scheduler if exec config has old k8s objects (#24117) +- ``TI.log_url`` fix for ``map_index`` (#24335) +- Fix migration ``0080_2_0_2`` - Replace null values before setting column not null (#24585) +- Patch ``sql_alchemy_conn`` if old Postgres schemes used (#24569) +- Seed ``log_template`` table (#24511) +- Fix deprecated ``log_id_template`` value (#24506) +- Fix toast messages (#24505) +- Add indexes for CASCADE deletes for ``task_instance`` (#24488) +- Return empty dict if Pod JSON encoding fails (#24478) +- Improve grid rendering performance with a custom tooltip (#24417, #24449) +- Check for ``run_id`` for grid group summaries (#24327) +- Optimize calendar view for cron scheduled DAGs (#24262) +- Use ``get_hostname`` instead of ``socket.getfqdn`` (#24260) +- Check that edge nodes actually exist (#24166) +- Fix ``useTasks`` crash on error (#24152) +- Do not fail re-queued TIs (#23846) +- Reduce grid view API calls (#24083) +- Rename Permissions to Permission Pairs. (#24065) +- Replace ``use_task_execution_date`` with ``use_task_logical_date`` (#23983) +- Grid fix details button truncated and small UI tweaks (#23934) +- Add TaskInstance State ``REMOVED`` to finished states and success states (#23797) +- Fix mapped task immutability after clear (#23667) +- Fix permission issue for dag that has dot in name (#23510) +- Fix closing connection ``dbapi.get_pandas_df`` (#23452) +- Check bag DAG ``schedule_interval`` match timetable (#23113) +- Parse error for task added to multiple groups (#23071) +- Fix flaky order of returned dag runs (#24405) +- Migrate ``jsx`` files that affect run/task selection to ``tsx`` (#24509) +- Fix links to sources for examples (#24386) +- Set proper ``Content-Type`` and ``chartset`` on ``grid_data`` endpoint (#24375) + +Doc only changes +^^^^^^^^^^^^^^^^ + +- Update templates doc to mention ``extras`` and format Airflow ``Vars`` / ``Conns`` (#24735) +- Document built in Timetables (#23099) +- Alphabetizes two tables (#23923) +- Clarify that users should not use Maria DB (#24556) +- Add imports to deferring code samples (#24544) +- Add note about image regeneration in June 2022 (#24524) +- Small cleanup of ``get_current_context()`` chapter (#24482) +- Fix default 2.2.5 ``log_id_template`` (#24455) +- Update description of installing providers separately from core (#24454) +- Mention context variables and logging (#24304) + +Misc/Internal +^^^^^^^^^^^^^ + +- Remove internet explorer support (#24495) +- Removing magic status code numbers from ``api_connexion`` (#24050) +- Upgrade FAB to ``4.1.2`` (#24619) +- Switch Markdown engine to ``markdown-it-py`` (#19702) +- Update ``rich`` to latest version across the board. (#24186) +- Get rid of ``TimedJSONWebSignatureSerializer`` (#24519) +- Update flask-appbuilder ``authlib``/ ``oauth`` dependency (#24516) +- Upgrade to ``webpack`` 5 (#24485) +- Add ``typescript`` (#24337) +- The JWT claims in the request to retrieve logs have been standardized: we use ``nbf`` and ``aud`` claims for + maturity and audience of the requests. Also "filename" payload field is used to keep log name. (#24519) +- Address all ``yarn`` test warnings (#24722) +- Upgrade to react 18 and chakra 2 (#24430) +- Refactor ``DagRun.verify_integrity`` (#24114) +- Upgrade FAB to ``4.1.1`` (#24399) +- We now need at least ``Flask-WTF 0.15`` (#24621) + + Airflow 2.3.2 (2022-06-04) -------------------------- @@ -7835,7 +7953,6 @@ There are five roles created for Airflow by default: Admin, User, Op, Viewer, an Breaking changes ~~~~~~~~~~~~~~~~ - * AWS Batch Operator renamed property queue to job_queue to prevent conflict with the internal queue from CeleryExecutor - AIRFLOW-2542 * Users created and stored in the old users table will not be migrated automatically. FAB's built-in authentication support must be reconfigured. * Airflow dag home page is now ``/home`` (instead of ``/admin``\ ). diff --git a/airflow/providers/elasticsearch/example_dags/__init__.py b/airflow/providers/elasticsearch/example_dags/__init__.py deleted file mode 100644 index 13a83393a9124..0000000000000 --- a/airflow/providers/elasticsearch/example_dags/__init__.py +++ /dev/null @@ -1,16 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. diff --git a/airflow/providers/elasticsearch/example_dags/example_elasticsearch_query.py b/airflow/providers/elasticsearch/example_dags/example_elasticsearch_query.py deleted file mode 100644 index d4a7b100f00f6..0000000000000 --- a/airflow/providers/elasticsearch/example_dags/example_elasticsearch_query.py +++ /dev/null @@ -1,50 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -from datetime import datetime, timedelta - -from airflow import DAG -from airflow.decorators import task -from airflow.providers.elasticsearch.hooks.elasticsearch import ElasticsearchHook - - -@task(task_id='es_print_tables') -def show_tables(): - """ - show_tables queries elasticsearch to list available tables - """ - es = ElasticsearchHook(elasticsearch_conn_id='production-es') - - # Handle ES conn with context manager - with es.get_conn() as es_conn: - tables = es_conn.execute('SHOW TABLES') - for table, *_ in tables: - print(f"table: {table}") - return True - - -# Using a DAG context manager, you don't have to specify the dag property of each task -with DAG( - 'elasticsearch_dag', - start_date=datetime(2021, 8, 30), - max_active_runs=1, - schedule_interval=timedelta(days=1), - default_args={'retries': 1}, # Default setting applied to all tasks - catchup=False, -) as dag: - - show_tables() diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index 5de2041c3314f..fd7a57694d350 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -111,6 +111,7 @@ Datasync DateFrame Datetimepicker Datetimes +Debounce Decrypt Decrypts DeidentifyContentResponse @@ -589,6 +590,7 @@ cfg cgi cgroups cgroupspy +chakra changelog charset checklicence diff --git a/newsfragments/24399.significant.rst b/newsfragments/24399.significant.rst deleted file mode 100644 index 7f1833a8e8c7e..0000000000000 --- a/newsfragments/24399.significant.rst +++ /dev/null @@ -1,31 +0,0 @@ -We've upgraded Flask Application Builder to a major version 4.*. - -Flask Application Builder is one of the important components of Airflow Webserver, as -it uses a lof of dependencies that are essential to run the webserver and integrate it -in enterprise environments - especially authentication. - -The FAB 4.* upgrades a number of dependencies to major releases, which upgrades them to versions -that have a number of security issues fixed. A lot of tests were performed to bring the dependencies -in a backwards-compatible way, however the dependencies themselves implement breaking changes in their -internals so it might be that some of those changes might impact the users in case they are using the -libraries for their onw purposes. - -One important change that you likely will need to apply to Oauth configuration is to add -``server_metadata_url`` or ``jwks_uri`` and you can read about it more -in `this issue `_. - -Here is the list of breaking changes in dependencies that comes together with FAB 4: - -* Flask from 1.X to 2.X `breaking changes `_ - -* flask-jwt-extended 3.X to 4.X `breaking changes: `_ - -* Jinja2 2.X to 3.X `breaking changes: `_ - -* Werkzeug 1.X to 2.X `breaking changes `_ - -* pyJWT 1.X to 2.X `breaking changes: `_ - -* Click 7.X to 8.X `breaking changes: `_ - -* itsdangerous 1.X to 2.X `breaking changes `_ diff --git a/newsfragments/24519.misc.rst b/newsfragments/24519.misc.rst deleted file mode 100644 index 799d9141d2a0a..0000000000000 --- a/newsfragments/24519.misc.rst +++ /dev/null @@ -1 +0,0 @@ -The JWT claims in the request to retrieve logs have been standardized: we use "nbf" and "aud" claims for maturity and audience of the requests. Also "filename" payload field is used to keep log name.