📖 Test Validation script for Airflow Repo with Updated Regex Package

[jhpyke]

User Story

As an airflow user
I want the validation script to spend less than 5 minutes running
So that I can get on with my day.

Value / Purpose

Currently the version of regex used for validation in Airflow is about 4 years old, and takes an extremely long time to install as part of workflows. As such, we should update to the latest version of regex (as in this PR) to save about 5 minute per run. This will save a significant amount of compute time, and as such, money for the business.

Useful Contacts

@jhpyke

User Types

No response

Hypothesis

If we update regex
Then validation will continue working the way we expect

Proposal

We only use fullmatch and compile from regex. As such, could compare the code for those functions between versions for functionality changes. Alternatively, we can try and manually violate each of the policies that rely on these functions and ensure they still work as intended.

Additional Information

Could be used as an opportunity to write unit tests for validation?

Definition of Done

• We are able to update the regex package version
• Validation script still performs as expected
• Support is prepared to respond to any issues arising from the upgrade

[jacobwoffenden]

Resolved by https://github.com/moj-analytical-services/airflow/pull/4186