Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create HostnameCallableRule to ease upgrade to Airflow 2.0 #11044

Closed
turbaszek opened this issue Sep 21, 2020 · 4 comments · Fixed by #12743
Closed

Create HostnameCallableRule to ease upgrade to Airflow 2.0 #11044

turbaszek opened this issue Sep 21, 2020 · 4 comments · Fixed by #12743
Assignees
Labels
area:upgrade Facilitating migration to a newer version of Airflow good first issue kind:feature Feature Requests upgrade-check upgrade-check CLI

Comments

@turbaszek
Copy link
Member

This issue is part of #8765

Rule

Create HostnameCallableRule which corresponds to

Unify hostname_callable option in core section

entry in UPDATING.md. This rule should allow users to check if their current configuration needs any adjusting
before migration to Airflow 2.0.

How to guide

To implement a new rule, create a class that inherits from airflow.upgrade.rules.base_rule.BaseRule.
It will be auto-registered and used by airflow upgrade-check command. The custom rule class has to have title,
description properties and should implement check method which returns a list of error messages in case of
incompatibility.

For example:

class ConnTypeIsNotNullableRule(BaseRule):
title = "Connection.conn_type is not nullable"
description = """\
The `conn_type` column in the `connection` table must contain content. Previously, this rule was \
enforced by application logic, but was not enforced by the database schema.
If you made any modifications to the table directly, make sure you don't have null in the conn_type column.\
"""
@provide_session
def check(self, session=None):
invalid_connections = session.query(Connection).filter(Connection.conn_type.is_(None))
return (
'Connection<id={}", conn_id={}> have empty conn_type field.'.format(conn.id, conn.conn_id)
for conn in invalid_connections
)

Remember to open the PR against v1-10-test branch.

@turbaszek turbaszek added area:upgrade Facilitating migration to a newer version of Airflow kind:feature Feature Requests good first issue labels Sep 21, 2020
@turbaszek turbaszek added this to the Airflow 1.10.13 milestone Sep 21, 2020
@snehal199
Copy link

Hey @turbaszek, can I pick this up?

@turbaszek
Copy link
Member Author

Sure @snehal199 !

@kaxil kaxil added the upgrade-check upgrade-check CLI label Oct 30, 2020
@turbaszek
Copy link
Member Author

Hey @snehal199 any update on this one?

dimberman added a commit that referenced this issue Dec 1, 2020
Creates a rule to ensure users have a 2.0 compatible hostname_callable
configuration in their airflow.cfg

addresses #11044
@dimberman dimberman linked a pull request Dec 1, 2020 that will close this issue
kaxil added a commit that referenced this issue Dec 3, 2020
Creates a rule to ensure users have a 2.0 compatible hostname_callable
configuration in their airflow.cfg

closes #11044


Co-authored-by: Ash Berlin-Taylor <[email protected]>
Co-authored-by: Kaxil Naik <[email protected]>
@kaxil
Copy link
Member

kaxil commented Dec 3, 2020

closed by #12743

@kaxil kaxil closed this as completed Dec 3, 2020
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this issue Mar 5, 2021
)

Creates a rule to ensure users have a 2.0 compatible hostname_callable
configuration in their airflow.cfg

closes apache#11044

Co-authored-by: Ash Berlin-Taylor <[email protected]>
Co-authored-by: Kaxil Naik <[email protected]>
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Jun 9, 2021
Creates a rule to ensure users have a 2.0 compatible hostname_callable
configuration in their airflow.cfg

closes apache/airflow#11044

Co-authored-by: Ash Berlin-Taylor <[email protected]>
Co-authored-by: Kaxil Naik <[email protected]>
GitOrigin-RevId: 326e0f910f2ee584af9fb2750868b5a2cfa56146
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Jun 9, 2021
Creates a rule to ensure users have a 2.0 compatible hostname_callable
configuration in their airflow.cfg

closes apache/airflow#11044

Co-authored-by: Ash Berlin-Taylor <[email protected]>
Co-authored-by: Kaxil Naik <[email protected]>
GitOrigin-RevId: 326e0f910f2ee584af9fb2750868b5a2cfa56146
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:upgrade Facilitating migration to a newer version of Airflow good first issue kind:feature Feature Requests upgrade-check upgrade-check CLI
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants