✨Adding GPU support to airflow

[rob-mcnaughter]

Describe the feature request.

Can GPU support/capability be added to airflow jobs please.

Describe the context.

Our current project involves converting text data to embeddings (utilising transformer based models (neural networks)) and using a machine learning model we have developed to make classification predictions on the embeddings. The process, in particular creating the embeddings which involves complex models, takes some time and would benefit from GPU compute to speed up processing times. We intend to process around 18000 texts each month.

Value / Purpose

This would hopefully speed up processing times.

User Types

No response