[Plugin] Support custom resource cleanup policy in Plugin interface

[clairemcginty]

Why would this plugin be helpful to the Flyte community
I'd like to propose adding a 5th method to the Plugin interface allowing users to customize how they want their resource to be cleaned up at the conclusion of the task. By default in Flytepropeller the task is cleaned up just by deleting the k8s resource; however for our use case we’d like to only delete selective resources and patch others. I put up a small branch [1] with an idea of what this could look like in plugins.go; and FlytePropeller could just defer to that method in its Abort method. Unfortunately because of Go’s inheritance rules it would be a breaking change for all plugin implementations. Per discussion with @EngHabu in #feature-discussions in the Flyte Slack instance, this could possibly be implemented as a "PluginV2" interface that users can choose whether to implement.

Type of Plugin

• Python/Java interface only plugin
• Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...)
• Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...)
• Customized Plugin using native kubernetes constructs
• Other

Can you help us with the implementation?

• Yes
• No

[welcome]

Thank you for opening your first issue here! 🛠