[Feature] Abstraction and standardization of query tasks

[wild-endeavor]

Motivation: Why do you think this is important?
Now that we have a Presto task and a Hive task in Flytekit, we should consider abstracting them both behind a more general Flytekit query task.

We should also change the way Hive queries currently run (as de facto dynamic tasks), and follow the pattern established by the new Presto plugin. It may not be possible to do this in a backwards compatible way however, especially considering legacy code at Lyft, so the development of a general query task that supports Hive may be independent from the existing Hive task.

Goal: What should the final outcome look like, ideally?
Flytekit tasks that are more general.

Describe alternatives you've considered
None.

Flyte component

• Overall
• Flyte Setup and Installation scripts
• Flyte Documentation
• Flyte communication (slack/email etc)
• FlytePropeller
• FlyteIDL (Flyte specification language)
• Flytekit (Python SDK)
• FlyteAdmin (Control Plane service)
• FlytePlugins
• DataCatalog
• FlyteStdlib (common libraries)
• FlyteConsole (UI)
• Other

Additional context
Original Presto task issue: #203

[wild-endeavor]

cc @lu4nm3 @kumare3

[kumare3]

I think this should be done by elevating SQL as an alternative to container in the task spec. IMO task spec contains a reference to the portable code, for which container is a great proxy. In case of queries you don’t need a container as the code itself is portable. And so we could capture the code inline

[github-actions]

Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏

[github-actions]

Hello 👋, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏

[kumare3]

this has been already done. Flyte TaskTemplates can have queries in them