[Feature]: Refactor AMS using function and process api

[majin1102]

Description

Three main goals for this feature:

1. Implementing a pluggable and extensible processing mechanism for lake table formats based on action and process api, after which users could define external or custom optimizing, syncing, expiring processes managed by AMS.
2. Optimize scheduling logic based on quota, which could better control cost and scheduling fairness for optimizing and other processes.
3. Migrate all task running logic from AMS to optimizers, including optimizing plan, commit and maintenance tasks, which will improve AMS stability greatly.
4. Use future model inside of process lifecycle management instead of table status watcher model for better readability and extensibility.

This top issue will be splitted into serveral sub issues.

Use case/motivation

Use action and process model for AMS and improve steability, readibility and extensibility

Describe the solution

No response

Subtasks

#2597
#3139

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct