From 622d874cd83e8c573346794d5c7e9cd753fb288d Mon Sep 17 00:00:00 2001 From: Raghav Mangla <97332401+RaghavMangla@users.noreply.github.com> Date: Wed, 9 Oct 2024 04:02:55 +0530 Subject: [PATCH] Update schedules.md delete documentation related to AWS scheduler configuration, as the flyte native scheduler should just work. reference slack thread: https://discuss.flyte.org/t/15852550/hi-team-i-have-created-a-simple-test-flyte-scheduler-using-b#929e7edd-c801-4dbb-bdbf-61cb36903727 Signed-off-by: Raghav Mangla <97332401+RaghavMangla@users.noreply.github.com> --- docs/user_guide/productionizing/schedules.md | 66 -------------------- 1 file changed, 66 deletions(-) diff --git a/docs/user_guide/productionizing/schedules.md b/docs/user_guide/productionizing/schedules.md index 1b72fd8bec..fa0a6eedb0 100644 --- a/docs/user_guide/productionizing/schedules.md +++ b/docs/user_guide/productionizing/schedules.md @@ -85,70 +85,4 @@ You can [archive/deactivate the launch plan](https://docs.flyte.org/projects/fly flytectl update launchplan -p flyteexamples -d development {{ name_of_lp }} --version --archive ``` -## Platform configuration changes for AWS scheduler - -The Scheduling feature can be run using the Flyte native scheduler which comes with Flyte. If you intend to use the AWS scheduler then it requires additional infrastructure to run, so these will have to be created and configured. The following sections are only required if you use the AWS scheme for the scheduler. You can still run the Flyte native scheduler on AWS. - -### Setting up scheduled workflows - -To run workflow executions based on user-specified schedules, you'll need to fill out the top-level `scheduler` portion of the flyteadmin application configuration. - -In particular, you'll need to configure the two components responsible for scheduling workflows and processing schedule event triggers. - -:::{note} -This functionality is currently only supported for AWS installs. -::: - -#### Event scheduler - -To schedule workflow executions, you'll need to set up an [AWS SQS](https://aws.amazon.com/sqs/) queue. A standard-type queue should suffice. The flyteadmin event scheduler creates [AWS CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/Create-CloudWatch-Events-Scheduled-Rule.html) event rules that invoke your SQS queue as a target. - -With that in mind, let's take a look at an example `eventScheduler` config section and dive into what each value represents: - -```bash -scheduler: - eventScheduler: - scheme: "aws" - region: "us-east-1" - scheduleRole: "arn:aws:iam::{{ YOUR ACCOUNT ID }}:role/{{ ROLE }}" - targetName: "arn:aws:sqs:us-east-1:{{ YOUR ACCOUNT ID }}:{{ YOUR QUEUE NAME }}" - scheduleNamePrefix: "flyte" -``` - -- **scheme**: in this case because AWS is the only cloud back-end supported for scheduling workflows, only `"aws"` is a valid value. By default, the no-op scheduler is used. -- **region**: this specifies which region initialized AWS clients should use when creating CloudWatch rules. -- **scheduleRole** This is the IAM role ARN with permissions set to `Allow` - : - `events:PutRule` - - `events:PutTargets` - - `events:DeleteRule` - - `events:RemoveTargets` -- **targetName** this is the ARN for the SQS Queue you've allocated to scheduling workflows. -- **scheduleNamePrefix** this is an entirely optional prefix used when creating schedule rules. Because of AWS naming length restrictions, scheduled rules are a random hash and having a shared prefix makes these names more readable and indicates who generated the rules. - -#### Workflow executor - -Scheduled events which trigger need to be handled by the workflow executor, which subscribes to triggered events from the SQS queue configured above. - -:::{NOTE} -Failure to configure a workflow executor will result in all your scheduled events piling up silently without ever kicking off workflow executions. -::: - -Again, let's break down a sample config: - -```bash -scheduler: - eventScheduler: - ... - workflowExecutor: - scheme: "aws" - region: "us-east-1" - scheduleQueueName: "{{ YOUR QUEUE NAME }}" - accountId: "{{ YOUR ACCOUNT ID }}" -``` - -- **scheme**: in this case because AWS is the only cloud back-end supported for executing scheduled workflows, only `"aws"` is a valid value. By default, the no-op executor is used and in case of sandbox we use `"local"` scheme which uses the Flyte native scheduler. -- **region**: this specifies which region AWS clients should use when creating an SQS subscriber client. -- **scheduleQueueName**: this is the name of the SQS Queue you've allocated to scheduling workflows. -- **accountId**: Your AWS [account id](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html#FindingYourAWSId). - [flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/master/examples/productionizing/