Skip to content

Commit

Permalink
Update schedules.md (#5826)
Browse files Browse the repository at this point in the history
delete documentation related to AWS scheduler configuration, as the flyte native scheduler should just work.

reference slack thread: https://discuss.flyte.org/t/15852550/hi-team-i-have-created-a-simple-test-flyte-scheduler-using-b#929e7edd-c801-4dbb-bdbf-61cb36903727

Signed-off-by: Raghav Mangla <[email protected]>
  • Loading branch information
RaghavMangla authored Oct 15, 2024
1 parent a8b16a5 commit 3523326
Showing 1 changed file with 0 additions and 66 deletions.
66 changes: 0 additions & 66 deletions docs/user_guide/productionizing/schedules.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,70 +85,4 @@ You can [archive/deactivate the launch plan](https://docs.flyte.org/projects/fly
flytectl update launchplan -p flyteexamples -d development {{ name_of_lp }} --version <foo> --archive
```

## Platform configuration changes for AWS scheduler

The Scheduling feature can be run using the Flyte native scheduler which comes with Flyte. If you intend to use the AWS scheduler then it requires additional infrastructure to run, so these will have to be created and configured. The following sections are only required if you use the AWS scheme for the scheduler. You can still run the Flyte native scheduler on AWS.

### Setting up scheduled workflows

To run workflow executions based on user-specified schedules, you'll need to fill out the top-level `scheduler` portion of the flyteadmin application configuration.

In particular, you'll need to configure the two components responsible for scheduling workflows and processing schedule event triggers.

:::{note}
This functionality is currently only supported for AWS installs.
:::

#### Event scheduler

To schedule workflow executions, you'll need to set up an [AWS SQS](https://aws.amazon.com/sqs/) queue. A standard-type queue should suffice. The flyteadmin event scheduler creates [AWS CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/Create-CloudWatch-Events-Scheduled-Rule.html) event rules that invoke your SQS queue as a target.

With that in mind, let's take a look at an example `eventScheduler` config section and dive into what each value represents:

```bash
scheduler:
eventScheduler:
scheme: "aws"
region: "us-east-1"
scheduleRole: "arn:aws:iam::{{ YOUR ACCOUNT ID }}:role/{{ ROLE }}"
targetName: "arn:aws:sqs:us-east-1:{{ YOUR ACCOUNT ID }}:{{ YOUR QUEUE NAME }}"
scheduleNamePrefix: "flyte"
```

- **scheme**: in this case because AWS is the only cloud back-end supported for scheduling workflows, only `"aws"` is a valid value. By default, the no-op scheduler is used.
- **region**: this specifies which region initialized AWS clients should use when creating CloudWatch rules.
- **scheduleRole** This is the IAM role ARN with permissions set to `Allow`
: - `events:PutRule`
- `events:PutTargets`
- `events:DeleteRule`
- `events:RemoveTargets`
- **targetName** this is the ARN for the SQS Queue you've allocated to scheduling workflows.
- **scheduleNamePrefix** this is an entirely optional prefix used when creating schedule rules. Because of AWS naming length restrictions, scheduled rules are a random hash and having a shared prefix makes these names more readable and indicates who generated the rules.

#### Workflow executor

Scheduled events which trigger need to be handled by the workflow executor, which subscribes to triggered events from the SQS queue configured above.

:::{NOTE}
Failure to configure a workflow executor will result in all your scheduled events piling up silently without ever kicking off workflow executions.
:::

Again, let's break down a sample config:

```bash
scheduler:
eventScheduler:
...
workflowExecutor:
scheme: "aws"
region: "us-east-1"
scheduleQueueName: "{{ YOUR QUEUE NAME }}"
accountId: "{{ YOUR ACCOUNT ID }}"
```

- **scheme**: in this case because AWS is the only cloud back-end supported for executing scheduled workflows, only `"aws"` is a valid value. By default, the no-op executor is used and in case of sandbox we use `"local"` scheme which uses the Flyte native scheduler.
- **region**: this specifies which region AWS clients should use when creating an SQS subscriber client.
- **scheduleQueueName**: this is the name of the SQS Queue you've allocated to scheduling workflows.
- **accountId**: Your AWS [account id](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html#FindingYourAWSId).

[flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/master/examples/productionizing/

0 comments on commit 3523326

Please sign in to comment.