[V2] Add Trigger service and Scheler

[machichima]

Tracking issue

Why are the changes needed?

Add Trigger functionality for task scheduling

What changes were proposed in this pull request?

1. Add Trigger and TriggerReservation table
2. Add Trigger service to get / list / update DB
3. Add scheduler that will sync from trigger DB and create action when time's up

Note:

• While we haven't release yet, I didn't add a new migration for the model change. Please remove DB tables to re-create tables
• Currently ListRuns API does not deal with filters correctly, therefor when viewing runs for a task / trigger, it will include irrelevant tasks. We will deal with this in a follow-up PR.

How was this patch tested?

Run on sandbox locally.

List Triggers

Cron string

Run sandbox locally and deploy trigger example task:

flyte deploy examples/triggers/basic.py -p flytesnacks -d development env

Ensure trigger tasks are being created and catch up works (after sandbox restart, should catch up all missed schedules)

Fixed rate

Create fixed rate example:

from datetime import datetime

import flyte

env = flyte.TaskEnvironment(
    name="fixed_rate_example",
)

every_minute_trigger = flyte.Trigger(
    "every_minute",
    flyte.FixedRate(1),
    inputs={"trigger_time": flyte.TriggerTime},
)


@env.task(triggers=every_minute_trigger)
def every_minute(trigger_time: datetime) -> str:
    return f"Task ran at {trigger_time.isoformat()}"


if __name__ == "__main__":
    flyte.init_from_config()
    flyte.deploy(env)

Deploy example with:

 flyte deploy examples/triggers/fixed_rate.py -p flytesnacks -d development env

Labels

Please add one or more of the following labels to categorize your PR:

• added: For new features.
• changed: For changes in existing functionality.
• deprecated: For soon-to-be-removed features.
• removed: For features being removed.
• fixed: For any bug fixed.
• security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

Docs link

• main
  • Flyte 2 WIP #6583
    • [V2] Add Trigger service and Scheler 👈

[machichima]

Remove this and rely on CreateAction logic. Also move the part of forming the task spec to run_service.go