Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Workflows backend prototype #542

Closed
wants to merge 44 commits into from
Closed

Workflows backend prototype #542

wants to merge 44 commits into from

Commits on Sep 5, 2024

  1. initialize dask cluster

    @andrii-i
    andrii-i committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    abedadf View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. add workflow handler and endpoint

    @andrii-i
    andrii-i committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    6c9de64 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. make execute_workflow a flow

    @andrii-i
    andrii-i committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    71aa7fe View commit details
    Browse the repository at this point in the history

  2. add execute_workflow option to DefaultExecutionManager

    @andrii-i
    andrii-i committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    c8e0bf6 View commit details
    Browse the repository at this point in the history

  3. Update job model to be used with workflows

    @andrii-i
    andrii-i committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    b79b917 View commit details
    Browse the repository at this point in the history

  4. Add workflow run and jobs endpoints

    @andrii-i
    andrii-i committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    4ef2373 View commit details
    Browse the repository at this point in the history

  5. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    d77310e View commit details
    Browse the repository at this point in the history

  6. split create_workflow and run_workflow endpoints and methods

    @andrii-i
    andrii-i committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    c5da5b0 View commit details
    Browse the repository at this point in the history

  7. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    c64a53b View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. add GET workflow/{id}

    @andrii-i
    andrii-i committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    b33191c View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    b07a765 View commit details
    Browse the repository at this point in the history

  3. add workflow/{workflow_id}/tasks endpoint

    @andrii-i
    andrii-i committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    044e3bb View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    5c351ac View commit details
    Browse the repository at this point in the history

  5. add create_workflow_task to handler and scheduler

    @andrii-i
    andrii-i committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    4aa3046 View commit details
    Browse the repository at this point in the history

  6. make CreateWorkflow.tasks optional

    @andrii-i
    andrii-i committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    c276385 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. add update workflow functionality

    @andrii-i
    andrii-i committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    2f6938b View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. execute notebook as task via DefaultExecutionManager

    @andrii-i
    andrii-i committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    3ae8997 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    b9c466b View commit details
    Browse the repository at this point in the history

  3. Use prefect task to execute jobs, download files. Add flow and run names

    @andrii-i
    andrii-i committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    69b34ca View commit details
    Browse the repository at this point in the history

  4. Remove Dask

    @andrii-i
    andrii-i committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    f18071e View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. change depends_on and tasks data type to not limit the size

    @andrii-i
    andrii-i committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    440dfd1 View commit details
    Browse the repository at this point in the history

  2. make workflow_id optional

    @andrii-i
    andrii-i committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    6f9ec65 View commit details
    Browse the repository at this point in the history

  3. remove distributed dependency

    @andrii-i
    andrii-i committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    5a29d03 View commit details
    Browse the repository at this point in the history

  4. don't require workflow_id in the task creation body, only as a part o… 

    …f URL
    @andrii-i
    andrii-i committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    cdafb4f View commit details
    Browse the repository at this point in the history

  5. add status to the workflow models

    @andrii-i
    andrii-i committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    650a566 View commit details
    Browse the repository at this point in the history

  6. add workflow definitions

    @andrii-i
    andrii-i committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    6466c48 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. add workflow execution handlers and endpoints

    @andrii-i
    andrii-i committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    9994767 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    be6a01f View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. serve workflow definition on schedule

    @andrii-i
    andrii-i committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    417f392 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. fix parameters passed to create_and_run_workflow when served with sch… 

    …edule
    @andrii-i
    andrii-i committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    5a2621b View commit details
    Browse the repository at this point in the history

  2. change worklow_definitions/{id}/activate endpoint to worklow_definiti… 

    …ons/{id}/deploy
    @andrii-i
    andrii-i committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    87306a8 View commit details
    Browse the repository at this point in the history

  3. add name, parameters fields to Workflow and WorkflowDefinition

    @andrii-i
    andrii-i committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    e35607f View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    78e0818 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. replace prefect with dask

    @andrii-i
    andrii-i committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    e9d22e1 View commit details
    Browse the repository at this point in the history

  2. add workflow runner

    @andrii-i
    andrii-i committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    14f5740 View commit details
    Browse the repository at this point in the history

  3. use dask instead of prefect

    @andrii-i
    andrii-i committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    9321d41 View commit details
    Browse the repository at this point in the history

  4. add trigger_rule enum, field to Jobs and JobDefinitions

    @andrii-i
    andrii-i committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    2b25c01 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. add DRAFT Status enum and set it as a default workflow status

    @andrii-i
    andrii-i committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    c2bcf13 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. add DEPLOYED status

    @andrii-i
    andrii-i committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    6e162cb View commit details
    Browse the repository at this point in the history

  2. rename activate_workflow_definition into deploy_workflow_definition

    @andrii-i
    andrii-i committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    8866d24 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. add create_time to Workflow model

    @andrii-i
    andrii-i committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    d49f930 View commit details
    Browse the repository at this point in the history

  2. create and run workflow definitions including on schedule

    @andrii-i
    andrii-i committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    ee9e067 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. Merge branch 'jupyter-server:main' into workflows-backend

    @andrii-i
    andrii-i authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    75e0e37 View commit details
    Browse the repository at this point in the history

  2. add more print statements

    @andrii-i
    andrii-i committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    97ac530 View commit details
    Browse the repository at this point in the history