Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: New Task Freshness Algorithm #73

Open
asg017 opened this issue Mar 30, 2020 · 0 comments
Open

WIP: New Task Freshness Algorithm #73

asg017 opened this issue Mar 30, 2020 · 0 comments

Comments

@asg017
Copy link
Owner

asg017 commented Mar 30, 2020

Assume we are in a cellDecorator. The decorator is currently at a task, t. These are the steps we should perform to determine if the task should be re-ran (and why).

  1. Create an input signature of the given inputs. For every dependency, if it is a task, get the filesignature of that task's targets. If the task has any "watch" files, get the filesignature of that as well. Then, hash all those filesignatures together, then save it.
CREATE TABLE TaskExecutions (
    run TEXT,
    cellName TEXT,
    ancestorCellHash TEXT,
    dependenciesFileHash TEXT,
    freshStatus TEXT, -- "out-of-date", "up-to-date", "does-not-exist"
    runProcessStart INTEGER,
    runProcessEnd INTEGER,
    runProcessExitCode INTEGER,
    runProcessPID INTEGER,
    expectProcessStart INTEGER,
    expectProcessEnd INTEGER,
    expectProcessExitCode INTEGER,
    expectProcessPID INTEGER,
    outputFileHash TEXT,
    runLog INTEGER,
    expectLog INTEGER,
    FOREIGN KEY (run) REFERENCE Runs(hash),
    FOREIGN KEY (log) REFERENCE Logs(rowid)
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant