add cli command to migrate job runs from one repo to another #9376
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ |
There was a problem hiding this comment.
This seems reasonable, though I'm not super well versed in the part of the code, so I think it could be worth it for @alangenfeld to take a look as well.
There was a problem hiding this comment.
the origin in the serialized DagsterRun will be out of sync, we use that as source of truth for idempotence checks.
So if you try to move a job that is hooked up to a sensor with run_key idempotence, the old run key runs will get discarded when we see a non matching origin id
|
|
||
| if should_migrate: | ||
| for record in tqdm(records): | ||
| with instance.run_storage.connect() as conn: |
There was a problem hiding this comment.
keep connection through loop - no need to reconnect each iteration
There was a problem hiding this comment.
little spicy depending on run_storage.connect here like this but i suppose the test helps protect (at least the one impl that the test uses)
prha
force-pushed
the
prha/run_repo_tag_migration_cli
branch
from
dd35a38 to
5dfafe3
Compare
prha
force-pushed
the
prha/run_repo_tag_migration_cli
branch
from
0a8ad73 to
1a38e26
Compare
| if not isinstance(instance.run_storage, SqlRunStorage): | ||
| return |
[INTERNAL_BRANCH=prha/run_migration]
prha
force-pushed
the
prha/run_repo_tag_migration_cli
branch
from
1a38e26 to
d9aa097
Compare
[INTERNAL_BRANCH=prha/run_migration]
prha
force-pushed
the
prha/run_repo_tag_migration_cli
branch
from
83e1c98 to
a889783
Compare
test #2 change to set cursor to latest materializations add cli command to migrate job runs from one repo to another (#9376) * add cli command to migrate job runs from one repo to another * extract migration into sql run storage * fix tests * remove prompt_required arg [INTERNAL_BRANCH=prha/run_migration] * rebase; raise; * fix rebase * fix bad rebase [INTERNAL_BRANCH=prha/run_migration]
Summary & Motivation
Context: #9317
Enables refactoring of repo code, while preserving history
How I Tested These Changes
BK