-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Test Plan: manual Reviewers: schrockn Differential Revision: https://dagster.phacility.com/D2970
- Loading branch information
1 parent
6b0ebea
commit bdd564d
Showing
7 changed files
with
92 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
FROM gitpod/workspace-full | ||
|
||
USER gitpod | ||
|
||
# Install wxPython dependencies | ||
RUN sudo apt-get -q update | ||
|
||
ENV DAGSTER_HOME="$HOME/dagster_home" | ||
RUN mkdir -p $DAGSTER_HOME |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
image: | ||
file: .gitpod.Dockerfile | ||
|
||
# This will expose all necessary ports needed for your VNC image | ||
ports: | ||
- port: 3000 | ||
onOpen: open-preview | ||
|
||
tasks: | ||
- command: | | ||
cd examples/dagster_examples/$EXAMPLE | ||
pip3 install -r requirements.txt | ||
[ -e README.md ] && gp open README.md | ||
clear && dagit | ||
- openMode: split-right | ||
command: echo SplitTerminal && clear |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
--- | ||
title: Dependency DSL | ||
description: How to create a custom DSL for defining pipelines | ||
--- | ||
|
||
# Dependency DSL | ||
|
||
This demo example is to demonstrate that it is possible to construct | ||
the dependencies of a pipeline definition from a YAML file or similar. | ||
This is not the "blessed" file format nor is it our recommendation for | ||
doing things. However we've been asked a number of times if this is possible | ||
and this should serve as a reasonable template to build upon. | ||
|
||
For example: | ||
|
||
```YAML | ||
pipeline: | ||
name: some_example | ||
description: blah blah blah | ||
solids: | ||
- def: add_one | ||
alias: A | ||
- def: add_one | ||
alias: B | ||
deps: | ||
num: | ||
solid: A | ||
- def: add_two | ||
alias: C | ||
deps: | ||
num: | ||
solid: A | ||
- def: subtract | ||
deps: | ||
left: | ||
solid: B | ||
right: | ||
solid: C | ||
``` | ||
# Open in Playground | ||
Open up this example in a playground using [Gitpod](https://gitpod.io) | ||
[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#EXAMPLE=dep_dsl/https://github.com/dagster-io/dagster) | ||
# Download Manually | ||
Download the example: | ||
``` | ||
curl https://codeload.github.com/dagster-io/dagster/tar.gz/master | tar -xz --strip=3 dagster-master/examples/dagster_examples/dep_dsl | ||
cd dep_dsl | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
repository: | ||
file: repo.py | ||
fn: define_repository |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
dagster | ||
dagit |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters