GFlows Packages

In addition to the native options for sharing code provided by Jsonnet and Ytt, GFlows provides a package format which lets you easily share library files and workflows between repos. The main advantages of a GFlows package are:

1. Compared to using Jsonnet's Jsonnet Bundler, GFlows Packages are lightweight, not requiring local bundles to be committed to source control or any additional tooling.

2. YTT lets you list individual remote files, but this means you need to know the structure of the library you're using. If it gets broken up into additional files or files get renamed, then the links you're using in your workflow config will break. GFlows Packages do not require this working knowledge of the package structure.

3. GFlows Packages define a directory structure which let you share workflows across repositories. If you want to share workflows (as opposed to library files) then you'll need to use GFlows Packages.

Creating a GFlows Package

A GFlows Package consists of:

1. A manifest. This is a simple JSON file listing the contents of the library. The file must be name gflowspkg.json, be in the root directory of the package, and must list the files using relative paths to the root. For example:

{
  "name": "my-lib",
  "files": [
    "workflows/my-lib-workflow.jsonnet",
    "libs/my-lib.libsonnet"
  ]
}

2. The library files. These should be hosted relative to the manifest. For example, if the above manifest is at https://example.com/my-lib/gflowspkg.json, then the workflow file listed should be available at https://example.com/my-lib/workflows/my-lib-workflow.jsonnet.

Using GFlows Packages

To add a package, add the root directory or URL to the dependencies section of the config. For example:

templates:
  engine: jsonnet
  defaults:
    dependencies:
    - https://example.com/my-lib

In the above example, the library could be used like this:

// my-workflow.jsonnet
local my_lib = import 'my-lib.libsonnet';
// etc

Git and Private Packages

If you want to use a private repository for your packages, you can list the repository as a dependency:

templates:
  engine: ytt
  defaults:
    dependencies:
    # If the .gflowspkg.json file is in the root of the repo:
    - [email protected]:my-org/my-pkg.git
    # If the .gflowspkg.json file is in a subdirectory:
    - [email protected]:my-org/my-pkg.git/path/to/package

If you use SSH for GitHub access on your dev machine, this should work for development without further changes. For CI on GitHub, you'll need to configure SSH. A good way to do this is:

1. Create a read only deploy key for the repo you need access to.
2. Add the private SSH key as a secret in the repository you use GFlows in.
3. Use an action such as webfactory/ssh-agent to configure SSH for the workflow.

For example, if you're using ytt, you might change your .gflows/workflows/gflows/gflows.yml template like this:

jobs:
  check_workflows:
    name: check-workflows
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
+   - uses: webfactory/[email protected]
+     with:
+       ssh-private-key: ${{ secrets.PRIVATE_DEPLOY_KEY }}
    - uses: jbrunton/setup-gflows@v1
      with:
        token: ${{ secrets.GITHUB_TOKEN }}
    - name: validate workflows
      env:
        GFLOWS_CONFIG: .gflows/config.yml
      run: ./gflows check

Examples

• For a Jsonnet example, see remote-lib-jsonnet.
• For a ytt example, see remote-lib-ytt.