snapi

snapi (/sneɪ.piˈaɪ/) is a simple yet flexible code generation framework. It uses Python, but can generate code for any programming language - or any text in general.

Why?

The general use case for this framework is to create code generation pipelines, where

• inputs are abstract, single-source-of-truth API definitions, and
• outputs are files with boilerplate code based on these inputs.

Here are some scenarios where an abstract API definition + code generation might be a good idea:

• Function declarations that are repeated multiple times, i.e. for interfaces, implementations, tests, ...
• Boilerplate wrappers that forward or convert between layers of an architecture
• SDKs for multiple programming languages using the same general API
• Semantic checks on API changes that require an easy to parse definition, i.e. to programmatically test for backward compatibility

Other existing code generators may require writing no code at all, but they often only produce specific outputs, and extending them is not always easy. For these scenarios, the goal of snapi is to make writing a custom generator as easy as possible.

Features

• Simple, opinionated framework to define inputs, processing steps and outputs
• Jinja2 integration for templates, with extension to mix generated and non-generated code in the same file
• Caching of re-used data
• Facilities for logging
• Useful utilities for naming conversions between multiple programming languages

Development status

This project is still work-in-progress. Current list of open tasks:

• Main features
• Documentation
• Examples
• Tests
• CI pipeline
• PyPI package

Installation

With pip:

$ cd pkg
$ pip install .

For usage in a container, see here.

A brief example

1. Design your API model in YAML/JSON

API specification (YAML):

services:
  - name: files
    functions:
      - name: upload
        args:
          - local_path: string
          - remote_path: string

      - name: download
        args:
          - remote_path: string
          - local_path: string

      - name: list_dir
        args:
          - remote_path: string = "/"
        returns: list<string>

2. Create templates using Jinja markup

Python template:

{% for fn in functions %}
def {{fn.name}}({{fn.args | format_py_args}}) -> {{fn.return_type}}:
  {% section fn.name + "_impl" %}
  # TODO: Implement.
  # Note: User code within a section is generally preserved between generator runs.
  {% endsection %}
{% endfor %}

Go template:

// TODO

Templates can use a transformed input model that contains language-specific data.

3. Implement generator with snapi

Create generator and declare inputs:

import snapi

g = snapi.Generator()

g.add_inputs(
  name="api_spec",
  impl=snapi.input_from_single_file,
  args={"path": "spec/api.yaml"}
)

Transform to input model for use in language-specific templates:

g.add_transformer(
  name="py_data",
  inputs="api_spec",
  impl=my_python_transformer
)

g.add_transformer(
  name="go_data",
  inputs="api_spec",
  impl=my_go_transformer
)

Declare outputs and run generator:

g.add_outputs(
  name="py",
  data="py_data",
  impl=write_py_outputs,
  args={"out_dir": "out/py", "tpl_dir": "templates/py"}
)

g.add_outputs(
  name="go",
  data="go_data",
  impl=write_go_outputs,
  args={"out_dir": "out/go", "tpl_dir": "templates/go"}
)

g.run()

An example output implementation:

def write_py_outputs(outputs: snapi.Outputs, services: List[models.py.Service], out_dir: str, tpl_dir: str):
  for service in services:
    outputs.to_file(
      os.path.join(out_dir, service.name + ".py"),
      template=f"{tpl_dir}/service.py.jinja",
      data={"service": service}
    )

Design rationale

Why not use gRPC/OpenAPI/Swagger/... instead?

If there is an existing tool or framework that already meets your requirements w.r.t. to supported input schemas and targeted output code, it is probably more effective to use that. The role of this framework is to help you implement requirements for which no ready-to-use solution exists yet.

Why require code instead of providing an approach that is fully declarative, i.e. rules defined in a YAML file to map inputs, templates and outputs.

Especially when targeting different programming languages, such mappings may require more than just a simple set of rules. For Python, a single module.yaml input might result in a single generated .py. For C++, there might be multiple header and source files with an entirely different layout. In general, conditionals, loops and other constructs would be required, so expressing this in Python code is the approach we prefer.

That being said, the goal is to make implementing a code generator as easy as possible. Eventually, this could mean including reusable components for common input schemas and template data models for common programming languages. All that would be left to the user is writing YAML specs, templates and mapping logic.