Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for Jinja in a destination node header and footer #119

Merged
merged 4 commits into from
Sep 4, 2024

Conversation

tomreitz
Copy link
Collaborator

This PR adds support for a destination node's header and footer to contain Jinja. The first row of data is passed in, so __row_data__.pop('__row_data__').keys() contains the column names, which can be used to define a dynamic header/footer. For example:

destinations:
  input_no_student_id_match:
    source: $transformations.input_no_student_id_match
    template: ./verbatim.csvt
    header: |
      {%raw%}{% for k in __row_data__.pop('__row_data__').keys() %}{{k}}{% if not loop.last %},{% endif %}{% endfor %}{%endraw%}

    extension: csv
    linearize: True

@tomreitz tomreitz requested a review from jayckaiser August 22, 2024 14:53
@tomreitz
Copy link
Collaborator Author

tomreitz commented Sep 4, 2024

@jayckaiser I updated this so the "first row" (and the corresponding __row_data__) is only computed (to be passed into the header/footer template) if either the header or footer in fact contain Jinja (via util.contains_jinja()). So in most cases we skip this potentially expensive step.

I also found and fixed a bug in the DEFAULT_TEMPLATE which was causing a Jinja parse error "circular reference detected" when it was actually used.

@tomreitz tomreitz merged commit ef9f882 into main Sep 4, 2024
@tomreitz tomreitz deleted the feature/jinja_in_destination_header_footer branch September 4, 2024 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants