Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: addl luau helper functions #2362

Merged
merged 3 commits into from
Dec 21, 2024
Merged

Conversation

jqnatividad
Copy link
Collaborator

feat(luau): add cumulative and lag helper functions

Implements additional helper functions for qsv's Lua DSL as suggested in #1782:

qsv_loadjson - load a JSON file into a Luau table

  • Cumulative functions:

    • qsv_cumsum: cumulative sum
    • qsv_cumprod: cumulative product
    • qsv_cumany: cumulative OR
    • qsv_cumall: cumulative AND
    • qsv_cummax: cumulative maximum
    • qsv_cummin: cumulative minimum
  • Lag/diff functions:

    • qsv_lag(col, k, default): Access previous values with configurable offset
    • qsv_diff(col, k, default): Difference between current and lagged values

qsv_loadjson is designed to work in CKAN so we can send package and resource info to DP+, so it can infer suggested metadata for the DRUF workflow, as we use the luau command for complex inferencing beyond the capabilities of MiniJinja.

This effectively gives us two DSLs for the DRUF workflow. MiniJinja for simple metadata inferences, and Luau for complex ones. Both MiniJinja and Luau are purpose-built as DSLs, so we don't need to use Python eval which leaves a gaping security hole.

The cumulative and lag/diff helpers enhances qsv's data analysis capabilities with R-like functionality.

Closes #1782

cc @ggrothendieck

@jqnatividad jqnatividad merged commit e87f3e9 into master Dec 21, 2024
13 checks passed
@jqnatividad jqnatividad deleted the 1782-addl-luau-helper-functions branch December 21, 2024 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

luau: additional helper functions
1 participant