Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add label polymorphism #4388

Merged
merged 40 commits into from
Apr 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
656fbf8
feat: Start on polymorphic labels
Jan 5, 2022
54a2d5c
test: Use expect_test for better string comparisons
Feb 21, 2022
6f304f0
refactor: Simplify Function unification
Feb 21, 2022
f4b7cf2
fix: record labels in the face of bound variables
Feb 21, 2022
4850af4
fix(poly-labels): Handle unbound variables
Feb 22, 2022
b44d90e
fix: Don't panic when converting converting zeroed ast::SourceLocation
Feb 22, 2022
df71f1e
test: Verify that we error on missing labels
Feb 22, 2022
caa4373
test: Check that labels get unified to strings
Feb 22, 2022
2016d25
docs: Add a Polymorphic label "RFC"
Feb 23, 2022
422edc9
feat: Try to get labels <: string unification working
Mar 29, 2022
7d8a4e0
chore: Fix the cffi tests
Mar 31, 2022
95c55a2
chore: Remove the no-package restriction for labels
Mar 31, 2022
0293eac
chore: Add a LabelPolymorphism feature flag
Apr 12, 2022
e35fa2f
chore: Handle errors when parsing property_key
Apr 13, 2022
1b92329
feat: Add some universe function to experimental with updated signatures
Apr 13, 2022
249e105
fix: Error when a label variable in a record label could not be made …
Apr 13, 2022
cd376cf
fix: Strings should not implement the Label kind
Apr 14, 2022
6f47855
chore: Fix some undefined reference errors in make fmt-flux
Apr 14, 2022
a1f867d
chore: Add the Label kind to the experimental/universe functions
Apr 14, 2022
b11ac06
chore: Add the Label kind to the flatbuffer representation
Apr 14, 2022
5f8fbb1
test: Make the columns test more accurate
Apr 14, 2022
cabb49c
chore: Update docs for experimental/universe
Apr 14, 2022
17ac065
feat: Represent label variables in the flatbuffer
Apr 19, 2022
f3bb91e
chore: make staticcheck
Apr 20, 2022
abb5e6d
test: Verify that label types work through variable bindings
Apr 20, 2022
61ea74a
chore: cargo clippy
Apr 20, 2022
59fbbb1
chore: make generate
Apr 20, 2022
cf36289
Apply suggestions from code review
Apr 22, 2022
344773f
chore: Update SPEC.md with label polymorphism
Apr 22, 2022
c74a199
chore: Ensure we error on unbound label variables
Apr 22, 2022
2b33c4e
fix: Remove the unbound label variable check
Apr 25, 2022
0331566
fix: Make copyMonoType work with record variables
Apr 25, 2022
19baaa5
chore: make generate
Apr 22, 2022
2b377e1
chore: Remove an unfinished sentence in docs/SPEC.md
Apr 26, 2022
4e8f6e4
feat: Use JSON to allow all kinds of features to be specified from th…
Apr 26, 2022
fa0b4ff
docs: Remove unneccessary todo
Apr 26, 2022
7a22672
chore: Address review comments
Apr 26, 2022
3b0e2d8
chore: Update the feature flag passing for make checkdocs
Apr 26, 2022
80453ed
chore: cargo clippy
Apr 27, 2022
1720617
chore: make generate
Apr 26, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions cmd/flux/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package main

import (
"context"
"encoding/json"
"fmt"
"io/ioutil"
"os"
Expand All @@ -23,7 +24,7 @@ var flags struct {
ExecScript bool
Trace string
Format string
Features []string
Features string
}

func runE(cmd *cobra.Command, args []string) error {
Expand Down Expand Up @@ -54,8 +55,10 @@ func runE(cmd *cobra.Command, args []string) error {
defer span.Finish()

flagger := executetest.TestFlagger{}
for _, feature := range flags.Features {
flagger[feature] = true
if len(flags.Features) != 0 {
if err := json.Unmarshal([]byte(flags.Features), &flagger); err != nil {
return errors.Newf(codes.Invalid, "Unable to unmarshal features as json: %s", err)
}
}
ctx = feature.Dependency{Flagger: flagger}.Inject(ctx)

Expand Down Expand Up @@ -115,7 +118,7 @@ func main() {
fluxCmd.Flags().StringVar(&flags.Trace, "trace", "", "Trace query execution")
fluxCmd.Flags().StringVarP(&flags.Format, "format", "", "cli", "Output format one of: cli,csv. Defaults to cli")
fluxCmd.Flag("trace").NoOptDefVal = "jaeger"
fluxCmd.Flags().StringSliceVar(&flags.Features, "feature", nil, "Adds a boolean feature flag. See internal/feature/flags.yml for a list of the current features")
fluxCmd.Flags().StringVar(&flags.Features, "feature", "", "JSON object specifying the features to execute with. See internal/feature/flags.yml for a list of the current features")

fmtCmd := &cobra.Command{
Use: "fmt",
Expand Down
28 changes: 25 additions & 3 deletions docs/SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -503,6 +503,25 @@ Note that an empty string is distinct from a _null_ value.

The length of a string is its size in bytes, not the number of characters, since a single character may be multiple bytes.

##### Label types (Upcoming/Feature flagged)

A _label type_ represents the name of a record field.
String literals may be treated as a label type instead of a `string` when used in a context that
expects a label type.

```
"a" // Can be treated as Label("a")
"xyz" // Can be treated as Label("xyz")
```
OfTheDelmer marked this conversation as resolved.
Show resolved Hide resolved

In effect, this allows functions accepting a record and a label to refer to specific properties of
the record.

```
// "mycolumn" is treated as Label("mycolumn") for when passed to `mean`
mean(column: "mycolumn") // Calculates the mean of `mycolumn`
OfTheDelmer marked this conversation as resolved.
Show resolved Hide resolved
```

##### Regular expression types

A _regular expression type_ represents the set of all patterns for regular expressions.
Expand All @@ -523,11 +542,9 @@ The length of an array is the number of elements in the array.
##### Record types

An _record type_ represents a set of unordered key and value pairs.
The key must always be a string.
The key can be a string or a [type variable](<#Type variables>).
The value may be any other type, and need not be the same as other values within the record.

Keys on a record may only be referenced statically.

Type inference will determine the properties that are present on a record.
If type inference determines all the properties on a record it is said to be bounded.
Not all keys may be known on the type of a record in which case the record is said to be unbounded.
Expand Down Expand Up @@ -555,6 +572,11 @@ The generated values may be of any other type but must all be the same type.
Flux functions can be polymorphic, meaning they can be applied to arguments of different types.
Flux supports parametric, record, and ad hoc polymorphism.

##### Type variables

Polymorphism are represented via "type variables" which are specified with a single uppercase
letter (`A`, `B`, etc).

##### Parametric Polymorphism

Parametric polymorphism is the notion that a function can be applied uniformly to arguments of any type.
Expand Down
Binary file not shown.
Binary file modified embed/stdlib/contrib/anaisdg/anomalydetection.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/anaisdg/statsmodels.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/bonitoo-io/alerta.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/bonitoo-io/servicenow.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/bonitoo-io/tickscript.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/bonitoo-io/victorops.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/bonitoo-io/zenoss.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/chobbs/discord.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/jsternberg/aggregate.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/jsternberg/influxdb.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/rhajek/bigpanda.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/sranka/opsgenie.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/sranka/sensu.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/sranka/teams.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/sranka/telegram.fc
Binary file not shown.
Binary file modified embed/stdlib/contrib/sranka/webexteams.fc
Binary file not shown.
Binary file modified embed/stdlib/date.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/aggregate.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/array.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/csv.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/geo.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/http/requests.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/oee.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/prometheus.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/query.fc
Binary file not shown.
Binary file added embed/stdlib/experimental/universe.fc
Binary file not shown.
Binary file modified embed/stdlib/experimental/usage.fc
Binary file not shown.
Binary file modified embed/stdlib/http.fc
Binary file not shown.
Binary file modified embed/stdlib/influxdata/influxdb/monitor.fc
Binary file not shown.
Binary file modified embed/stdlib/influxdata/influxdb/sample.fc
Binary file not shown.
Binary file modified embed/stdlib/influxdata/influxdb/schema.fc
Binary file not shown.
Binary file modified embed/stdlib/influxdata/influxdb/tasks.fc
Binary file not shown.
Binary file modified embed/stdlib/influxdata/influxdb/v1.fc
Binary file not shown.
Binary file modified embed/stdlib/internal/location.fc
Binary file not shown.
Binary file modified embed/stdlib/internal/promql.fc
Binary file not shown.
Binary file modified embed/stdlib/pagerduty.fc
Binary file not shown.
Binary file modified embed/stdlib/pushbullet.fc
Binary file not shown.
Binary file modified embed/stdlib/sampledata.fc
Binary file not shown.
Binary file modified embed/stdlib/slack.fc
Binary file not shown.
Binary file modified embed/stdlib/testing.fc
Binary file not shown.
Binary file modified embed/stdlib/timezone.fc
Binary file not shown.
Binary file modified embed/stdlib/universe.fc
Binary file not shown.
14 changes: 12 additions & 2 deletions internal/fbsemantic/semantic.fbs
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,18 @@ table Argument {
optional:bool;
}

union RecordLabel {
Concrete,
Var,
}

table Concrete {
id:string /*(required)*/;
}

table Prop {
k:string /*(required)*/;
v:MonoType /*(required)*/;
k:RecordLabel /*(required)*/;
v:MonoType /*(required)*/;
}

table PolyType {
Expand All @@ -118,6 +127,7 @@ enum Kind : ubyte {
Numeric,
Comparable,
Equatable,
Label,
Nullable,
Record,
Negatable,
Expand Down
121 changes: 106 additions & 15 deletions internal/fbsemantic/semantic_generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions internal/feature/flags.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions internal/feature/flags.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,9 @@
key: vectorizeOperators
default: false
contact: Markus Westerlind

- name: Label polymorphism
description: Enables label polymorphism in the type system
key: labelPolymorphism
default: false
contact: Markus Westerlind
Loading