Add recording and slides from DebConf16 presentation by James Shubin #52
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -75,6 +75,8 @@ We'd love to have your patches! Please send them by email, or as a pull request. | |||
| * Felix Frank; blog: [Puppet Powered Mgmt (puppet to mgmt tl;dr)](https://ffrank.github.io/features/2016/06/19/puppet-powered-mgmt/) | |||
| * James Shubin; blog: [Automatic clustering in mgmt](https://ttboj.wordpress.com/2016/06/20/automatic-clustering-in-mgmt/) | |||
| * James Shubin; video: [Recording from CoreOSFest 2016](https://www.youtube.com/watch?v=KVmDCUA42wc&html5=1) | |||
| * James Shubin; video: [Recording from DebConf16: Next Generation Config Mgmt](http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/Next_Generation_Config_Mgmt.webm) | |||
There was a problem hiding this comment.
"Recording from DebConf16" is probably sufficient. It's all about mgmt.
|
@jacksgt Great, thank you! Your patch is failing because of the trailing whitespace. Can you put it all on one line, and rebase please? |
|
Sure, done! |
|
On Wed, Jul 6, 2016 at 4:02 PM, jacksgt [email protected] wrote:
Looks great, merged thanks! Cheers, |
purpleidea
pushed a commit
that referenced
this pull request
Apr 25, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 7, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 7, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 12, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 15, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 17, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 23, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 28, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 28, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 28, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 28, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
May 28, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 1, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 1, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 1, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 1, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 7, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 25, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 25, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 25, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
gelisam
added a commit
to gelisam/mgmt
that referenced
this pull request
Jun 25, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message purpleidea#3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message purpleidea#4:
lang: move FuncValue to its own package
This is the commit message purpleidea#5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message purpleidea#6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message purpleidea#7:
lang: ExprFunc does not need to store its value
This is the commit message purpleidea#8:
spelling
This is the commit message purpleidea#9:
convert between SimpleFn, FuncValue, and Func
This is the commit message purpleidea#10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message purpleidea#11:
ExprFunc.Func()'s three cases
This is the commit message purpleidea#12:
FunctionFunc is no longer used
This is the commit message purpleidea#13:
ExprBool.MergedGraph()
This is the commit message purpleidea#14:
ExprIf.MergedGraph()
This is the commit message purpleidea#15:
ExprFunc.MergedGraph()
This is the commit message purpleidea#16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message purpleidea#17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message purpleidea#18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message purpleidea#19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message purpleidea#20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message purpleidea#21:
restore ExprFunc::Graph() and Func()
This is the commit message purpleidea#22:
fix type errors
This is the commit message purpleidea#23:
ExprCall.MergedGraph()
This is the commit message purpleidea#24:
a more precise type for args
This is the commit message purpleidea#25:
GraphTxn
This is the commit message purpleidea#26:
ReversibleTxn.AddGraph
This is the commit message purpleidea#27:
ExprFunc.MergedGraph()
This is the commit message purpleidea#28:
ExprVar.MergedGraph()
This is the commit message purpleidea#29:
sub-graph is spelled subgraph in this codebase
This is the commit message purpleidea#30:
ExprFunc.mkFunc() was unused
This is the commit message purpleidea#31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message purpleidea#32:
move ConstFunc to lang.funcs
This is the commit message purpleidea#33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message purpleidea#34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message purpleidea#35:
FuncValue.Call()
This is the commit message purpleidea#36:
drop MergedGraph's unused txn parameter
This is the commit message purpleidea#37:
move CallFunc to funcs.simple
This is the commit message purpleidea#38:
Func from channel
This is the commit message purpleidea#39:
ChannelBased{Source,Sink}Func
This is the commit message purpleidea#40:
MapFunc
This is the commit message purpleidea#41:
remove unused CallFunc field
This is the commit message purpleidea#42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message purpleidea#43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message purpleidea#44:
extend environment with StmtProg's local variables
This is the commit message purpleidea#45:
delete unused VarFunc
This is the commit message purpleidea#46:
appease govet
This is the commit message purpleidea#47:
add imported variables to the environment
This is the commit message purpleidea#48:
add builtins to the environment
This is the commit message purpleidea#49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message purpleidea#50:
XXX: wip new function engine
This is the commit message purpleidea#51:
comments
This is the commit message purpleidea#52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message purpleidea#53:
f(...) support for ExprCall
This is the commit message purpleidea#54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message purpleidea#55:
don't recreate the subgraphInput
This is the commit message purpleidea#56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message purpleidea#57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message purpleidea#58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 27, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 29, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 29, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 29, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 29, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jun 30, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jul 3, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jul 15, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jul 19, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jul 27, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Jul 30, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 8, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 9, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 9, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 13, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 20, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 21, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 22, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 22, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 27, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 28, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 28, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 28, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 29, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 30, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Aug 31, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Sep 4, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Sep 4, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Sep 19, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Sep 19, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Sep 19, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
purpleidea
pushed a commit
that referenced
this pull request
Sep 19, 2023
This is the 1st commit message:
lang: Split FuncValue into FuncValue and SimpleFn
Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.
Here is why this is a mistake:
The output of a function like
$f = fn(x) {
Shell(Sprintf("seq %d", x))
}
varies over time, while a single Value does not. Thus, code like
Map($f, list(1, 2))
would first produce the value list("1", "1"), but then it would
_not_ update to list("1", "2") when "seq 2" produces its second
line. That's because with the mistaken design, when Map receives a
new FuncValue or a new input list of N elements, Map calls the
function from Value to Value N times and produces a single output
list of N elements.
Here is why the corrected design is better:
Here's what happens with this new design when Map receives a new
FuncValue or a new input list of N elements.
First, Map constructs N item-input nodes, each of which extracts a
different entry from the list. Then, Map calls the function from
Vertex to Vertex N times, once for each item-input node, and thus
obtain N item-output nodes. Finally, Map constructs an
item-collecting node which constructs a list out of all of its
inputs, and Map connects the N item-output nodes to the
item-collecting node. This item-collecting node is the output of
Map.
The Vertex to Vertex function constructs and connects its own nodes;
in this case, it constructs an Sprintf node and connects the
item-input node to it, and then constructs a Shell node and connects
the Sprintf node to it, and then returns the Shell node as the
item-output node.
The two Shell node in this sub-graph emit a first value "1", which
propagates to the item-collecting node and causes it to output a
first value list("1", "1"). Then, the second Shell node emits a
second value "2", which propagates to the item-collecting node and
causes it to output a second value list("1", "2"), as desired.
Here is how this commit brings us closer to the above plan:
Changing FuncValue throughout the codebase is a big change. One of
the difficulties is that it is not just nodes which are emitting
FuncValues, there are also many other places in the code where
FuncValue is used to hold a golang function from Value to Value.
Some of those places now need to hold a golang function from Vertex
to Vertex, but other places still need to hold a golang function
from Vertex to Vertex.
Thus, as a first step, we need to split FuncValue into two types.
This commit splits the old FuncValue into two types:
1. The new FuncValue will hold a function from Vertex to Vertex.
FuncValue is a Value.
2. A new type named "SimpleFn" will hold a function from Value to
Value. SimpleFn is not a Value.
This commit replaces occurrences of the old FuncValue with one of
those two new types, as appropriate. This commit does not yet adapt
the surrounding code to make use of the new representation; that
will be done in future commits. I have annotated the missing parts
with the following panic message in order to make it easy to find
which parts still need to be implemented. The "..." part explains
what needs to be implemented.
panic("TODO [SimpleFn]: ...");
Here's where I need help:
One part of the code which is not clear to me are the parts which
use reflection. I don't understand the purpose of that code well
enough to explain what needs to be implemented. I have annotated
those "known unknown" parts of the remaining work with the following
panic message in order to make it easy to find which parts still
need more thinking and planning:
panic("TODO [SimpleFn] [Reflect]: ...");
This is the commit message #2:
lang: Add the core func graph Txn API
This will eventually let functions change the running graph via a
transaction API.
At the moment the core Lock and Unlock primitives aren't implemented.
This is the commit message #3:
lang: A reversible wrapper around Txn
This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.
This is the commit message #4:
lang: move FuncValue to its own package
This is the commit message #5:
lang: combine lang/func/structs and ast
Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.
This is the commit message #6:
lang: CallExpr must generate a subgraph
FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.
This is the commit message #7:
lang: ExprFunc does not need to store its value
This is the commit message #8:
spelling
This is the commit message #9:
convert between SimpleFn, FuncValue, and Func
This is the commit message #10:
no need for ExprCall.argVertices
it's the exact same thing as ExprCall.Args
This is the commit message #11:
ExprFunc.Func()'s three cases
This is the commit message #12:
FunctionFunc is no longer used
This is the commit message #13:
ExprBool.MergedGraph()
This is the commit message #14:
ExprIf.MergedGraph()
This is the commit message #15:
ExprFunc.MergedGraph()
This is the commit message #16:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.
This is the commit message #17:
lang: ast, interfaces: Add MergedGraph signature and implementation
This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.
This is the commit message #18:
Use MergedGraph signature and implementation
This puts it into play, but doesn't initialize the input args at all.
This is the commit message #19:
Restore ExprBool.Graph() and Func()
This reverts commit 3ea3845.
This is the commit message #20:
Restore ExprIf.Graph()
This reverts commit a62889e.
This is the commit message #21:
restore ExprFunc::Graph() and Func()
This is the commit message #22:
fix type errors
This is the commit message #23:
ExprCall.MergedGraph()
This is the commit message #24:
a more precise type for args
This is the commit message #25:
GraphTxn
This is the commit message #26:
ReversibleTxn.AddGraph
This is the commit message #27:
ExprFunc.MergedGraph()
This is the commit message #28:
ExprVar.MergedGraph()
This is the commit message #29:
sub-graph is spelled subgraph in this codebase
This is the commit message #30:
ExprFunc.mkFunc() was unused
This is the commit message #31:
CallFunc.Stream() should add Funcs to the graph, not Exprs
This is the commit message #32:
move ConstFunc to lang.funcs
This is the commit message #33:
move conversion functions to lang.funcs.simple
I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.
This is the commit message #34:
it is the responsibility of the function engine to call Init on the nodes
This is the commit message #35:
FuncValue.Call()
This is the commit message #36:
drop MergedGraph's unused txn parameter
This is the commit message #37:
move CallFunc to funcs.simple
This is the commit message #38:
Func from channel
This is the commit message #39:
ChannelBased{Source,Sink}Func
This is the commit message #40:
MapFunc
This is the commit message #41:
remove unused CallFunc field
This is the commit message #42:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #43:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #44:
extend environment with StmtProg's local variables
This is the commit message #45:
delete unused VarFunc
This is the commit message #46:
appease govet
This is the commit message #47:
add imported variables to the environment
This is the commit message #48:
add builtins to the environment
This is the commit message #49:
accumulate all the imported variables
previously, we were only keeping the imported variables from the last
import statement.
This is the commit message #50:
XXX: wip new function engine
This is the commit message #51:
comments
This is the commit message #52:
remove Engine.{Lock,Unlock} placeholders
This is the commit message #53:
f(...) support for ExprCall
This is the commit message #54:
[REVERT ME] remove unrelated files to make VS Code happy
This is the commit message #55:
don't recreate the subgraphInput
This is the commit message #56:
CallFunc now takes a single input, the FuncValue
Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.
This is the commit message #57:
GraphTxn must take a pointer in order to modify the graph
This is the commit message #58:
GraphTxn is unused
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Just watched the recording from DebConf, checked out the repo and saw that the video link was missing.
Fixed 👍