Somehow add metaprogramming to type parameters

[sampersand]

Overview

Currently in RBS, methods and types take a fixed amount of known parameters. This makes sense for the majority of cases, but occasionally is insufficient (especially when dealing with code that forwards arguments/reuse of RBS interfaces, etc.). The syntax is bikesheddable.

Motivating Examples

1. Kernel#open and friends

The Kernel#open function, along with things like IO.new, File.read, etc. all have a (fairly lengthy) common set of keyword arguments they accept. For example, Kernel#open (slightly abridged):

module Kernel
  def self?.open: (
                  path file,
                  ?(int | string)? vmode,
                  ?int vperm,
                  ?mode: (int | string)?,
                  ?perm: int?,
                  ?flags: int?,
                  ?textmode: boolish,
                  ?binmode: boolish,
                  ?encoding: encoding?,
                  ?extenc: encoding?,
                  ?intenc: encoding?,
                  ) -> File
                | [T] (
                  path file,
                  ?(int | string)? vmode,
                  ?int vperm,
                  ?mode: (int | string)?,
                  ?perm: int?,
                  ?flags: int?,
                  ?textmode: boolish,
                  ?binmode: boolish,
                  ?encoding: encoding?,
                  ?extenc: encoding?,
                  ?intenc: encoding?,
                  ) { (File) -> T } -> T
end

Repeating them across all similar functions is both error-prone, and difficult to maintain. An alternative would be something like this:

module Kernel
  type open_kwargs = {
    ?mode: (int | string)?,
    ?perm: int?,
    ?flags: int?,
    ?textmode: boolish,
    ?binmode: boolish,
    ?encoding: encoding?,
    ?extenc: encoding?,
    ?intenc: encoding?,
  }

  def self?.open: (path file, ?(int | string)? vmode, ?int vperm, open_kwargs**) -> File
                | [T] (path file, ?(int | string)? vmode, ?int vperm, open_kwargs**) { (File) -> T } -> T
end

2. Creating a _Call interface

We currently have special syntactic support for Procs (^(...) -> ...), but not things that define .call. Most of the time in Ruby, you don't care about explicitly taking a Proc, but rather something that defines .call(...), which can also encompass Methods and custom types. It would be nice if we could have support for _Call (and friends, like _Index and _IndexAssign). Here's an example (like before, syntax bikesheddable):

interface _Call[P, K, B, R]
  def call: (P*, K**, &B) -> R
end

You could then use the interface like _Call[[Integer], { fizz: String, buzz: String }, nil, String]. Like before, the syntax can be modified, but I think the idea of allowing people to "forward" arguments from interfaces has merit.

3. Add full support for "forwarding methods"

Similar to point #1, this would make it easier for end users to create their own "forwarding methods" (eg def debug(...) = log(:debug, ...):

class MyLogger
  type level = :debug | :info | :warn | :error | :fatal

  def log: (level, ?String message, ?String call_site, ?cause: Exception?, **untyped extra_info) ?{ () -> String } -> void
  def debug: (?String message, ?String call_site, ?cause: Exception?, **untyped extra_info) ?{ () -> String } -> void
  def info: (?String message, ?String call_site, ?cause: Exception?, **untyped extra_info) ?{ () -> String } -> void
  def error: (?String message, ?String call_site, ?cause: Exception?, **untyped extra_info) ?{ () -> String } -> void
  def fatal: (?String message, ?String call_site, ?cause: Exception?, **untyped extra_info) ?{ () -> String } -> bot
end

class MyLogger
  type level = :debug | :info | :warn | :error | :fatal

  type pargs = [?String, ?String]
  type kwargs = { ?cause: Exception?, **untyped extra_info }
  type block = ?{ () -> String }
  type ret = void

  def log: (level, pargs*, kwargs**, &block) -> ret
  def debug: (pargs*, kwargs**, &block) -> ret
  def info: (pargs*, kwargs**, &block) -> ret
  def error: (pargs*, kwargs**, &block) -> ret
  def fatal: (pargs*, kwargs**, &block) -> bot
end

Syntax possibilities

As you've seen before, one of the possibilities we could use would be to make use of our tuple/record types, and simply have some special syntax to indicate that we want to "splat" them out into the arguments for the function.

[** < typename]

...

Another idea I've been kicking around is making use of .... Currently, in RBS you can already use ... (such as def foo: ... | <more definitions here>), which is used to mean "accept any signatures that parent classes define, in addition to any other definitions I'll add."

I want to somehow use the ... syntax as part of RBS as well, like Ruby does. (Like ruby, ... can only be used once per function signature, as the last argument). Here's just a few ideas of how it could be used:

Method signatures

def base: (Integer, ?Integer, ?a: Symbol) ?{ () -> String } -> Integer

# Use `base`'s signature exactly; no return type means use `base`'s.
def forward_all: (...base)

# Using a return type will replace `base`'s
def custom_return: (...base) -> String

# Like in ruby, you can add additional positional parameters
def also_take_args: (Complex foo, ?Rational bar, ...base)

# But you can also use it to add additional keyword arguments
def has_radix: (Complex foo, ?Rational bar, ?radix: Numeric, ...base)

# You can supply a block to replace the return value
def requires_a_block_now: (...base) { () -> _ToS }

# You can use `nil` to indicate you don't want to forward an aspect of the base.
def dont_forward_positionals: (*nil, ...base)
def dont_forward_kwargs: (**nil, ...base) #NB: This may conflict with ruby's `**nil` syntax; todo: look into it.
def dont_forward_block: (&nil, ...base)
def dont_forward_return: (...base) -> UseACustomReturnTypeHere

# In this example, there's no reason in using `...base`, as we indicate we don't
# want any of `base`'s values being forwarded.'
def no_reason_to_use_forwarding_syntax: (*nil, **nil, &nil, ...base) -> CustomType

# You can use `*nil` and friends to only copy positional arguments:
def only_forward_kwargs: (myarg1, ?myarg2, *nil, &nil, ...base) -> MyReturn

# Call `super` with the rest of your args:
class Human < Animal
  def initialize: (String name, ...super) -> void
end

Upsides:

• Fairly intuitive to use
• Matches Ruby's style
• Reuses the ... syntax
• Doesn't allow for super complicated signatures

Downsides:

• Using **nil might conflict with ruby's **nil
• We now need a parser for method paths (eg def open: (<positional args here>, ...IO#open) -> File needs to not parse # as a comment)
• The -> foo syntax is now not always required for functions (which means we can't use omitting -> x as an alias for -> void, or other purposes)

Generics forwarding

We can also use ... to forward generic parameters:

interface _Call[...Fwd]
  def call: (...Fwd)
end

interface _IndexAssign[...Fwd]
  def []=: (&nil, ...Fwd) # don't accept keyword arguments
end

There's probably more examples of generics using this, i'll add them when I think of them.

[ParadoxV5]

Whether pargs* and kwargs** are ugly or not aside, note that

def debug: (pargs*, kwargs**, &block) -> ret

Inconsistently, it’s &block rather than block&
[P.S. and ...base rather than base...]

[ParadoxV5]

&block conflicts with #1622 just like pargs* vs. *pargs.

[ParadoxV5]

2. Creating a _Call interface

Relevant: #1223 (comment)

[ParadoxV5]

Resolves: #420

[sampersand]

One problem with the ...base syntax is: how'd we typecheck the logger example? Is def debug: (?string, ?String, *nil, ...base) acceptable?

[ParadoxV5]

Forwarding can by default mean forwarding “the rest of the args” over, so something like that (syntax pending) is fine.

[ParadoxV5]

# You can use `nil` to indicate you don't want to forward an aspect of the base.
def dont_forward_positionals: (*nil, ...base)
def dont_forward_kwargs: (**nil, ...base) #NB: This may conflict with ruby's `**nil` syntax; todo: look into it.
def dont_forward_block: (&nil, ...base)
def dont_forward_return: (...base) -> UseACustomReturnTypeHere

This isn’t intuitive. Ability to selectively forward *, **, & and -> … would be clearer. Imagine:

def only_forward_return: (*nil, **nil, &nil, ...base)

[ParadoxV5]

Maybe something like base*, base** and base&

[ParadoxV5]

[** < typename]

This is my idea, originally dumped in the Discord. Samp would like me to compile the thoughts over here.

[seat reservation]

[ParadoxV5]

Synopsis

module Kernel
  type open_args = [path file, …]
  type open_kwargs = {?mode: (int | string)?, …}
  # Unchanged from Samp’s syntax,
  # but note that both requires support for optional fields in explicity tuples and records

  def self?.open: [* < open_args, ** < open_kwargs] (*, **) -> File
                | [* < open_args, ** < open_kwargs, T] (*, **) { (File) -> T } -> T
end

Explaination

Essentially, this idea gives *args, **kwargs and &block an alternate type specification syntax by letting them act similar to type arguments. That is:

• [* < Array[E]] (*) is the same as (*E)
• [** < Hash[K, V]] (**) is the same as (**K => V)
• [& < ^…] (&) is the same as () { … }(See also #1622) But each vararg part may use heterogenous types or even DRYly source from atypedefinition. In addition,...may be a combo meal of all three, with the right side of<` be either a proc subytype or the name of a method (delegation)

_Call

interface _Call[A, K, B, R]
  def call: [* < A, ** < K, & < B] (*, **, &) -> R
end

(OP example did not constraint K to a `Hash[untyped, untyped] and etc., so I’ll leave that implicity to another discussion.)

[ParadoxV5]

Context: This idea originated from the following syntax idea about constraints on self and other things, i.e. “where type”:

def require: [$LOAD_PATH < Array[string]] (path path) -> bool # Kernel
# Constraint on `self` which might have unique singleton stuff
def <: [self < _WithSpaceshipOperator] (untyped other) -> bool # Comparable
# Also:
# * instance (inferred from the context module/class; i.e., considers singleton classes as different)
# * class

[ParadoxV5]

There are several niche cases that having type splatting solves:

• Add Support for Non-Symbol Keyowrds #1645
  • E.g., def add_exponentiated_numbers: (Hash[Integer base, Integer exponent]**) -> Integer
• Order-sensitive vararg / type arg
  • E.g., FFI::Struct.layout: ([Symbol field, type, ?Integer offset]*) -> StructLayout

---

It’s actually almost possible to use the same char to splat, since

• only Arrays/tuples can become varargs
• only Hashes/records can become variable kwargs
• only Procs can become blocks

This is like @sampersand’s original idea – fragments splattable using %.
Problems include:

• All combinations of unions between arrays, hashes and procs – for starters, bot and untyped – will cause ambiguity. Same if nil is treated uniquely.
• What would the syntax to “splat” the return be?

We could restrict the fragments to literal typs only, since

• Array[T]* is the same as *T
• Hash[K, V]** is the same as **K => V with Add Support for Non-Symbol Keyowrds #1645
• Proc& is the same as
  • &untyped with Add in &untyped #1622
  • { (?) -> void } with Untyped block args #1335
  • { (́…) -> … } with give _ToProc#to_proc a return type #1223 (comment)
• It takes no effort to just write out the return -> R

---

Splatting Array[T]s and Hash[Symbol, T]es can also indicate varargs and var-kwargs.

• Suggestion/discussion about typing with the splat operator (*, **, "rest" or "varargs") #322

We shouldn’t touch the existing syntaxes, however, and thus unfortunately * and ** prefixes can’t be repurposed to splatting, for this is a wild backward incompatibility that breaks things everywhere.

All existing proposals are more or less ugly, if not limited. They almost look like macros to some extent, yet they aren’t the kind of macros that solve WET issues.
Examples:

• Annotate members of Struct and Data classes (alternative: struct/data class literals) [ping: @DanielPechersky]
• FFI::AbstractMemory’s matrix of methods – https://github.com/ParadoxV5/ruby-ffi/blob/rbs-init/sig/ffi/abstract_memory.rbs

If we are gonna add something that looks like macros and quacks like macros, we might as well go all in and actually add macros.
Here’s a wild, title-says-it-all idea: Support ERB?

[ParadoxV5]

New, better idea!

module Kernel
  type open_args = (
    path file,
    ?(int | string)? vmode,
    ?int vperm,
    ?mode: (int | string)?,
    ?perm: int?,
    ?flags: int?,
    ?textmode: boolish,
    ?binmode: boolish,
    ?encoding: encoding?,
    ?extenc: encoding?,
    ?intenc: encoding?,
  )

  def self?.open: open_args -> File
                | [T] open_args { (File) -> T } -> T
end

Yes, open_args is a _parameters_.
Surprisingly, (…) is kinda underused in RBS.

original new idea

module Kernel
  # Yes, this is a method body type: no `^`
  type open[R, B] = (
    path file,
    ?(int | string)? vmode,
    ?int vperm,
    ?mode: (int | string)?,
    ?perm: int?,
    ?flags: int?,
    ?textmode: boolish,
    ?binmode: boolish,
    ?encoding: encoding?,
    ?extenc: encoding?,
    ?intenc: encoding?,
  ) { B } -> R

  def self?.open: open[File, nil]
                | [T] open[T, (File) -> T] # this is also a “method body”: no `^{ … }`
end

• Ruby &nil means “pass no block” explicitly, mainly for super(… &nil)
  • There’s also a (suspended) proposal for &nil in method signature.
    We could support it as { nil }, but I prefer error on extraneous blocks instead.
• Alo note that this makes Suggestion: how about & for overloaded method type separator, instead of |? #566’s point clear.

[ParadoxV5]

class MyLogger
  type level = :debug | :info | :warn | :error | :fatal
  # How about we include the `_block?_` as well? Like `**kwargs`, the `&blk` is also a part of the args.
  type log_args = (?String message, ?String call_site, ?cause: Exception?, **untyped extra_info) ?{ () -> String }

  # nested `_parameters_` auto-splat?
  def log: (level, log_args) -> void
  # so `(args)` is the same as `args`?
  def debug: (log_args) -> void
  …

[ParadoxV5]

module Kernel
  type open_args = [path file, …]
  type open_kwargs = {?mode: (int | string)?, …}
  # Unchanged from Samp’s syntax,
  # but note that both requires support for optional fields in explicity tuples and records
  
  def self?.open: (*?open_args, **?open_kwargs) -> File
                | [R] (*?open_args, **?open_kwargs) { (File) -> R } -> R
end

class MyLogger
  type level = :debug | :info | :warn | :error | :fatal
  type log_args = [?String message, ?String call_site, ?cause: Exception?]
  
  def log: (level, *?log_args, **untyped extra_info) ?{ () -> String } -> void
  def debug: (*?log_args, **untyped extra_info) ?{ () -> String } -> void
  …

抛砖引玉; inspired by

rbs/stdlib/json/0/json.rbs

Line 992 in 36f84b7

def self?.load_file: (string path, ?json_options opts) -> untyped