Dereferencing a ref type buggy

[timotheecour]

after #11751 was merged, the behavior of type dereferencing seems buggy

• the compiler SIGSEGV for when compiles(Foo2[])
• dereferencing a ref type (ref to a generic) doesn't work (see below: type Goo1d = Goo1c[])
• some dereferencing behavior seem suspicious; in general we shouldn't conflate dereferencing (only for ref / ptr would make sense IMO) vs accessing generic arguments (for which other apis should be used eg typetraits.genericHead, and extractGeneric ([superseded] new macro for generic reflection: extractGeneric(Foo2[float, string], 0) is float #8554) otherwise it gives ambiguities
• inconsistent, eg:

doAssert (Bar[]) is string # passes
type Bar2 = Bar[] # Error: illformed AST

Example

type
  MyType = object
    a: int32
    b: int32
    c: int32

  MyRefType = ref MyType

echo sizeof(MyRefType[])
echo MyRefType[]

type Foo2 = object
  x: int

doAssert MyRefType[] is MyType
# echo sizeof(Foo2[]) # SIGSEGV: Illegal storage access
# when compiles(Foo2[]):discard # even checking that gives: SIGSEGV: Illegal storage access

type
  MyType2 = ref object
    a: int32
    b: int32
    c: int32
  MyRefType2 = ref MyType2

type Bar = seq[string]
doAssert (Bar[]) is string # is that ok? there's other ways to access generic parameters
# type Bar2 = Bar[] # Error: illformed AST: Bar[]
# doAssert string is (Bar[]) # Error: illformed AST: Bar[]

doAssert (ref ref string)[][] is string

# these pass, i guess they're ok?
doAssert (static string)[] is string
doAssert string[] is char
# doAssert char is string[] # Error: illformed AST: string[]
doAssert string isnot (ref char)

type Goo1[T] = object
  # x: T
type Goo1b = Goo1[float]
echo Goo1b[] # BUG: prints as 
echo Goo1b[].default
doAssert Goo1b[] is Goo1 # bug: runtime error if you uncomment x: T
# doAssert Goo1 is Goo1b[] # Error: illformed AST: Goo1b[] (same as above cases)

type Goo1c = ref Goo1b
doAssert Goo1c[] is Goo1b
# type Goo1d = Goo1c[] # Error: illformed AST: Goo1c[]; this should definitely work: Goo1c was a `ref X`

Current Output

inline in the example

Possible Solution

make [] only compile for ref T or ptr T
other use cases (eg generic access, including for special seq) should use extractGeneric

[mratsim]

I already mentioned my skepticism on IRC but I think we should have a deref for typedesc[ref or ptr] in typetraits rather than overloading this already overloaded [] operator nim-lang/RFCs#154

[zah]

Here is the solution I would use:

template `[]`*(T: typedesc): typedesc =
  typeof(default(T)[])

This would also work for user-defined smart pointer types that have a deref operation.

[mratsim]

But that requires an instantiation.

[zah]

@mratsim, I've seen this misconception being mentioned before. It's wrong. The expression given to typeof() is not evaluated, it's just analyzed by the compiler front-end in order to determine its type.

[Araq]

I reverted this PR.