ret-bang variant trips llvm assert "Invalid constantexpr cast!"

[jruderman]

fn my_err(s: str) -> ! { log_err s; fail; }
fn main() { my_err("bye") == 3u; }

Assertion failed: (CastInst::castIsValid(opc, C, Ty) && "Invalid constantexpr cast!"), function getCast, file Constants.cpp, line 1217.

[jruderman]

Switch the operands around and we get a different error message:

fn my_err(s: str) -> ! { log_err s; fail; }
fn main() { 3u == my_err("bye"); }

Assertion failed: (getOperand(0)->getType() == getOperand(1)->getType() && "Both operands to ICmp instruction are not of the same type!"), function ICmpInst, file /Users/jruderman/code/llvm/include/llvm/Instructions.h, line 643.

[jruderman]

Here's a version that uses early-ret instead of bang:

fn wsucc(n: int) -> int {
    { ret n + 1 } == 0;
}
fn main() {}

It's interesting that if I replace "==" with "+" I get "error: binary operation + cannot be applied to type _|_". Why is that the case for "+" but not for "=="?

[catamorphism]

"error: binary operation + cannot be applied to type _|_" is a typechecker bug -- values of type _|_ are usable in any context. I'll look into that too.

[catamorphism]

The first two programs enjoyed different results because trans was assuming both sides of the == had the same type, and was choosing what to do based on the RHS type.

If it chooses _|_, then we see the first error message (about the bad cast) because it was falling into cmp glue, since _|_ is not a scalar type.

If it chooses uint, then we see the second error message, because it tries to generate an unsigned int comparison but my_err has the wrong return type.

So far as I can tell, the correct thing to do is to just not generate code for the comparison if one operand is divergently typed. I have a fix which I'm about to check in.

Next I'll look at the early-ret version.