wac plug error: "error: encoding produced a component that failed validation"

[estk]

Repro

https://github.com/estk/hotswap-example

The error

wac plug --plug types.wasm salutation.wasm -o tysal.wasm
error: encoding produced a component that failed validation

Caused by:
    instance not valid to be used as export (at offset 0x36d3bf)

Why this is surprising to me

Since the user-interface interface is local to the hotswap::salutation package and the plugged world types exports the user-interface interface, I would have thought that the imports were satisfied and that the underlying types in user-interface would be automatically re-exported.

WIT at a glance

In the salutation package we have the following:

package hotswap:[email protected];

interface salutation {
    use user-interface.{user};

    variant formal-honorific {
        sir,
        maam,
        sir-maam,
        custom(string)
    } 
    get-formal-honorific: func(u: user) -> formal-honorific;
    greet: func(u: user) -> string;
}

world app {
    import user-interface;
    export salutation;
}

...

interface salutation {
    use user-interface.{user};

    variant formal-honorific {
        sir,
        maam,
        sir-maam,
        custom(string)
    } 
    get-formal-honorific: func(u: user) -> formal-honorific;
    greet: func(u: user) -> string;
}

world app {
    import user-interface;
    export salutation;
}

[estk]

Based on this issue: bytecodealliance/wasm-tools#1798
I tried changing:

world app {
    import user-interface;
    export user-interface;
    export salutation;
}

Which results in the following with cargo component build -p salutation

The problem resolving this is that we now have a number of different Users all of which should actually be resolved to the same User. I'm not really sure how to tell wit-bindgen about this and my attempts to use the bindgen macro have been a dead end so far as well.

error[E0053]: method `greet` has an incompatible type for trait
   --> crates/salutation/src/lib.rs:13:17
    |
13  |     fn greet(u: User) -> String {
    |                 ^^^^ expected `User`, found a different `User`
    |
note: type in trait
   --> crates/salutation/src/bindings.rs:406:33
    |
406 |                     fn greet(u: User) -> _rt::String;
    |                                 ^^^^
    = note: expected signature `fn(exports::hotswap::salutation::user_interface::User) -> String`
               found signature `fn(bindings::hotswap::salutation::user_interface::User) -> String`
help: change the parameter type to match the trait
    |
13  |     fn greet(u: exports::hotswap::salutation::user_interface::User) -> String {
    |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

error[E0053]: method `get_formal_honorific` has an incompatible type for trait
   --> crates/salutation/src/lib.rs:24:32
    |
24  |     fn get_formal_honorific(u: User) -> FormalHonorific {
    |                                ^^^^ expected `User`, found a different `User`
    |
note: type in trait
   --> crates/salutation/src/bindings.rs:405:48
    |
405 |                     fn get_formal_honorific(u: User) -> FormalHonorific;
    |                                                ^^^^
    = note: expected signature `fn(exports::hotswap::salutation::user_interface::User) -> FormalHonorific`
               found signature `fn(bindings::hotswap::salutation::user_interface::User) -> FormalHonorific`
help: change the parameter type to match the trait
    |
24  |     fn get_formal_honorific(u: exports::hotswap::salutation::user_interface::User) -> FormalHonorific {

[alexcrichton]

To possibly expand on this from the context of bytecodealliance/wasm-tools#1798 given this input in types.wit:

package a:b;

interface api {
    record foo {
        x: string,
    }

    record bar {
        x: foo,
    }

    f1: func() -> bar;
}

interface use-api {
  use api.{bar};

  f2: func() -> bar;
}

world a {
  export api;
}

world b {
  export use-api;
}

the error can be reproduced with:

$ wasm-tools component embed --dummy ./types.wit --world a | \
  wasm-tools component new -o a.wasm
$ wasm-tools component embed --dummy ./types.wit --world b | \
  wasm-tools component new -o b.wasm
$ wac plug b.wasm --plug a.wasm -o foo.wasm
error: encoding produced a component that failed validation

Caused by:
    instance not valid to be used as export (at offset 0x634)

the problem being that use-api is referring to a type that was provided in an interface from api, but the import into the world b was satisfied from the first component. That means that there's no actual interface to write down in the final component corresponding to the api interface.

How best to solve this I'm not sure, it's a bit tricky. One option is to have a first-class error for this situation and just not support it. Another would possible be to synthesize an import api in the final component with just the types. That only works if resources aren't present, though, so I'm not sure if there's a general-purpose solution.

[estk]

The example you gave above was super helpful in exploring a bit more what is a valid composition, glad to know those tools exist!

I spent a little more time with this running thru your example with some future use cases. For me, the workaround of extracting the types into a separate interface is totally workable.

If its easy, adding a first class error and a note in the docs would definitely be helpful. After that, given unlimited time and resources I suppose it would be nice to avoid somewhat arbitrarily splitting types and functions into separate interfaces.

Thanks so much for your time and attention resolving this issue. 😄

(I'm not sure if you'd like to leave this open for reference or close it out, so I'll leave it up to you)

[sammyne]

I made another types-only component as here, triggering the similar errors as

error: encoding produced a component that failed validation

Caused by:
    instance not valid to be used as export (at offset 0x19310)
make: *** [Makefile:25: /github.com/sammyne/wasm-component-broken-compose/_out/calculator-composed.wasm] Error 1

Glimpse of WIT

adder.wit

package docs:[email protected];

interface api {
    use api-types.{wallet};
    
    incr: func(self: wallet, value: u64) -> result<wallet, string>;
}

interface api-types {
    use types.{balance};
    
    variant wallet {
        btc(balance),
        eth(balance),
    }
}

interface types {
    record balance {
        value: u64,
    }
}

world docs-adder {
    export api;
    export api-types;
    export types;
}

calculator.wit

package docs:[email protected];

interface hi {
  use types.{btc-or-eth-wallet};

  variant player {
    alice(btc-or-eth-wallet),
    bob(btc-or-eth-wallet),
  } 
}

interface types {
  use docs:adder/[email protected].{wallet};

  record btc-or-eth-wallet {
    w: wallet,
  }
}

world xml-hello {
  export hi;
  export types;
}

Removing the line export types in world xml-hello would fix it.
Really appreciate if this could be explained also~

[estk]

@sammyne maybe you need to remove api-types from the docs-adder

[estk]

@alexcrichton , @sammyne 's comment brought up another question for me, would (and how would) the workaround also work if api-types was located in a separate package?

[alexcrichton]

I believe so yeah, the package part is mostly an organization in WIT as opposed to the component model, so the important bit is having the types in an extra interface rather than which package it's in.