Incompatible types when using beet.bignum in a fixed size array

[MHHukiewitz]

Hi guys,

I get the following error when compiling TS:

Type 'ElementCollectionBeet & BeetBase & BeetReadWrite<bignum[], Partial<bignum>[]>' is not assignable to type 'FixedSizeBeet<bignum[], (bignum | undefined)[]>'.
  Type 'ElementCollectionBeet & BeetBase & BeetReadWrite<bignum[], Partial<bignum>[]>' is not assignable to type 'ElementCollectionFixedSizeBeet<bignum[], (bignum | undefined)[]>'.
    Type 'ElementCollectionBeet & BeetBase & BeetReadWrite<bignum[], Partial<bignum>[]>' is not assignable to type 'BeetReadWrite<bignum[], (bignum | undefined)[]>'.
      Types of property 'write' are incompatible.
        Type '(buf: Buffer, offset: number, value: bignum[] | Partial<bignum>[]) => void' is not assignable to type '(buf: Buffer, offset: number, value: bignum[] | (bignum | undefined)[]) => void'.
          Types of parameters 'value' and 'value' are incompatible.
            Type 'bignum[] | (bignum | undefined)[]' is not assignable to type 'bignum[] | Partial<bignum>[]'.
              Type '(bignum | undefined)[]' is not assignable to type 'bignum[] | Partial<bignum>[]'.
                Type '(bignum | undefined)[]' is not assignable to type 'bignum[]'.
                  Type 'bignum | undefined' is not assignable to type 'bignum'.
                    Type 'undefined' is not assignable to type 'bignum'.

18   [['bytes', beet.uniformFixedSizeArray(beet.u64, 5)]],
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Caused by this generated code:

/**
 * This code was GENERATED using the solita package.
 * Please DO NOT EDIT THIS FILE, instead rerun solita to update it or write a wrapper to add functionality.
 *
 * See: https://github.com/metaplex-foundation/solita
 */

import * as beet from '@metaplex-foundation/beet'
export type FieldElementZC = {
  bytes: beet.bignum[] /* size: 5 */
}

/**
 * @category userTypes
 * @category generated
 */
export const fieldElementZCBeet = new beet.BeetArgsStruct<FieldElementZC>(
  [['bytes', beet.uniformFixedSizeArray(beet.u64, 5)]],
  'FieldElementZC'
)

From given IDL:

    {
      "name": "FieldElementZC",
      "type": {
        "kind": "struct",
        "fields": [
          {
            "name": "bytes",
            "type": {
              "array": [
                "u64",
                5
              ]
            }
          }
        ]
      }
    },

[MHHukiewitz]

Works fine with u32 and lower (using number instead of BN/bignum)

[thlorenz]

Hi, sorry for the late reply ... could you please provide me a complete repro sample IDL so I could easier get started on fixing this please?
The rust code that generated that IDL would be helpful as well.

Thanks.

[MHHukiewitz]

No worries:
https://gist.github.com/MHHukiewitz/1e7d63764d360466223f826a8745bd5b

As I'm not from the Switchboard team, but merely working with them, I cannot provide the source code.
Thanks!

[thlorenz]

Looking into this I get another error actually:

[ERR_ASSERTION]: Rendering struct for AggregatorAddJobParams should have at least 1 field

..

[ERR_ASSERTION]: Rendering struct for AggregatorLockParams should have at least 1 field

..

[ERR_ASSERTION]: Rendering struct for AggregatorSetAuthorityParams should have at least 1 field


..

[ERR_ASSERTION]: Rendering struct for AggregatorSetHistoryBufferParams should have at least 1 field

..

lots more similar cases

Fixed them one by one by adding some field for those and then things work fine as in I cannot repro your issue.

Are you sure you supplied the correct IDL?

[MHHukiewitz]

Yes, we modified Beet to accommodate for this. But the issue I'm referring to here is harder than we expected, so we decided to ask for help.

If you like to, we could create a PR with the modifications we did, which increase compatibility with some IDLs (we tested and improved on at least two different ones).

EDIT: And you tried compiling a project with the given generated code?

[thlorenz]

Well in either case I'd like to reproduce the issue first, but the IDL you provided doesn't do that.
Even after I work around the accommodations that this beet version doesn't have since then it compiles fine.

I'm happy to look at a PR which allows empty structs ideally with a use-case/sample included.

And you tried compiling a project with the given generated code?

Yes, I created an integration test for this which renders the code and then checks it in multiple ways (including TS compilation).

To be double sure I even went into the generated folder and did tsc -p tsconfig.json and it compiled fine.

[thlorenz]

I'm now thinking that maybe either your version of beet is out of date with this one and/or the modifications you did are causing these issues.
I double checked the solita/test/integration/output/issue-bignum-in-fixedsize-array/generated/types/FieldElementZC.ts after running the test and see no issues there at all.

[MHHukiewitz]

It could indeed be out of date, as we forked a version of Beet, which is back from June. We'll try to use the newest version now and I'll keep you posted

[DavidBarrick]

node_modules/@metaplex-foundation/beet/dist/types/src/types.d.ts:194:81 - error TS2677: A type predicate's type must be assignable to its parameter's type.
  Type 'FixedSizeBeet<T, Partial<T>>' is not assignable to type 'Beet<T, V>'.
    Type 'ScalarFixedSizeBeet<T, Partial<T>>' is not assignable to type 'Beet<T, V>'.
      Type 'ScalarFixedSizeBeet<T, Partial<T>>' is not assignable to type 'ElementCollectionFixedSizeBeet<T, V>'.
        Type 'ScalarFixedSizeBeet<T, Partial<T>>' is not assignable to type 'BeetReadWrite<T, V>'.
          Types of property 'write' are incompatible.
            Type '(buf: Buffer, offset: number, value: T | Partial<T>) => void' is not assignable to type '(buf: Buffer, offset: number, value: T | V) => void'.
              Types of parameters 'value' and 'value' are incompatible.
                Type 'T | V' is not assignable to type 'T | Partial<T>'.
                  Type 'V' is not assignable to type 'T | Partial<T>'.

194 export declare function isFixedSizeBeet<T, V = Partial<T>>(x: Beet<T, V>): x is FixedSizeBeet<T>;
                                                                                    ~~~~~~~~~~~~~~~~

  node_modules/@metaplex-foundation/beet/dist/types/src/types.d.ts:194:44
    194 export declare function isFixedSizeBeet<T, V = Partial<T>>(x: Beet<T, V>): x is FixedSizeBeet<T>;
                                                   ~~~~~~~~~~~~~~
    This type parameter might need an 'extends T | Partial<T>' constraint.

node_modules/@metaplex-foundation/beet/dist/types/src/struct.d.ts:41:5 - error TS2416: Property 'write' in type 'BeetStruct<Class, Args>' is not assignable to the same property in base type 'ScalarFixedSizeBeet<Class, Partial<Class>>'.
  Type '(buf: Buffer, offset: number, value: Args) => void' is not assignable to type '(buf: Buffer, offset: number, value: Class | Partial<Class>) => void'.
    Types of parameters 'value' and 'value' are incompatible.
      Type 'Class | Partial<Class>' is not assignable to type 'Args'.
        'Args' could be instantiated with an arbitrary type which could be unrelated to 'Class | Partial<Class>'.

41     write(buf: Buffer, offset: number, value: Args): void;
       ~~~~~

  node_modules/@metaplex-foundation/beet/dist/types/src/struct.d.ts:15:33
    15 export declare class BeetStruct<Class, Args = Partial<Class>> implements ScalarFixedSizeBeet<Class> {
                                       ~~~~~
    This type parameter might need an 'extends Args' constraint.


Found 2 errors.

I think I'm getting a similar error when building TS.

[millaron]

/**
 * Arguments used to create {@link UserEntrantData}
 * @category Accounts
 * @category generated
 */
export type UserEntrantDataArgs = {
  front: number
  rear: number
  entrantsQueue: web3.PublicKey[] /* size: 5 */
}

/* ...... */


/**
 * @category Accounts
 * @category generated
 */
export const userEntrantDataBeet = new beet.BeetStruct<
  UserEntrantData,
  UserEntrantDataArgs & {
    accountDiscriminator: number[] /* size: 8 */
  }
>(
  [
    ['accountDiscriminator', beet.uniformFixedSizeArray(beet.u8, 8)],
    ['front', beet.u32],
    ['rear', beet.u32],
    ['entrantsQueue', beet.uniformFixedSizeArray(beetSolana.publicKey, 5)],
    /* -------------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ */
  ],
  UserEntrantData.fromArgs,
  'UserEntrantData'
)

TS2322: 
Type 'ElementCollectionBeet & BeetBase & BeetReadWrite<PublicKey[], Partial<PublicKey>[]>' is not assignable to type 'FixedSizeBeet<number | PublicKey[] | number[], Partial<number | PublicKey[] | number[]>>'.
Type 'ElementCollectionBeet & BeetBase & BeetReadWrite<PublicKey[], Partial<PublicKey>[]>' is not assignable to type 'ElementCollectionFixedSizeBeet<number | PublicKey[] | number[], Partial<number | PublicKey[] | number[]>>'.
Type 'ElementCollectionBeet & BeetBase & BeetReadWrite<PublicKey[], Partial<PublicKey>[]>' is not assignable to type 'BeetReadWrite<number | PublicKey[] | number[], Partial<number | PublicKey[] | number[]>>'.
Types of property 'write' are incompatible.
Type '(buf: Buffer, offset: number, value: PublicKey[] | Partial<PublicKey>[]) => void' is not assignable to type '(buf: Buffer, offset: number, value: PublicKey[] | number[] | Partial<number | PublicKey[] | number[]>) => void'.
Types of parameters 'value' and 'value' are incompatible.
Type 'PublicKey[] | number[] | Partial<number | PublicKey[] | number[]>' is not assignable to type 'PublicKey[] | Partial<PublicKey>[]'.
Type 'number' is not assignable to type 'PublicKey[] | Partial<PublicKey>[]'.

beet/types.ts
write(buf: Buffer, offset: number, value: T | V): void;
seems to be causing the problem.

Ignoring Partial as a temporary workaround
['entrantsQueue', beet.uniformFixedSizeArray<PublicKey,PublicKey>(beetSolana.publicKey, 5)]

[thlorenz]

@DavidBarrick @millaron thanks for reporting those related cases.
Given that this was generated via solita from an IDL it would help me immensely if you could post that IDL here.
Or at least the complete code of the module that has the type issue.

Then I can have a look on how to fix those.

[dmshvetsov]

node_modules/@metaplex-foundation/beet/dist/types/src/types.d.ts:194:81 - error TS2677: A type predicate's type must be assignable to its parameter's type.
  Type 'FixedSizeBeet<T, Partial<T>>' is not assignable to type 'Beet<T, V>'.
    Type 'ScalarFixedSizeBeet<T, Partial<T>>' is not assignable to type 'Beet<T, V>'.
      Type 'ScalarFixedSizeBeet<T, Partial<T>>' is not assignable to type 'ElementCollectionFixedSizeBeet<T, V>'.
        Type 'ScalarFixedSizeBeet<T, Partial<T>>' is not assignable to type 'BeetReadWrite<T, V>'.
          Types of property 'write' are incompatible.
            Type '(buf: Buffer, offset: number, value: T | Partial<T>) => void' is not assignable to type '(buf: Buffer, offset: number, value: T | V) => void'.
              Types of parameters 'value' and 'value' are incompatible.
                Type 'T | V' is not assignable to type 'T | Partial<T>'.
                  Type 'V' is not assignable to type 'T | Partial<T>'.

194 export declare function isFixedSizeBeet<T, V = Partial<T>>(x: Beet<T, V>): x is FixedSizeBeet<T>;
                                                                                    ~~~~~~~~~~~~~~~~

  node_modules/@metaplex-foundation/beet/dist/types/src/types.d.ts:194:44
    194 export declare function isFixedSizeBeet<T, V = Partial<T>>(x: Beet<T, V>): x is FixedSizeBeet<T>;
                                                   ~~~~~~~~~~~~~~
    This type parameter might need an 'extends T | Partial<T>' constraint.

node_modules/@metaplex-foundation/beet/dist/types/src/struct.d.ts:41:5 - error TS2416: Property 'write' in type 'BeetStruct<Class, Args>' is not assignable to the same property in base type 'ScalarFixedSizeBeet<Class, Partial<Class>>'.
  Type '(buf: Buffer, offset: number, value: Args) => void' is not assignable to type '(buf: Buffer, offset: number, value: Class | Partial<Class>) => void'.
    Types of parameters 'value' and 'value' are incompatible.
      Type 'Class | Partial<Class>' is not assignable to type 'Args'.
        'Args' could be instantiated with an arbitrary type which could be unrelated to 'Class | Partial<Class>'.

41     write(buf: Buffer, offset: number, value: Args): void;
       ~~~~~

  node_modules/@metaplex-foundation/beet/dist/types/src/struct.d.ts:15:33
    15 export declare class BeetStruct<Class, Args = Partial<Class>> implements ScalarFixedSizeBeet<Class> {
                                       ~~~~~
    This type parameter might need an 'extends Args' constraint.


Found 2 errors.

I think I'm getting a similar error when building TS.

has the same TSC errors, to find a work around I downgraded typescript from 4.9.5 to 4.7.4. I used "@metaplex-foundation/js": "0.18.1" and "@solana/web3.js": "1.73.2", which is the latest versions of these packages for today

{
  "dependencies": {
    "@metaplex-foundation/js": "0.18.1",
    "@solana/web3.js": "1.73.2",
    "typescript": "4.7.4"
  }
}

note the rest of deps and package.json fields are omitted

[staccDOTsol]

It looks like the issue is with the beet.bignum type not being compatible with the bignum | undefined type in the generated code. One possible solution is to change the type of the bytes field in the IDL to u64[] instead of u64[5]. This will allow the beet.uniformArray function to generate a beet.UniformArray type instead of a beet.UniformFixedSizeArray type, which should be compatible with the beet.bignum type.

Here's an example of what the updated IDL would look like:

{
  "name": "FieldElementZC",
  "type": {
    "kind": "struct",
    "fields": [
      {
        "name": "bytes",
        "type": {
          "array": "u64"
        }
      }
    ]
  }
}

After making this change, you'll need to regenerate the code using solita to update the generated code.