update or delete on table "token_entity" violates foreign key constraint

[vikiival]

Describe the bug

I ran a test subsquid for arrow squid

To Reproduce

run indexer at arrow squid branch.

update or delete on table "token_entity" violates foreign key constraint "FK_060d0f515d293fac1d81ee61a79" on table "nft_entity"DELETE FROM "token_entity" WHERE id = $1
{"detail":"Key (id)=(95-e43f299e96cf1b93d196654669ee5457) is still referenced from table \"nft_entity\".","state_code":"23503"}

QueryFailedError: update or delete on table "token_entity" violates foreign key constraint "FK_060d0f515d293fac1d81ee61a79" on table "nft_entity" at PostgresQueryRunner.query (/squid/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async DataSource.query (/squid/node_modules/typeorm/data-source/DataSource.js:345:20) at async rollbackBlock (/squid/node_modules/@subsquid/typeorm-store/lib/hot.js:112:17) at async /squid/node_modules/@subsquid/typeorm-store/lib/database.js:82:17 at async EntityManager.transaction (/squid/node_modules/typeorm/entity-manager/EntityManager.js:73:28) at async TypeormDatabase.submit (/squid/node_modules/@subsquid/typeorm-store/lib/database.js:151:24) at async Runner.withProgressMetrics (/squid/node_modules/@subsquid/util-internal-processor-tools/lib/runner.js:183:22) at async Runner.handleFinalizedBlocks (/squid/node_modules/@subsquid/util-internal-processor-tools/lib/runner.js:140:9) at async Runner.processFinalizedBlocks (/squid/node_modules/@subsquid/util-internal-processor-tools/lib/runner.js:124:25)

Expected behavior

Indexer should not stuck

Screenshots

Additional context
Add any other context about the problem here.

[vikiival]

Assigning @daiagi as he implemented the TokenEntities

[daiagi]

hi @vikiival
I'm trying to debug the issue and it seems that syncing the chain in arrow-squid branch is much much slower then main branch
is it just me?

another thing, seems that stop and start processing starts it again from beginning, not from last block (on arrow-squid branch)

also, you encountered the bug while syncing kusama or polkadot?

[vikiival]

I'm trying to debug the issue and it seems that syncing the chain in arrow-squid branch is much much slower then main branch

ArrowSquid should be faster. Maybe internet.

another thing, seems that stop and start processing starts it again from beginning, not from last block (on arrow-squid branch)

It processes from last batch IMO, so depends which command you are using

also, you encountered the bug while syncing kusama or polkadot?

Polkadot ^-^

[daiagi]

@vikiival
i started the indexer the old fashioned way (squid up && sqd process)

and passed said block, did not get stuck

ill' try again with just upd && just process

[daiagi]

ill' try again with just upd && just process

can't reproduce. all seems ok

[vikiival]

processor 02.03.2024 08:57:36 ERROR sqd:processor QueryFailedError: update or delete on table "token_entity" violates foreign key constraint "FK_060d0f515d293fac1d81ee61a79" on table "nft_entity" at PostgresQueryRunner.query (/squid/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async DataSource.query (/squid/node_modules/typeorm/data-source/DataSource.js:345:20) at async rollbackBlock (/squid/node_modules/@subsquid/typeorm-store/lib/hot.js:112:17) at async /squid/node_modules/@subsquid/typeorm-store/lib/database.js:82:17 at async EntityManager.transaction (/squid/node_modules/typeorm/entity-manager/EntityManager.js:73:28) at async TypeormDatabase.submit (/squid/node_modules/@subsquid/typeorm-store/lib/database.js:151:24) at async Runner.withProgressMetrics (/squid/node_modules/@subsquid/util-internal-processor-tools/lib/runner.js:183:22) at async Runner.handleFinalizedBlocks (/squid/node_modules/@subsquid/util-internal-processor-tools/lib/runner.js:140:9) at async Runner.processFinalizedBlocks (/squid/node_modules/@subsquid/util-internal-processor-tools/lib/runner.js:124:25)
db 02.03.2024 08:57:36 ERROR update or delete on table "token_entity" violates foreign key constraint "FK_060d0f515d293fac1d81ee61a79" on table "nft_entity" {"statement":"DELETE FROM "token_entity" WHERE id = $1","state_code":"23503","detail":"Key (id)=(106-c3d6cc2c7d0af60bca5e4c3eceed0737:e4395888a1273967a2de3883df29ed4b) is still referenced from table "nft_entity"."}

[vikiival]

@daiagi can you please check?

[vikiival]

SPECK (5771669)

update or delete on table "token_entity" violates foreign key constraint "FK_060d0f515d293fac1d81ee61a79" on table "nft_entity"DELETE FROM "token_entity" WHERE id = $1
{"state_code":"23503","detail":"Key (id)=(106-c3d6cc2c7d0af60bca5e4c3eceed0737:e4395888a1273967a2de3883df29ed4b) is still referenced from table \"nft_entity\"."}

STICK (6534910)

update or delete on table "token_entity" violates foreign key constraint "FK_060d0f515d293fac1d81ee61a79" on table "nft_entity"DELETE FROM "token_entity" WHERE id = $1
{"detail":"Key (id)=(379-74a2f2ffa672e31bc0830ebb0ec445e2:f7794b23b5e0c4bb1aed22136ceaa19b) is still referenced from table \"nft_entity\".","state_code":"23503"}

query {
  nftEntities(where: { token: { id_eq: "106-c3d6cc2c7d0af60bca5e4c3eceed0737:e4395888a1273967a2de3883df29ed4b" } }) {
    id
    name
    image
  }
}

{
  "data": {
    "nftEntities": [
      {
        "id": "106-18",
        "name": "Refraction",
        "image": "ipfs://ipfs/bafybeidioqzawurwzqsllolngyqg3vxpwisial5lqd6ejafxb4h4b54k2a"
      }
    ]
  }
}

[yangwao]

So what is status on this?

[vikiival]

Guess quickfix was merged, no one has opened a deploy PR

Will do when I come

[vikiival]

• 🔖 Speck fix #234
• 🔖 Stick fix #235

[vikiival]

@daiagi still an issue

https://assethub-polkadot.subscan.io/block/5858315

update or delete on table "token_entity" violates foreign key constraint "FK_060d0f515d293fac1d81ee61a79" on table "nft_entity"DELETE FROM "token_entity" WHERE id = $1

{
  "detail": "Key (id)=(112-61dbec5f9f3ee7d511b16253dd4d43dd:ca874f6b5847ad42bb0bc05dba3f7a5f) is still referenced from table \"nft_entity\".",
  "state_code": "23503"
}

[daiagi]

@vikiival
nothing here makes sense

1. delete is wrapped in try/catch. how could it possibly crash the indexer?

2. https://assethub-polkadot.subscan.io/block/5858315

only extrinsic here is set_price which is handled by nfts/list which does nothing with TokenEntity

3. local logs:
   3.1: only thing about this token id is creation
   

3.2 no mention of delete in this block, only LIST, as expected

[vikiival]

You have access to the https://app.subsquid.io/squids so please observe logs

[daiagi]

yes i know it's there. I'm not claiming it isn't

but i can't make any sense of it