Bug : Indexer Returning the chunk data as null

[nabeel99]

Hey near team,
I have integrated the near indexer into my project but I am running into a issue where sometimes the chunk property returned by the lake indexer seems to be null, this is a problem for me since I need a way to retrieve the tx hash where the receiver id is my s.c ,
So looking for possible fixes or suggestions around this.
One way was getting the receipt id but I found no way to figure out how to go from receipt id -> the original tx hash which originated this receipt since the chunk data was returning null I could not just listen for tx towards my contract
Note :Not sure if this is a js specific issue on the lake framework

[khorolets]

@nabeel99 hey there! Some blocks really misses chunks. It happens when chunk producer couldn't produce it in time. In that cases the precious's block chunks are included in the current.

On the indexer side we have decided we don't want to include duplicates.

I don't know your exact case but consider keeping the previous chunk in local cache if you really need it 🤷‍♂️

[nabeel99]

Hmm, so if i understand correctly, say there sre two blocks, A and B, if block B misses its chunks, block A chunks are duplicated into it, but the lake indexer as a design decisons dsnt do this, if so how do i achieve the logic of listening for tx specifically for my receiver contract if the chunk will be empty, the only thing I will have is receipt outcomes, but via that there is no possible near rpc that allows me to get the original tx back, nor is there a rpc apinthat fetches details given a receipt id.
That is why I need a way to get the tx of a receipt executed some x blocks after the tx originally got created, technically i just need to fetch the logs of the receipt id in question but vis the rpc api some how to achieve double validation.

[khorolets]

Hey @nabeel99,

Just to recap, if a block is missing, new transactions won't be included. As for finding the parent transaction of a receipt, it's a common issue without a straightforward solution that I know of.

A while back, I shared an example indexer that monitors transactions and can locate a parent for a particular account or contract. It's written in Rust, but you might find it helpful for inspiration. Check it out at https://github.com/near-examples/indexer-tx-watcher-example-lake.

[khorolets]

By the way, I am working on a big high-level update of the Lake Framework https://github.com/near/near-lake-framework-rs/milestone/3

I plan to work on introducing a simple solution to the problem mentioned in my previous comment. It's going be implemented for Rust at first and then ported to JS. So stay tuned!

[nabeel99]

Hey @nabeel99,

Just to recap, if a block is missing, new transactions won't be included. As for finding the parent transaction of a receipt, it's a common issue without a straightforward solution that I know of.

A while back, I shared an example indexer that monitors transactions and can locate a parent for a particular account or contract. It's written in Rust, but you might find it helpful for inspiration. Check it out at https://github.com/near-examples/indexer-tx-watcher-example-lake.

A block missing isnt the same thing as a chunk missing right ? I did see your rust solution and used thst on an application where i could use rust, I tried to do the same thing here, but this is where i started getting null chunks, if I understand you correctly of a chunk is null, does that mean no tx was accepted into the network for that chunk, and if i see a tx in a certain block on the explorer, is it guranteed that one of those chunks the lake indexer sends me has thst tx , am using shard and chunk interchangbly here, as each shard has a single chunk which itself has all the tx

[khorolets]

@nabeel99 I am currently on a sick leave. Could you please post a bit information for debugging meantime?

Block heights you miss chunks for, tx hashes you are interested in. Also, point what chain id that are for.

I will check it in a week once I'm hopefully back.