RPC: No good way to poll recent signatures for an address

[jstarry]

Problem

The getSignaturesForAddress RPC method doesn't support an after option for fetching signatures for transactions that have been processed more recently than the queried signature. This makes it difficult to poll for more recent signatures.

Proposed Solution

Add an after option to getSignaturesForAddress

[CriesofCarrots]

I think that after already exists in the form of the until parameter.
Namely: using until with no before should return all confirmed/finalized (depending on commitment) signatures more recent than the provided signature.

[CriesofCarrots]

Is the issue that the search starts from the most recent signature, so that the limit may kick in before the until signature is reached?

[jstarry]

Is the issue that the search starts from the most recent signature, so that the limit may kick in before the until signature is reached?

Yes, exactly

[CriesofCarrots]

Hmm, I see. This will definitely require some new plumbing, since typically we work backward in time and only check long-term storage if whatever search is not fully satisfied by the recent ledger in blockstore. I'll think on a performant way to do this. (May want to bundle this plumbing with the blockstore-column rework discussed here: #19515 (comment))

Also, I think it will be somewhat confusing to users to have/disambiguate until and after, so let's think on the rpc interface a little. If I understand correctly, we'd want after to be mutually exclusive with before/until, right?

[jstarry]

Hmm, I see. This will definitely require some new plumbing, since typically we work backward in time and only check long-term storage if whatever search is not fully satisfied by the recent ledger in blockstore. I'll think on a performant way to do this. (May want to bundle this plumbing with the blockstore-column rework discussed here: #19515 (comment))

Got it, I don't think the need for this is too strong so I think it would be ok if it gets deprioritized for now, especially since it requires new plumbing and it can be worked around with the current API.

If I understand correctly, we'd want after to be mutually exclusive with before/until, right?

Yes, that's what I was thinking.

[chandraprakash]

Hi All,
I have got a case for above issue by @jstarry , which needs after option.

Use case:
A Solana node (no vote) is running and we poll each new blocks to find relevant account transactions and record it.

If our node goes down for some time and on restarting unable to catch up to the latest slot, we need to start with snapshot (remove --no-snapshot-fetch flag).
Above action leaves gaps in ledger in our node and will miss some relevant transactions if happened.

As suggested in docs, we could use getSignaturesForAddress to fetch all the relevant transactions (if) happened during the gap period from public node or node which was healthy during that time.

Problem is, there is no 'after' parameter which from our records put the last tx that we know before the gap period.
That means, we need to unnecessarily put a bigger limit or send multiple rpc stressing the public node/another active node.

Please let me know if there is a better way to handle this if I am missing something.
If not, then having this parameter will be useful.

Thanks for the good work!

[jstarry]

I don't believe there is a better way until this issue gets resolved

[CriesofCarrots]

When this PR lands, we'll be in a position to start reworking the AddressSignatures column.

[chandraprakash]

Looks like the dependent PR is merged now. Any plans of picking this one in near future?

[CriesofCarrots]

Looks like the dependent PR is merged now. Any plans of picking this one in near future?

I have started designing the new AddressSignatures column. There's still quite a bit of lower-level work there before we can start adjusting RPC parameters and behavior