[TS SDK] Improve `IndexerClient` to support tokenv2 sorting and new queries #9395

0xmaayan · 2023-07-31T20:51:50Z

Description

Per @kent-white's request in aptos-labs/explorer#578

Export indexer types export * from "./indexer/generated/types";
Support for token v2 activities
Aggregate query for token v2 activities
Support for sorting indexer queries
Support for get owned tokens by token_data_id

Test Plan

ts sdk indexer tests are passing

kent-white

noice

xbtmatt · 2023-08-01T01:00:13Z

ecosystem/typescript/sdk/src/providers/indexer.ts

@@ -146,31 +168,52 @@ export class IndexerClient {
  }

  /**
-   * Queries a token activities by token id hash
+   * Queries a token activities by token address (token data id)


Maybe to be a little clearer: * Queries a token activities by token address or token data id

yea... so token data id is token address, indexer uses token_data_id key but token address might be more understandable for users..? open for suggestions

it's either the token_data_id hash or the token address, the former for v1 and latter for v2, right? that was my understanding

token_data_id is still a thing in v2 https://cloud.hasura.io/public/graphiql?endpoint=https%3A%2F%2Findexer.mainnet.aptoslabs.com%2Fv1%2Fgraphql&query=query+MyQuery+%7B%0A++token_activities_v2%28%0A++++where%3A+%7Btoken_data_id%3A+%7B_eq%3A+%220x5a8e6dd0531df0b68603d0fe6fd57924b34fc3bb7d26c74cfd4955d809ef30cb%22%7D%7D%0A++++distinct_on%3A+transaction_version%0A++%29+%7B%0A++++token_data_id%0A++++token_standard%0A++++from_address%0A++++to_address%0A++%7D%0A%7D%0A

yeah, but it's just a naming convention to make v1 and v2 queries coalesce, the value for v2 is just the token address, whereas with token v1 the value is the hash of the token name + creator + collection name (or maybe something else, I could be wrong, maybe it's a table handle or something), but I've only seen the indexer using it

Sorry if you already knew this, don't mean to explain it to you if you did, but just saying because some people might not understand that the field for the query works for both v1 and v2

I see what you mean! yes I was just thinking about sdk vs indexer naming - I think what you are saying makes sense, will update the code! thanks!

Yeah some of the names are very confusing for the indexer. At first glance you'd think all the _v2 queries are specifically for token v2, but they're not, they're just more robust versions of the "v1" queries haha (although maybe the v1 queries don't handle v2, I don't know anymore 🤣)

That's the main reason I suggest it, because then it's a little clearer to people what the purpose of the "_v2" queries are

yes, I think the idea was that all _v2 queries can handle v1 and v2 data, and all v1 queries can only handle v1 data

xbtmatt · 2023-08-01T01:02:59Z

ecosystem/typescript/sdk/src/providers/indexer.ts

+  }
+
+  /**
+   * Queries account's current owned tokens by the token address (token data id).


same here, (and maybe anywhere else..?) token address or token data id

github-actions · 2023-08-02T00:53:00Z

✅ Forge suite `compat` success on `aptos-node-v1.5.1` ==> `6be384298ed93f66e896f3d38df0d2b8893f0328`

Compatibility test results for aptos-node-v1.5.1 ==> 6be384298ed93f66e896f3d38df0d2b8893f0328 (PR)
1. Check liveness of validators at old version: aptos-node-v1.5.1
compatibility::simple-validator-upgrade::liveness-check : committed: 4101 txn/s, latency: 6992 ms, (p50: 6600 ms, p90: 10400 ms, p99: 19300 ms), latency samples: 176360
2. Upgrading first Validator to new version: 6be384298ed93f66e896f3d38df0d2b8893f0328
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1743 txn/s, latency: 16620 ms, (p50: 19200 ms, p90: 22200 ms, p99: 22500 ms), latency samples: 90640
3. Upgrading rest of first batch to new version: 6be384298ed93f66e896f3d38df0d2b8893f0328
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1884 txn/s, latency: 15529 ms, (p50: 19200 ms, p90: 21800 ms, p99: 22300 ms), latency samples: 92320
4. upgrading second batch to new version: 6be384298ed93f66e896f3d38df0d2b8893f0328
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 2936 txn/s, latency: 9896 ms, (p50: 10200 ms, p90: 13800 ms, p99: 15000 ms), latency samples: 126280
5. check swarm health
Compatibility test for aptos-node-v1.5.1 ==> 6be384298ed93f66e896f3d38df0d2b8893f0328 passed
Test Ok

Grafana dashboard
Humio Logs
Test runner output
Test run is land-blocking

github-actions · 2023-08-02T00:55:41Z

✅ Forge suite `realistic_env_max_load` success on `6be384298ed93f66e896f3d38df0d2b8893f0328`

two traffics test: inner traffic : committed: 6258 txn/s, latency: 6258 ms, (p50: 6000 ms, p90: 7800 ms, p99: 11100 ms), latency samples: 2710020
two traffics test : committed: 100 txn/s, latency: 2872 ms, (p50: 2700 ms, p90: 3600 ms, p99: 6900 ms), latency samples: 1720
Max round gap was 1 [limit 4] at version 1273813. Max no progress secs was 3.808821 [limit 10] at version 1273813.
Test Ok

Grafana dashboard
Humio Logs
Test runner output
Test run is land-blocking

github-actions · 2023-08-02T01:02:58Z

✅ Forge suite `framework_upgrade` success on `aptos-node-v1.5.1` ==> `6be384298ed93f66e896f3d38df0d2b8893f0328`

Compatibility test results for aptos-node-v1.5.1 ==> 6be384298ed93f66e896f3d38df0d2b8893f0328 (PR)
Upgrade the nodes to version: 6be384298ed93f66e896f3d38df0d2b8893f0328
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 4294 txn/s, latency: 7393 ms, (p50: 7800 ms, p90: 10500 ms, p99: 11100 ms), latency samples: 167480
5. check swarm health
Compatibility test for aptos-node-v1.5.1 ==> 6be384298ed93f66e896f3d38df0d2b8893f0328 passed
Test Ok

Grafana dashboard
Humio Logs
Test runner output
Test run is land-blocking

)

0xmaayan requested a review from kent-white July 31, 2023 20:51

0xmaayan requested review from gregnazario, banool and 0xmigo as code owners July 31, 2023 20:51

0xmaayan changed the title ~~improve indexer client to support tokenv2 sorting and new queries~~ [TS SDK] Improve IndexerClient to support tokenv2 sorting and new queries Jul 31, 2023

0xmaayan mentioned this pull request Jul 31, 2023

[TS SDK] Use account_transactions and account_transaction_aggregate queries #9403

Merged

0xmaayan force-pushed the indexer_improv branch from 3bc4197 to f104ceb Compare July 31, 2023 23:09

kent-white approved these changes Jul 31, 2023

View reviewed changes

xbtmatt reviewed Aug 1, 2023

View reviewed changes

xbtmatt approved these changes Aug 1, 2023

View reviewed changes

0xmaayan force-pushed the indexer_improv branch 3 times, most recently from bc40449 to 9e0b2e6 Compare August 1, 2023 21:52

improve indexer client to support tokenv2 sorting and new queries

ed53f07

0xmaayan force-pushed the indexer_improv branch from 9e0b2e6 to ed53f07 Compare August 1, 2023 22:14

0xmaayan enabled auto-merge (squash) August 1, 2023 22:16

This comment has been minimized.

Sign in to view

fix tests

6be3842

This comment has been minimized.

Sign in to view

0xmaayan merged commit f0de1b3 into main Aug 2, 2023

0xmaayan deleted the indexer_improv branch August 2, 2023 01:03

gedigi pushed a commit that referenced this pull request Aug 2, 2023

improve indexer client to support tokenv2 sorting and new queries (#9395

2bc2134

)

xbtmatt pushed a commit that referenced this pull request Aug 13, 2023

improve indexer client to support tokenv2 sorting and new queries (#9395

51326b4

)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TS SDK] Improve `IndexerClient` to support tokenv2 sorting and new queries #9395

[TS SDK] Improve `IndexerClient` to support tokenv2 sorting and new queries #9395

0xmaayan commented Jul 31, 2023 •

edited

Loading

kent-white left a comment

xbtmatt Aug 1, 2023

0xmaayan Aug 1, 2023 •

edited

Loading

xbtmatt Aug 1, 2023

0xmaayan Aug 1, 2023

xbtmatt Aug 1, 2023 •

edited

Loading

0xmaayan Aug 1, 2023

xbtmatt Aug 1, 2023 •

edited

Loading

0xmaayan Aug 1, 2023

xbtmatt Aug 1, 2023

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Aug 2, 2023

github-actions bot commented Aug 2, 2023

github-actions bot commented Aug 2, 2023

[TS SDK] Improve IndexerClient to support tokenv2 sorting and new queries #9395

[TS SDK] Improve IndexerClient to support tokenv2 sorting and new queries #9395

Conversation

0xmaayan commented Jul 31, 2023 • edited Loading

Description

Test Plan

kent-white left a comment

Choose a reason for hiding this comment

xbtmatt Aug 1, 2023

Choose a reason for hiding this comment

0xmaayan Aug 1, 2023 • edited Loading

Choose a reason for hiding this comment

xbtmatt Aug 1, 2023

Choose a reason for hiding this comment

0xmaayan Aug 1, 2023

Choose a reason for hiding this comment

xbtmatt Aug 1, 2023 • edited Loading

Choose a reason for hiding this comment

0xmaayan Aug 1, 2023

Choose a reason for hiding this comment

xbtmatt Aug 1, 2023 • edited Loading

Choose a reason for hiding this comment

0xmaayan Aug 1, 2023

Choose a reason for hiding this comment

xbtmatt Aug 1, 2023

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Aug 2, 2023

✅ Forge suite compat success on aptos-node-v1.5.1 ==> 6be384298ed93f66e896f3d38df0d2b8893f0328

github-actions bot commented Aug 2, 2023

✅ Forge suite realistic_env_max_load success on 6be384298ed93f66e896f3d38df0d2b8893f0328

github-actions bot commented Aug 2, 2023

✅ Forge suite framework_upgrade success on aptos-node-v1.5.1 ==> 6be384298ed93f66e896f3d38df0d2b8893f0328

[TS SDK] Improve `IndexerClient` to support tokenv2 sorting and new queries #9395

[TS SDK] Improve `IndexerClient` to support tokenv2 sorting and new queries #9395

0xmaayan commented Jul 31, 2023 •

edited

Loading

0xmaayan Aug 1, 2023 •

edited

Loading

xbtmatt Aug 1, 2023 •

edited

Loading

xbtmatt Aug 1, 2023 •

edited

Loading

✅ Forge suite `compat` success on `aptos-node-v1.5.1` ==> `6be384298ed93f66e896f3d38df0d2b8893f0328`

✅ Forge suite `realistic_env_max_load` success on `6be384298ed93f66e896f3d38df0d2b8893f0328`

✅ Forge suite `framework_upgrade` success on `aptos-node-v1.5.1` ==> `6be384298ed93f66e896f3d38df0d2b8893f0328`