Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to use Token Metadata (and latest Bubblegum) Rust clients #168

Merged
merged 6 commits into from
Feb 2, 2024

Conversation

danenbm
Copy link
Contributor

@danenbm danenbm commented Feb 1, 2024

Update 2/2/2024

  • Added latest blockbuster 1.0.0
  • Tested regular load generator working
  • Look at asset, asset_creators, asset_grouping, asset_data, and asset_v1_account_attachments tables and looks like expected data from load generation.
  • Can also index some bubblegum transactions correctly (using txn forwarder and some devnet txns with known database results for asset, asset_creators, asset_grouping, asset_data, and cl_items tables).

Notes

  • Use Token Metadata and latest Bubblegum Rust clients.
  • Use new Blockbuster which also uses these Rust clients (note its currently pointed to hash of commit from Update to use Token Metadata (and latest Bubblegum) Rust clients blockbuster#35 until I can merge and publish the change).
  • Remove candy machine and candy guard dependencies that are not used.
  • Improve MasterEditionV1 and MasterEditionV2 indexing.
    • These were switched in blockbuster before, so things came through as the opposite version of what they were supposed to be.
    • Also the attachment_type column was hardcoded to master_edition_v1, which is now fixed to use the function argument.
    • This won't affect output as print_current_supply and print_max_supply are currently hardcoded in asset_to_rpc anyways, and even if data was being used, the same info is being stored no matter what the key is.
    • See Update to use Token Metadata (and latest Bubblegum) Rust clients blockbuster#35 for more info on the blockbuster side of this issue.

Testing

  • Runs in docker with load generator running and Token Metadata NFTs index. See below for some before and after values from asset_data indexing. But I was also able to see a load generated NFT indexed in asset, asset_creators, and asset_grouping tables.

Mainline asset data from load generator.

solana=# select * from asset_data where id = '\x293b4e8c515dd7df21546dee85ee6a311fa9a209f25dd4719936d97090b212ca';
-[ RECORD 1 ]---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
id                    | \x293b4e8c515dd7df21546dee85ee6a311fa9a209f25dd4719936d97090b212ca
chain_data_mutability | immutable
chain_data            | {"name": "fake", "symbol": "fake", "edition_nonce": 254, "token_standard": "NonFungible", "primary_sale_happened": false}
metadata_url          | https://usd363wqbeq4xmuyddhbicmvm5yzegh4ulnsmp67jebxi6mqe45q.arweave.net/pIe_btAJIcuymBjOFAmVZ3GSGPyi2yY_30kDdHmQJzs
metadata_mutability   | mutable
metadata              | {"name": "Handalf", "image": "https://arweave.net/UicDlez8No5ruKmQ1-Ik0x_NNxc40mT8NEGngWyXyMY", "attributes": [], "properties": {"files": ["https://arweave.net/UicDlez8No5ruKmQ1-Ik0x_NNxc40mT8NEGngWyXyMY"], "category": null}, "description": "The Second NFT ever minted from justmint.xyz", "external_url": ""}
slot_updated          | 30
reindex               | f
raw_name              | \x66616b6500000000000000000000000000000000000000000000000000000000
raw_symbol            | \x66616b65000000000000
base_info_seq         | 0

This PR asset data from load generator

solana=# select * from asset_data where id = '\x293b4e8c515dd7df21546dee85ee6a311fa9a209f25dd4719936d97090b212ca';
-[ RECORD 1 ]---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
id                    | \x293b4e8c515dd7df21546dee85ee6a311fa9a209f25dd4719936d97090b212ca
chain_data_mutability | immutable
chain_data            | {"name": "fake", "symbol": "fake", "edition_nonce": 254, "token_standard": "NonFungible", "primary_sale_happened": false}
metadata_url          | https://usd363wqbeq4xmuyddhbicmvm5yzegh4ulnsmp67jebxi6mqe45q.arweave.net/pIe_btAJIcuymBjOFAmVZ3GSGPyi2yY_30kDdHmQJzs
metadata_mutability   | mutable
metadata              | {"name": "Handalf", "image": "https://arweave.net/UicDlez8No5ruKmQ1-Ik0x_NNxc40mT8NEGngWyXyMY", "attributes": [], "properties": {"files": ["https://arweave.net/UicDlez8No5ruKmQ1-Ik0x_NNxc40mT8NEGngWyXyMY"], "category": null}, "description": "The Second NFT ever minted from justmint.xyz", "external_url": ""}
slot_updated          | 30
reindex               | f
raw_name              | \x66616b6500000000000000000000000000000000000000000000000000000000
raw_symbol            | \x66616b65000000000000
base_info_seq         | 0

  • MasterEditionV2 indexing is now as expected (see before and after data below).

Mainline MasterEditionV2 data indexed before this change

solana=# select * from asset_v1_account_attachments;
                                 id                                 | asset_id |  attachment_type  | initialized |                           data                           | slot_updated 
--------------------------------------------------------------------+----------+-------------------+-------------+----------------------------------------------------------+--------------
 \x0959c32095ea8e4efe3cd3085da48fe76315fb964d75adb35ef77385c4824e8b |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           10
 \x2536a1214ca39b157ced46c63e811af89533a0bbb50138dbfd55116caa25dc04 |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           16
 \xdf01db9dae8c86894a11e82a5613628009e2adcecf2f1fc370396c9e350a3a84 |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |            4
 \x038c060efc5b0d94a7e9e3b18a5412d3f8f8ae2f6f2696bf4397db66577d2d77 |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           22
 \x1e03d3ca5cc64798343f89b7b2cfabdd1812da638c8e68c7e667b8e452de35e1 |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           29
 \x448647109f2d2fde9328d77e24ff6105b751e3ae45b1f31ac277ecb41a66c561 |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           35
 \xcf2138727f907817109103c14605015fcde92629aeed463a836cfc8c27a2a15b |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           41
 \xa494deba3d99b376a507c729055775ce88cb16a0d9724743e9975858ef86a03f |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           47
 \x92abf2c0864539ace0afa35001574fdf68f9fd31f5cd532c349ddd40149cb0bc |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           53
 \x0e170152ff4c0c4e29bd56e74ecf0bcea026ca45203ff274ccd00167e9d46305 |          | master_edition_v1 | f           | {"key": "MasterEditionV1", "supply": 0, "max_supply": 0} |           60

MasterEditionV2 data indexed after this PR

solana=# select * from asset_v1_account_attachments;
                                 id                                 | asset_id |  attachment_type  | initialized |                           data                           | slot_updated 
--------------------------------------------------------------------+----------+-------------------+-------------+----------------------------------------------------------+--------------
 \x8bf1923e9bf91e1b7c2f6dd4e34263830b3b3db02f9dc82206a50bf46a970080 |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           10
 \xde3d8ebef3abb24a2aa44f01d1e6de63548263407177a4c59b4e2533d37174ac |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           16
 \x0a8855858539effa63c5de182d916e0156f91761f4e8da0bcdbe62082edc8c1a |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |            4
 \xd6dff4dfda6bd12d505b77da353030ee17e56124cbf4f99a8b7caeb94aa96b5a |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           22
 \x979ae025e5a22c24c501339c7ce6d343e09263e1f2909abe3cffd122ad66d9ef |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           28
 \x50a30345bcfe07060e6dccb94c0bb5f98d2a19cbd6f3a9218827aefc40d11c46 |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           35
 \xd15f5f5b8705c1f82e2ace27091b6688967feff8cdb9306447fc10c4a87129e6 |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           41
 \x74d7878a89e5855199b6a38912aadf7b745dd5fd62dd542df78b519a49778fb0 |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           47
 \x4b7a46e1a727960ff068cf1e565768bc0f89c7c1cc4c3c0c5abb0033dde2de6f |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           53
 \xbae47b4cd09bdfad08363e9e9dddb18a7c79326067c1901913e96b9c4ebebd09 |          | master_edition_v2 | f           | {"key": "MasterEditionV2", "supply": 0, "max_supply": 0} |           60

Copy link
Collaborator

@niks3089 niks3089 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, tested and works great

@danenbm danenbm merged commit 748862f into main Feb 2, 2024
3 checks passed
@danenbm danenbm deleted the danenbm/tm-rust-client branch February 2, 2024 23:38
@danenbm danenbm mentioned this pull request Feb 5, 2024
niks3089 added a commit to helius-labs/digital-asset-rpc-infrastructure that referenced this pull request Feb 19, 2024
* Fix cNFT update metadata indexing (metaplex-foundation#167)

* Update to use Token Metadata (and latest Bubblegum) Rust clients (metaplex-foundation#168)

* Use Token Metadata and latest Bubblegum Rust clients, new blockbuster

* Remove candy machine and candy guard

* Update lock file

* Improve Master Edition V1 and V2 indexing

* Update to use blockbuster published crate

* Update lock file

* Update Token Metadata and Blockbuster deps (metaplex-foundation#170)

* Update to latest token metadata and blockbuster crates

* Update Cargo lock file

* Add DAS integration tests (metaplex-foundation#169)

* Add das integration tests

* Delete unused snapshots

* Delete unnecessary test data

* Clean up dependencies

* Nit

* Address comments

* Add README.md

* Fix warnings

* feat(ops): add ops crate for bundling misc tools. add a tree backfiller based on cl_audits_v2.

* Fix NFT race condition (metaplex-foundation#172)

* Add das integration tests

* Delete unused snapshots

* Delete unnecessary test data

* Clean up dependencies

* Nit

* Address comments

* Add README.md

* Fix warnings

* Fix asset race condition

* Add missing files

* Fix warnings

* Fix missing slot updated

* Address comments

* Nit

* Update Rust version and missing dirs in docker build (metaplex-foundation#173)

---------

Co-authored-by: Nicolas Pennie <[email protected]>
Co-authored-by: Michael Danenberg <[email protected]>
Co-authored-by: pmantica11 <[email protected]>
Co-authored-by: Kyle Espinola <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants