Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change creator and collection verification to use upserts to support out of order #87

Merged
merged 44 commits into from
Jul 27, 2023

Conversation

danenbm
Copy link
Contributor

@danenbm danenbm commented Jul 13, 2023

Notes

  • This PR adds support for creator verification to be indexed with out of order transactions.
  • It adds support for collection verification to be indexed with out of order transactions.
  • It adds an extra sequence number and a verified flag to asset_grouping to support out-of-order.

Testing

  • Verified by running a multiple sequences in order and in reverse order. See table below for details.
  • Verified that when a collection is unverified, getAsset returns empty grouping. Then after the collection is verified, getAsset returns the collection as a grouping. This is to remain consistent with existing API behavior.
  • See Enable out-of-order transaction processing for asset table #77 (comment) for more info on overall testing process.
Transaction sequence Database result getAsset result
CreateTree/Mint, VerifyCreator OK - asset, asset_creators, cl_items match when instructions run forwards and backwards using PR code. PR code's cl_items table consistent with mainline code when instructions run forwards. asset and asset_creators incorrect on main when instructions run backwards. OK - Before verified, all creators listed as unverified. After verified, one creator listed as verified
CreateTree/Mint, VerifyCollection OK - asset, asset_grouping, cl_items match when instructions run forwards and backwards using PR code. PR code's asset and cl_items tables consistent with mainline code when instructions run forwards. asset_grouping incorrect on main. OK - Using PR code, no grouping returned before verified, grouping returned after verified.
CreateTree/Mint, VerifyCollection, UnverifyCollection OK - asset, asset_grouping, cl_items match when instructions run forwards and backwards using PR code. OK - Using PR code, grouping returned after verified, no longer returned after unverified.
CreateTree/Mint, SetAndVerifyCollection SetAndVerifyCollection does not parse properly due to Blockbuster error. Not a regression. SetAndVerifyCollection does not parse properly due to Blockbuster error. Not a regression.
CreateTree/MintToCollection, UnverifyCollection OK - asset, asset_grouping, cl_items match when instructions run forwards and backwards using PR code. OK - Using PR code, grouping returned after verified, no longer returned after unverified.

danenbm added 30 commits May 24, 2023 14:27
* This allows out-of-order Bubblegum transactions to
create and update the asset table.
* Upsert leaf schema, owner, delegate, and seq separately
since those are updated by all instructions and gated
by sequence number to ensure freshest value.
* Mint, burn, and decompress happen without regard to
sequence number because they operate on unique fields.
* Mint and burn have been updated but Decompress still
needs to be fixed to handle out of order transactions.
* Also remove unused 'filling' variable.
Also remove not null constraints for asset fields without defaults.
Also update redeem and decompress to not use
leaf schema events.
compressed_seq won't work because decompression doesn't
create a cl_event.
@danenbm danenbm marked this pull request as ready for review July 13, 2023 07:13
@danenbm danenbm force-pushed the danenbm/update-creators branch from f678f24 to 2bc2946 Compare July 17, 2023 22:09
@danenbm danenbm requested a review from NicolasPennie July 18, 2023 21:52
@danenbm danenbm force-pushed the danenbm/update-creators branch from e4af3a2 to c91977a Compare July 19, 2023 05:21
Copy link
Collaborator

@linuskendall linuskendall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one comment on the updating of position

.to_owned(),
)
.build(DbBackend::Postgres);
txn.execute(query).await?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this query need a where check for example on SlotUpdated?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it cannot be reordered, then is the update needed atg all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is still needed to be an update because if 'verify_creator` is processed first, then there will be a record but it won't actually have all this base information from the time of minting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this query need a where check for example on SlotUpdated?

It does not need protection because the base creator info is ONLY filled in at time of mint.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will likely be updating this to support changes to creator array and collection.

@danenbm danenbm changed the base branch from danenbm/update-asset to main July 27, 2023 00:21
danenbm added 2 commits July 26, 2023 17:43
…of order processing (#90)

* Adding verified flag to asset_grouping table

* Regenerate Sea ORM types

* Remove null constraints on asset_grouping table

* Regenerate Sea ORM types

* Update digital asset types and ingester based on new Sea ORM objects

* Setting new verified flag in asset_grouping table to be non null with default

Also regenerating Sea ORM types

* Separate out collection insert in mintV1 into separate upserts

* Fix error message

* Separate update collection base info from collection verified

* Add group info seq to asset_grouping table

* Regenerate Sea ORM types

* Add group_info_seq checks to collection base info upsert

* Add check for verified = true in grouping for Read API

* Fix conditions for asset grouping updates

* Require grouping to verified to be returned from API in all cases
@danenbm danenbm changed the title Change creator verification to use upserts to support out of order Change creator and collection verification to use upserts to support out of order Jul 27, 2023
@danenbm danenbm merged commit 3ca1303 into main Jul 27, 2023
muhitrhn pushed a commit to muhitrhn/digital-asset-rpc-infrastructure that referenced this pull request Oct 6, 2023
muhitrhn pushed a commit to muhitrhn/digital-asset-rpc-infrastructure that referenced this pull request Oct 6, 2023
muhitrhn pushed a commit to muhitrhn/digital-asset-rpc-infrastructure that referenced this pull request Oct 6, 2023
@danenbm danenbm deleted the danenbm/update-creators branch October 9, 2023 06:07
@danenbm danenbm restored the danenbm/update-creators branch October 9, 2023 06:07
@danenbm danenbm deleted the danenbm/update-creators branch October 9, 2023 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants