Define TODO list to complete NFT support

[fedsten]

The goal of this discussion is to define the list of steps that need to be taken in order to complete the support for NFTs and help coordinate their development.
Initial support for NFTs should already be there. There's a summary discussion which points to LNPBP 0021. The discusson on data containers also seems relevant.
With version 0.8 of the components being released in the latest months and to kick off the effort to complete NTF support in RGB, what are the steps that remain to be done as of September 2022?

[fedsten]

As discussed in the community dev call on the 21/09/2022 the desired features for an NFT schema should be:

• Support for data attachments.
• Support for engraving (i.e. state owner can add extra attachments).
• No decimal precision allowed.

If someone believes there are other desired features we should support please comment below.

[cryptoquick]

So, I have a number of thoughts around NFTs, and while they might not be directly applicable to the protocol, the protocol should be designed in such a way that there is an obvious means of supporting these features within wallets and layers that the protocol would support.

Supply and Precision

It's unclear if it should be assumed necessarily that NFTs will only be 1-of-1 NFTs, or, unique assets. For reference, see this chart:

For collective assets, it would be nice to have a serializable field for each token, such as, a means to indicate its order, similar to @casey's Ordinals project. Sort of how vehicles or baseball cards have a factory serial number.

In addition, although in some ways, precision is merely cosmetic, this is factored in upon token creation. For example, if I wanted fractional ownership of 1 special bitcoin, I would specify a supply of 1 and a precision of 8, so there could be 100M shares of this bitcoin. Internally, of course, this value would be normalized to an integer.

Provenance

Provenance, containing a list of previous owners, unless they wish to stay anoymous, prices of how many sats or tokens this was swapped for (if specified), and dates of sale. While some of this could be determined from state transitions contained within consignments, a convenience method for retrieving this data from the schema would be nice.

Attachments / Hashes

Attachments could be embedded within the contract, though being able to provide a mime type would be preferable over magic number detection. Also, restricting the types of attachments might also be appropriate. These could be more for thumbnailing and previews within wallets.

Hashes are more complex, since they also need a provider. A provider could be an identifier on a DHT which is quite ephemeral but more trustworthy, since only peers that have the content would respond to requests for the content hash. A provider could also be a set of "bootstrap peers" that is more durable and can be specified in a field in the asset manifest, but not necessarily reliable.

Manifest

Perhaps it's desirable to reduce the amount of metadata embedded within the contract to just a single hash. If only a single hash need be provided, ideally it should be a reference to a manifest file with a number of different fields that can be optionally retrieved by clients that support those fields and types. The alternative is to embed the manifest within the contract, to skip a lookup step.

Ideally the fields in this manifest format are agreed upon in advance by stakeholders within a documented specification, and is also versioned to evolve over time, with the understanding that not all fields need to be parsed and handled by clients.

Content

Asset content could be in a number of different mixed types, and could include images, video, audio, text, and files or code (in tar format), and any one of these could be associated with the other (such as cover art for a music album). Executable binary blobs are probably best avoided.

The contexts in which this media would appear should also be specified, so as to provide icons, previews, or thumbnails. There could also be a quality field, for original, lossless, desktop, web, or mobile-optimized formats. Dimensions should also be associated with each of these, (width/height, length, depth). Finally, there should be an explicit mime type, and magic number detection should need to be relied upon.

Carbonado

I'd like to raise attention to a flat file format I've devised for the purpose of persisting, replicating, and transmitting valuable data in a durable, encrypted, and reliable way. I call it Carbonado, and more on it can be found here: https://github.com/diba-io/carbonado

One interesting feature is that it's encrypted by default, and one property of ecies encryption is that a public key can be used to encrypt a file, but a secret key is needed to decrypt it. This is the opposite behavior than libsodium cryptobox provides, but both are a form of authenticated encryption.

The wallet providing keys for Carbonado files would have different derivation paths for:

1. Private files, where the keys are never expected to be shared
2. Dropbox / Write-only files, where only the public key is shared
3. Owned files, where only the public key hash is shared, with the assumption that the public key will be shared selectively
4. Public files where the private key to decrypt it is shared in advance, usually specified in a manifest

All files have a Bao hash, and this is used to verify data integrity, in addition to verifying hash-stream replication proofs. The 32-byte Blake3/Bao hash also makes the file content-addressable, and is usually what the file name would contain, in addition to 10 bits for the number of bits needed to pad the file, a number between 0 and 1023, which corresponds to the Bao slice format. This is then removed by Zfec forward error correction.

Updating of fields

Similar to Renumeration, a means of indicating to a client that there is new information via the use of a UTXO specified with a single use seal should also be possible for updating metadata fields for a particular asset. Whether this is for all fields or only specific ones should also be considered, since it'd be unfortunate if we intended manifest hash fields to be capable of being updated, but instead, a token issuer just dramatically increases supply to the point of inflating the token, unbeknownst to existing tokenholders. For precision this isn't so bad, since it's more like a "split", though this is confined only to decimal (factor of 10) splits.

Dynamic NFTs

For dynamic NFTs, things like associating a "parent" NFT would be useful, in addition to a timelock, and maybe some AluVM code that could execute upon certain actions, times, or other intrinsic inputs. There could also be an option for the owner of the NFT to be able to update certain properties of it themselves using their own keys, in addition to the issuer having that capability.

Anyway, those are all my thoughts regarding NFTs.

[Gigi3d]

Provenance
I think it is important for all RGB assets remain private by default. However, public timestamps could be used to show provenance. In addition to that, marketplaces and users of such platforms can choose how much info they want to reveal.

[cryptoquick]

The thing about provenance is that it doesn't have to be revealed publicly, just to those that owned it. But if we build enough privacy in, individual UTXOs will be insufficient to de-anonymize anyone who wishes to be anonymous.

[josediegorobles]

Could be that only blinded utxos are retrieve in provenance? That could saved the privacy, and left to the user what he wants to reveal.

[fedsten]

From the consignment history it is already possible to retrieve the history of previous TXOs owning the asset (they cannot be blinded for verification purposes), but by themselves they do not reveal the identity of previous owners. If there is a desire to have a proof of previous ownership of a particular individual we should have an identity system that links online identities to a set of TXOs, but this seem like a project outside of the RGB scope and can be achieved by adapting existing identity systems (e.g. you sign messages with the TXO's private key and you PGP key to prove they are owned by the same person, and the wallet verifies such signatures).

[josediegorobles]

Ok, It's true, there aren't purpose in hide what the consignments reveal.

[fedsten]

Currently in the RGB20 genesis schema we have the following fields:

metadata: type_map! {
            FieldType::Ticker => Once,
            FieldType::Name => Once,
            FieldType::Precision => Once,
            FieldType::Timestamp => Once,
            // We need this field in order to be able to verify pedersen
            // commitments
            FieldType::IssuedSupply => Once
        },
        owned_rights: type_map! {
            OwnedRightType::Inflation => NoneOrMore,
            OwnedRightType::OpenEpoch => NoneOrOnce,
            OwnedRightType::Assets => NoneOrMore,
            OwnedRightType::Renomination => NoneOrOnce
        },
        public_rights: none!(),
    }

To support all the NFT types listed by @cryptoquick probably the easiest way is to leave Precision and IssuedSupply the same as in RGB20.
For Attachment and Hash (or Manifest) I think we can create the new respective fields as metadata, while engraving (i.e. new attachments) can be added as owned_rights NoneOrMore

The Dynamic NFT features seem to me as adding a lot of complexity, so it would probably make sense to reserve it for a separate future schema.

Looking forward for your thoughts on this.

[josediegorobles]

I think that for static NFTs that must be enough.
For dynamic things, AluVM code probably is better to reserve for another schema or future version of this shcema (btw, maybe it's interesting to have a field of "version" of the schema, so it can be updated: what do you think?), but assign a "parent" NFT could be made with a NoneOrMore, for example.

[fedsten]

As @dr-orlovsky was pointing out in the call, to use versioning or create new schema doesn't change much, wallet developers will still have to implement the new schema to support the asset in both cases. However I expect the static NFT schema to remain needed also when the dynamic one is available (not all wallets will want to support them), so relegating the static schema as an old version of the NFT schema may cause some confusion.

[cryptoquick]

I think in the very least, a ParentToken field with an Option would be sufficient for the most obvious use cases for dynamic NFTs. It seems like such a small thing, but it essentially allows building a DAG of NFTs, and if we could get that as a base feature for RGB21, that could be used to establish very powerful emergent applications.

[fedsten]

Yes if it doesn't affect the validation logic an extra field shouldn't increase the complexity and if it enables extra use cases that's nice. Would you put it under metadata or owned_rights?

[cryptoquick]

That's actually a really good question! So, thinking it through some more, I think the Parent can be specified NoneOrOnce in the metadata, because you can't change your parent. But if you want to change something similar, I think a good name for a sort of "mutable parent" would be Authority. And that could be NoneOrOnce under owned_rights.

I think symmetry is an important thing to consider when designing protocols. And then we can establish another dimension on the graph above for "immutable" and "mutable" tokens. Much like there are Dynamic NFTs, there could be Mutable UDAs. My intuition leads me to believe having both properties will be important for RGB21.

[fedsten]

For reference, this is the RGB21 draft that @dr-orlovsky wrote few years ago. Looks like it already has most of the features we are looking for. We should probably just reintroduce Precision and IssuedSupply from RGB20 plus adding the ParentToken field, and update it with the necessary fixes to make it aligned with the latest version of RGB20.

[cryptoquick]

I think a helpful reference for what metadata should be included within the RGB21 metadata could be the OpenGraph spec for social previews:
https://ogp.me
(it's the metadata embedded in a webpage that's scraped whenever a link is posted on a social platform)
It's very comprehensive and would help suit all the use-cases we have planned for audio and video also. I can contribute a schema with the specific datatypes I'd think would be best for such metadata.

The one thing I would change over the opengraph spec is to avoid URLs and instead use hashes or provide the actual content itself.

Also, I don't think Ticker is a good field for an NFT, maybe it should be "Identifier". Name should be fine though.

I'm not sure if Renomination is applicable, but it might be nice to have a owned_rights field that can be used to update the metadata. An optional UTXO could be made and spent with an anchor to a new set of metadata.

Maybe we could use strict encoding for this more rich "extended metadata" (perhaps MetadataExt ?), and then a hash of that data could be used for the identifier in the metadata field.

It should be developed with wallet compatibility, future extensibility, and finally, with the consideration that web URLs and centralized locators of that nature shouldn't be included or relied upon.

Ideally, either the content is included, or a hash is provided, along with a list of bootstrap peers to look that hash up, and a protocol defined in which to do so.

[fedsten]

We created a draft of the RGB21 repo, in particular the schema has the following fields:

    GenesisSchema {
        metadata: type_map! {
            FieldType::Name => Once,
            FieldType::Description => NoneOrOnce,
            FieldType::Data => NoneOrMore,
            FieldType::DataFormat => NoneOrOnce,
            FieldType::Precision => Once,
            FieldType::Timestamp => Once,
            FieldType::IssuedSupply => Once,
            FieldType::ParentId => NoneOrOnce
        },
        owned_rights: type_map! {
            OwnedRightType::Assets => NoneOrMore,
            OwnedRightType::Engraving => NoneOrMore
        },
        public_rights: none!(),
    }

Please provide your feedback for fields that we could add, remove or change.

[cryptoquick]

Those are all actually really good choices!

[fedsten]

We finalised the RGB21 proposal, in particular the schema looks as follow:

Schema {
        rgb_features: none!(),
        root_id: none!(),
        genesis: genesis(),
        type_system: type_system(),
        extensions: none!(),
        transitions: type_map! {
            TransitionType::Issue => issue(),
            TransitionType::Transfer => TransitionSchema {
                metadata: none!(),
                closes: type_map! {
                    OwnedRightType::Assets => OnceOrMore
                },
                owned_rights: type_map! {
                    OwnedRightType::Assets => NoneOrMore
                },
                public_rights: none!()
            },
            TransitionType::Engraving => TransitionSchema {
                metadata: type_map! {
                    FieldType::Data => NoneOrMore,
                    FieldType::DataFormat => NoneOrOnce
                },
                closes: type_map! {
                    OwnedRightType::Assets => OnceOrMore,
                    OwnedRightType::Engraving => NoneOrMore
                },
                owned_rights: type_map! {
                    OwnedRightType::Assets => NoneOrMore,
                    OwnedRightType::Engraving => NoneOrMore
                },
                public_rights: none!()
            }
        },
        field_types: type_map! {
            // Rational: if we will use just 26 letters of English alphabet (and
            // we are not limited by them), we will have 26^8 possible tickers,
            // i.e. > 208 trillions, which is sufficient amount
            FieldType::Name => TypeRef::ascii_string(),
            FieldType::RicardianContract => TypeRef::ascii_string(),
            FieldType::Description => TypeRef::ascii_string(),
            FieldType::Data => TypeRef::bytes(),
            FieldType::DataFormat => TypeRef::u16(),
            // Contract text may contain URL, text or text representation of
            // Ricardian contract, up to 64kb. If the contract doesn't fit, a
            // double SHA256 hash and URL should be used instead, pointing to
            // the full contract text, where hash must be represented by a
            // hexadecimal string, optionally followed by `\n` and text URL
            FieldType::Precision => TypeRef::u8(),
            // We need this b/c allocated amounts are hidden behind Pedersen
            // commitments
            FieldType::IssuedSupply => TypeRef::u64(),
            // While UNIX timestamps allow negative numbers; in context of RGB
            // Schema, assets can't be issued in the past before RGB or Bitcoin
            // even existed; so we prohibit all the dates before RGB release
            // This timestamp is equal to 10/10/2020 @ 2:37pm (UTC)
            FieldType::Timestamp => TypeRef::i64(),
            FieldType::ParentId => TypeRef::ascii_string()
        },
        owned_right_types: type_map! {
            // How much issuer can issue tokens on this path. If there is no
            // limit, than `core::u64::MAX` / sum(inflation_assignments)
            // must be used, as this will be a de-facto limit to the
            // issuance
            OwnedRightType::Assets => StateSchema::DiscreteFiniteField(DiscreteFiniteFieldFormat::Unsigned64bit),
            OwnedRightType::Engraving => StateSchema::DataContainer
        },
        public_right_types: none!(),
        script: ValidationScript::Embedded,
        override_rules: OverrideRules::AllowAnyVm,
    }

Notice that the engraving transition is possible only for asset owners, it cannot be a standalone right separated from the asset. It looked to us the safest and more sensible way to implement it. Also if the engraving is used with a de facto fungible asset you can end up with separate transition histories with different engravings, which if merged will result in an asset balance with multiple engraving attached to it, and I am not sure how the wallet should interpret it. The way we see it, the engraving feature make sense mostly for assets with supply 1 and no precision, but if people use it for fungible it will still work (while maybe being confusing).

Please let us know if this proposal satisfies your expectations for RGB21. If it doesn't we still intend to start using the schema as proposed, but we can give it a different name (e.g. RGB210) and reserve the RGB21 name for a future more advanced schema.

[cryptoquick]

Do you mind if we also get an OwnedRight under owned_right_types for OwnerId, so this can enable dynamic NFT applications to override a ParentId whenever there's a new owner?

[cryptoquick]

Also, with regards to this comment:

// While UNIX timestamps allow negative numbers; in context of RGB
// Schema, assets can't be issued in the past before RGB or Bitcoin
// even existed; so we prohibit all the dates before RGB release
// This timestamp is equal to 10/10/2020 @ 2:37pm (UTC)
FieldType::Timestamp => TypeRef::i64(),

In this case, wouldn't it make more sense to make the timestamp a u64? Also, be sure to specify if this in milliseconds, and provide an explicit timestamp in the spec for the epoch.

[fedsten]

@dr-orlovsky can you add your comments here on what you would like to change in the proposed RGB21 schema?

[fedsten]

It would be great to have your feedback on both. Can you point out what do you see as incorrect?

[dr-orlovsky]

I wrote some explanations here, but there are more issues than was stated: RGB-WG/rgb-std#6 (comment)

Some kind of summary (potentially incomplete):

1. Engraving (in the way it is implemented) is not engraving and can't be used in any meaningful way;
2. Assets are just fungible assets; a very restricted form of RGB20 (no re-naming, secondary issue); there is nothing "NFT" about them;
3. Assets owners will not be able to create blank transitions, i.e. if they will allocate RGB20 and RGB21 to the same output, their wallet would not be able to spend neither of them in an automatic way. Only manual construction of the necessary state transitions is possible.
4. No NFT specific validation happens.

So, briefly, it is not an NFT schema, it is implemented in an incorrect way (leading to locking the assets) and does some random stuff. It uses RGB things in a wrong way, not doing proper validation, rights management, which leads to the end-user problems.

The solution I see is to do a new proper NFT schema -- and hardcode into the RGB Core re-coding of this old schema ID into a new proper NFT schema. It will be the first schema bug requiring a fix at the consensus level to prevent the loss of already issued assets :(

This all has brought me to understanding that it is impossible to develop a new schema without either me - or proper tooling and documentation; and that is why I have started work on that as one of my priorities for the nearest feature (otherwise independent guys will do a lot of invalid schemata, issue assets - and end-users will not be able to use them, blaming RGB and not asset issuers/schema developers). It is not fault of anybody, just a natural consequence of having new complex technology w/o proper documentation and training; like using CRISPR gene engineering without ability to study genetics (like if there were no books to read).

[dr-orlovsky]

That's why I have started writing things like:

• https://github.com/RGB-WG/rgb-node/discussions/219
• https://github.com/RGB-WG/rgb-node/discussions/218
• Basics of languague design RGB-WG/contractum-lang#1
• Language grammar RGB-WG/contractum-lang#2

[fedsten]

1. Can you be more specific on how do you expect the engraving feature to be implemented? In the current proposal the right owner can add a file attachment to the asset, which enable wallets to interpret the new attachment in a number of ways (e.g. proof of previous owner, edits to the file etc).
2. Please look at the discussion above, this has been discussed in depth and fungibility has been considered as a necessary feature. I think the problem is more with using the term NFT instead of a more generic collectibles, which better describes the scope of the schema. If people wants to issue an NFT they can still use this schema and set total supply to 1, but there is no valid reason to exclude use cases that need collectibles with some level of fungibility.
3. As a general design principle, do you believe that these kind of automations should be always handled by RGB node rather then the wallet?
4. Do you mean prohibiting supply larger than 1 here or something else?

Regarding the tooling to build new schema, I personally do not expect a large amount of schema to be ever developed and adopted, I would be surprised if more than 10 get actually ever used in a wallet. Probably a better allocation of time would be to only document it better rather than building ad hoc tooling (at least for the foreseeable future).

[dr-orlovsky]

1. With this schema the engraving doesn't "add a file attachment to the asset", since it has no relation with assets at all. And yes, that is how I understood engraving (adding data to specific assets).
2. (and 4) Multiple assets != fungibility. NFT can't be fungible: like with the engraving, an asset having engraving must be distinguishable - and not fungible - with the asset not having; etc. Each asset must have a unique media data or information behind etc. This is not the case with this schema.
3. It can't: there could be no API between RGB Node and wallet capable of providing such level of flexibility. That level of flexibility will actually mean that each wallet has to become its own implementation of "an RGB node", which will significantly increase amount of work the wallet devs have to do. Also, a dedicated wallet will have to be developed for each new schema. I.e. this makes things much more complicated for the wallet devs.

On docs vs tooling. First, it can't be separated in a good way: designing some features require "design via implementation" to ensure they work at all. Second, I expect a cambrian explosion of the BiFi schemata like "wrapped bitcoins", "algostablecoins", dual-linked assets, decentralized identities, also linked to assets etc. Third, that's why I propose to separate schema from the "interface", with RGB20/21 etc becoming "interfaces" and not "schema". The difference between interface, schema and contract will be the same as the difference between trait, struct and instance in Rust, or interface, class and instance in most of OOP languages. Thus, while interfaces ("standards") will be not many (<10), schema development will flourish quite soon (and I am afraid we can't prevent that from happening - like it happened in this case of NFT schema).

[fedsten]

Due to evidence that the previously proposed schema does not meet everyone's expectations (e.g. @dr-orlovsky expressed the need to not allow fungible collectibles and to have the engraving feature implemented in a more strict way), we renamed the schema as RGB121, so that we can avoid collisions and confusion with a future RGB21 schema which will provide the requested features.

[cryptoquick]

That's kind of strange to me, because if DIBA and Iris use RGB121, who will use RGB21?

[fedsten]

Whoever needs the features that the future RGB21 schema will support, I think it is reasonable to have multiple schema that target different use cases. In Iris I believe that we will support any schema that has sufficient interest from issuers and users. It doesn't need to be either RGB21 or RGB121, they can coexist to serve different needs. For example RGB21 could become the schema for pure NFTs, while with RGB121 people will issue collectibles that require some level of fungibility (e.g. digital equivalent of baseball or Pokemon cards).

[dr-orlovsky]

With new interfaces schema would not be RGB*, they will be just schema. RGB21 (and 20 and others) will become pure standards and multiple schemata can correspond to them. So the renaming doesn't make sense as for my opinion.

[dr-orlovsky]

I did a RGB21 interface proposal which covers "unique", "collective", "fractional" and "fungible" types of collectible items: #137

BTW the main difference between "fungible" collectible and fungible financial asset is the fact that fungible collectible tracks its identity, so proper name would be "collective with fractional ownership".