Introduce NIP-59 gift wrap

[staab]

A modified version of #468 focused on the wrapping standard, and omitting DM-specific stuff. @v0l please feel free to incorporate this diff into the original PR and I can close this one, or close the original and we can continue from here, it's up to you.

[vitorpamplona]

1. If we want to break the new protocols down into multiple NIPs, the text here should be much smaller.

Just GiftWrap + Seals of unsigned objects.

No whys. Only hows. Don't discuss applications of them. Then, we can cite them in the other PRs.

2. I don't understand the need for the kind:author wrapper->[recipient1,recipient2] notation.

[staab]

Sure, I'll try to trim the verbage down.

I don't understand the need for the kind:author wrapper->[recipient1,recipient2] notation.

You will after you read my groups proposal. With double-wrappers you have 1. kind, 2. author, 3. wrapper key, and 4. recipient. It's very hard to clearly explain all that in prose, the notation makes it much more easily skimmable.

Compare:

Any group member may post event to the group by creating a rumor of any kind, sealed by their own key and signed by the shared key. The recipients tagged on the gift wrap MUST be empty.

Any group member may post to the group using any:author shared_key->[].

[vitorpamplona]

Any group member may post to the group using any:author shared_key->[].

Sorry, I want to like this, but this is not better than the prose in the same example. There is a lot of hidden information in that notation. It will just complicate things for new NIP readers.

[jb55]

+ "content": "{\"ciphertext\":\"f5Lfq6a9laLW9PWuYatTTnrGOSHcGJUPnb5OQ/gEEVjtDnrOzDB9VFKdUkoImEMiO6Ddu+HYn0R3hUtqgTRppOjs4zNFTLNAP/3hkN3hAaKjvUzBVYVmU1xrBS749kUhCt8v/8l/tBeen9BYB4s1luAUqdS10h8jwM+NqE8aYrv9sEU9AaLuErbI1hwq0ZUEGl5unfxHpv+QjINbrPYs1x6bGUU7D416Lp0gV6fr/wD/JLkceb/quAE2UkgzVRS4gauWL8/PyaPz3332h+vH/tMWOQd2no4sDG3ftXPO8y0zXs5p/NvcPa2hIXsOsNOPzOfCFvBk7Rfv9JjwCN0K/LztDHZOzgryELAjcm+mDWPeYBw+PYWfwayFkDbbLCsQJ1KLYRBDS7BEMBOLdd1v1vF4J196tGaN\",\"nonce\":\"s6DBwU6gq9h14+9hRwnts1qZ6BdyPgHn\",\"v\":1}", You shouldn't have to parse this payload on a relay unless you're decrypting the payload, it's intended to be opaque.

If you say so. The database library I'm building is meant for clients and will be processing notes on behalf of client UIs, and you can query it like a relay. It seems like you guys have already made up your minds on this point so I will not waste my time further.

[staab]

The database library I'm building is meant for clients

No, that's a good point. I don't think we can put this in tags because it needs to be passed to decrypt functions in signer applications, but would CSV or something else make your life easier compared to JSON?

[staab]

Any group member may post to the group using any:author shared_key->[].

Sorry, I want to like this, but this is not better than the prose in the same example. There is a lot of hidden information in that notation. It will just complicate things for new NIP readers.

I'll move it into NIP 87 and we can re-introduce it at some point if it seems helpful.

[jb55]

On Fri, Aug 11, 2023 at 09:21:59AM -0700, Jon Staab wrote: > The database library I'm building is meant for clients No, that's a good point. I don't think we can put this in tags because it needs to be passed to `decrypt` functions in signer applications, but would CSV or something else make your life easier compared to JSON?

It's not a big deal, I just think the trend of putting JSON in the content makes everything a lot more annoying on the processing side. Parsing JSON is *slow* and blocks UIs, so you almost always need to put it in a worker thread somewhere. This is why in my database library I'm doing all the heavy lifting and processing for client UIs so they don't have to worry about it: profiles are automatically decoded into flatbuffers and stored in the DB. In these cases, JSON actually makes sense, because there are some many fields that can change. For things as simple as 3 standard fields, tags just makes way more sense and are much simpler. We have tags, why not use them!? Again, if ya'll already have it implemented as JSON and don't want to change it, then fine, I will deal with it. I just wanted to voice my distaste for JSON in content yet again xD

[alexgleason]

This is a very cool and interesting MR. It took some effort to understand why it has to go through 3 rounds of events. It's excessive. But I think it's a good solution.

[mikedilger]

I just read this NIP and I fully understand it. Great design guys @v0l @vitorpamplona. Well written @staab. I also like separating this from particular DM implementations which can be NIPs on top of this.

[v0l]

We should encourage gift wraps to have PoW, problems for relays accepting gift wraps when the only reliable info is the recipient

[staab]

Agreed, that's in there:

Relays may choose not to store gift wrapped events due to them not being publicly useful. Clients MAY choose to attach a certain amount of proof-of-work to the wrapper event per NIP-13 in a bid to demonstrate that the event is not spam or a denial-of-service attack.

[vitorpamplona]

Dont we need some type of ring signature scheme for Relays? To the best of my knowledge, a ring signature would allow a user to sign a message that proves he/she is part of a set of white-listed keys without revealing which one. The GiftWrap would then have an additional tag that contains a ring signature to prove the sender is either a paying account or a white-listed user. I am not sure what's needed to create the "ring" tough.

We can also build a delete-by-tag for GiftWraps only. If I receive a GiftWrap and ask to delete it, the Relay could oblige.

[paulmillr]

Ring sigs have questionable privacy. There have been countless attacks on related currencies. Not sure if it's even worth pursuing. Also, how would one mix it, interactively, or not, would the keys be real, or decoys

[vitorpamplona]

Ok, I am thinking of ways to hide the receiver. What about this addition:

---

Hidden Alias Event

Hidden Alias Events use kind:10059 replaceable events to privately inform peers of the set of pub keys a user is subscribing their GiftWrap filters with. The keys listed as p tags in this event can be used as p tags in GiftWraps to the user.

{
  "id": "<id>",
  "pubkey": "<Author's Main PubKey>",
  "content": "",
  "kind": 10059,
  "created_at": 1686840217,
  "tags": [
    ["p", "<Pubkey Alias 1>"]
    ["p", "<Pubkey Alias 2>"]
    ["p", "<Pubkey Alias 3>"]
    ["expiration", "1600000000"] // optional
  ]
}

Hidden Alias Events MUST be transferred inside GiftWraps: kind:10059 events MUST be unsigned, sealed, and gift-wrapped to each receiver individually.

Clients MUST encrypt the content to the user's main pubkey while creating the event with a randomly-picked alias as the p tag. The use of the user's main private key to decrypt the event sustains a trust-minimized approach where aliases cannot be used as a hidden tool to facilitate leaks later.

Clients SHOULD randomize alias when sending sequential messages to the same user.

Clients SHOULD NOT assume the user has control of the private key of such keys.

Clients SHOULD NOT assume the alias set received is the complete set a user uses.

Clients SHOULD assume constant rotation of these aliases.

Users MAY send the same alias set to relays in order to allow-list those addresses.

Clients/Users MAY store ALL aliases if they want to recover messages in the future.

A NIP-40 expiration tag informs Clients when to stop using these aliases.

[arthurfranca]

Ok, I am thinking of ways to hide the receiver.

@vitorpamplona What if the receiver used a different filter instead of #p with his pubkey to get events sent to him? Like an identifier that him and many other random relay users (but not all users) have in common. Then he will receive some garbage he can't decrypt along with the right events.

For instance, the receiver's pubkey "a" char count may be between 0 and 64 and would split the user base in 65 groups. If needed we could split the relay user base in more or less groups by choosing a different encoding. Like if we convert the pubkey to binary and count the 0s there will be up to 257 groups of users. The fewer the groups, the higher the number of received garbage and added privacy (though used download bandwidth and time to process messages also increases).

Then instead of adding "tags": [["p", "166bf3765ebd1fc55decfe395beff2ea3b2a4e0a8946e7eb578512b555737c99"]] to the gift wrap event, one would add "tags": [["c", "3"]] if hex (or ["c", "109"] if binary).

[staab]

I like this idea, and while I don't think @arthurfranca's anonymity set idea is the best we can do (it's been proposed many times, several of them by me), using something other than a pubkey could be cool if we can avoid tracking more state. For example, if there were a way to create a shared secret with a known salt that the recipient could recognize. So for example if the sender could do getSharedId(myPrivateKey, recipientPublicKey, 'challenge') and the recipient could do getSharedId(myPrivateKey, senderPublicKey, 'challenge') and get the same result. The sender could then tag that id and the recipient could search for it. The challenge string would be a set of ~100 known strings defined in an NIP somewhere so every user has 100 state-free aliases per other user. @paulmillr would something like this be possible?

That said, I think using p tags is probably the best idea, because the anonymity set is all pubkeys + all aliases, and you can't tell by which tag the sender uses whether a pubkey or alias is being used.

[paulmillr]

How does the whole flow look like?

1. Sender Alice/recipient Bob do id = getSharedId()
2. They query a relay for the id tag in p, or in any other way
3. Relay obviously keeps all the logs to itself and sees that both Alice and Bob queried for id from IPs where they queried their own messages with other ppl
4. Auditors deduce all the actual conversations

Meh

[staab]

Relay obviously keeps all the logs to itself and sees that both Alice and Bob queried for id from IPs where they queried their own messages with other ppl

Does the negotiated aliases approach not have this problem? Relay surveillance seems very hard to get around. Relays can also correlate DMs with public notes too in the same way.

[vitorpamplona]

Does the negotiated aliases approach not have this problem?

Aliases are not negotiated, they are just advertised. Clients can just listen in to changes in them via the usual GiftWrap filter. Relays won't know which package is an Alias or which package is a message.

However, because aliases will be part of a filter that comes from the same IP as the main PubKey, whoever sees that filter can see the alias group. So, the relay used to receive GiftWraps is trusted to know all your aliases.

Aliases don't protect against the relay you use to receive GiftWraps. They protect against the public.

That's why I named them "Hidden" and not "Private"

[staab]

Aliases are not negotiated, they are just advertised

I sort of see it as the same thing, another question I had is how you envision these being shared? Can someone request an alias? Or are they just sent after the first message to hide subsequent messages? IOW you'd still have at least 1 note addressed to the recipient's pubkey.

Anyway, this is very clean and minimal, out of band alias sharing could be added easily.

[vitorpamplona]

@staab I was playing around and I think this defines better responsibilities between functions for the example.

import { bytesToHex } from "@noble/hashes/utils"
import type {EventTemplate, UnsignedEvent, Event} from "nostr-tools"
import {getPublicKey, getEventHash, nip19, nip44, finalizeEvent, generateSecretKey} from "nostr-tools"

type Rumor = UnsignedEvent & {id: string}

const TWO_DAYS = 2 * 24 * 60 * 60

const now = () => Math.round(Date.now() / 1000)
const randomNow = () => Math.round(now() - (Math.random() * TWO_DAYS))

const nip44ConversationKey = (privateKey: Uint8Array, publicKeyHex: string) => 
  nip44.v2.utils.getConversationKey(bytesToHex(privateKey), publicKeyHex)

const nip44Encrypt = (data: EventTemplate, privateKey: Uint8Array, publicKeyHex: string) => 
  nip44.v2.encrypt(JSON.stringify(data), nip44ConversationKey(privateKey, publicKeyHex))

const nip44Decrypt = (data: Event, privateKey: Uint8Array) => 
  JSON.parse(nip44.v2.decrypt(data.content, nip44ConversationKey(privateKey, data.pubkey)))

const createRumor = (event: Partial<UnsignedEvent>, privateKey: Uint8Array) => {
  const rumor = {
    created_at: now(),
    content: "",
    tags: [],
    ...event,
    pubkey: getPublicKey(privateKey),
  } as any

  rumor.id = getEventHash(rumor)

  return rumor as Rumor
}

const createSeal = (rumor: Rumor, privateKey: Uint8Array, toPublicKeyHex: string) => {
  return finalizeEvent(
    {
      kind: 13,
      content: nip44Encrypt(rumor, privateKey, toPublicKeyHex),
      created_at: randomNow(),
      tags: [], 
    }, 
    privateKey
  ) as Event
}

const createWrap = (event: Event, toPublicKeyHex: string) => {
  const randomKey = generateSecretKey()

  return finalizeEvent(
    {
      kind: 1059,
      content: nip44Encrypt(event, randomKey, toPublicKeyHex),
      created_at: randomNow(),
      tags: [["p", toPublicKeyHex]], 
    }, 
    randomKey
  ) as Event
}

// Test case using the above example
const senderPrivateKey = nip19.decode(`nsec1p0ht6p3wepe47sjrgesyn4m50m6avk2waqudu9rl324cg2c4ufesyp6rdg`).data
const recipientPrivateKey = nip19.decode(`nsec1uyyrnx7cgfp40fcskcr2urqnzekc20fj0er6de0q8qvhx34ahazsvs9p36`).data
const recipientPublicKeyHex = getPublicKey(recipientPrivateKey)

const rumor = createRumor(
  {
    kind: 1,
    content: "Are you going to the party tonight?",
  }, 
  senderPrivateKey
)

const seal = createSeal(rumor, senderPrivateKey, recipientPublicKeyHex)
const wrap = createWrap(seal, recipientPublicKeyHex)

// Receiver unwraps with his/her private key.

const unwrappedSeal = nip44Decrypt(wrap, recipientPrivateKey)
const unsealedRumor = nip44Decrypt(unwrappedSeal, recipientPrivateKey)

console.log(unsealedRumor)

[vitorpamplona]

BTW, I believe all these utility functions should be provided by nostr-tools.

[staab]

I think this defines better responsibilities between functions for the example.

What's the difference aside from order of arguments?

BTW, I believe all these utility functions should be provided by nostr-tools.

Maybe we can add them in a new nip 59 file.

[vitorpamplona]

What's the difference aside from order of arguments?

Small things like forcing tags to be empty in the Seal, fixed p-tag in the Wrap, new random key inside the wrap, forced pubkey in the rumor, using nsecs instead of hexes as well as updates it to use the finalizeEvent method

[monlovesmango]

overall this is a neat idea.

I do have one dumb question, how does the recipient obtain the ephemeral wrapper key to unwrap the seal? Is there another NIP I need to read up on to understand how this works?

[vitorpamplona]

I do have one dumb question, how does the recipient obtain the ephemeral wrapper key to unwrap the seal? Is there another NIP I need to read up on to understand how this works?

You unwrap with the receiver's private key. That's the only key you need.

It works because the wrap is encrypted to the recepient's pub key directly, just like nip04 did. In other words, the conversation key of the random private key + the recipient pub key is equal to the conversation key of the random pubkey + recipient private key.

[monlovesmango]

ahh ok its making a conversation key with the random key and recipients public key, very slick. thank you!

[mikedilger]

Weren't we going to make it such that GiftWrap events could be deleted by the 'p' tagged person rather than the author? I don't see it in there. Was there some problem with that idea? #945 Shared Key DM proposal sends a private key for this purpose which I like less.

[vitorpamplona]

I think half of us didn't like the fact that the p-tag deletion was a rule made only for GiftWraps (but I do think it is a great rule) and the other half was waiting for the resolution to the #539 idea. Both can work for GiftWraps.

[staab]

p tag for deletion can't work if there is a shared key (e.g. with my nip 87 groups proposal).

[vitorpamplona]

Other than the deletion debate, I think we are ready to merge this.

[staab]

Squashed and rebased, ready to merge.

[arthurfranca]

My usage of gift wraps doesn't include a p tag because the pubkey isn't random (It was informed previously to the receiver). So "MUST" may be too restrictive.

See here https://github.com/nostr-protocol/nips/pull/978/files#diff-58e2c23e46271957d48ea789043bb41d6c096df68176c1e46573bb5711498d7dR151-R152

[vitorpamplona]

Or, since there are privacy implications, we may just need to define a separate wrap kind where tags are always empty and the pubkey is not random.

The random pubkey kind is like UDP (just push events) and what you want is a TCP-like kind (ack-first-then-push).

Either wrap can carry the same DM kinds. And you can define your authorization-to-receive-scheme in the wrap kind itself so that I can be reused for any other event kind.

[staab]

Removed this part and added some clarifications that NIP 59 shouldn't be used on its own.

[mikedilger]

I've already approved, but I like the changes and reiterate my +1

[staab]

@fiatjaf ok to merge? I don't want to merge my own PR

[vitorpamplona]

@fiatjaf can we merge?

[fiatjaf]

Now we have a chain of dependencies in the NIPs. Soon we will need npm, the NIP Possibilities Manager CLI tool and a nip.json file.