Skip to content
This repository has been archived by the owner on Jul 9, 2021. It is now read-only.

Signature generalisation #376

Merged
merged 26 commits into from
Mar 1, 2018
Merged

Conversation

recmo
Copy link
Contributor

@recmo recmo commented Feb 7, 2018

Description

  • Make signatures opaque bytes with a one-byte flag
  • Implement different signature types:
    • Current eth_sign
    • EIP712
    • msg.sender
    • External contract
  • Make message hash compatible with EIP712
  • Adjust internal and external interfaces for this change

Motivation and Context

This tries to implement on:

  • ZEIP-1: Off-chain order generation by smart contracts
  • ZEIP-7: On-chain order generation by smart contracts

It does not try to implement, but is related to:

  • ZEIP-3: Cross-relayer orders
  • ZEIP-9: Allow generation of multiple orders with a single signature
  • ZEIP-10: multihash/multiformat message format.

How Has This Been Tested?

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Change requires a change to the documentation.
  • Added tests to cover my changes.

@recmo recmo changed the title Signature abstraction _ logic between modules (depends on PR #364) Signature abstraction and refactors (depends on PR #364) Feb 7, 2018
@coveralls
Copy link

coveralls commented Feb 7, 2018

Coverage Status

Coverage remained the same at 90.238% when pulling 8ef7087 on feature/contracts/module-refactors into 732a46a on v2-prototype.

@recmo recmo changed the title Signature abstraction and refactors (depends on PR #364) [WIP] Signature abstraction and refactors (depends on PR #364) Feb 8, 2018
@recmo recmo force-pushed the feature/contracts/module-refactors branch from 27f32f3 to 53c3971 Compare February 8, 2018 18:30
@recmo recmo changed the title [WIP] Signature abstraction and refactors (depends on PR #364) [WIP] Signature abstraction and refactors Feb 8, 2018
@recmo recmo force-pushed the feature/contracts/module-refactors branch from 6391dbe to c4efdae Compare February 8, 2018 19:12
@recmo recmo changed the title [WIP] Signature abstraction and refactors [WIP] Signature generalisation Feb 8, 2018

// Signed using web3.eth_sign
} else if (stype == SignatureType.Ecrecover) {
require(signature.length == 67);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be 66?

@@ -0,0 +1,9 @@
pragma solidity ^0.4.19;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove _v1 from the filename.

s
order.orderHash ^ 0x1, // Domain separator maker/taker
order.taker,
takerSignature
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The taker signature should be optional (so if takerSignature.length is 0, assume taker is msg.sender).
  • We also need to update a mapping to prevent replay attacks.
  • I personally prefer the more generalized approaches described in the ZEIP 18 comments. Maybe since we're pretty undecided on the best approach for this, we just make this a separate function for now?

address signer,
bytes signature)
public view
returns (bool valid)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change named return to isValid.

v,
r,
s
order.orderHash ^ 0x1, // Domain separator maker/taker
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have a different message format or add a salt, we can probably remove the separator.


contract ISigner {

function validate(bytes32 hash, bytes signature)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not rename this isValidSignature for consistency?

@recmo recmo force-pushed the feature/contracts/module-refactors branch 3 times, most recently from e8becc2 to d193b5c Compare February 9, 2018 02:02
@BMillman19 BMillman19 added the WIP label Feb 9, 2018
@recmo recmo force-pushed the feature/contracts/module-refactors branch 3 times, most recently from 8ef7087 to 2d186f6 Compare February 13, 2018 23:30
@recmo recmo changed the title [WIP] Signature generalisation Signature generalisation Feb 13, 2018
@recmo recmo force-pushed the feature/contracts/module-refactors branch from 2d186f6 to d5cfe98 Compare February 14, 2018 00:41
// TODO: Domain separation: make hash depend on role. (Taker sig should not be valid as maker sig, etc.)

require(signature.length >= 1);
SignatureType stype = SignatureType(uint8(signature[0]));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: can we rename stype to sigType? Camel-case and less ambiguous.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@recmo our convention is the thumbs up a comment once it's been addressed.

bytes32 s;

// Zero is always an invalid signature
if (stype == SignatureType.Invalid) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this case necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not necessary, but it seemed a good idea from a defense programing perspective. Default values are usually zero. Let's not make them accidentally be some sort of signature.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

isValid = false;
return;

// Implicitely signed by caller
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: Implicitly

return;

// Signed using web3.eth_sign
} else if (stype == SignatureType.Ecrecover) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super cool that V2 will keep backward compatibility support for EC sigs 🥇

// Anything else is illegal
} else {
revert();

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awkward new line. Can we remove?

private pure
returns (bytes32 result)
{
// require(b.length >= index + 32);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to be commented out still?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's commented for performance reasons. This check is more costly than the actual function. That should not concert us at the moment though, and it would be better to have more checks while we keep breaking stuff.

@recmo recmo force-pushed the feature/contracts/module-refactors branch from d5cfe98 to fdc6cde Compare February 14, 2018 23:26

// Implicitely signed by caller
} else if (stype == SignatureType.Caller) {
require(signature.length == 1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give an example of how this would be used? It seems like msg.sender will always be order.maker in this case, which seems a bit strange to me.

bytes32 s;

// Zero is always an invalid signature
if (stype == SignatureType.Invalid) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

require(order.takerTokenAmount > 0);
require(takerTokenCancelAmount > 0);
require(isValidSignature(
keccak256(orderSchemaHash, order.orderHash),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems a bit strange to me that we are always including the orderSchemaHash, even when we aren't using an EIP712 signature. What's the logic for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reasoning is that EIP712 is not a standard for signing a hash, but a standard for generating a hash from structured data. Since none of the other signing methods have requirements on the hashing algorithm, we may as well use EIP712 everywhere.

In fact, we should probably use an EIP712 hash for order.orderHash.

return;

// Implicitly signed by caller
} else if (signatureType == SignatureType.Caller) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide an example of how this is used? It seems to me that in the current contracts, msg.sender will always be order.maker.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current PR excludes taker abstraction, so for a successful fillOrder, we always have msg.sender == order.taker. When taker abstract is added, this will no longer hold true, and this signature type can be used by either maker or taker if they call fillOrder.

In the current PR the cancelOrder function also takes a signature. If maker calls cancelOrder herself she can set the signature to SignatureType.Caller.

bytes32 s;

// Zero is always an invalid signature
if (signatureType == SignatureType.Invalid) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When would someone pass in an intentionally invalid signature? Why not just revert if the signature is malformed?

Copy link
Contributor Author

@recmo recmo Feb 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to reserve 0x00 as an invalid value, because it is the default value and might show up accidentally.

You are right, it should revert() instead of throwing, to be consistent with other invalid values.

Would there be value in adding a signature schema that rejects (returns false) always? It's pretty much free to add, does hurt and might be useful somehow?

@@ -113,26 +105,19 @@ contract MixinWrapperFunctions is
/// @param orderAddresses Array of address arrays containing individual order addresses.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The length of the inputs here are no longer guaranteed to be 484 bytes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the significance of 484? I updated fillOrderNoThrow to take arbitrary lengths.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, it wouldn't let me comment on the correct line for some reason. The delegatecall takes the input length, and we're still passing in 484.

/// @param s Array of ECDSA signature s parameters.
function batchFillOrdersNoThrow(

function fillOrdersUpTo(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this function, we renamed to marketFillOrders.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore my previous comment, looks like you accidentally renamed batchFillOrdersNoThrow to fillOrdersUpTo.

/// @return Validity of order signature.
function isValidSignature(
bytes32 digest,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really have a preference for digest over hash, but this arg is still named hash in ISigner.

@@ -159,7 +165,8 @@ contract MixinExchangeCore is
function cancelOrder(
address[5] orderAddresses,
uint256[6] orderValues,
uint256 takerTokenCancelAmount)
uint256 takerTokenCancelAmount,
bytes signature)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This signature does not include a nonce, so cancelOrders with this signature mechanism are prone to replays.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the current implementation we don't even need a signature here since msg.sender is required to be the maker.

assembly {
result := mload(add(b, index))
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add an explicit return here.

s
);
isValid = signer == recovered;
return;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change all of these returns to return isValid for clarity.

@@ -100,29 +96,39 @@ contract MixinExchangeCore is
expirationTimestampInSec: orderValues[4],
orderHash: getOrderHash(orderAddresses, orderValues)
});

// Validate order and maker only if first time seen
if (filled[order.orderHash] == 0 && cancelled[order.orderHash] == 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any data on how expensive is the signature validation? Two state reads also consume some gas.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An ecrecover is at least 3000 gas (and your gas profiler actually showed it being one of the most expensive parts of the function). An sload is only 200 I believe.

Copy link
Contributor Author

@recmo recmo Feb 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! ecrecover is 3000 gas + 36 for the hash + couple hundred overhead. State read is 200 gas. Let's ignore (partially) canceled orders for now and look at two scenarios:

  1. Unfilled order: Read both filled and cancelled, then verify signature ~ 4000 gas.
  2. Partially filled order: Read filled, short circuit, skip verify ~ 200 gas.

So depending on the ratio partial filled / new orders the expect gas cost is 4000 x + 200 (1-x) =
3800 x + 200.

Status quo is a fixed cost of about 3500. Setting these equal and solving for x gives 87%.

So if less than 87% of the fill orders are new orders, this method saves gas.

@tomhschmidt : Do we have empirical data on this?

But!

We need to optimize for the case where the order is valid. In this case, we would need to read both state variables anyway to process the order, so reading it earlier should not add gas cost. Making the optimization essentially free.

At the moment we don't optimize for this, and we read the same state again in getUnavailableTakerTokenAmount on line 124(244) and again on line 138. This is of course
redundant, as the state can not change in between.

I added a comment to remind us to optimize this.

// signature array with invalid type or length. We may as well make
// it an explicit option. This aids testing and analysis. It is
// also the initialization value for the enum type.
if (signatureType == SignatureType.Illegal) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All those ifs add to understanding but increase the gas consumption. Will we comment them out in a production version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not planning on that, and we need some way to distinguish the cases. What do you suggest instead?

A simple trick is to sort the branches by popularity.


// Signature verified by signer contract
} else if (signatureType == SignatureType.Contract) {
isValid = ISigner(signer).isValidSignature(hash, signature);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isValidSignature seems too generic. Can we change to isValid0xOrderSignature or smth longer?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's intended to be generic though. I could see people using the same function for other purposes as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it is, it is pretty generic. Right now we only use it for maker signature, we will also use it for taker signature and maybe controller signature, etc. These would then sign different objects, not Orders's but FillOrder's and possibly other structures.

public
returns (bool success, uint256 takerTokenFilledAmount)
{
bytes4 FILL_ORDER_FUNCTION_SIGNATURE = bytes4(keccak256("fillOrder(address[5],uint256[6],uint256,uint8,bytes32,bytes32)"));

// Input size is padded to a 4 + n * 32 byte boundary
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is not a WIP any more we need more comments. Much more comments.
I'm unable to read that. Let's reference to some solidity documentation or other sources that can help people understand that. (Talking about the whole function. Let's also have it as a rule to have a very good and verbose explanation before every assembly trick. How much does it save, why is it absolutely neccessary? not against assembly in general, but I want other people to be able to understand that even if they're not so experienced.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to ignore it for now. It's still a prototype.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm not happy with the complexity of this assembly at all. It's not even tested and probably contains bugs. It does show that there is a solution though. We can reimplement it using byte[] input so we can construct the input arguments in Solidity instead of assembly.

Copy link
Contributor Author

@recmo recmo Feb 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a // TODO

@recmo
Copy link
Contributor Author

recmo commented Feb 24, 2018

TODO

  • Tezor signatures

Copy link
Member

@abandeali1 abandeali1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of like adding more explicit support for 0xProject/ZEIPs#7, since this requires one less external call than SignatureType.Contract. Thoughts on adding SignatureType.Approved?

Also, it occurs to me that we should probably be returning the address of the signer rather than a bool in all cases. With taker abstraction, we don't always know the address of the taker in advance (and therefore can't pass in signer).

@recmo
Copy link
Contributor Author

recmo commented Feb 26, 2018

I kind of like adding more explicit support for 0xProject/ZEIPs#7, since this requires one less external call than SignatureType.Contract. Thoughts on adding SignatureType.Approved?

How about this:

contract MixinSignatureValidator {

    enum SignatureType {
        // ....
        Presigned
   }

    mapping ((hash, signer) => bool) presigned;

    function preSign(bytes32 hash, address signer, bytes signature) {
         require(signature.type != Presigned); // Not really necessary
         if(isValidSignature(hash, signer, signature)) {
              presigned[hash, signer] = true;
         }
    }

    function isValidSignature(bytes32 hash, address signer, bytes signature) {
         // ...
         if(signature.type == Presign && presigned[hash, signer])
              return true;
    }

  // ...
}

In the EIP-7 scenario the maker would call

preSign(orderHash, this, SignatureType.Caller);

and the taker would call

fillOrder(/* ... */, SignatureType.Presign);

If the goal is to create an on-chain order book, you can add a wrapper function that includes order details:

function preSignOrder(Order order) {
   preSign(orderHash(order), msg.sender, SignatureType.Caller);
   LogOnchainOrder(order);
}

@recmo
Copy link
Contributor Author

recmo commented Feb 26, 2018

Also, it occurs to me that we should probably be returning the address of the signer rather than a bool in all cases. With taker abstraction, we don't always know the address of the taker in advance (and therefore can't pass in signer).

Hmm... So far my solution was to add address taker, bytes takerSignature to the fillOrder arguments. In the case of SignatureType.Caller this is redundant. In the case of SignatureType.Ecrecover it is a bit more complex, as an invalid signature would always be accepted, but result in a random address (which will likely not have tokens and thus ultimately fail the TX, but I prefer if it fails at signature verification stage already). In the case of SignatureType.Contract and the above proposed SignatureType.Presign we absolutely need a provided address, but this could be provided in the bytes signature.

I'm not sure which is more elegant. Can we revisit this discussion when we PR the taker abstraction?

@abandeali1
Copy link
Member

abandeali1 commented Feb 26, 2018

Re presigning an order, why not just use a mapping of orderHash => signer?

I also don't think that presigning should actually require a signature (we can simply calculate the orderHash and use that, since maker is part of the hash).

@recmo
Copy link
Contributor Author

recmo commented Feb 26, 2018

Re presigning an order, why not just use a mapping of orderHash => signer?

I try to keep isValidSignature agnostic to its use case. Currently it supports multiple signers for the same message. This mapping would break that assumption by allowing only one signer per orderHash. Now that I think of it, that may also cause a race condition if we receive two preSigns for the same message hash with different signers.

I also don't think that presigning should actually require a signature (we can simply calculate the orderHash and use that, since maker is part of the hash).

This would make it specific to the maker signatures, it would not work for taker signatures and potential other kind of signature uses.


I'll open a separate PR for this feature, as it is quite a significant extension.

// (3): (32 - len(signature)) mod 32
// (4): 420 + len(signature) + (32 - len(signature)) mod 32
//
// [1]: https://solidity.readthedocs.io/en/develop/abi-spec.html
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r = get32(signature, 2);
s = get32(signature, 34);
recovered = ecrecover(
keccak256("\x19Ethereum Signed Message:\n32", hash),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are really trying to push the community towards ethereum/EIPs#712


// Signature from Trezor hardware wallet
// It differs from web3.eth_sign in the encoding of message length
// (Bitcoin varint encoding vs ascii-decimal, the later is not
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

later -> latter

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants