dag-cbor - number conversion different in go and js implementations?

[marcinczenko]

» jsipfs --version
0.34.4
» ipfs --version
ipfs version 0.4.19
» node --version
v11.6.0

OS: Mac OS X v10.14.3 (18D109).

Seems like the js and go implementations serialise numbers differently when using dag interface.

Take the following json content:

{
  "a": 1,
}

and let's put it to IPFS using the following command:

$ echo "{\"a\": 1}" | ipfs dag put --format=dag-cbor --hash=sha2-256
zdpuAxTWBh3d49ZCgPP44BT8AAa9qiqxB167yZCFdxHxVg6gN

Then the same content, but now using jsipfs with the help of the following script:

const ipfsClient = require('ipfs-http-client')

// or connect with multiaddr
const ipfs = ipfsClient('/ip4/127.0.0.1/tcp/5001')

const obj = {
  a: 1
}

ipfs.dag.put(obj, { format: 'dag-cbor', hashAlg: 'sha2-256' }, (err, cid) => {
  console.log(cid.toBaseEncodedString())
  process.exit()
})

process.stdin.resume()

The output is: zdpuB2H7FsgxU1PVuGcwk4TDYGgv7xrQ7naFJ1YHfRV71t7Rf.

Finally, when I try to reconstruct the CID myself using:

const dagCBOR = require("ipld-dag-cbor")
const multihashes = require("multihashes")
const sha256 = require("js-sha256").sha256
const CID = require("cids")

const obj = {
  a: 1
}

const getDagCBOR = obj => new Promise((resolve, reject) => {
    dagCBOR.util.serialize(obj, (err, serialized) => {
      if (err) {
        reject(err)
      }
      resolve(serialized)
    })
})

const serialized = await getDagCBOR(obj)
const hash = Buffer.from(sha256.arrayBuffer(serialized))
const mhash = multihashes.encode(hash, 'sha2-256')
const cid = new CID(1, 'dag-cbor', mhash)
const encodedCid = cid.toBaseEncodedString()
console.log('encodedCid=', encodedCid)

I get zdpuB2H7FsgxU1PVuGcwk4TDYGgv7xrQ7naFJ1YHfRV71t7Rf as well (so the same as via ipfs-http-client).

Now, if I change the content to be:

{
  "a": "1",
}

then all three methods return the same CID: zdpuAqnunWR8B95ZbsDpCrHewXcVyRgixLwbThCdjZ37PCx8G

[alanshaw]

@vmx @rvagg any chance you can look into this? CBOR encoding seems to differ between JS/Go.

ipfs-http-client does the serialization client side so the two JS code examples above are effectively the same.

Exploring these two gives identical CID info with a differing hash:

https://explore.ipld.io/#/explore/zdpuB2H7FsgxU1PVuGcwk4TDYGgv7xrQ7naFJ1YHfRV71t7Rf

https://explore.ipld.io/#/explore/zdpuAxTWBh3d49ZCgPP44BT8AAa9qiqxB167yZCFdxHxVg6gN

[vmx]

Problems with numbers is CBOR are a known issue:

• Specify encoding of numbers in DAG-CBOR ipld/specs#80
• Numeric types ipld/interface-ipld-format#9 (comment)

[marcinczenko]

Thanks @vmx for pointing out. I will be following.

[achingbrain]

This is a known issue with CBOR, please see the links further up the thread.