Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer: add {read|write}Big[U]Int64{BE|LE} methods #19691

Closed
wants to merge 12 commits into from
4 changes: 4 additions & 0 deletions benchmark/buffers/buffer-read.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
const common = require('../common.js');

const types = [
'BigUInt64LE',
'BigUInt64BE',
'BigInt64LE',
'BigInt64BE',
'UInt8',
'UInt16LE',
'UInt16BE',
Expand Down
21 changes: 21 additions & 0 deletions benchmark/buffers/buffer-write.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
const common = require('../common.js');

const types = [
'BigUInt64LE',
'BigUInt64BE',
'BigInt64LE',
'BigInt64BE',
'UInt8',
'UInt16LE',
'UInt16BE',
Expand Down Expand Up @@ -32,11 +36,17 @@ const INT8 = 0x7f;
const INT16 = 0x7fff;
const INT32 = 0x7fffffff;
const INT48 = 0x7fffffffffff;
const INT64 = 0x7fffffffffffffffn;
const UINT8 = 0xff;
const UINT16 = 0xffff;
const UINT32 = 0xffffffff;
const UINT64 = 0xffffffffffffffffn;

const mod = {
writeBigInt64BE: INT64,
writeBigInt64LE: INT64,
writeBigUInt64BE: UINT64,
writeBigUInt64LE: UINT64,
writeInt8: INT8,
writeInt16BE: INT16,
writeInt16LE: INT16,
Expand Down Expand Up @@ -67,12 +77,23 @@ function main({ n, buf, type }) {

if (!/\d/.test(fn))
benchSpecialInt(buff, fn, n);
else if (/BigU?Int/.test(fn))
benchBigInt(buff, fn, BigInt(n));
else if (/Int/.test(fn))
benchInt(buff, fn, n);
else
benchFloat(buff, fn, n);
}

function benchBigInt(buff, fn, n) {
const m = mod[fn];
bench.start();
for (var i = 0n; i !== n; i++) {
buff[fn](i & m, 0);
}
bench.end(Number(n));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's minor, but I had trouble choosing between the following options:

  1. n and i are Numbers, convert i to BigInt in the call (BigInt(i) & m).
  2. n is Number, i is BigInt, use != in the loop.
  3. n and i are BigInts, convert to Number before bench.end() call.

It's now using option 3, but I'm not sure it's the best.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine to me. You could do const nn = Number(n) before bench.start() if you're worried about the coercion influencing the benchmark numbers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's already converted to BigInt in main().

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean the coercion in bench.end(Number(n)).

}

function benchInt(buff, fn, n) {
const m = mod[fn];
bench.start();
Expand Down
90 changes: 90 additions & 0 deletions doc/api/buffer.md
Original file line number Diff line number Diff line change
Expand Up @@ -1546,6 +1546,46 @@ deprecated: v8.0.0

The `buf.parent` property is a deprecated alias for `buf.buffer`.

### buf.readBigInt64BE([offset])
### buf.readBigInt64LE([offset])
<!-- YAML
added: REPLACEME
-->

* `offset` {integer} Number of bytes to skip before starting to read. Must
satisfy: `0 <= offset <= buf.length - 8`. **Default:** `0`.
* Returns: {bigint}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems MDN have not a BigInt doc yet, but maybe we can add a link to spec or proposal in customTypesMap for now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's put it off for now. Maybe it will be on MDN when it's time to land.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not on MDN. What do others think about adding a link to the proposal?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would link to the proposal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a precedent:

'AsyncIterator': 'https://github.com/tc39/proposal-async-iteration',

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in #27101


Reads a signed 64-bit integer from `buf` at the specified `offset` with
the specified endian format (`readBigInt64BE()` returns big endian,
`readBigInt64LE()` returns little endian).

Integers read from a `Buffer` are interpreted as two's complement signed values.

### buf.readBigUInt64BE([offset])
### buf.readBigUInt64LE([offset])
<!-- YAML
added: REPLACEME
-->

* `offset` {integer} Number of bytes to skip before starting to read. Must
satisfy: `0 <= offset <= buf.length - 8`. **Default:** `0`.
* Returns: {bigint}

Reads an unsigned 64-bit integer from `buf` at the specified `offset` with
specified endian format (`readBigUInt64BE()` returns big endian,
`readBigUInt64LE()` returns little endian).

```js
const buf = Buffer.from([0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff]);

console.log(buf.readBigUInt64BE(0));
// Prints: 4294967295n

console.log(buf.readBigUInt64LE(0));
// Prints: 18446744069414584320n
```

### buf.readDoubleBE([offset])
### buf.readDoubleLE([offset])
<!-- YAML
Expand Down Expand Up @@ -2149,6 +2189,56 @@ console.log(`${len} bytes: ${buf.toString('utf8', 0, len)}`);
// Prints: 12 bytes: ½ + ¼ = ¾
```

### buf.writeBigInt64BE(value[, offset])
### buf.writeBigInt64LE(value[, offset])
<!-- YAML
added: REPLACEME
-->

* `value` {bigint} Number to be written to `buf`.
* `offset` {integer} Number of bytes to skip before starting to write. Must
satisfy: `0 <= offset <= buf.length - 8`. **Default:** `0`.
* Returns: {integer} `offset` plus the number of bytes written.

Writes `value` to `buf` at the specified `offset` with specified endian
format (`writeBigInt64BE()` writes big endian, `writeBigInt64LE()` writes little
endian).

`value` is interpreted and written as a two's complement signed integer.

```js
const buf = Buffer.allocUnsafe(8);
BridgeAR marked this conversation as resolved.
Show resolved Hide resolved

buf.writeBigInt64BE(0x0102030405060708n, 0);

console.log(buf);
// Prints: <Buffer 01 02 03 04 05 06 07 08>
```

### buf.writeBigUInt64BE(value[, offset])
### buf.writeBigUInt64LE(value[, offset])
<!-- YAML
added: REPLACEME
-->

* `value` {bigint} Number to be written to `buf`.
* `offset` {integer} Number of bytes to skip before starting to write. Must
satisfy: `0 <= offset <= buf.length - 8`. **Default:** `0`.
* Returns: {integer} `offset` plus the number of bytes written.

Writes `value` to `buf` at the specified `offset` with specified endian
format (`writeBigUInt64BE()` writes big endian, `writeBigUInt64LE()` writes
little endian).

```js
const buf = Buffer.allocUnsafe(8);

buf.writeBigUInt64LE(0xdecafafecacefaden, 0);

console.log(buf);
// Prints: <Buffer de fa ce ca fe fa ca de>
```

### buf.writeDoubleBE(value[, offset])
### buf.writeDoubleLE(value[, offset])
<!-- YAML
Expand Down
147 changes: 147 additions & 0 deletions lib/internal/buffer.js
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,82 @@ function boundsError(value, length, type) {
}

// Read integers.
function readBigUInt64LE(offset = 0) {
validateNumber(offset, 'offset');
const first = this[offset];
const last = this[offset + 7];
if (first === undefined || last === undefined)
boundsError(offset, this.length - 8);

const lo = first +
this[++offset] * 2 ** 8 +
this[++offset] * 2 ** 16 +
this[++offset] * 2 ** 24;

const hi = this[++offset] +
this[++offset] * 2 ** 8 +
this[++offset] * 2 ** 16 +
last * 2 ** 24;

return BigInt(lo) + (BigInt(hi) << 32n);
}

function readBigUInt64BE(offset = 0) {
validateNumber(offset, 'offset');
const first = this[offset];
const last = this[offset + 7];
if (first === undefined || last === undefined)
boundsError(offset, this.length - 8);

const hi = first * 2 ** 24 +
this[++offset] * 2 ** 16 +
this[++offset] * 2 ** 8 +
this[++offset];

const lo = this[++offset] * 2 ** 24 +
this[++offset] * 2 ** 16 +
this[++offset] * 2 ** 8 +
last;

return (BigInt(hi) << 32n) + BigInt(lo);
}

function readBigInt64LE(offset = 0) {
validateNumber(offset, 'offset');
const first = this[offset];
const last = this[offset + 7];
if (first === undefined || last === undefined)
boundsError(offset, this.length - 8);

const val = this[offset + 4] +
this[offset + 5] * 2 ** 8 +
this[offset + 6] * 2 ** 16 +
(last << 24); // Overflow
return (BigInt(val) << 32n) +
BigInt(first +
this[++offset] * 2 ** 8 +
this[++offset] * 2 ** 16 +
this[++offset] * 2 ** 24);
}

function readBigInt64BE(offset = 0) {
validateNumber(offset, 'offset');
const first = this[offset];
const last = this[offset + 7];
if (first === undefined || last === undefined)
boundsError(offset, this.length - 8);

const val = (first << 24) + // Overflow
this[++offset] * 2 ** 16 +
this[++offset] * 2 ** 8 +
this[++offset];
return (BigInt(val) << 32n) +
BigInt(this[++offset] * 2 ** 24 +
this[++offset] * 2 ** 16 +
this[++offset] * 2 ** 8 +
last);
}

function readUIntLE(offset, byteLength) {
if (offset === undefined)
throw new ERR_INVALID_ARG_TYPE('offset', 'number', offset);
Expand Down Expand Up @@ -473,6 +549,68 @@ function readDoubleForwards(offset = 0) {
}

// Write integers.
function writeBigU_Int64LE(buf, value, offset, min, max) {
checkInt(value, min, max, buf, offset, 7);

let lo = Number(value & 0xffffffffn);
buf[offset++] = lo;
lo = lo >> 8;
buf[offset++] = lo;
lo = lo >> 8;
buf[offset++] = lo;
lo = lo >> 8;
buf[offset++] = lo;
let hi = Number(value >> 32n & 0xffffffffn);
buf[offset++] = hi;
hi = hi >> 8;
buf[offset++] = hi;
hi = hi >> 8;
buf[offset++] = hi;
hi = hi >> 8;
buf[offset++] = hi;
return offset;
BridgeAR marked this conversation as resolved.
Show resolved Hide resolved
}

function writeBigUInt64LE(value, offset = 0) {
return writeBigU_Int64LE(this, value, offset, 0n, 0xffffffffffffffffn);
}

function writeBigU_Int64BE(buf, value, offset, min, max) {
checkInt(value, min, max, buf, offset, 7);

let lo = Number(value & 0xffffffffn);
buf[offset + 7] = lo;
lo = lo >> 8;
buf[offset + 6] = lo;
lo = lo >> 8;
buf[offset + 5] = lo;
lo = lo >> 8;
buf[offset + 4] = lo;
let hi = Number(value >> 32n & 0xffffffffn);
buf[offset + 3] = hi;
hi = hi >> 8;
buf[offset + 2] = hi;
hi = hi >> 8;
buf[offset + 1] = hi;
hi = hi >> 8;
buf[offset] = hi;
return offset + 8;
}

function writeBigUInt64BE(value, offset = 0) {
return writeBigU_Int64BE(this, value, offset, 0n, 0xffffffffffffffffn);
}

function writeBigInt64LE(value, offset = 0) {
return writeBigU_Int64LE(
this, value, offset, -0x8000000000000000n, 0x7fffffffffffffffn);
}

function writeBigInt64BE(value, offset = 0) {
return writeBigU_Int64BE(
this, value, offset, -0x8000000000000000n, 0x7fffffffffffffffn);
}

function writeUIntLE(value, offset, byteLength) {
if (byteLength === 6)
return writeU_Int48LE(this, value, offset, 0, 0xffffffffffff);
Expand Down Expand Up @@ -790,6 +928,15 @@ function writeFloatBackwards(val, offset = 0) {
class FastBuffer extends Uint8Array {}

function addBufferPrototypeMethods(proto) {
proto.readBigUInt64LE = readBigUInt64LE,
proto.readBigUInt64BE = readBigUInt64BE,
proto.readBigInt64LE = readBigInt64LE,
proto.readBigInt64BE = readBigInt64BE,
proto.writeBigUInt64LE = writeBigUInt64LE,
proto.writeBigUInt64BE = writeBigUInt64BE,
proto.writeBigInt64LE = writeBigInt64LE,
proto.writeBigInt64BE = writeBigInt64BE,

proto.readUIntLE = readUIntLE;
proto.readUInt32LE = readUInt32LE;
proto.readUInt16LE = readUInt16LE;
Expand Down
51 changes: 51 additions & 0 deletions test/parallel/test-buffer-bigint64.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
'use strict';
require('../common');
const assert = require('assert');

const buf = Buffer.allocUnsafe(8);

['LE', 'BE'].forEach(function(endianness) {
// Should allow simple BigInts to be written and read
let val = 123456789n;
buf['writeBigInt64' + endianness](val, 0);
let rtn = buf['readBigInt64' + endianness](0);
assert.strictEqual(val, rtn);

// Should allow INT64_MAX to be written and read
val = 0x7fffffffffffffffn;
buf['writeBigInt64' + endianness](val, 0);
rtn = buf['readBigInt64' + endianness](0);
assert.strictEqual(val, rtn);

// Should read and write a negative signed 64-bit integer
val = -123456789n;
buf['writeBigInt64' + endianness](val, 0);
assert.strictEqual(val, buf['readBigInt64' + endianness](0));

// Should read and write an unsigned 64-bit integer
val = 123456789n;
buf['writeBigUInt64' + endianness](val, 0);
assert.strictEqual(val, buf['readBigUInt64' + endianness](0));

// Should throw a RangeError upon INT64_MAX+1 being written
assert.throws(function() {
const val = 0x8000000000000000n;
buf['writeBigInt64' + endianness](val, 0);
}, RangeError);

// Should throw a RangeError upon UINT64_MAX+1 being written
assert.throws(function() {
const val = 0x10000000000000000n;
buf['writeBigUInt64' + endianness](val, 0);
}, RangeError);

// Should throw a TypeError upon invalid input
assert.throws(function() {
buf['writeBigInt64' + endianness]('bad', 0);
}, TypeError);

// Should throw a TypeError upon invalid input
assert.throws(function() {
buf['writeBigUInt64' + endianness]('bad', 0);
}, TypeError);
});