Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify heartbeat signature scheme #1537

Merged
merged 6 commits into from
Nov 29, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/shaggy-points-collect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@api3/airnode-node': minor
---

Simplify heartbeat scheme
61 changes: 29 additions & 32 deletions packages/airnode-node/src/reporting/heartbeat.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,11 @@ describe('reportHeartbeat', () => {
HTTP_SIGNED_DATA_GATEWAY_URL: httpSignedDataGatewayUrl,
AIRNODE_WALLET_PRIVATE_KEY: fixtures.getAirnodeWalletPrivateKey(),
});
const timestamp = 1661582890984;
const systemTimestamp = 1661582890984;
const expectedTimestamp = 1661582891;

beforeEach(() => {
jest.spyOn(Date, 'now').mockImplementation(() => timestamp);
jest.spyOn(Date, 'now').mockImplementation(() => systemTimestamp);
});

it('does nothing if the heartbeat is disabled', async () => {
Expand All @@ -40,13 +41,14 @@ describe('reportHeartbeat', () => {
const config = fixtures.buildConfig();
const coordinatorId = randomHexString(16);
const state = coordinatorState.create(config, coordinatorId);
const heartbeatPayload = {
const heartbeatPayload = JSON.stringify({
timestamp: expectedTimestamp,
stage: state.config.nodeSettings.stage,
cloud_provider: state.config.nodeSettings.cloudProvider.type,
http_gateway_url: 'http://localhost:3000/http-data',
http_signed_data_gateway_url: 'http://localhost:3000/http-signed-data',
};
const signature = await heartbeat.signHeartbeat(heartbeatPayload, timestamp);
});
const signature = await heartbeat.signHeartbeat(heartbeatPayload);
const res = await heartbeat.reportHeartbeat(state);
expect(res).toEqual([
{ level: 'INFO', message: 'Sending heartbeat...' },
Expand All @@ -60,9 +62,8 @@ describe('reportHeartbeat', () => {
'airnode-heartbeat-api-key': '3a7af83f-6450-46d3-9937-5f9773ce2849',
},
data: {
...heartbeatPayload,
payload: heartbeatPayload,
signature,
timestamp,
},
timeout: 5_000,
});
Expand All @@ -73,13 +74,14 @@ describe('reportHeartbeat', () => {
const config = fixtures.buildConfig();
const coordinatorId = randomHexString(16);
const state = coordinatorState.create(config, coordinatorId);
const heartbeatPayload = {
const heartbeatPayload = JSON.stringify({
timestamp: expectedTimestamp,
stage: state.config.nodeSettings.stage,
cloud_provider: state.config.nodeSettings.cloudProvider.type,
http_gateway_url: 'http://localhost:3000/http-data',
http_signed_data_gateway_url: 'http://localhost:3000/http-signed-data',
};
const signature = await heartbeat.signHeartbeat(heartbeatPayload, timestamp);
});
const signature = await heartbeat.signHeartbeat(heartbeatPayload);
const logs = await heartbeat.reportHeartbeat(state);
expect(logs).toEqual([
{ level: 'INFO', message: 'Sending heartbeat...' },
Expand All @@ -93,9 +95,8 @@ describe('reportHeartbeat', () => {
'airnode-heartbeat-api-key': '3a7af83f-6450-46d3-9937-5f9773ce2849',
},
data: {
...heartbeatPayload,
payload: heartbeatPayload,
signature,
timestamp,
},
timeout: 5_000,
});
Expand Down Expand Up @@ -159,14 +160,15 @@ describe('reportHeartbeat', () => {
const region = 'us-east1';
config.nodeSettings.cloudProvider = { type: 'aws', disableConcurrencyReservations: false, region };
const state = coordinatorState.create(config, 'coordinatorId');
const heartbeatPayload = {
const heartbeatPayload = JSON.stringify({
timestamp: expectedTimestamp,
stage: state.config.nodeSettings.stage,
cloud_provider: state.config.nodeSettings.cloudProvider.type,
region,
http_gateway_url: httpGatewayUrl,
http_signed_data_gateway_url: httpSignedDataGatewayUrl,
};
const signature = await heartbeat.signHeartbeat(heartbeatPayload, timestamp);
});
const signature = await heartbeat.signHeartbeat(heartbeatPayload);
const logs = await heartbeat.reportHeartbeat(state);

expect(logs).toEqual([
Expand All @@ -181,9 +183,8 @@ describe('reportHeartbeat', () => {
'airnode-heartbeat-api-key': '3a7af83f-6450-46d3-9937-5f9773ce2849',
},
data: {
...heartbeatPayload,
payload: heartbeatPayload,
signature,
timestamp,
},
timeout: 5_000,
});
Expand All @@ -194,13 +195,14 @@ describe('reportHeartbeat', () => {
const config = cloneDeep(baseConfig);
config.nodeSettings.cloudProvider = { type: 'local', gatewayServerPort: 8765 };
const state = coordinatorState.create(config, 'coordinatorId');
const heartbeatPayload = {
const heartbeatPayload = JSON.stringify({
timestamp: expectedTimestamp,
stage: state.config.nodeSettings.stage,
cloud_provider: state.config.nodeSettings.cloudProvider.type,
http_gateway_url: 'http://localhost:8765/http-data',
http_signed_data_gateway_url: 'http://localhost:8765/http-signed-data',
};
const signature = await heartbeat.signHeartbeat(heartbeatPayload, timestamp);
});
const signature = await heartbeat.signHeartbeat(heartbeatPayload);
const logs = await heartbeat.reportHeartbeat(state);

expect(logs).toEqual([
Expand All @@ -215,9 +217,8 @@ describe('reportHeartbeat', () => {
'airnode-heartbeat-api-key': '3a7af83f-6450-46d3-9937-5f9773ce2849',
},
data: {
...heartbeatPayload,
payload: heartbeatPayload,
signature,
timestamp,
},
timeout: 5_000,
});
Expand All @@ -226,24 +227,20 @@ describe('reportHeartbeat', () => {

describe('signHearbeat', () => {
const airnodeAddress = fixtures.getAirnodeWallet().address;
const heartbeatPayload = {
const heartbeatPayload = JSON.stringify({
timestamp: expectedTimestamp,
stage: 'test',
cloud_provider: 'local',
http_gateway_url: httpGatewayUrl,
http_signed_data_gateway_url: httpSignedDataGatewayUrl,
};
});

it('signs verifiable heartbeat', async () => {
const signature = await heartbeat.signHeartbeat(heartbeatPayload, timestamp);
const signerAddress = ethers.utils.verifyMessage(
ethers.utils.arrayify(
ethers.utils.solidityKeccak256(['uint256', 'string'], [timestamp, JSON.stringify(heartbeatPayload)])
),
signature
);
const signature = await heartbeat.signHeartbeat(heartbeatPayload);
const signerAddress = ethers.utils.verifyMessage(heartbeatPayload, signature);

expect(signature).toEqual(
'0x3d63a776155b1ea16eff0a8399fe16ed25e1f207300136918979074e80bd2cab75ee200bb4148e294a0867a42520eeed1a2214f190e3369914b5d2116c2cde141b'
'0x941f0ed9f7990f26c9b98434cc3dc094d35607a9087f0e6a71f58659d3383dce6d19c64a85a11714287cbb2cec8e1718e6fe40e4cd81d5658b000b40fa75c0a91c'
);
expect(signerAddress).toEqual(airnodeAddress);
});
Expand Down
35 changes: 13 additions & 22 deletions packages/airnode-node/src/reporting/heartbeat.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { ethers } from 'ethers';
import { execute } from '@api3/airnode-adapter';
import { logger, PendingLog } from '@api3/airnode-utilities';
import { go } from '@api3/promise-utils';
Expand All @@ -22,23 +21,10 @@ export function getHttpSignedDataGatewayUrl(config: Config) {
return getEnvValue('HTTP_SIGNED_DATA_GATEWAY_URL');
}

export const signHeartbeat = (
heartbeatPayload: {
cloud_provider: string;
stage: string;
region?: string;
http_gateway_url?: string;
httpSignedDataGatewayUrl?: string;
},
timestamp: number
) => {
export const signHeartbeat = (heartbeatPayload: string) => {
const airnodeWallet = getAirnodeWalletFromPrivateKey();

return airnodeWallet.signMessage(
ethers.utils.arrayify(
ethers.utils.solidityKeccak256(['uint256', 'string'], [timestamp, JSON.stringify(heartbeatPayload)])
)
);
return airnodeWallet.signMessage(heartbeatPayload);
};

export async function reportHeartbeat(state: CoordinatorState): Promise<PendingLog[]> {
Expand All @@ -56,16 +42,22 @@ export async function reportHeartbeat(state: CoordinatorState): Promise<PendingL
const httpGatewayUrl = getHttpGatewayUrl(config);
const httpSignedDataGatewayUrl = getHttpSignedDataGatewayUrl(config);

const heartbeatPayload = {
const timestamp = Math.round(Date.now() / 1_000);

/*
The heartbeat payload is serialised as JSON and then serialised again in JSON as the value of the payload field.
The reason this is done is to avoid any inconsistencies between different JSON serialisation implementations.
*/
const heartbeatPayload = JSON.stringify({
Copy link
Member

@andreogle andreogle Nov 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's worth adding a comment explaining why this is stringified. i.e. to avoid any minor differences that might occur when encoding/decoding in other environments. The payload/message needs to be exactly the same everywhere

Copy link
Contributor Author

@aquarat aquarat Nov 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the following comment, LMK if you disagree:

  /*
  The heartbeat payload is serialised as JSON and then serialised again in JSON as the value of the payload field.
  The reason this is done is to avoid any inconsistencies between different JSON serialisation implementations.
   */

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Maybe it would be wise to make sure that the payload content fields are always sorted. JSON.stringify doesn't do that. Maybe something like [json-stable-stringify](https://github.com/ljharb/json-stable-stringify] would be better. It's not like JSON.stringify will stringify it randomly, the fields will be in the order they are presented in when creating the object. But just to be on the safe side?

Copy link
Contributor Author

@aquarat aquarat Nov 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we serialise it like this we will always get a consistent result.
Serialising a serialised payload gets rid of the unstable-ordering-related issue and any other potential issues as you're comparing the signature with a byte array effectively. Adding an additional library to ensure serialisation ordering consistency seems unnecessary to me and introduces a new dependency 🤷

timestamp,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I'd suggest using the UNIX timestamp (in seconds) and not the JS timestamp which is in ms... But this is a breaking change so we might just not do it.

Copy link
Member

@andreogle andreogle Nov 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Unix time would be much better imo. And just to confirm, is this in UTC or is it the server timezone?

Why is it a breaking change?

Copy link
Contributor

@Siegrift Siegrift Nov 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you rely on the heartbeat timestamp being in ms? (That's how it worked until now). Changing that to seconds would mean the old values would need to be migrated.

EDIT: But tou will be implementing new handling for this anyway, so maybe it's not a big problem. But I am not sure if there aren't any other users of heartbeat that would be affected. (But they will be affected either way by the packing removal).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChainAPI hasn't been using the timestamp until now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made the change by using Math.round(Date.now()) 👌

stage,
cloud_provider: cloudProvider.type,
...(cloudProvider.type !== 'local' ? { region: cloudProvider.region } : {}),
...(httpGatewayUrl ? { http_gateway_url: httpGatewayUrl } : {}),
...(httpSignedDataGatewayUrl ? { http_signed_data_gateway_url: httpSignedDataGatewayUrl } : {}),
};
const timestamp = Date.now();
});

const goSignHeartbeat = await go(() => signHeartbeat(heartbeatPayload, timestamp));
const goSignHeartbeat = await go(() => signHeartbeat(heartbeatPayload));
if (!goSignHeartbeat.success) {
const log = logger.pend('ERROR', 'Failed to sign heartbeat', goSignHeartbeat.error);
return [log];
Expand All @@ -78,9 +70,8 @@ export async function reportHeartbeat(state: CoordinatorState): Promise<PendingL
'airnode-heartbeat-api-key': apiKey,
},
data: {
...heartbeatPayload,
payload: heartbeatPayload,
signature: goSignHeartbeat.data,
timestamp,
},
timeout: 5_000,
};
Expand Down