Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add meta code verify #311

Merged
merged 37 commits into from
Apr 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
a0ab233
first commit
spiritbroski Mar 9, 2023
41fde68
Merge branch 'main' into meta-code-verify
spiritbroski Mar 27, 2023
5d311ab
done and need review
spiritbroski Mar 28, 2023
24b3969
add all of em
spiritbroski Mar 28, 2023
3c92c59
fix lint
spiritbroski Mar 28, 2023
ed25046
adding origin to ipfs, also check whether origin is the same
spiritbroski Apr 2, 2023
f008c94
fix lint
spiritbroski Apr 2, 2023
2fc75cf
remove yarn.lock and put test inside .github root folder
spiritbroski Apr 4, 2023
291a13d
Merge branch 'develop' into meta-code-verify
spiritbroski Apr 4, 2023
45f9736
add README.md and delete unnecessary files
spiritbroski Apr 5, 2023
45c9746
Merge branch 'develop' into meta-code-verify
spiritbroski Apr 5, 2023
15f2cc0
rename build to built
spiritbroski Apr 6, 2023
f206b22
fix the docs
spiritbroski Apr 11, 2023
8b9125f
refactor(i18n): add i18nAllowlist message
spiritbroski Apr 18, 2023
7347ef0
fix(manifest): update manifest permissions
spiritbroski Apr 18, 2023
82b072d
feat(manifest): add new permissions and host permissions
spiritbroski Apr 18, 2023
58fd3e8
style(popup): add allowlist section to menu
spiritbroski Apr 18, 2023
3d97dd0
feat(background): add support for validating meta company manifests
spiritbroski Apr 18, 2023
958f449
fix(config): change KVSTORE origin to localhost
spiritbroski Apr 18, 2023
33ec312
add allowlist done
spiritbroski Apr 19, 2023
b2d14b3
Merge branch 'meta-code-verify' of github.com:spiritbroski/human-prot…
spiritbroski Apr 19, 2023
ceeb02b
Merge branch 'develop' into meta-code-verify
spiritbroski Apr 19, 2023
c2fa0f3
Update README.md
spiritbroski Apr 19, 2023
869754f
Update README.md
spiritbroski Apr 19, 2023
f1b54a3
change ipfs_cid to version
spiritbroski Apr 19, 2023
04e97c1
fix extension always validate even though its risk
spiritbroski Apr 19, 2023
30d12fb
Update README.md
spiritbroski Apr 19, 2023
0299156
change jest to vitest, change js to ts
spiritbroski Apr 20, 2023
420dafe
Merge branch 'meta-code-verify' of github.com:spiritbroski/human-prot…
spiritbroski Apr 20, 2023
036890e
fix test
spiritbroski Apr 21, 2023
8b4cb6f
build: update package.json for meta-code-verify
spiritbroski Apr 21, 2023
e575714
Update ci-test-meta-code-verify.yml
spiritbroski Apr 21, 2023
d8d2b42
Update README.md
spiritbroski Apr 22, 2023
5fc5ef3
Update README.md
spiritbroski Apr 22, 2023
920abfe
adding generateMerkleTree.ts test
spiritbroski Apr 22, 2023
49816a2
fix lint
spiritbroski Apr 23, 2023
4fbf7fd
Merge branch 'develop' into meta-code-verify
spiritbroski Apr 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/ci-test-meta-code-verify.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: meta-code-verify CI

on:
push:
branches:
- "main"
pull_request:
workflow_dispatch:

jobs:
meta-code-verify-test:
name: meta-code-verify Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: npm install --global yarn && yarn
name: Install dependencies
- run: yarn compile
name: Compile smart contracts
working-directory: ./packages/core
- run: yarn meta-code-verify:test
name: Run meta-code-verify test
6 changes: 4 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,10 @@
"sdk:lint": "yarn workspace @human-protocol/sdk lint",
"basemodels:test": "yarn workspace @human-protocol/basemodels test",
"basemodels:lint": "yarn workspace @human-protocol/basemodels lint",
"test": "concurrently npm:core:test npm:subgraph:test npm:escrow-dashboard:test npm:fortune:test npm:sdk:test npm:basemodels:test npm:faucet-server:test",
"lint": "concurrently npm:core:lint npm:subgraph:lint npm:escrow-dashboard:lint npm:fortune:lint npm:sdk:lint npm:basemodels:lint npm:faucet-server:lint",
"meta-code-verify:test": "yarn workspace @human-protocol/meta-code-verify test",
"meta-code-verify:lint": "yarn workspace @human-protocol/meta-code-verify lint",
"test": "concurrently npm:core:test npm:subgraph:test npm:escrow-dashboard:test npm:fortune:test npm:sdk:test npm:basemodels:test npm:faucet-server:test npm:meta-code-verify:test",
"lint": "concurrently npm:core:lint npm:subgraph:lint npm:escrow-dashboard:lint npm:fortune:lint npm:sdk:lint npm:basemodels:lint npm:faucet-server:lint npm:meta-code-verify:lint",
"prepare": "husky install"
},
"workspaces": {
Expand Down
10 changes: 9 additions & 1 deletion packages/apps/escrow-dashboard/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,15 @@ You may also see any lint errors in the console.

### `yarn run build`

Builds the app for production to the `build` folder.\
Before building you need to set environment variable in `.env`:

```
VITE_APP_NFT_STORAGE_API=
VITE_APP_FAUCET_SERVER_URL=
VITE_APP_WALLETCONNECT_PROJECT_ID=
```

Builds the app for production to the `dist` folder.\
It correctly bundles React in production mode and optimizes the build for the best performance.

The build is minified and the filenames include the hashes.
Expand Down
1 change: 1 addition & 0 deletions packages/apps/escrow-dashboard/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,6 @@
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
<script id="binary-transparency-manifest" type="application/json"></script>
</body>
</html>
6 changes: 6 additions & 0 deletions packages/apps/escrow-dashboard/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,15 @@
"devDependencies": {
"@testing-library/jest-dom": "^5.16.5",
"@testing-library/react": "^14.0.0",
"@types/crypto-js": "^4.1.1",
"@types/glob": "^8.1.0",
"@types/numeral": "^2.0.2",
"@types/react": "^18.0.25",
"@types/react-dom": "^18.0.9",
"@types/react-test-renderer": "^18.0.0",
"@vitejs/plugin-react": "^3.1.0",
"crypto-js": "^4.1.1",
"dotenv": "^16.0.3",
"eslint-config-react-app": "^7.0.1",
"eslint-import-resolver-typescript": "^3.5.2",
"eslint-plugin-import": "^2.26.0",
Expand All @@ -49,7 +53,9 @@
"happy-dom": "^8.2.6",
"identity-obj-proxy": "^3.0.0",
"jsdom": "^21.1.0",
"merkletreejs": "^0.3.9",
"resize-observer-polyfill": "^1.5.1",
"sinon": "^15.0.4",
"vite": "^4.1.4",
"vite-plugin-node-polyfills": "^0.7.0",
"vitest": "^0.29.2"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
import * as fs from 'node:fs';
import * as path from 'path';
import { NFTStorage } from 'nft.storage';
import sinon from 'sinon';
import tmp from 'tmp';
import { test, assert } from 'vitest';
import generateMerkleTree from '../generateMerkleTree';
const mockToken = 'test-token';

test('generateMerkleTree should return a valid Merkle tree JSON', async () => {
// Create a temporary directory for the dist/assets folder
const tmpDir = tmp.dirSync();
const tmpDistAssetsPath = path.join(tmpDir.name, 'dist/assets');
fs.mkdirSync(tmpDistAssetsPath, { recursive: true });

// Create some JS files in the temporary dist/assets folder
fs.writeFileSync(
path.join(tmpDistAssetsPath, 'test.js'),
'console.log("Hello, world!");'
);
fs.writeFileSync(
path.join(tmpDistAssetsPath, 'test2.js'),
'console.log("Another file!");'
);

// Mock NFTStorage.storeBlob method
const fakeCid = 'bafybeih42y6g7zkr76j6ax7z6wjc5d56xazsrtxzp6f7j6fsk67djnppmq';
sinon.stub(NFTStorage.prototype, 'storeBlob').resolves(fakeCid);

const origin = 'https://example.com';

const merkleTreeJson = await generateMerkleTree(
origin,
mockToken,
tmpDistAssetsPath
);
const merkleTreeData = JSON.parse(merkleTreeJson);

// Cleanup and restore the actual file system and NFTStorage.storeBlob method
fs.rmdirSync(tmpDir.name, { recursive: true });
sinon.restore();

// Assertions
assert(
merkleTreeData.hasOwnProperty('version'),
'Merkle tree JSON should have a "version" property'
);
assert(
merkleTreeData.version === fakeCid,
'Merkle tree JSON "version" should match the fake CID'
);
assert(
merkleTreeData.hasOwnProperty('root'),
'Merkle tree JSON should have a "root" property'
);
assert(
merkleTreeData.hasOwnProperty('leaves'),
'Merkle tree JSON should have a "leaves" property'
);
assert(
merkleTreeData.root.startsWith('0x'),
'Merkle tree root hash should start with "0x"'
);
assert(
merkleTreeData.leaves.length > 0,
'Merkle tree leaves should not be empty'
);
assert(
merkleTreeData.leaves.every((leaf) => leaf.startsWith('0x')),
'Each Merkle tree leaf hash should start with "0x"'
);
});
51 changes: 51 additions & 0 deletions packages/apps/escrow-dashboard/scripts/generateMerkleTree.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import * as fs from 'node:fs';
import * as path from 'path';
import { SHA256 } from 'crypto-js';
import * as glob from 'glob';
import { MerkleTree } from 'merkletreejs';
import { NFTStorage } from 'nft.storage';

export default async function generateMerkleTree(
origin: string,
token: string,
buildPath?: string
): Promise<string> {
if (!buildPath) {
buildPath = path.join(__dirname, '../dist/assets');
}

const allFiles = glob.sync('**/*.js', { cwd: buildPath });
const NFT_STORAGE_CLIENT = new NFTStorage({
token,
});
const fileHashes = allFiles.map((file) => {
const filePath = path.join(buildPath, file);
const fileContent = fs.readFileSync(filePath, 'utf-8');
return SHA256(fileContent).toString();
});

const merkleTree = new MerkleTree(fileHashes, SHA256);
const merkleRoot = '0x' + merkleTree.getRoot().toString('hex');

// Add the '0x' prefix to each leaf
const leaves = merkleTree
.getLeaves()
.map((leaf) => '0x' + leaf.toString('hex'));

const someData = new Blob([
JSON.stringify({
origin,
root_hash: merkleRoot,
published_date: Date.now(),
}) as string,
]);
const cid = await NFT_STORAGE_CLIENT.storeBlob(someData);

const merkleTreeJson = JSON.stringify({
version: cid,
root: merkleRoot,
leaves: leaves,
});

return merkleTreeJson;
}
75 changes: 50 additions & 25 deletions packages/apps/escrow-dashboard/vite.config.ts
Original file line number Diff line number Diff line change
@@ -1,35 +1,60 @@
/// <reference types="vitest" />
/// <reference types="vite/client" />

import * as fs from 'fs';
import path from 'path';
import react from '@vitejs/plugin-react';
import dotenv from 'dotenv';
import { defineConfig } from 'vite';
import { nodePolyfills } from 'vite-plugin-node-polyfills';
import generateMerkleTree from './scripts/generateMerkleTree';

dotenv.config();
// https://vitejs.dev/config/
export default defineConfig({
plugins: [
react({ fastRefresh: false }),
nodePolyfills({
// Whether to polyfill `node:` protocol imports.
protocolImports: true,
}),
],
worker: {
plugins: [react()],
},
resolve: {
alias: [{ find: 'src', replacement: path.resolve(__dirname, 'src') }],
},
test: {
globals: true,
environment: 'happy-dom',
setupFiles: './tests/setup.ts',
coverage: {
reporter: ['text', 'json', 'html'],
export default defineConfig(({ mode }) => {
return {
plugins: [
react({ fastRefresh: false }),
nodePolyfills({
// Whether to polyfill `node:` protocol imports.
protocolImports: true,
}),
{
name: 'generate-merkle-tree',
apply: 'build',
async writeBundle() {
const merkleTreeJson = await generateMerkleTree(
mode === 'development'
? 'localhost'
: 'dashboard.humanprotocol.org',
process.env.VITE_APP_NFT_STORAGE_API as string
);

const indexPath = path.resolve(__dirname, './dist/index.html');
const indexContent = fs.readFileSync(indexPath, 'utf-8');
const newIndexContent = indexContent.replace(
'<script id="binary-transparency-manifest" type="application/json"></script>',
`<script id="binary-transparency-manifest" type="application/json">${merkleTreeJson}</script>`
);
fs.writeFileSync(indexPath, newIndexContent);
},
},
],
worker: {
plugins: [react()],
},
resolve: {
alias: [{ find: 'src', replacement: path.resolve(__dirname, 'src') }],
},
test: {
globals: true,
environment: 'happy-dom',
setupFiles: './tests/setup.ts',
coverage: {
reporter: ['text', 'json', 'html'],
},
},
server: {
port: 3002,
},
},
server: {
port: 3002,
},
};
});
21 changes: 21 additions & 0 deletions packages/apps/meta-code-verify/.eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"root": true,
"env": {
"browser": true,
"es2021": true,
"jest": true,
"webextensions": true
},
"parserOptions": { "project": ["./tsconfig.json"] },
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/recommended"
],
"parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint"],
"rules": {
"@typescript-eslint/no-unused-vars": ["error", {"argsIgnorePattern": "^_"}],
"@typescript-eslint/no-empty-function": 0,
"@typescript-eslint/no-extra-semi": 0
}
}
5 changes: 5 additions & 0 deletions packages/apps/meta-code-verify/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
dist/chrome
dist/edge
dist/firefox
node_modules
.env
12 changes: 12 additions & 0 deletions packages/apps/meta-code-verify/.prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"semi": true,
"singleQuote": true,
"trailingComma": "es5",
"bracketSpacing": true,
"jsxBracketSameLine": false,
"arrowParens": "avoid",
"proseWrap": "preserve"
}
Loading