Skip to content

Commit

Permalink
Blockscout scraping (#1092)
Browse files Browse the repository at this point in the history
* Typo

* Fix new Blockscout UI scraping

Since the new Blockscout uses client side rendering, we need a browser
to scrape the page contents.
Use puppeteer to scrape.

Also update `txRegex` to be an array to support trying multiple regexes.

* Install puppeteer dependencies

* Install puppeteer dependencies on new branch

* Lower lib-sourcify coverage limits

* Move from `nyc` to `c8`

After adding `puppeteer` we started getting `cov_{id} is not defined`
errors.

This was caused by `nyc` which hasn't been updated in years.
It would make sense to move to a more up-to-date coverage library.

See istanbuljs/nyc#1327

* Replace additional commands with `nyc`

* Replace `nyc` with `c8` in `bytecode-utils`

* Build before test in `bytecode-utils`

* Replace `nyc` with `c8` in `contract-call-decoder`

* Silence the coverage report in test results

* Replace `nyc` with `c8` in `lib-sourcify`

* Remove nyc from server .gitignore and package-lock

* Fix package-lock's

* Fix package-locks

* Fix 'Please use only .send() or only .field() & .attach()'
  • Loading branch information
kuzdogan authored Jul 11, 2023
1 parent 99f7d7d commit 1658af5
Show file tree
Hide file tree
Showing 20 changed files with 33,875 additions and 29,105 deletions.
3 changes: 3 additions & 0 deletions .circleci/continue_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,9 @@ jobs:
- run:
name: install dependencies
command: npm install && npx lerna bootstrap
- run:
name: install puppeteer dependencies
command: sudo apt-get update && sudo apt-get -y install xvfb gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
- run:
name: lint
command: npm run lint
Expand Down
4 changes: 4 additions & 0 deletions .circleci/new_branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ jobs:
- run:
name: install dependencies
command: npm install && npx lerna bootstrap
- run:
name: install puppeteer dependencies
command: sudo apt-get update && sudo apt-get -y install xvfb gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget

- run:
name: lint
command: npm run lint
Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,5 @@ metacoin-source-verify/
logs
**/too_big.txt
chain-tests-report/
.nyc_output
coverage
!src/server/controllers/repository
Loading

0 comments on commit 1658af5

Please sign in to comment.