Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Human mainrepo update #51

Merged
merged 67 commits into from
Apr 18, 2024
Merged

Human mainrepo update #51

merged 67 commits into from
Apr 18, 2024

Conversation

macnablocky
Copy link
Collaborator

Description

This merge request does following things:

  • updates main human repo develop to the latest version
  • merges human develop branch

mrhouzlane and others added 30 commits March 21, 2024 16:53
* refactor reputation

* add web3 auth
remove createWeb3User

* fix tests

* fix mocks

* resolve comments

* add migration

* add migration

* fix auth refactor

* fix rep oracle tests

---------

Co-authored-by: portuu3 <[email protected]>
* Used fund amount in quick launch

* Resolved comments
* Handle escrow_created event in Exchange Oracle

* Fix exchange oracle tests

* Get jobs endpoint in Exchange Oracle (humanprotocol#1712)

* Modify jobDto to camelCase

* Check jobType filter in getJobList

* Add create assignment and get assignments endpoints in Exchange Oracle (humanprotocol#1725)

* [Fortune] Add endpoint for workers stats  (humanprotocol#1717)

* add endpoint

* add assignment repo

---------

Co-authored-by: Francisco López <[email protected]>

* exchange oracle fixes

* comment cors

* replace npm by yarn

* Add cron job module (humanprotocol#1739)

* update yarn.lock

* Add endpoint to getOracleStats and optimize getAssignmentStats

* Fix database.module.ts path

* add docker file to exo

* Add heath module in Exchange Oracle

* test blueprint render

* move render file to root folder

* fixes exchange oracle

* Fix job launcher webhook type and exchange oracle jwt auth

* Fix case in signature guard

* Fix outgoing webhook success check

* fix kyc status check and add constants

* Add exchange oracle unit tests

* Add API docs for exchange oracle and fix some bugs

* Fix exchange oracle webhook

* remove unused render blueprint

* delete unuseful test

---------

Co-authored-by: Mehdi <[email protected]>
Co-authored-by: portuu3 <[email protected]>
Bumps [express](https://github.com/expressjs/express) from 4.18.3 to 4.19.2.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](expressjs/express@4.18.3...4.19.2)

---
updated-dependencies:
- dependency-name: express
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [hardhat-gas-reporter](https://github.com/cgewecke/hardhat-gas-reporter) from 1.0.10 to 2.0.2.
- [Release notes](https://github.com/cgewecke/hardhat-gas-reporter/releases)
- [Changelog](https://github.com/cgewecke/hardhat-gas-reporter/blob/master/CHANGELOG.md)
- [Commits](https://github.com/cgewecke/hardhat-gas-reporter/commits/v2.0.2)

---
updated-dependencies:
- dependency-name: hardhat-gas-reporter
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [winston](https://github.com/winstonjs/winston) from 3.11.0 to 3.13.0.
- [Release notes](https://github.com/winstonjs/winston/releases)
- [Changelog](https://github.com/winstonjs/winston/blob/master/CHANGELOG.md)
- [Commits](winstonjs/winston@v3.11.0...v3.13.0)

---
updated-dependencies:
- dependency-name: winston
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#1770)

* Added `IMAGE_BOXES_FROM_POINTS` and `IMAGE_SKELETONS_FROM_BOXES` job types for CVAT.
API schema for `/job/cvat` endpoint is changed.
Slightly changed the way manifest is being created for a CVAT job.

* Use `ApiPropertyOptional` instead of `IsOptional()`

* Adding missing `IsOptional` decorators

* Add the new IMAGE_BOXES_FROM_POINTS and IMAGE_SKELETONS_FROM_BOXES job types in JL Frontend

* Added missing migration.
Minor fixes

* Send nodes only for IMAGE_SKELETONS_FROM_BOXES

* Tests fix

---------

Co-authored-by: Sergey Dzeranov <[email protected]>
* Fix job event handling when no assignments exist

* Implement boxes from points task creation

* Refactor cloud storage api

* Implement job downloading

* Estimate max bbox

* Little refactoring

* Copy some shared code from excor to recor

* Refactor task creation and export

* Implement boxes from points validation

* Integrate SDK updates

* Fix escrow access

* [Exchange/Recoring oracles] Add GCS support

* Remove local dev mocks

* Remove some extra changes

* Update tests

* Update some tests

* Remove extra launcher change

* Refactor code

* Add google-cloud-storage dependency

* Fix exception classes

* Set env variables in docker-compose.test.yml

* [CVAT] Points to boxes task (humanprotocol#1560)

* Fix job event handling when no assignments exist

* Implement boxes from points task creation

* Refactor cloud storage api

* Implement job downloading

* Estimate max bbox

* Little refactoring

* Copy some shared code from excor to recor

* Refactor task creation and export

* Implement boxes from points validation

* Integrate SDK updates

* Fix escrow access

* Remove local dev mocks

* Remove some extra changes

* Update tests

* Update some tests

* Remove extra launcher change

* Use virtual hosted bucket style

* Fix creating CS in CVAT

* Fix tests

* Add basic implementation for skeletons from boxes

* Refactor and fix some errors

* Implement job uploading

* Implement downloading

* Fix extra gt boxes in merged results from excor

* Implement skeleton matching

* Fix labels in merged annotations

* Update project events handling in excor

* Improve oks sigma comment

* Remove local testing assets

* Fix label mapping

* Add extra skeleton validations in manifest

* Update .env template

* Update several comments

* Fix failing tests

* Update enum name

* Add more annotation validations for skeletons

* Extend input annotations validation

* Fix quality computation, refactor

* Unify assignment accuracy checks between different job types

* Refactor GT downloading for validation

* Add gcs support - fixes (#1)

* Update code formatting

* Update poetry lock

* Refactor

* Use dict for gcs file contents

* Update class name

* Add backward compatibility for data bucket env var

* Remove extra changes

* Remove extra import

* Fix enum name

* Update bucket access info parsing

* Fix error type in cloud provider parsing

* Update tests

* Add job annotation mode param into assignment links

* Add CD trigger for experimental cvat oracles

* Update code formatting

* Fix test

* Format code

* Fix bucket uses

* Fix result sending

* Remove extra changes

* Fix extra code

* Remove unused code

* Align enum naming convention

* Use relative paths in bucket in task creation

* Improve error messages

* Improve skeleton and bbox validation

* Fix escrow manifest downloading

* Update tests

* Fix tests

* Implement general image bans

* Fix import

* Make default gt ban threshold more strict

* Fix comparison for absent points

* Fix comparison for omitted points in jobs

* Finish escrows with too many unverifiable assignments

* Check if an increased healthcheck interval will fix unhealthy container (humanprotocol#1700)

* Fix validations for boxes from points task creation

* Clean code

* disable roi estimation for unreliable cases

* Fix manifest parsing

* Remove m0 launcher stubs

* Remove m0 rep or stubs

* Make max assignment time optional in manifest

* Make label type fully optional in manifest

* Fix test

* Enable and fix a disabled m1 task creation test

* Fix RecOr tests

---------

Co-authored-by: maya <[email protected]>
Co-authored-by: Ivan <[email protected]>
Co-authored-by: Dzeranov <[email protected]>
* Fixed typo in `/webhook/process` endpoint

* Fixed typo in `handleWebhook` method
* Fix job event handling when no assignments exist

* Implement boxes from points task creation

* Refactor cloud storage api

* Implement job downloading

* Estimate max bbox

* Little refactoring

* Copy some shared code from excor to recor

* Refactor task creation and export

* Implement boxes from points validation

* Integrate SDK updates

* Fix escrow access

* [Exchange/Recoring oracles] Add GCS support

* Remove local dev mocks

* Remove some extra changes

* Update tests

* Update some tests

* Remove extra launcher change

* Refactor code

* Add google-cloud-storage dependency

* Fix exception classes

* Set env variables in docker-compose.test.yml

* [CVAT] Points to boxes task (humanprotocol#1560)

* Fix job event handling when no assignments exist

* Implement boxes from points task creation

* Refactor cloud storage api

* Implement job downloading

* Estimate max bbox

* Little refactoring

* Copy some shared code from excor to recor

* Refactor task creation and export

* Implement boxes from points validation

* Integrate SDK updates

* Fix escrow access

* Remove local dev mocks

* Remove some extra changes

* Update tests

* Update some tests

* Remove extra launcher change

* Use virtual hosted bucket style

* Fix creating CS in CVAT

* Fix tests

* Add basic implementation for skeletons from boxes

* Refactor and fix some errors

* Implement job uploading

* Implement downloading

* Fix extra gt boxes in merged results from excor

* Implement skeleton matching

* Fix labels in merged annotations

* Update project events handling in excor

* Improve oks sigma comment

* Remove local testing assets

* Fix label mapping

* Add extra skeleton validations in manifest

* Update .env template

* Update several comments

* Fix failing tests

* Update enum name

* Add more annotation validations for skeletons

* Extend input annotations validation

* Fix quality computation, refactor

* Unify assignment accuracy checks between different job types

* Refactor GT downloading for validation

* Add gcs support - fixes (#1)

* Update code formatting

* Update poetry lock

* Refactor

* Use dict for gcs file contents

* Update class name

* Add backward compatibility for data bucket env var

* Remove extra changes

* Remove extra import

* Fix enum name

* Update bucket access info parsing

* Fix error type in cloud provider parsing

* Update tests

* Add job annotation mode param into assignment links

* Add CD trigger for experimental cvat oracles

* Update code formatting

* Fix test

* Format code

* Fix bucket uses

* Fix result sending

* Remove extra changes

* Fix extra code

* Remove unused code

* Align enum naming convention

* Use relative paths in bucket in task creation

* Improve error messages

* Improve skeleton and bbox validation

* Fix escrow manifest downloading

* Update tests

* Fix tests

* Implement general image bans

* Fix import

* Make default gt ban threshold more strict

* Fix comparison for absent points

* Fix comparison for omitted points in jobs

* Finish escrows with too many unverifiable assignments

* Check if an increased healthcheck interval will fix unhealthy container (humanprotocol#1700)

* Fix validations for boxes from points task creation

* Clean code

* disable roi estimation for unreliable cases

* Fix manifest parsing

* Remove m0 launcher stubs

* Remove m0 rep or stubs

* Make max assignment time optional in manifest

* Make label type fully optional in manifest

* Fix test

* Enable and fix a disabled m1 task creation test

* Don't fail on too many unused GT in boxes from points

* Make joints optional in skeletons from boxes manifests

* Review gt exclusion errors

* Fix RecOr tests

---------

Co-authored-by: maya <[email protected]>
Co-authored-by: Ivan <[email protected]>
Co-authored-by: Dzeranov <[email protected]>
Enabled signature for CVAT oracles
Check not only for 201 status but for 200 too when webhook is being sent
* Fix project cancellation for skeletons from boxes jobs

* Remove m0 launcher compatibility stub in webhook receiving of exchange oracle

* Fix reported assignment size
* Changed the way of working with env variables

* Config services splitted into different files

* Remove `publicKey` from `auth.modules.ts`

* Fixed tests, implemented configs

* Added server config

* Resolved comments

---------

Co-authored-by: Eugene Voronov <[email protected]>
* Implemented get elements count for cvat

* Slightly changed the way of calculating `jobBounty` for `IMAGE_BOXES_FROM_POINTS` and `IMAGE_SKELETONS_FROM_BOXES` job types

* Updated job interface, updated get elements count method, improved calculate job bounty method

* Updated job service

* Removed env config

---------

Co-authored-by: Sergey Dzeranov <[email protected]>
…ol#1798)

* Fixed create incoming webhook method

* Tests fix

---------

Co-authored-by: Sergey Dzeranov <[email protected]>
)

Bumps [hardhat-dependency-compiler](https://github.com/ItsNickBarry/hardhat-dependency-compiler) from 1.1.3 to 1.1.4.
- [Commits](ItsNickBarry/hardhat-dependency-compiler@v1.1.3...v1.1.4)

---
updated-dependencies:
- dependency-name: hardhat-dependency-compiler
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@stripe/stripe-js](https://github.com/stripe/stripe-js) from 2.4.0 to 3.1.0.
- [Release notes](https://github.com/stripe/stripe-js/releases)
- [Commits](stripe/stripe-js@v2.4.0...v3.1.0)

---
updated-dependencies:
- dependency-name: "@stripe/stripe-js"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [concurrently](https://github.com/open-cli-tools/concurrently) from 7.6.0 to 8.2.2.
- [Release notes](https://github.com/open-cli-tools/concurrently/releases)
- [Commits](open-cli-tools/concurrently@v7.6.0...v8.2.2)

---
updated-dependencies:
- dependency-name: concurrently
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env) from 7.24.1 to 7.24.3.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.24.3/packages/babel-preset-env)

---
updated-dependencies:
- dependency-name: "@babel/preset-env"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* add reputation oracle, make sengrid optional and use env for setup scripts

* Add setup for fortune oracles

* Fix some bugs to run localhost fortune

* remove duplicated code

* Remove yarn start:watch and fix encryption conditionals

---------

Co-authored-by: portuu3 <[email protected]>
…1806)

* Refactored calculate job bounty method

* Updated create datset actions

* Resolved comments
)

* Updated getCvatElementsCount method

* Imlemented generateUrls method, refactored calculateJobBounty method, updated unit tests
flopez7 and others added 28 commits April 5, 2024 09:22
* Improve config service in reputation oracle

* Use disabled as default value for SENDGRID_API_KEY

* Create new KYC entry when synaps apiKey disabled

* Fix tests
* Refactor the way to handle bucket urls

* add missing envs
Modify getResults by adding the id in path instead of query
Fix getJobList call in job launcher client
* Improve config service in reputation oracle

* Update s3-config.service.ts

* Add missing cron job  and stats tests

* Add postgres_url for render deployments
…l#1830)

* Filter ambiguous boxes and skeletons in GT preparation
* Increase the threshold for unused or discarded GT in new tasks
* reputation oracle serverless

* fix get reputation serverless function
* Update jwt times from milliseconds to seconds
Remove parseEthers in jl front since it is already done in server

* Remove bignumber.ts utils file
* Implemented auth workflow

* add database and envs to e2e tests

* fix env load

* [CVAT-M2] Add more logging to recording oracle (humanprotocol#1813)

* Add more logs to recording oracle

* fix default echo value

* [CVAT-M2] Fix validation when no GT stats to be updated (humanprotocol#1815)

* Fix recording oracle failure when no gt stats to be updated

* Add trace loglevel for db queries logging

* Reputation Oracle - add hcaptcha token to signin dto (humanprotocol#1816)

add hcaptcha token to sign in

* Removed spaces in jwt keys

* Revert "Removed spaces in jwt keys"

This reverts commit 35e91b8.

* Revert "Reputation Oracle - add hcaptcha token to signin dto (humanprotocol#1816)"

This reverts commit d571157.

* Revert "[CVAT-M2] Fix validation when no GT stats to be updated (humanprotocol#1815)"

This reverts commit e7632c7.

* Removed spaces in jwt keys

* Revert "fix env load"

This reverts commit 354b96a.

* Revert "[CVAT-M2] Add more logging to recording oracle (humanprotocol#1813)"

This reverts commit 90f0eca.

* Fix env load

* add set NODE_ENV

---------

Co-authored-by: portuu3 <[email protected]>
Co-authored-by: Maxim Zhiltsov <[email protected]>
Co-authored-by: portuu3 <[email protected]>
Bumps [stripe](https://github.com/stripe/stripe-node) from 14.23.0 to 14.24.0.
- [Release notes](https://github.com/stripe/stripe-node/releases)
- [Changelog](https://github.com/stripe/stripe-node/blob/master/CHANGELOG.md)
- [Commits](stripe/stripe-node@v14.23.0...v14.24.0)

---
updated-dependencies:
- dependency-name: stripe
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [tenderly](https://github.com/Tenderly/hardhat-tenderly) from 0.7.0 to 0.9.1.
- [Release notes](https://github.com/Tenderly/hardhat-tenderly/releases)
- [Changelog](https://github.com/Tenderly/hardhat-tenderly/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Tenderly/hardhat-tenderly/compare/[email protected]@0.9.1)

---
updated-dependencies:
- dependency-name: tenderly
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
)

* Implemented e2e quick launch workflow

* Added extra checks

* Updated tests
* Added restore password e2e workflow

* Added e2e file
* Exclude input GT files from the data for assignments

* Use the original GT during merging final results for new task types

* Fix gt preparation for skeletons_from_boxes

* Fix debug message
…ocol#1817)

Temporarily disable sending webhooks to Recording Oracle
* Display partially paid escrows on Job Launcher

* Fix subgraph tests
* modify getReputationNetworkOperators

* update test

* test fixed

* job_types not optional

* update

* modify python sdk

* change schema

* modify subgraph schema

* update subgraph

* update subgraph

* update python sdk

* remove job type as param

* undo changes

* make fixes

* changes

* fix sdks

* fix test

* undo subgraph

* change subgraph url in tests

---------

Co-authored-by: portuu3 <[email protected]>
* deploy contracts on Amoy

* polygon amoy

* Fix SDK failing tests

* fix jl client for amoy network

---------

Co-authored-by: Francisco López <[email protected]>
Bumps [@sendgrid/mail](https://github.com/sendgrid/sendgrid-nodejs) from 7.7.0 to 8.1.3.
- [Release notes](https://github.com/sendgrid/sendgrid-nodejs/releases)
- [Changelog](https://github.com/sendgrid/sendgrid-nodejs/blob/main/CHANGELOG.md)
- [Upgrade guide](https://github.com/sendgrid/sendgrid-nodejs/blob/main/UPGRADE.md)
- [Commits](sendgrid/sendgrid-nodejs@7.7.0...8.1.3)

---
updated-dependencies:
- dependency-name: "@sendgrid/mail"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@types/react-dom](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react-dom) from 18.2.24 to 18.2.25.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react-dom)

---
updated-dependencies:
- dependency-name: "@types/react-dom"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [openpgp](https://github.com/openpgpjs/openpgpjs) from 5.10.2 to 5.11.1.
- [Release notes](https://github.com/openpgpjs/openpgpjs/releases)
- [Commits](openpgpjs/openpgpjs@v5.10.2...v5.11.1)

---
updated-dependencies:
- dependency-name: openpgp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
humanprotocol#1879)

* Return only projects with active assignments in /tasks?wallet_address

* Update test
Fix bigint parsing and disable mumbai and goerli
@macnablocky macnablocky merged commit 885168a into develop Apr 18, 2024
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants