Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server-side matchcode processing #224

Closed
6 tasks done
JonoYang opened this issue Nov 17, 2023 · 1 comment
Closed
6 tasks done

Server-side matchcode processing #224

JonoYang opened this issue Nov 17, 2023 · 1 comment
Assignees

Comments

@JonoYang
Copy link
Member

JonoYang commented Nov 17, 2023

We want to run the entire matchcode process from the server side, instead of having the client-server setup we have with scancode.io and purldb. This involves integrating scio and purldb together and updating the matching pipeline to not use API calls but directly accessing the purldb models.

We have a purldb and scancode.io app that cooperate:

  • "client" scancode.io is where a user creates a project and codebase and requests a matching pipeline. This pipeline will post a scan to the server, check if the match is done, and then process match results when available.
  • "server" scancode.io, that we call matchcode, receives a POST request from the scancode.io client with codebase scan
  • matchcode runs a pipeline that performs Package matching on the purldb

The new matching endpoint will be at /api/matching POST
a. Request:
- ScanCode scan
- Webhook
b. Response:
- Matching-id if matching is successfully started else error message
c. Call the webhook with enhanced scan result
d. /api/matching/ - Shows the status and result of match instance

@JonoYang JonoYang self-assigned this Nov 17, 2023
@JonoYang JonoYang added this to the PurlDB API update milestone Nov 17, 2023
JonoYang added a commit that referenced this issue Nov 18, 2023
    * Update dependencies

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Nov 21, 2023
JonoYang added a commit that referenced this issue Nov 22, 2023
    * Update dependencies

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Nov 22, 2023
JonoYang added a commit that referenced this issue Nov 22, 2023
    * Add scancodeio under extra_requires

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Nov 23, 2023
    * Drop aboutcode-toolkit version

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Nov 30, 2023
    * Create MatchingViewSet

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Nov 30, 2023
JonoYang added a commit that referenced this issue Dec 1, 2023
JonoYang added a commit that referenced this issue Dec 1, 2023
    * Use JSON input to build codebase

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 4, 2023
JonoYang added a commit that referenced this issue Dec 4, 2023
    * Fix urls.py

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 5, 2023
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Add scancodeio under extra_requires

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Drop aboutcode-toolkit version

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Create MatchingViewSet

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 13, 2023
JonoYang added a commit that referenced this issue Dec 13, 2023
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Use JSON input to build codebase

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 13, 2023
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Fix urls.py

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 13, 2023
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Add scancodeio under extra_requires

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Drop aboutcode-toolkit version

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Create MatchingViewSet

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Dec 13, 2023
JonoYang added a commit that referenced this issue Dec 13, 2023
JonoYang added a commit that referenced this issue Dec 13, 2023
    * Use JSON input to build codebase

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Jan 12, 2024
JonoYang added a commit that referenced this issue Jan 12, 2024
JonoYang added a commit that referenced this issue Jan 12, 2024
JonoYang added a commit that referenced this issue Jan 12, 2024
JonoYang added a commit that referenced this issue Jan 13, 2024
JonoYang added a commit that referenced this issue Jan 15, 2024
JonoYang added a commit that referenced this issue Jan 15, 2024
JonoYang added a commit that referenced this issue Jan 15, 2024
JonoYang added a commit that referenced this issue Jan 15, 2024
JonoYang added a commit that referenced this issue Jan 15, 2024
    * Route auth and auth_token to packagedb

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Jan 15, 2024
JonoYang added a commit that referenced this issue Jan 15, 2024
JonoYang added a commit that referenced this issue Jan 16, 2024
JonoYang added a commit that referenced this issue Jan 17, 2024
JonoYang added a commit that referenced this issue Jan 26, 2024
    * Declare external network differently
    * Remove production docker-compose file

Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Jan 26, 2024
JonoYang added a commit that referenced this issue Jan 27, 2024
Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Jan 29, 2024
JonoYang added a commit that referenced this issue Jan 29, 2024
JonoYang added a commit that referenced this issue Jan 30, 2024
Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Jan 30, 2024
Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Jan 30, 2024
JonoYang added a commit that referenced this issue Jan 30, 2024
Signed-off-by: Jono Yang <[email protected]>
JonoYang added a commit that referenced this issue Jan 30, 2024
JonoYang added a commit that referenced this issue Jan 30, 2024
JonoYang added a commit that referenced this issue Jan 31, 2024
@pombredanne
Copy link
Member

Closing in favor of the duplicated #238

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants