Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Adds custom project's metadata in the webserver API #4421

Merged
merged 53 commits into from
Jul 7, 2023

Conversation

pcrespov
Copy link
Member

@pcrespov pcrespov commented Jun 26, 2023

What do these changes do?

Implements get/update_project_metadata operations on web-server side for this exchange with the public-api to manage custom metadata (i.e. user-defined) on a project

sequenceDiagram
        participant API
        participant AS as api-server
        participant WS as web-server
        link API: OAS @ https://editor.swagger.io/?url=https://raw.githubusercontent.com/ITISFoundation/osparc-simcore/master/services/api-server/openapi.json
        link WS: OAS @ https://editor.swagger.io/?url=https://raw.githubusercontent.com/ITISFoundation/osparc-simcore/master/services/web/server/src/simcore_service_webserver/api/v0/openapi.yaml
    
        API->>+AS: get/update_job_metadata(JobMetadataReplace)
        AS->>+WS: get/update_project_metadata(ProjectMetadataReplace)
        WS-->>-AS: ProjectMetadataGet
        AS-->>-API: JobMetadata
Loading

The connection with the api-server will be done in a separate PR for clarity.

The project metadata is then stored in the database in a new table projects_metadata. This table should keep "third-party" metadata attached to a project. This info SHOULD NOT be actual properties of the project (e.g. uuid, name etc) but rather extra information attached by third-parties that "decorate" or qualify a project resource. e.g. things like 'stars', 'quality', 'classifiers', 'dev', etc (or any kind of stats) should be moved here. This is an ERD with some of the tables associated to the projects table

tables

Highlights

  • Adds projects/{project_id}/metadata/custom project's sub-resource webserver API (v0.24.0)
    • implemented as a singleton sub-resource of projects
    • splits controller-service-repository, i.e. _handlers, _api, _db
    • Added api for "project's access rights"
  • 🗃️ Adds "joinable" projects_metadata to include (for now) user custom metadata.
    • Limited to key-value map

Related issue/s

How to test

  • Driving test for database/repository
    • packages/postgres-database/tests/test_utils_projects_metadata.py
  • Driving test for web-server:
    • services/web/server/tests/unit/with_dbs/02/test_projects_metadata_handlers.py

DevOps

  • 🗃️ New projects_metadata table

NOTE: copy & paste diagrams in https://mermaid.live/

@codecov
Copy link

codecov bot commented Jun 26, 2023

Codecov Report

Merging #4421 (9d263d8) into master (afc1eb0) will decrease coverage by 0.4%.
The diff coverage is 90.5%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #4421     +/-   ##
========================================
- Coverage    82.8%   82.5%   -0.4%     
========================================
  Files         944     898     -46     
  Lines       40385   39595    -790     
  Branches      972     584    -388     
========================================
- Hits        33452   32671    -781     
- Misses       6723    6787     +64     
+ Partials      210     137     -73     
Flag Coverage Δ
integrationtests 61.3% <66.4%> (+21.6%) ⬆️
unittests 80.8% <90.5%> (-1.8%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...library/api_schemas_webserver/projects_metadata.py 0.0% <0.0%> (ø)
...c/simcore_service_api_server/api/routes/solvers.py 51.2% <ø> (ø)
...core_service_webserver/exporter/_formatter/_sds.py 56.9% <50.0%> (+29.1%) ⬆️
...mcore_service_webserver/projects/_handlers_crud.py 80.9% <50.0%> (+12.8%) ⬆️
...e_service_webserver/projects/_access_rights_api.py 88.8% <88.8%> (ø)
...simcore_service_webserver/projects/_metadata_db.py 92.0% <92.0%> (ø)
...e_service_webserver/projects/_metadata_handlers.py 95.2% <95.2%> (ø)
...core_postgres_database/models/projects_metadata.py 100.0% <100.0%> (ø)
...mcore_postgres_database/utils_projects_metadata.py 100.0% <100.0%> (ø)
...re_service_webserver/projects/_access_rights_db.py 100.0% <100.0%> (ø)
... and 9 more

... and 260 files with indirect coverage changes

Copy link
Contributor

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good 👍

@pcrespov pcrespov self-assigned this Jun 27, 2023
@pcrespov pcrespov force-pushed the is4110/projects_jobs_n_meta branch from 35a49a4 to 46f0d19 Compare June 27, 2023 18:41
@pcrespov pcrespov added this to the Watermelon milestone Jun 27, 2023
@pcrespov pcrespov added the a:database associated to postgres service and postgres-database package label Jun 27, 2023
@pcrespov pcrespov marked this pull request as draft June 27, 2023 18:47
Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please ask for a review again when this is done.

@pcrespov pcrespov force-pushed the is4110/projects_jobs_n_meta branch from e2d55a0 to 623c1b0 Compare July 4, 2023 12:14
@pcrespov pcrespov force-pushed the is4110/projects_jobs_n_meta branch 3 times, most recently from ec2d472 to e61ece9 Compare July 6, 2023 15:30
@pcrespov pcrespov marked this pull request as ready for review July 6, 2023 16:53
@pcrespov pcrespov requested a review from mguidon July 6, 2023 16:53
@pcrespov pcrespov changed the title ✨ Adds custom projects metadata in the wbserver ✨ Adds custom project's metadata in the wbserver API Jul 6, 2023
@pcrespov pcrespov enabled auto-merge (squash) July 6, 2023 17:23
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice, please consider my comments

@pcrespov pcrespov force-pushed the is4110/projects_jobs_n_meta branch from 3ae0229 to c489d20 Compare July 7, 2023 13:51
Copy link
Contributor

@bisgaard-itis bisgaard-itis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pcrespov pcrespov changed the title ✨ Adds custom project's metadata in the wbserver API ✨ Adds custom project's metadata in the webserver API Jul 7, 2023
@codeclimate
Copy link

codeclimate bot commented Jul 7, 2023

Code Climate has analyzed commit 9d263d8 and detected 0 issues on this pull request.

View more on Code Climate.

@sonarcloud
Copy link

sonarcloud bot commented Jul 7, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@pcrespov pcrespov merged commit 9e9e234 into ITISFoundation:master Jul 7, 2023
@pcrespov pcrespov deleted the is4110/projects_jobs_n_meta branch July 7, 2023 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:database associated to postgres service and postgres-database package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants