Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: dedt_lumi provider #1119

Merged
merged 33 commits into from
Apr 30, 2024
Merged

feat: dedt_lumi provider #1119

merged 33 commits into from
Apr 30, 2024

Conversation

sbrunato
Copy link
Collaborator

@sbrunato sbrunato commented Apr 29, 2024

Adds dedt_lumi as new provider

Destination Earth Digital Twin Outputs from LUMI through Polytope API

Updates Download order mechanism

jlahovnik and others added 30 commits April 26, 2024 18:36
add polytope as a provider + create new search plugin
required for WEkEO with PostJsonSearch
* rename de_dt_lumi to dedt_lumi
Copy link
Contributor

github-actions bot commented Apr 29, 2024

Test Results

    4 files  ±0      4 suites  ±0   4m 35s ⏱️ +7s
  508 tests ±0    505 ✅ ±0   3 💤 ±0  0 ❌ ±0 
2 032 runs  ±0  1 942 ✅ ±0  90 💤 ±0  0 ❌ ±0 

Results for commit 9e7bfb3. ± Comparison against base commit bd79ff7.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Apr 29, 2024

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         303      51  83.17%   60, 646-692, 794-845, 849
config.py                                      338      27  92.01%   81-83, 92, 100, 104-106, 179, 191, 441-443, 507-510, 557-558, 567-568, 647, 716-721, 723
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    739      76  89.72%   88-97, 370, 597, 641-644, 682, 787, 791-796, 822, 892, 969, 1082, 1172-1184, 1224, 1226, 1230, 1253, 1257-1268, 1281-1287, 1377-1380, 1409-1429, 1486, 1503-1506, 1518-1521, 1889, 1922-1928, 2193, 2197-2200, 2214-2216, 2251
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        215      34  84.19%   53-60, 64-66, 170-177, 261-262, 352, 367-368, 393, 454, 468-471, 484, 508-511, 554-560
api/product/metadata_mapping.py                653      84  87.14%   67-69, 130-132, 233, 265-266, 312-313, 323-335, 337, 348, 354-366, 407-410, 447, 468-471, 494, 502-503, 579-580, 604-605, 611-614, 629-630, 779, 825, 942-947, 1078, 1092-1112, 1132, 1137, 1247, 1269, 1283, 1296-1315, 1354, 1406, 1444-1448, 1467
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       3  86.96%   25, 48, 55
plugins/manager.py                             132      14  89.39%   49-51, 105-110, 156, 195, 215, 241, 280-281
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           98      15  84.69%   47-55, 156-158, 205-206, 232-234
plugins/apis/usgs.py                           172      36  79.07%   59-64, 128, 201, 235, 270-272, 277, 303-304, 309, 339-346, 357-362, 384-390, 392-398, 421
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       103      64  37.86%   39-41, 138-150, 154-172, 180-222, 228-237, 246-286, 291-299, 304-305
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 86      17  80.23%   35-37, 86, 99, 101, 116-128, 170-173
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      10  80.00%   32-34, 51-52, 71, 80-83, 85, 92-95
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        491     165  66.40%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 458-459, 465-469, 502, 537, 541, 548, 578-586, 590, 628-636, 643-645, 686-760, 778-839, 850-855, 871-884, 913, 928-930, 933, 943-951, 959-972, 982-1001, 1008-1020, 1061, 1087, 1132-1134, 1354
plugins/download/base.py                       260      57  78.08%   58-64, 145, 180, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 688, 710, 718
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       507     118  76.73%   85-91, 124, 202-212, 214-215, 250-253, 315-318, 320-321, 328-333, 347, 349-350, 402, 409-415, 430, 444, 458, 461-463, 477-482, 494, 512, 551-555, 612, 671-677, 713-774, 792, 825-834, 860-861, 891, 899, 915, 932-933, 960-961, 968, 1029-1035, 1090-1091, 1097, 1107, 1173, 1177, 1191-1207
plugins/download/s3rest.py                     117      27  76.92%   55-58, 121, 162, 196, 226-233, 236-238, 242, 255-261, 269-270, 273-277, 300, 321-324
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         127       9  92.91%   49-54, 108, 112, 275, 295, 378
plugins/search/build_search_result.py          179      24  86.59%   62, 97, 142-143, 149, 160, 296-299, 333, 390-407, 469, 472, 482, 499, 527, 529
plugins/search/creodias_s3.py                   54       3  94.44%   55, 73, 107
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          197      65  67.01%   52, 89-92, 108, 120, 124-125, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 280-283, 291-302, 319, 321, 328-329, 331-332, 350-354, 387, 394, 405, 418, 424-436, 441
plugins/search/qssearch.py                     552      58  89.49%   96, 368-374, 382-383, 490-496, 551-554, 627-628, 669, 687, 702, 755, 776, 779-780, 789, 800, 809, 832, 892-897, 901-902, 930, 1001, 1048, 1124-1128, 1194-1195, 1216, 1241-1253, 1260, 1291-1292, 1302-1308, 1351, 1365, 1385, 1475
plugins/search/static_stac_search.py            47       3  93.62%   39-40, 82
rest/__init__.py                                 5       2  60.00%   21-22
rest/core.py                                   206      16  92.23%   66-72, 261, 345, 605, 607, 610-612, 684, 691-695
rest/server.py                                 299      54  81.94%   80-81, 107, 130-131, 242-244, 260, 300-301, 313-329, 415-420, 451, 612-619, 651, 694-695, 783-785, 802-807, 837, 839, 843-844, 848-849
rest/stac.py                                   436      89  79.59%   60-62, 272, 294, 313-319, 366, 403-405, 428, 463-464, 550-595, 640, 660-661, 841, 906-908, 1127, 1137-1149, 1162-1184, 1198-1243, 1402-1403
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185       9  95.14%   52-55, 231-235, 288, 291, 359
rest/types/stac_queryables.py                   30       2  93.33%   28, 114
rest/types/stac_search.py                      122      10  91.80%   48-51, 167, 182-184, 192, 196
rest/utils/__init__.py                          96      13  86.46%   51, 108-109, 128-130, 183, 193-207
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       3  90.32%   78, 90, 92
types/__init__.py                               76       6  92.11%   53, 87, 154, 174, 179, 187
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              502      43  91.43%   84, 89, 110-112, 193-194, 203-230, 233, 247, 329-333, 409-413, 434-436, 518, 523, 533, 571-572, 968-971, 979-980, 1021-1022, 1102, 1186, 1204, 1380
utils/constraints.py                           123      42  65.85%   31, 84-93, 134, 139, 143, 154, 177-178, 189-197, 206, 220-236, 245-256
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/rest.py                                   28       1  96.43%   49
utils/stac_reader.py                            91      28  69.23%   55-56, 63-86, 93-95, 99, 141, 155-158
TOTAL                                         8812    1560  82.30%

Diff against develop

Filename                                 Stmts    Miss  Cover
-------------------------------------  -------  ------  -------
config.py                                  +20       0  +0.50%
api/product/metadata_mapping.py             +4      +2  -0.23%
plugins/authentication/token.py             +5      +4  -3.72%
plugins/download/base.py                    -1       0  -0.08%
plugins/download/http.py                   +49     +24  -2.75%
plugins/search/build_search_result.py       +1      +2  -1.05%
plugins/search/qssearch.py                  +4      +4  -0.66%
rest/core.py                                +7      +1  -0.23%
rest/stac.py                                -1       0  -0.04%
utils/__init__.py                           +5       0  +0.08%
TOTAL                                      +93     +37  -0.23%

Results for commit: 9e7bfb3

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

Copy link
Contributor

github-actions bot commented Apr 29, 2024

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         303      51  83.17%   60, 646-692, 794-845, 849
config.py                                      338      28  91.72%   81-83, 92, 100, 104-106, 179, 191, 441-443, 507-510, 557-558, 567-568, 647, 681, 716-721, 723
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    739      76  89.72%   88-97, 370, 597, 641-644, 682, 787, 791-796, 822, 892, 969, 1082, 1172-1184, 1224, 1226, 1230, 1253, 1257-1268, 1281-1287, 1377-1380, 1409-1429, 1486, 1503-1506, 1518-1521, 1889, 1922-1928, 2193, 2197-2200, 2214-2216, 2251
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        215      34  84.19%   53-60, 64-66, 170-177, 261-262, 352, 367-368, 393, 454, 468-471, 484, 508-511, 554-560
api/product/metadata_mapping.py                653      85  86.98%   67-69, 130-132, 233, 265-266, 312-313, 323-335, 337, 348, 354-366, 407-410, 447, 468-471, 494, 502-503, 579-580, 604-605, 611-614, 629-630, 779, 825, 942-947, 1078, 1092-1112, 1132, 1137, 1247, 1269, 1283, 1296-1315, 1354, 1406, 1429, 1444-1448, 1467
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       4  82.61%   25, 48, 55, 68
plugins/manager.py                             132      14  89.39%   49-51, 105-110, 156, 195, 215, 241, 280-281
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           98      15  84.69%   47-55, 156-158, 205-206, 232-234
plugins/apis/usgs.py                           172      36  79.07%   59-64, 128, 201, 235, 270-272, 277, 303-304, 309, 339-346, 357-362, 384-390, 392-398, 421
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       103      64  37.86%   39-41, 138-150, 154-172, 180-222, 228-237, 246-286, 291-299, 304-305
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 86      17  80.23%   35-37, 86, 99, 101, 116-128, 170-173
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      35  30.00%   32-34, 48-53, 69-114
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        491     165  66.40%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 458-459, 465-469, 502, 537, 541, 548, 578-586, 590, 628-636, 643-645, 686-760, 778-839, 850-855, 871-884, 913, 928-930, 933, 943-951, 959-972, 982-1001, 1008-1020, 1061, 1087, 1132-1134, 1354
plugins/download/base.py                       260      59  77.31%   58-64, 145, 180, 250-252, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 688, 710, 718
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       507     118  76.73%   85-91, 124, 202-212, 214-215, 250-253, 315-318, 320-321, 328-333, 347, 349-350, 402, 409-415, 430, 444, 458, 461-463, 477-482, 494, 512, 551-555, 612, 671-677, 713-774, 792, 825-834, 860-861, 891, 899, 915, 932-933, 960-961, 968, 1029-1035, 1090-1091, 1097, 1107, 1173, 1177, 1191-1207
plugins/download/s3rest.py                     117      27  76.92%   55-58, 121, 162, 196, 226-233, 236-238, 242, 255-261, 269-270, 273-277, 300, 321-324
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         127       9  92.91%   49-54, 108, 112, 275, 295, 378
plugins/search/build_search_result.py          179      31  82.68%   62, 97, 142-143, 149, 160, 296-299, 333, 390-407, 469, 472, 482, 499, 519-534
plugins/search/creodias_s3.py                   54       3  94.44%   55, 73, 107
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          197      65  67.01%   52, 89-92, 108, 120, 124-125, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 280-283, 291-302, 319, 321, 328-329, 331-332, 350-354, 387, 394, 405, 418, 424-436, 441
plugins/search/qssearch.py                     552      82  85.14%   96, 368-374, 382-383, 490-496, 551-554, 627-628, 669, 687, 702, 755, 776, 779-780, 789, 800, 809, 832, 892-897, 901-902, 930, 1001, 1048, 1124-1128, 1194-1195, 1216, 1241-1253, 1260, 1291-1292, 1302-1308, 1351, 1365, 1385, 1435-1504
plugins/search/static_stac_search.py            47       3  93.62%   39-40, 82
rest/__init__.py                                 5       2  60.00%   21-22
rest/core.py                                   206      95  53.88%   66-72, 153, 155, 157, 163-164, 185-193, 199-205, 252-306, 319-355, 505-536, 554, 604-643, 684, 691-695
rest/server.py                                 299     299  0.00%    18-862
rest/stac.py                                   436     148  66.06%   60-62, 214, 272, 294, 313-319, 366, 403-405, 428, 463-464, 550-595, 640, 648-649, 653-661, 783, 841, 906-908, 925-927, 935-937, 950-952, 966-983, 993-1014, 1024-1046, 1054-1071, 1094-1117, 1127, 1137-1149, 1162-1184, 1198-1243, 1396-1422
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185      18  90.27%   52-55, 231-235, 268-270, 288, 291, 297, 301, 359, 371-374
rest/types/stac_queryables.py                   30       6  80.00%   28, 53-58, 114
rest/types/stac_search.py                      122      12  90.16%   48-51, 167, 182-184, 192, 196, 240, 243
rest/utils/__init__.py                          96      31  67.71%   51, 79-85, 105, 108-109, 128-130, 143, 150, 176-184, 191-212
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       5  83.87%   73-74, 78, 90, 92
types/__init__.py                               76      11  85.53%   53, 87, 127, 154, 164-166, 174, 179, 187, 197
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              502      44  91.24%   84, 89, 110-112, 193-194, 203-230, 233, 247, 329-333, 409-413, 434-436, 518, 523, 533, 571-572, 968-971, 979-980, 1021-1022, 1102, 1185-1186, 1204, 1380
utils/constraints.py                           123      42  65.85%   31, 84-93, 134, 139, 143, 154, 177-178, 189-197, 206, 220-236, 245-256
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/rest.py                                   28       1  96.43%   49
utils/stac_reader.py                            91      28  69.23%   55-56, 63-86, 93-95, 99, 141, 155-158
TOTAL                                         8812    2045  76.79%

Diff against develop

Filename                                 Stmts    Miss  Cover
-------------------------------------  -------  ------  -------
config.py                                  +20       0  +0.53%
api/product/metadata_mapping.py             +4      +2  -0.23%
plugins/authentication/token.py             +5      +4  -3.72%
plugins/download/base.py                    -1       0  -0.08%
plugins/download/http.py                   +49     +24  -2.75%
plugins/search/build_search_result.py       +1      +2  -1.03%
plugins/search/qssearch.py                  +4      +4  -0.63%
rest/core.py                                +7      +7  -1.90%
rest/stac.py                                -1       0  -0.07%
utils/__init__.py                           +5       0  +0.09%
TOTAL                                      +93     +43  -0.25%

Results for commit: 9e7bfb3

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@sbrunato sbrunato merged commit be8b4ee into develop Apr 30, 2024
10 checks passed
@sbrunato sbrunato deleted the 924-polytope-2 branch April 30, 2024 07:26
alambare pushed a commit that referenced this pull request May 2, 2024
Co-authored-by: Julia Lahovnik <[email protected]>
Co-authored-by: LAMBARE Aubin <[email protected]>
sbrunato added a commit that referenced this pull request May 2, 2024
Co-authored-by: Julia Lahovnik <[email protected]>
Co-authored-by: LAMBARE Aubin <[email protected]>
sbrunato added a commit that referenced this pull request May 13, 2024
Co-authored-by: Julia Lahovnik <[email protected]>
Co-authored-by: LAMBARE Aubin <[email protected]>
@sbrunato sbrunato added this to the 3.0.0b1 milestone Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants