feat(plugins): flatten_top_dirs true by default

[sbrunato]

Set flatten_top_dirs download plugins setting to True by default
(flatten directory structure, removes unnecessary nested parent folders of downloaded products)

[github-actions]

Test Results

    4 files  ±0      4 suites  ±0   5m 57s ⏱️ +3s
  547 tests ±0    544 ✅ ±0    3 💤 ±0  0 ❌ ±0 
2 188 runs  ±0  2 088 ✅ ±0  100 💤 ±0  0 ❌ ±0 

Results for commit 7c835d7. ± Comparison against base commit ccf150d.

♻️ This comment has been updated with latest results.

[github-actions]

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     10       0  100.00%
cli.py                                         302      51  83.11%   60, 644-690, 792-843, 847
config.py                                      350      27  92.29%   81-83, 92, 100, 104-106, 183, 195, 459-461, 525-528, 575-576, 585-586, 665, 734-739, 741
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    762      84  88.98%   89-98, 372, 597, 641-644, 682, 719, 792, 796-801, 827, 923, 1000, 1113, 1203-1215, 1255, 1257, 1261, 1284, 1288-1299, 1312-1318, 1408-1411, 1440-1460, 1517, 1534-1537, 1549-1552, 1569-1576, 1944, 1977-1983, 2248, 2252-2255, 2269-2271, 2306
api/search_result.py                            50       6  88.00%   33-35, 70, 79, 86, 100
api/product/__init__.py                          5       0  100.00%
api/product/_assets.py                          44       6  86.36%   27-29, 79, 119, 129
api/product/_product.py                        188      27  85.64%   56-63, 67-69, 239-240, 322, 351, 412, 426-429, 442, 466-469, 512-518
api/product/metadata_mapping.py                678      83  87.76%   67-69, 131-133, 234, 266-267, 313-314, 324-336, 338, 349, 355-367, 412-413, 450, 471-474, 497, 505-506, 582-583, 607-608, 614-617, 632-633, 782, 828, 1002-1007, 1138, 1152-1172, 1192, 1197, 1307, 1329, 1343, 1356-1375, 1414, 1466, 1504-1508, 1527
api/product/drivers/__init__.py                  5       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 23       3  86.96%   25, 48, 55
plugins/manager.py                             133      15  88.72%   49-51, 105-110, 156, 197, 219, 223, 248, 287-288
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           99      15  84.85%   46-54, 150-152, 199-200, 226-228
plugins/apis/usgs.py                           175      36  79.43%   60-65, 132, 205, 239, 274-276, 281, 307-308, 313, 343-350, 361-366, 388-394, 396-402, 425
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                21       1  95.24%   28
plugins/authentication/keycloak.py              49       6  87.76%   33-35, 132, 156-161
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       186      19  89.78%   41-43, 121, 135-160, 168, 322-325, 349
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 90      17  81.11%   35-37, 79, 107, 109, 131-143, 198-201
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      10  80.00%   32-34, 51-52, 71, 80-83, 85, 92-95
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      19  72.06%   28-30, 33, 72-75, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        492     165  66.46%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 459-460, 466-470, 503, 538, 542, 549, 579-587, 591, 629-637, 644-646, 687-761, 779-840, 851-856, 872-885, 914, 929-931, 934, 944-952, 960-973, 983-1002, 1009-1021, 1062, 1088, 1133-1135, 1355
plugins/download/base.py                       260      57  78.08%   58-64, 145, 180, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 688, 710, 718
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       533     133  75.05%   85-91, 124, 203-213, 215-216, 251-254, 318-321, 323-324, 331-336, 354-369, 386-388, 400, 448, 455-461, 479, 493, 507, 515-517, 533-538, 549, 567, 609-613, 635, 675, 720, 734-740, 776-840, 858, 891-900, 926-927, 954-959, 965, 968, 984, 1001-1002, 1029-1030, 1037, 1099-1105, 1160-1161, 1167, 1177, 1211, 1247, 1265-1281
plugins/download/s3rest.py                     118      27  77.12%   55-58, 121, 162, 197, 227-234, 237-239, 243, 256-262, 270-271, 274-278, 301, 322-325
plugins/search/__init__.py                      26       3  88.46%   27-31
plugins/search/base.py                         133      13  90.23%   48-53, 104, 108, 121, 271, 291, 350-351, 371, 380
plugins/search/build_search_result.py          183      24  86.89%   62, 97, 141-142, 148, 159, 295-298, 327, 384-401, 463, 466, 476, 493, 521, 523
plugins/search/cop_marine.py                   179      40  77.65%   43-44, 55, 65-66, 109-111, 123-124, 164-173, 177, 182, 197, 233, 237, 252, 256, 260, 264, 268-272, 278-281, 284-297, 314, 337, 344
plugins/search/creodias_s3.py                   55       3  94.55%   56, 74, 108
plugins/search/csw.py                          108      83  23.15%   44-46, 58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          198      65  67.17%   53, 90-93, 109, 121, 125-126, 137, 142, 147, 154, 167-170, 224-225, 229, 239-245, 250, 276-279, 287-298, 315, 317, 324-325, 327-328, 346-350, 383, 390, 401, 414, 420-432, 437
plugins/search/qssearch.py                     646      69  89.32%   102, 386, 390-396, 404-405, 511-517, 567, 583, 593, 620, 622, 665-668, 743-744, 787, 806, 821, 879, 900, 903-904, 913, 924, 933, 956, 1020-1025, 1029-1030, 1063, 1137, 1186, 1260-1264, 1330-1331, 1352, 1380-1392, 1399, 1431-1433, 1443-1449, 1493, 1508, 1528, 1631-1641
plugins/search/static_stac_search.py            73      12  83.56%   37-38, 95-122, 138, 151
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  23       0  100.00%
rest/constants.py                                7       0  100.00%
rest/core.py                                   237      30  87.34%   83-87, 279, 363, 478, 705, 712-760
rest/server.py                                 286      55  80.77%   81-82, 108, 131-133, 246-248, 304-305, 317-333, 425-430, 458, 625-632, 661, 705-706, 729, 802-804, 821-826, 855, 857, 861-862, 866-867
rest/stac.py                                   465     123  73.55%   64-66, 319, 341, 389-392, 416-442, 447-453, 476-478, 501, 536-537, 565, 578, 623-663, 691-707, 786-799, 806, 859-860, 924, 992-994, 1213, 1223-1235, 1248-1270, 1284-1329, 1483-1484
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     184       9  95.11%   52-55, 235-239, 292, 295, 363
rest/types/queryables.py                        58       2  96.55%   38, 172
rest/types/stac_search.py                      127      11  91.34%   50-53, 132, 178, 193-195, 203, 207
rest/utils/__init__.py                          95      13  86.32%   51, 108-109, 128-130, 182, 192-206
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       3  90.32%   78, 90, 92
types/__init__.py                              110      11  90.00%   53, 70, 129-132, 199, 214-218, 245, 258
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              501      39  92.22%   84, 89, 110-112, 193-194, 203-230, 233, 247, 329-333, 409-413, 434-436, 518, 533, 571-572, 968-971, 979-980, 1021-1022, 1204
utils/constraints.py                           119      37  68.91%   94-103, 144, 149, 153, 164, 190-192, 202, 216-232, 241-252
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/requests.py                               55      11  80.00%   69, 96, 98, 100, 102, 104, 123, 128-130, 138
utils/rest.py                                   27       1  96.30%   49
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 143, 159-162, 215-224, 234-264
TOTAL                                         9552    1702  82.18%

Diff against develop

Filename                    Stmts    Miss  Cover
------------------------  -------  ------  --------
plugins/download/http.py       +2       0  +0.10%
TOTAL                          +2       0  +100.00%

Results for commit: 7c835d7

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[github-actions]

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     10       0  100.00%
cli.py                                         302      51  83.11%   60, 644-690, 792-843, 847
config.py                                      350      28  92.00%   81-83, 92, 100, 104-106, 183, 195, 459-461, 525-528, 575-576, 585-586, 665, 699, 734-739, 741
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    762      84  88.98%   89-98, 372, 597, 641-644, 682, 719, 792, 796-801, 827, 923, 1000, 1113, 1203-1215, 1255, 1257, 1261, 1284, 1288-1299, 1312-1318, 1408-1411, 1440-1460, 1517, 1534-1537, 1549-1552, 1569-1576, 1944, 1977-1983, 2248, 2252-2255, 2269-2271, 2306
api/search_result.py                            50       6  88.00%   33-35, 70, 79, 86, 100
api/product/__init__.py                          5       0  100.00%
api/product/_assets.py                          44       6  86.36%   27-29, 79, 119, 129
api/product/_product.py                        188      27  85.64%   56-63, 67-69, 239-240, 322, 351, 412, 426-429, 442, 466-469, 512-518
api/product/metadata_mapping.py                678      83  87.76%   67-69, 131-133, 234, 266-267, 313-314, 324-336, 338, 349, 355-367, 412-413, 450, 471-474, 497, 505-506, 582-583, 607-608, 614-617, 632-633, 782, 828, 1002-1007, 1138, 1152-1172, 1192, 1197, 1307, 1329, 1343, 1356-1375, 1414, 1466, 1504-1508, 1527
api/product/drivers/__init__.py                  5       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 23       4  82.61%   25, 48, 55, 68
plugins/manager.py                             133      15  88.72%   49-51, 105-110, 156, 197, 219, 223, 248, 287-288
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           99      15  84.85%   46-54, 150-152, 199-200, 226-228
plugins/apis/usgs.py                           175      36  79.43%   60-65, 132, 205, 239, 274-276, 281, 307-308, 313, 343-350, 361-366, 388-394, 396-402, 425
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                21       1  95.24%   28
plugins/authentication/keycloak.py              49       6  87.76%   33-35, 132, 156-161
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       186      19  89.78%   41-43, 121, 135-160, 168, 322-325, 349
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 90      17  81.11%   35-37, 79, 107, 109, 131-143, 198-201
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      35  30.00%   32-34, 48-53, 69-114
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      19  72.06%   28-30, 33, 72-75, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        492     165  66.46%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 459-460, 466-470, 503, 538, 542, 549, 579-587, 591, 629-637, 644-646, 687-761, 779-840, 851-856, 872-885, 914, 929-931, 934, 944-952, 960-973, 983-1002, 1009-1021, 1062, 1088, 1133-1135, 1355
plugins/download/base.py                       260      59  77.31%   58-64, 145, 180, 250-252, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 688, 710, 718
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       533     133  75.05%   85-91, 124, 203-213, 215-216, 251-254, 318-321, 323-324, 331-336, 354-369, 386-388, 400, 448, 455-461, 479, 493, 507, 515-517, 533-538, 549, 567, 609-613, 635, 675, 720, 734-740, 776-840, 858, 891-900, 926-927, 954-959, 965, 968, 984, 1001-1002, 1029-1030, 1037, 1099-1105, 1160-1161, 1167, 1177, 1211, 1247, 1265-1281
plugins/download/s3rest.py                     118      27  77.12%   55-58, 121, 162, 197, 227-234, 237-239, 243, 256-262, 270-271, 274-278, 301, 322-325
plugins/search/__init__.py                      26       3  88.46%   27-31
plugins/search/base.py                         133      18  86.47%   48-53, 104, 108, 121, 271, 291, 350-351, 371, 374-382, 384
plugins/search/build_search_result.py          183      31  83.06%   62, 97, 141-142, 148, 159, 295-298, 327, 384-401, 463, 466, 476, 493, 513-528
plugins/search/cop_marine.py                   179      40  77.65%   43-44, 55, 65-66, 109-111, 123-124, 164-173, 177, 182, 197, 233, 237, 252, 256, 260, 264, 268-272, 278-281, 284-297, 314, 337, 344
plugins/search/creodias_s3.py                   55       3  94.55%   56, 74, 108
plugins/search/csw.py                          108      83  23.15%   44-46, 58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          198      65  67.17%   53, 90-93, 109, 121, 125-126, 137, 142, 147, 154, 167-170, 224-225, 229, 239-245, 250, 276-279, 287-298, 315, 317, 324-325, 327-328, 346-350, 383, 390, 401, 414, 420-432, 437
plugins/search/qssearch.py                     646      97  84.98%   102, 386, 390-396, 404-405, 511-517, 567, 570, 583, 593, 612-627, 665-668, 743-744, 787, 806, 821, 879, 900, 903-904, 913, 924, 933, 956, 1020-1025, 1029-1030, 1063, 1137, 1186, 1260-1264, 1330-1331, 1352, 1380-1392, 1399, 1431-1433, 1443-1449, 1493, 1508, 1528, 1590-1661
plugins/search/static_stac_search.py            73      12  83.56%   37-38, 95-122, 138, 151
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33      22  33.33%   35-37, 44-70
rest/config.py                                  23       4  82.61%   34-36, 64
rest/constants.py                                7       0  100.00%
rest/core.py                                   237     152  35.86%   83-87, 167-246, 270-324, 337-373, 409-447, 467-483, 506-517, 526-562, 581, 627-666, 705, 712-760
rest/server.py                                 286     286  0.00%    18-878
rest/stac.py                                   465     394  15.27%   64-66, 127-133, 141-164, 195-200, 232, 252-375, 385-454, 472-511, 527-580, 590-613, 623-663, 691-707, 717, 733-736, 755-825, 842-873, 902-927, 935-951, 959-964, 974-1001, 1011-1013, 1021-1023, 1036-1038, 1052-1069, 1079-1100, 1110-1132, 1140-1157, 1180-1203, 1213, 1223-1235, 1248-1270, 1284-1329, 1337-1354, 1365-1514
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     184      19  89.67%   52-55, 125, 235-239, 272-274, 292, 295, 301, 305, 363, 375-378
rest/types/queryables.py                        58      14  75.86%   38, 51-52, 58-59, 65-66, 95-100, 109-110, 172
rest/types/stac_search.py                      127      22  82.68%   50-53, 130-132, 156-157, 162-163, 178, 193-195, 203, 207, 250-255
rest/utils/__init__.py                          95      31  67.37%   51, 79-85, 105, 108-109, 128-130, 143, 150, 175-183, 190-211
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       6  80.65%   70, 73-74, 78, 90, 92
types/__init__.py                              110      36  67.27%   53, 66-70, 81-93, 120-122, 129-132, 172, 199, 209-221, 226, 245, 250, 258, 268
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              501      41  91.82%   84, 89, 110-112, 193-194, 203-230, 233, 247, 329-333, 409-413, 434-436, 518, 533, 571-572, 968-971, 979-980, 1021-1022, 1055, 1204, 1380
utils/constraints.py                           119      37  68.91%   94-103, 144, 149, 153, 164, 190-192, 202, 216-232, 241-252
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/requests.py                               55      11  80.00%   69, 96, 98, 100, 102, 104, 123, 128-130, 138
utils/rest.py                                   27       1  96.30%   49
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 143, 159-162, 215-224, 234-264
TOTAL                                         9552    2495  73.88%

Diff against develop

Filename                    Stmts    Miss  Cover
------------------------  -------  ------  -------
plugins/download/http.py       +2       0  +0.10%
TOTAL                          +2       0  +0.01%

Results for commit: 7c835d7

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results