Skip to content

Masked detection crops vs regular detection crops for ReID feature extraction

Mike edited this page Dec 22, 2022 · 11 revisions

Yolov5m-seg masked crops

HOTA: exp254-pedestrian            HOTA      DetA      AssA      DetRe     DetPr     AssRe     AssPr     LocA      OWTA      HOTA(0)   LocA(0)   HOTALocA(0)
MOT17-04-FRCNN                     46.268    40.507    53.084    43.65     77.509    57.073    82.028    84.142    48.113    60.324    76.513    46.156    
MOT17-05-FRCNN                     39.402    39.461    39.945    45.312    63.063    48.639    63.758    76.969    42.436    57.683    66.837    38.553    
MOT17-09-FRCNN                     51.775    55.197    48.674    61.194    76.153    53.973    78.013    83.43     54.574    67.121    78.622    52.772    
MOT17-10-FRCNN                     46.507    45.036    48.21     48.834    72.091    51.766    75.005    79.146    48.512    64.044    72.683    46.549    
MOT17-11-FRCNN                     55.933    49.806    63.148    59.653    69.347    68.28     83.685    84.404    61.35     71.674    76.957    55.158    
MOT17-13-FRCNN                     40.322    32.507    50.289    34.156    76.089    55.286    75.748    81.533    41.408    51.569    76.745    39.577    
COMBINED                           46.762    41.864    52.629    45.911    74.093    57.331    79.778    82.589    49.1      61.404    75.366    46.277    

CLEAR: exp254-pedestrian           MOTA      MOTP      MODA      CLR_Re    CLR_Pr    MTR       PTR       MLR       sMOTA     CLR_TP    CLR_FN    CLR_FP    IDSW      MT        PT        ML        Frag      
MOT17-04-FRCNN                     39.693    83.844    39.847    48.081    85.378    20.482    40.964    38.554    31.926    22866     24691     3916      73        17        34        32        458       
MOT17-05-FRCNN                     32.745    74.701    34.263    53.058    73.843    19.549    63.91     16.541    19.322    3670      3247      1300      105       26        85        22        216       
MOT17-09-FRCNN                     62.085    81.533    62.629    71.493    88.969    46.154    50        3.8462    48.882    3807      1518      472       29        12        13        1         63        
MOT17-10-FRCNN                     52.582    75.999    53.127    60.433    89.215    28.07     49.123    22.807    38.078    7759      5080      938       70        16        28        13        314       
MOT17-11-FRCNN                     45.369    84.02     45.581    65.801    76.494    32        29.333    38.667    34.854    6209      3227      1908      20        24        22        29        51        
MOT17-13-FRCNN                     38.808    78.53     39.272    42.08     93.743    20.909    34.545    44.545    29.773    4899      6743      327       54        23        38        49        176       
COMBINED                           42.68     81.24     43.055    52.51     84.741    24.38     45.455    30.165    32.829    49210     44506     8861      351       118       220       146       1278      

Identity: exp254-pedestrian        IDF1      IDR       IDP       IDTP      IDFN      IDFP      
MOT17-04-FRCNN                     53.813    42.059    74.684    20002     27555     6780      
MOT17-05-FRCNN                     51.232    44.022    61.268    3045      3872      1925      
MOT17-09-FRCNN                     64.89     58.516    72.821    3116      2209      1163      
MOT17-10-FRCNN                     61.441    51.53     76.072    6616      6223      2081      
MOT17-11-FRCNN                     64.866    60.333    70.137    5693      3743      2424      
MOT17-13-FRCNN                     52.359    37.932    84.501    4416      7226      810       
COMBINED                           56.511    45.764    73.854    42888     50828     15183     

Count: exp254-pedestrian           Dets      GT_Dets   IDs       GT_IDs    
MOT17-04-FRCNN                     26782     47557     145       83        
MOT17-05-FRCNN                     4970      6917      155       133       
MOT17-09-FRCNN                     4279      5325      59        26        
MOT17-10-FRCNN                     8697      12839     104       57        
MOT17-11-FRCNN                     8117      9436      146       75        
MOT17-13-FRCNN                     5226      11642     113       110       
COMBINED                           58071     93716     722       484

Yolov5m-seg regular crops

HOTA: exp256-pedestrian            HOTA      DetA      AssA      DetRe     DetPr     AssRe     AssPr     LocA      OWTA      HOTA(0)   LocA(0)   HOTALocA(0)
MOT17-04-FRCNN                     46.381    40.529    53.315    43.664    77.554    57.052    83.461    84.151    48.225    60.434    76.539    46.255    
MOT17-05-FRCNN                     40.092    39.293    41.478    45.22     62.77     49.566    64.273    76.926    43.215    59.495    66.605    39.626    
MOT17-09-FRCNN                     50.924    54.162    48.013    60.443    75.271    52.002    78.02     83.274    53.867    66.859    77.846    52.047    
MOT17-10-FRCNN                     45.089    44.888    45.425    48.711    71.943    50.429    73.232    79.11     47.033    62.66     72.604    45.494    
MOT17-11-FRCNN                     56.292    49.841    63.923    59.633    69.443    68.814    84.06     84.407    61.713    72.01     76.989    55.44     
MOT17-13-FRCNN                     40.33     32.339    50.557    33.991    76.014    55.015    76.675    81.553    41.42     51.681    76.742    39.661    
COMBINED                           46.667    41.77     52.539    45.83     74.005    57.111    80.403    82.578    49.012    61.43     75.293    46.253    

CLEAR: exp256-pedestrian           MOTA      MOTP      MODA      CLR_Re    CLR_Pr    MTR       PTR       MLR       sMOTA     CLR_TP    CLR_FN    CLR_FP    IDSW      MT        PT        ML        Frag      
MOT17-04-FRCNN                     39.691    83.839    39.853    48.077    85.393    20.482    40.964    38.554    31.921    22864     24693     3911      77        17        34        32        452       
MOT17-05-FRCNN                     32.485    74.768    34.133    53.087    73.691    20.301    60.902    18.797    19.091    3672      3245      1311      114       27        81        25        211       
MOT17-09-FRCNN                     61.972    81.676    62.498    71.399    88.915    46.154    50        3.8462    48.889    3802      1523      474       28        12        13        1         61        
MOT17-10-FRCNN                     52.372    76.006    53.034    60.371    89.164    28.07     49.123    22.807    37.887    7751      5088      942       85        16        28        13        317       
MOT17-11-FRCNN                     45.358    84.03     45.644    65.759    76.577    32        29.333    38.667    34.856    6205      3231      1898      27        24        22        29        50        
MOT17-13-FRCNN                     38.473    78.574    39.065    41.891    93.68     20        35.455    44.545    29.497    4877      6765      329       69        22        39        49        182       
COMBINED                           42.582    81.261    43.009    52.468    84.725    24.38     44.835    30.785    32.75     49171     44545     8865      400       118       217       149       1273      

Identity: exp256-pedestrian        IDF1      IDR       IDP       IDTP      IDFN      IDFP      
MOT17-04-FRCNN                     53.708    41.973    74.551    19961     27596     6814      
MOT17-05-FRCNN                     51.126    43.979    61.048    3042      3875      1941      
MOT17-09-FRCNN                     63.368    57.127    71.141    3042      2283      1234      
MOT17-10-FRCNN                     59.967    50.284    74.267    6456      6383      2237      
MOT17-11-FRCNN                     65.374    60.757    70.752    5733      3703      2370      
MOT17-13-FRCNN                     52.089    37.691    84.287    4388      7254      818       
COMBINED                           56.173    45.48     73.441    42622     51094     15414     

Count: exp256-pedestrian           Dets      GT_Dets   IDs       GT_IDs    
MOT17-04-FRCNN                     26775     47557     147       83        
MOT17-05-FRCNN                     4983      6917      153       133       
MOT17-09-FRCNN                     4276      5325      64        26        
MOT17-10-FRCNN                     8693      12839     109       57        
MOT17-11-FRCNN                     8103      9436      153       75        
MOT17-13-FRCNN                     5206      11642     121       110       
COMBINED                           58036     93716     747       484

Conclusion

  • Boosts on all metrics for segmentation models. HOTA (+0.095) MOTA (+0.098) IDF1 (+0.507)
  • No noticeable slowdown after masking effectivization

Example crops with masking fed to ReID models

Screenshot from 2022-12-18 23-22-56 Screenshot from 2022-12-18 23-23-17