Skip to content

Latest commit

 

History

History

SWE-bench_Lite_golden

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 

Benchmark results

Patch Apply Success

Resolved Count Rate
Yes 318 98.45%
Partially 0 0%
No 5 1.55%

Patch Apply Success + Failure

Resolved Count Rate
Yes 318 98.45%
Partially 0 0%
No 5 1.55%

Benchmark instances

Applied but not resolved

Instance ID Repository Testbed version
django__django-13660 django/django 3.2
pydata__xarray-4094 pydata/xarray 0.12
pydata__xarray-4493 pydata/xarray 0.12
pydicom__pydicom-1413 pydicom/pydicom 2.1
sympy__sympy-13177 sympy/sympy 1.1

Resolved

Instance ID Repository Testbed version
astropy__astropy-12907 astropy/astropy 4.3
astropy__astropy-14182 astropy/astropy 5.1
astropy__astropy-14365 astropy/astropy 5.1
astropy__astropy-14995 astropy/astropy 5.2
astropy__astropy-6938 astropy/astropy 1.3
astropy__astropy-7746 astropy/astropy 1.3
django__django-10914 django/django 3.0
django__django-10924 django/django 3.0
django__django-11001 django/django 3.0
django__django-11019 django/django 3.0
django__django-11039 django/django 3.0
django__django-11049 django/django 3.0
django__django-11099 django/django 3.0
django__django-11133 django/django 3.0
django__django-11179 django/django 3.0
django__django-11283 django/django 3.0
django__django-11422 django/django 3.0
django__django-11564 django/django 3.1
django__django-11583 django/django 3.0
django__django-11620 django/django 3.0
django__django-11630 django/django 3.0
django__django-11742 django/django 3.0
django__django-11797 django/django 3.1
django__django-11815 django/django 3.1
django__django-11848 django/django 3.1
django__django-11905 django/django 3.1
django__django-11910 django/django 3.1
django__django-11964 django/django 3.1
django__django-11999 django/django 3.1
django__django-12113 django/django 3.1
django__django-12125 django/django 3.1
django__django-12184 django/django 3.1
django__django-12284 django/django 3.1
django__django-12286 django/django 3.1
django__django-12308 django/django 3.1
django__django-12453 django/django 3.1
django__django-12470 django/django 3.1
django__django-12497 django/django 3.1
django__django-12589 django/django 3.1
django__django-12700 django/django 3.1
django__django-12708 django/django 3.1
django__django-12747 django/django 3.1
django__django-12856 django/django 3.2
django__django-12908 django/django 3.2
django__django-12915 django/django 3.2
django__django-12983 django/django 3.2
django__django-13028 django/django 3.2
django__django-13033 django/django 3.2
django__django-13158 django/django 3.2
django__django-13220 django/django 3.2
django__django-13230 django/django 3.2
django__django-13265 django/django 3.2
django__django-13315 django/django 3.2
django__django-13321 django/django 3.2
django__django-13401 django/django 3.2
django__django-13447 django/django 4.0
django__django-13448 django/django 3.2
django__django-13551 django/django 3.2
django__django-13590 django/django 3.2
django__django-13658 django/django 3.2
django__django-13710 django/django 4.0
django__django-13757 django/django 3.2
django__django-13768 django/django 3.2
django__django-13925 django/django 4.0
django__django-13933 django/django 4.0
django__django-13964 django/django 4.0
django__django-14016 django/django 4.0
django__django-14017 django/django 4.0
django__django-14155 django/django 4.0
django__django-14238 django/django 4.0
django__django-14382 django/django 4.0
django__django-14411 django/django 4.0
django__django-14534 django/django 4.0
django__django-14580 django/django 4.0
django__django-14608 django/django 4.0
django__django-14667 django/django 4.0
django__django-14672 django/django 4.0
django__django-14730 django/django 4.0
django__django-14752 django/django 4.0
django__django-14787 django/django 4.1
django__django-14855 django/django 4.0
django__django-14915 django/django 4.1
django__django-14997 django/django 4.1
django__django-14999 django/django 4.1
django__django-15061 django/django 4.1
django__django-15202 django/django 4.1
django__django-15213 django/django 4.1
django__django-15252 django/django 4.1
django__django-15320 django/django 4.1
django__django-15347 django/django 4.1
django__django-15388 django/django 4.1
django__django-15400 django/django 4.1
django__django-15498 django/django 4.1
django__django-15695 django/django 4.1
django__django-15738 django/django 4.2
django__django-15781 django/django 4.2
django__django-15789 django/django 4.2
django__django-15790 django/django 4.2
django__django-15814 django/django 4.2
django__django-15819 django/django 4.2
django__django-15851 django/django 4.2
django__django-15902 django/django 4.2
django__django-15996 django/django 4.2
django__django-16041 django/django 4.2
django__django-16046 django/django 4.2
django__django-16139 django/django 4.2
django__django-16229 django/django 4.2
django__django-16255 django/django 4.2
django__django-16379 django/django 4.2
django__django-16400 django/django 4.2
django__django-16408 django/django 5.0
django__django-16527 django/django 5.0
django__django-16595 django/django 5.0
django__django-16816 django/django 5.0
django__django-16820 django/django 5.0
django__django-16873 django/django 5.0
django__django-16910 django/django 5.0
django__django-17051 django/django 5.0
django__django-17087 django/django 5.0
marshmallow-code__marshmallow-1343 marshmallow-code/marshmallow 2.20
marshmallow-code__marshmallow-1359 marshmallow-code/marshmallow 3.0
matplotlib__matplotlib-18869 matplotlib/matplotlib 3.3
matplotlib__matplotlib-22711 matplotlib/matplotlib 3.5
matplotlib__matplotlib-22835 matplotlib/matplotlib 3.5
matplotlib__matplotlib-23299 matplotlib/matplotlib 3.5
matplotlib__matplotlib-23314 matplotlib/matplotlib 3.5
matplotlib__matplotlib-23476 matplotlib/matplotlib 3.5
matplotlib__matplotlib-23562 matplotlib/matplotlib 3.5
matplotlib__matplotlib-23563 matplotlib/matplotlib 3.5
matplotlib__matplotlib-23913 matplotlib/matplotlib 3.6
matplotlib__matplotlib-23964 matplotlib/matplotlib 3.6
matplotlib__matplotlib-23987 matplotlib/matplotlib 3.6
matplotlib__matplotlib-24149 matplotlib/matplotlib 3.6
matplotlib__matplotlib-24265 matplotlib/matplotlib 3.6
matplotlib__matplotlib-24334 matplotlib/matplotlib 3.6
matplotlib__matplotlib-24970 matplotlib/matplotlib 3.6
matplotlib__matplotlib-25079 matplotlib/matplotlib 3.6
matplotlib__matplotlib-25311 matplotlib/matplotlib 3.7
matplotlib__matplotlib-25332 matplotlib/matplotlib 3.7
matplotlib__matplotlib-25433 matplotlib/matplotlib 3.7
matplotlib__matplotlib-25442 matplotlib/matplotlib 3.7
matplotlib__matplotlib-25498 matplotlib/matplotlib 3.7
matplotlib__matplotlib-26011 matplotlib/matplotlib 3.7
matplotlib__matplotlib-26020 matplotlib/matplotlib 3.7
mwaskom__seaborn-2848 mwaskom/seaborn 0.12
mwaskom__seaborn-3010 mwaskom/seaborn 0.12
mwaskom__seaborn-3190 mwaskom/seaborn 0.12
mwaskom__seaborn-3407 mwaskom/seaborn 0.13
pallets__flask-4045 pallets/flask 2.0
pallets__flask-4992 pallets/flask 2.3
pallets__flask-5063 pallets/flask 2.3
psf__requests-1963 psf/requests 2.3
psf__requests-2148 psf/requests 2.3
psf__requests-2317 psf/requests 2.4
psf__requests-2674 psf/requests 2.7
psf__requests-3362 psf/requests 2.10
psf__requests-863 psf/requests 0.14
pvlib__pvlib-python-1072 pvlib/pvlib-python 0.7
pvlib__pvlib-python-1154 pvlib/pvlib-python 0.8
pvlib__pvlib-python-1606 pvlib/pvlib-python 0.8
pvlib__pvlib-python-1707 pvlib/pvlib-python 0.9
pvlib__pvlib-python-1854 pvlib/pvlib-python 0.9
pydata__xarray-3364 pydata/xarray 0.12
pydata__xarray-4248 pydata/xarray 0.12
pydata__xarray-5131 pydata/xarray 0.12
pydicom__pydicom-1139 pydicom/pydicom 2.0
pydicom__pydicom-1256 pydicom/pydicom 2.1
pydicom__pydicom-1694 pydicom/pydicom 2.3
pydicom__pydicom-901 pydicom/pydicom 1.3
pylint-dev__astroid-1196 pylint-dev/astroid 2.12
pylint-dev__astroid-1268 pylint-dev/astroid 2.9
pylint-dev__astroid-1333 pylint-dev/astroid 2.10
pylint-dev__astroid-1866 pylint-dev/astroid 2.13
pylint-dev__astroid-1978 pylint-dev/astroid 2.14
pylint-dev__pylint-5859 pylint-dev/pylint 2.13
pylint-dev__pylint-6506 pylint-dev/pylint 2.14
pylint-dev__pylint-7080 pylint-dev/pylint 2.15
pylint-dev__pylint-7114 pylint-dev/pylint 2.15
pylint-dev__pylint-7228 pylint-dev/pylint 2.15
pylint-dev__pylint-7993 pylint-dev/pylint 2.15
pytest-dev__pytest-11143 pytest-dev/pytest 8.0
pytest-dev__pytest-11148 pytest-dev/pytest 8.0
pytest-dev__pytest-5103 pytest-dev/pytest 4.5
pytest-dev__pytest-5221 pytest-dev/pytest 4.4
pytest-dev__pytest-5227 pytest-dev/pytest 4.4
pytest-dev__pytest-5413 pytest-dev/pytest 4.6
pytest-dev__pytest-5495 pytest-dev/pytest 4.6
pytest-dev__pytest-5692 pytest-dev/pytest 5.0
pytest-dev__pytest-6116 pytest-dev/pytest 5.2
pytest-dev__pytest-7168 pytest-dev/pytest 5.4
pytest-dev__pytest-7220 pytest-dev/pytest 5.4
pytest-dev__pytest-7373 pytest-dev/pytest 5.4
pytest-dev__pytest-7432 pytest-dev/pytest 5.4
pytest-dev__pytest-7490 pytest-dev/pytest 6.0
pytest-dev__pytest-8365 pytest-dev/pytest 6.3
pytest-dev__pytest-8906 pytest-dev/pytest 7.0
pytest-dev__pytest-9359 pytest-dev/pytest 7.0
pyvista__pyvista-4315 pyvista/pyvista 0.39
scikit-learn__scikit-learn-10297 scikit-learn/scikit-learn 0.20
scikit-learn__scikit-learn-10508 scikit-learn/scikit-learn 0.20
scikit-learn__scikit-learn-10949 scikit-learn/scikit-learn 0.20
scikit-learn__scikit-learn-11040 scikit-learn/scikit-learn 0.20
scikit-learn__scikit-learn-11281 scikit-learn/scikit-learn 0.20
scikit-learn__scikit-learn-12471 scikit-learn/scikit-learn 0.21
scikit-learn__scikit-learn-13142 scikit-learn/scikit-learn 0.21
scikit-learn__scikit-learn-13241 scikit-learn/scikit-learn 0.21
scikit-learn__scikit-learn-13439 scikit-learn/scikit-learn 0.21
scikit-learn__scikit-learn-13496 scikit-learn/scikit-learn 0.21
scikit-learn__scikit-learn-13497 scikit-learn/scikit-learn 0.21
scikit-learn__scikit-learn-13584 scikit-learn/scikit-learn 0.21
scikit-learn__scikit-learn-13779 scikit-learn/scikit-learn 0.22
scikit-learn__scikit-learn-14087 scikit-learn/scikit-learn 0.22
scikit-learn__scikit-learn-14092 scikit-learn/scikit-learn 0.22
scikit-learn__scikit-learn-14894 scikit-learn/scikit-learn 0.22
scikit-learn__scikit-learn-14983 scikit-learn/scikit-learn 0.22
scikit-learn__scikit-learn-15512 scikit-learn/scikit-learn 0.22
scikit-learn__scikit-learn-15535 scikit-learn/scikit-learn 0.22
scikit-learn__scikit-learn-25500 scikit-learn/scikit-learn 1.3
scikit-learn__scikit-learn-25570 scikit-learn/scikit-learn 1.3
scikit-learn__scikit-learn-25638 scikit-learn/scikit-learn 1.3
scikit-learn__scikit-learn-25747 scikit-learn/scikit-learn 1.3
sphinx-doc__sphinx-10325 sphinx-doc/sphinx 5.0
sphinx-doc__sphinx-10451 sphinx-doc/sphinx 5.1
sphinx-doc__sphinx-11445 sphinx-doc/sphinx 7.1
sphinx-doc__sphinx-7686 sphinx-doc/sphinx 3.1
sphinx-doc__sphinx-7738 sphinx-doc/sphinx 3.1
sphinx-doc__sphinx-7975 sphinx-doc/sphinx 3.2
sphinx-doc__sphinx-8273 sphinx-doc/sphinx 3.3
sphinx-doc__sphinx-8282 sphinx-doc/sphinx 3.3
sphinx-doc__sphinx-8435 sphinx-doc/sphinx 3.4
sphinx-doc__sphinx-8474 sphinx-doc/sphinx 3.4
sphinx-doc__sphinx-8506 sphinx-doc/sphinx 3.4
sphinx-doc__sphinx-8595 sphinx-doc/sphinx 3.5
sphinx-doc__sphinx-8627 sphinx-doc/sphinx 3.5
sphinx-doc__sphinx-8713 sphinx-doc/sphinx 4.0
sphinx-doc__sphinx-8721 sphinx-doc/sphinx 3.5
sphinx-doc__sphinx-8801 sphinx-doc/sphinx 3.5
sqlfluff__sqlfluff-1517 sqlfluff/sqlfluff 0.6
sqlfluff__sqlfluff-1625 sqlfluff/sqlfluff 0.6
sqlfluff__sqlfluff-1733 sqlfluff/sqlfluff 0.6
sqlfluff__sqlfluff-1763 sqlfluff/sqlfluff 0.6
sqlfluff__sqlfluff-2419 sqlfluff/sqlfluff 0.8
sympy__sympy-11400 sympy/sympy 1.0
sympy__sympy-11870 sympy/sympy 1.1
sympy__sympy-11897 sympy/sympy 1.0
sympy__sympy-12171 sympy/sympy 1.0
sympy__sympy-12236 sympy/sympy 1.0
sympy__sympy-12419 sympy/sympy 1.0
sympy__sympy-12454 sympy/sympy 1.0
sympy__sympy-12481 sympy/sympy 1.0
sympy__sympy-13031 sympy/sympy 1.1
sympy__sympy-13043 sympy/sympy 1.1
sympy__sympy-13146 sympy/sympy 1.1
sympy__sympy-13437 sympy/sympy 1.1
sympy__sympy-13471 sympy/sympy 1.1
sympy__sympy-13480 sympy/sympy 1.1
sympy__sympy-13647 sympy/sympy 1.1
sympy__sympy-13773 sympy/sympy 1.1
sympy__sympy-13895 sympy/sympy 1.1
sympy__sympy-13915 sympy/sympy 1.1
sympy__sympy-13971 sympy/sympy 1.1
sympy__sympy-14024 sympy/sympy 1.1
sympy__sympy-14308 sympy/sympy 1.1
sympy__sympy-14317 sympy/sympy 1.1
sympy__sympy-14396 sympy/sympy 1.1
sympy__sympy-14774 sympy/sympy 1.1
sympy__sympy-14817 sympy/sympy 1.1
sympy__sympy-15011 sympy/sympy 1.2
sympy__sympy-15308 sympy/sympy 1.4
sympy__sympy-15345 sympy/sympy 1.4
sympy__sympy-15346 sympy/sympy 1.4
sympy__sympy-15609 sympy/sympy 1.4
sympy__sympy-15678 sympy/sympy 1.4
sympy__sympy-16106 sympy/sympy 1.4
sympy__sympy-16281 sympy/sympy 1.4
sympy__sympy-16503 sympy/sympy 1.5
sympy__sympy-16792 sympy/sympy 1.5
sympy__sympy-16988 sympy/sympy 1.5
sympy__sympy-17022 sympy/sympy 1.5
sympy__sympy-17139 sympy/sympy 1.5
sympy__sympy-17630 sympy/sympy 1.5
sympy__sympy-17655 sympy/sympy 1.5
sympy__sympy-18057 sympy/sympy 1.6
sympy__sympy-18087 sympy/sympy 1.6
sympy__sympy-18189 sympy/sympy 1.6
sympy__sympy-18199 sympy/sympy 1.6
sympy__sympy-18532 sympy/sympy 1.6
sympy__sympy-18621 sympy/sympy 1.6
sympy__sympy-18698 sympy/sympy 1.6
sympy__sympy-18835 sympy/sympy 1.6
sympy__sympy-19007 sympy/sympy 1.6
sympy__sympy-19254 sympy/sympy 1.7
sympy__sympy-19487 sympy/sympy 1.7
sympy__sympy-20049 sympy/sympy 1.7
sympy__sympy-20154 sympy/sympy 1.7
sympy__sympy-20212 sympy/sympy 1.7
sympy__sympy-20322 sympy/sympy 1.8
sympy__sympy-20442 sympy/sympy 1.8
sympy__sympy-20590 sympy/sympy 1.7
sympy__sympy-20639 sympy/sympy 1.8
sympy__sympy-21055 sympy/sympy 1.8
sympy__sympy-21171 sympy/sympy 1.8
sympy__sympy-21379 sympy/sympy 1.9
sympy__sympy-21612 sympy/sympy 1.9
sympy__sympy-21614 sympy/sympy 1.9
sympy__sympy-21627 sympy/sympy 1.9
sympy__sympy-21847 sympy/sympy 1.9
sympy__sympy-22005 sympy/sympy 1.9
sympy__sympy-22714 sympy/sympy 1.10
sympy__sympy-22840 sympy/sympy 1.10
sympy__sympy-23117 sympy/sympy 1.11
sympy__sympy-23191 sympy/sympy 1.11
sympy__sympy-23262 sympy/sympy 1.11
sympy__sympy-24066 sympy/sympy 1.12
sympy__sympy-24102 sympy/sympy 1.12
sympy__sympy-24152 sympy/sympy 1.12
sympy__sympy-24213 sympy/sympy 1.12
sympy__sympy-24909 sympy/sympy 1.13