20240402_sweagent_gpt4

Benchmark results

Patch Apply Success

Resolved	Count	Rate
Yes	54	19.08%
Partially	10	3.53%
No	219	77.39%

Patch Apply Success + Failure

Resolved	Count	Rate
Yes	54	19.01%
Partially	10	3.52%
No	220	77.46%

Benchmark instances

Generated but not applied

Instance ID	Repository	Testbed version
django__django-12497	django/django	3.1

Applied but not resolved

Instance ID	Repository	Testbed version
astropy__astropy-12907	astropy/astropy	4.3
astropy__astropy-14182	astropy/astropy	5.1
astropy__astropy-14365	astropy/astropy	5.1
astropy__astropy-6938	astropy/astropy	1.3
astropy__astropy-7746	astropy/astropy	1.3
django__django-10924	django/django	3.0
django__django-11001	django/django	3.0
django__django-11019	django/django	3.0
django__django-11049	django/django	3.0
django__django-11179	django/django	3.0
django__django-11283	django/django	3.0
django__django-11422	django/django	3.0
django__django-11564	django/django	3.1
django__django-11620	django/django	3.0
django__django-11630	django/django	3.0
django__django-11742	django/django	3.0
django__django-11797	django/django	3.1
django__django-11815	django/django	3.1
django__django-11848	django/django	3.1
django__django-11905	django/django	3.1
django__django-11910	django/django	3.1
django__django-11964	django/django	3.1
django__django-11999	django/django	3.1
django__django-12113	django/django	3.1
django__django-12125	django/django	3.1
django__django-12184	django/django	3.1
django__django-12284	django/django	3.1
django__django-12308	django/django	3.1
django__django-12470	django/django	3.1
django__django-12589	django/django	3.1
django__django-12700	django/django	3.1
django__django-12708	django/django	3.1
django__django-12747	django/django	3.1
django__django-12856	django/django	3.2
django__django-12908	django/django	3.2
django__django-12915	django/django	3.2
django__django-13028	django/django	3.2
django__django-13033	django/django	3.2
django__django-13158	django/django	3.2
django__django-13220	django/django	3.2
django__django-13315	django/django	3.2
django__django-13321	django/django	3.2
django__django-13448	django/django	3.2
django__django-13551	django/django	3.2
django__django-13590	django/django	3.2
django__django-13660	django/django	3.2
django__django-13710	django/django	4.0
django__django-13757	django/django	3.2
django__django-13768	django/django	3.2
django__django-13925	django/django	4.0
django__django-13933	django/django	4.0
django__django-14016	django/django	4.0
django__django-14017	django/django	4.0
django__django-14155	django/django	4.0
django__django-14411	django/django	4.0
django__django-14534	django/django	4.0
django__django-14580	django/django	4.0
django__django-14608	django/django	4.0
django__django-14667	django/django	4.0
django__django-14730	django/django	4.0
django__django-14997	django/django	4.1
django__django-15061	django/django	4.1
django__django-15202	django/django	4.1
django__django-15252	django/django	4.1
django__django-15320	django/django	4.1
django__django-15388	django/django	4.1
django__django-15400	django/django	4.1
django__django-15498	django/django	4.1
django__django-15695	django/django	4.1
django__django-15781	django/django	4.2
django__django-15789	django/django	4.2
django__django-15790	django/django	4.2
django__django-15819	django/django	4.2
django__django-15902	django/django	4.2
django__django-15996	django/django	4.2
django__django-16041	django/django	4.2
django__django-16229	django/django	4.2
django__django-16400	django/django	4.2
django__django-16408	django/django	5.0
django__django-16816	django/django	5.0
django__django-16820	django/django	5.0
django__django-16910	django/django	5.0
django__django-17051	django/django	5.0
django__django-17087	django/django	5.0
matplotlib__matplotlib-18869	matplotlib/matplotlib	3.3
matplotlib__matplotlib-22711	matplotlib/matplotlib	3.5
matplotlib__matplotlib-22835	matplotlib/matplotlib	3.5
matplotlib__matplotlib-23299	matplotlib/matplotlib	3.5
matplotlib__matplotlib-23314	matplotlib/matplotlib	3.5
matplotlib__matplotlib-23476	matplotlib/matplotlib	3.5
matplotlib__matplotlib-23562	matplotlib/matplotlib	3.5
matplotlib__matplotlib-23563	matplotlib/matplotlib	3.5
matplotlib__matplotlib-23913	matplotlib/matplotlib	3.6
matplotlib__matplotlib-23987	matplotlib/matplotlib	3.6
matplotlib__matplotlib-24149	matplotlib/matplotlib	3.6
matplotlib__matplotlib-24334	matplotlib/matplotlib	3.6
matplotlib__matplotlib-24970	matplotlib/matplotlib	3.6
matplotlib__matplotlib-25079	matplotlib/matplotlib	3.6
matplotlib__matplotlib-25311	matplotlib/matplotlib	3.7
matplotlib__matplotlib-25332	matplotlib/matplotlib	3.7
matplotlib__matplotlib-25433	matplotlib/matplotlib	3.7
matplotlib__matplotlib-25442	matplotlib/matplotlib	3.7
matplotlib__matplotlib-25498	matplotlib/matplotlib	3.7
mwaskom__seaborn-2848	mwaskom/seaborn	0.12
mwaskom__seaborn-3190	mwaskom/seaborn	0.12
mwaskom__seaborn-3407	mwaskom/seaborn	0.13
pallets__flask-4045	pallets/flask	2.0
pallets__flask-4992	pallets/flask	2.3
pallets__flask-5063	pallets/flask	2.3
psf__requests-1963	psf/requests	2.3
psf__requests-2148	psf/requests	2.3
pydata__xarray-3364	pydata/xarray	0.12
pydata__xarray-4094	pydata/xarray	0.12
pydata__xarray-4248	pydata/xarray	0.12
pydata__xarray-4493	pydata/xarray	0.12
pydata__xarray-5131	pydata/xarray	0.12
pylint-dev__pylint-6506	pylint-dev/pylint	2.14
pylint-dev__pylint-7080	pylint-dev/pylint	2.15
pylint-dev__pylint-7114	pylint-dev/pylint	2.15
pylint-dev__pylint-7228	pylint-dev/pylint	2.15
pylint-dev__pylint-7993	pylint-dev/pylint	2.15
pytest-dev__pytest-11148	pytest-dev/pytest	8.0
pytest-dev__pytest-5221	pytest-dev/pytest	4.4
pytest-dev__pytest-5413	pytest-dev/pytest	4.6
pytest-dev__pytest-5495	pytest-dev/pytest	4.6
pytest-dev__pytest-6116	pytest-dev/pytest	5.2
pytest-dev__pytest-7220	pytest-dev/pytest	5.4
pytest-dev__pytest-7373	pytest-dev/pytest	5.4
pytest-dev__pytest-7432	pytest-dev/pytest	5.4
pytest-dev__pytest-7490	pytest-dev/pytest	6.0
pytest-dev__pytest-8365	pytest-dev/pytest	6.3
pytest-dev__pytest-8906	pytest-dev/pytest	7.0
pytest-dev__pytest-9359	pytest-dev/pytest	7.0
scikit-learn__scikit-learn-10508	scikit-learn/scikit-learn	0.20
scikit-learn__scikit-learn-10949	scikit-learn/scikit-learn	0.20
scikit-learn__scikit-learn-11040	scikit-learn/scikit-learn	0.20
scikit-learn__scikit-learn-11281	scikit-learn/scikit-learn	0.20
scikit-learn__scikit-learn-12471	scikit-learn/scikit-learn	0.21
scikit-learn__scikit-learn-13142	scikit-learn/scikit-learn	0.21
scikit-learn__scikit-learn-13439	scikit-learn/scikit-learn	0.21
scikit-learn__scikit-learn-13496	scikit-learn/scikit-learn	0.21
scikit-learn__scikit-learn-13497	scikit-learn/scikit-learn	0.21
scikit-learn__scikit-learn-13779	scikit-learn/scikit-learn	0.22
scikit-learn__scikit-learn-14087	scikit-learn/scikit-learn	0.22
scikit-learn__scikit-learn-14092	scikit-learn/scikit-learn	0.22
scikit-learn__scikit-learn-14983	scikit-learn/scikit-learn	0.22
scikit-learn__scikit-learn-15512	scikit-learn/scikit-learn	0.22
scikit-learn__scikit-learn-25500	scikit-learn/scikit-learn	1.3
scikit-learn__scikit-learn-25570	scikit-learn/scikit-learn	1.3
scikit-learn__scikit-learn-25638	scikit-learn/scikit-learn	1.3
sphinx-doc__sphinx-10325	sphinx-doc/sphinx	5.0
sphinx-doc__sphinx-10451	sphinx-doc/sphinx	5.1
sphinx-doc__sphinx-11445	sphinx-doc/sphinx	7.1
sphinx-doc__sphinx-7686	sphinx-doc/sphinx	3.1
sphinx-doc__sphinx-7738	sphinx-doc/sphinx	3.1
sphinx-doc__sphinx-7975	sphinx-doc/sphinx	3.2
sphinx-doc__sphinx-8273	sphinx-doc/sphinx	3.3
sphinx-doc__sphinx-8282	sphinx-doc/sphinx	3.3
sphinx-doc__sphinx-8435	sphinx-doc/sphinx	3.4
sphinx-doc__sphinx-8474	sphinx-doc/sphinx	3.4
sphinx-doc__sphinx-8506	sphinx-doc/sphinx	3.4
sphinx-doc__sphinx-8595	sphinx-doc/sphinx	3.5
sphinx-doc__sphinx-8627	sphinx-doc/sphinx	3.5
sphinx-doc__sphinx-8801	sphinx-doc/sphinx	3.5
sympy__sympy-11400	sympy/sympy	1.0
sympy__sympy-11870	sympy/sympy	1.1
sympy__sympy-11897	sympy/sympy	1.0
sympy__sympy-12171	sympy/sympy	1.0
sympy__sympy-12236	sympy/sympy	1.0
sympy__sympy-12419	sympy/sympy	1.0
sympy__sympy-12454	sympy/sympy	1.0
sympy__sympy-12481	sympy/sympy	1.0
sympy__sympy-13031	sympy/sympy	1.1
sympy__sympy-13043	sympy/sympy	1.1
sympy__sympy-13177	sympy/sympy	1.1
sympy__sympy-13437	sympy/sympy	1.1
sympy__sympy-13471	sympy/sympy	1.1
sympy__sympy-13773	sympy/sympy	1.1
sympy__sympy-13895	sympy/sympy	1.1
sympy__sympy-13915	sympy/sympy	1.1
sympy__sympy-14024	sympy/sympy	1.1
sympy__sympy-14308	sympy/sympy	1.1
sympy__sympy-14317	sympy/sympy	1.1
sympy__sympy-14396	sympy/sympy	1.1
sympy__sympy-15011	sympy/sympy	1.2
sympy__sympy-15308	sympy/sympy	1.4
sympy__sympy-15345	sympy/sympy	1.4
sympy__sympy-15346	sympy/sympy	1.4
sympy__sympy-15609	sympy/sympy	1.4
sympy__sympy-15678	sympy/sympy	1.4
sympy__sympy-16106	sympy/sympy	1.4
sympy__sympy-16281	sympy/sympy	1.4
sympy__sympy-16503	sympy/sympy	1.5
sympy__sympy-16792	sympy/sympy	1.5
sympy__sympy-17022	sympy/sympy	1.5
sympy__sympy-17139	sympy/sympy	1.5
sympy__sympy-17630	sympy/sympy	1.5
sympy__sympy-17655	sympy/sympy	1.5
sympy__sympy-18057	sympy/sympy	1.6
sympy__sympy-18087	sympy/sympy	1.6
sympy__sympy-18189	sympy/sympy	1.6
sympy__sympy-18199	sympy/sympy	1.6
sympy__sympy-18532	sympy/sympy	1.6
sympy__sympy-18621	sympy/sympy	1.6
sympy__sympy-18698	sympy/sympy	1.6
sympy__sympy-18835	sympy/sympy	1.6
sympy__sympy-19007	sympy/sympy	1.6
sympy__sympy-19254	sympy/sympy	1.7
sympy__sympy-19487	sympy/sympy	1.7
sympy__sympy-20049	sympy/sympy	1.7
sympy__sympy-20154	sympy/sympy	1.7
sympy__sympy-20212	sympy/sympy	1.7
sympy__sympy-20322	sympy/sympy	1.8
sympy__sympy-20590	sympy/sympy	1.7
sympy__sympy-20639	sympy/sympy	1.8
sympy__sympy-21055	sympy/sympy	1.8
sympy__sympy-21171	sympy/sympy	1.8
sympy__sympy-21379	sympy/sympy	1.9
sympy__sympy-21612	sympy/sympy	1.9
sympy__sympy-21614	sympy/sympy	1.9
sympy__sympy-21627	sympy/sympy	1.9
sympy__sympy-21847	sympy/sympy	1.9
sympy__sympy-22005	sympy/sympy	1.9
sympy__sympy-22714	sympy/sympy	1.10
sympy__sympy-22840	sympy/sympy	1.10
sympy__sympy-23117	sympy/sympy	1.11
sympy__sympy-24066	sympy/sympy	1.12
sympy__sympy-24102	sympy/sympy	1.12
sympy__sympy-24909	sympy/sympy	1.13

Resolved

Instance ID	Repository	Testbed version
astropy__astropy-14995	astropy/astropy	5.2
django__django-10914	django/django	3.0
django__django-11039	django/django	3.0
django__django-11099	django/django	3.0
django__django-11133	django/django	3.0
django__django-11583	django/django	3.0
django__django-12286	django/django	3.1
django__django-12453	django/django	3.1
django__django-12983	django/django	3.2
django__django-13230	django/django	3.2
django__django-13401	django/django	3.2
django__django-13447	django/django	4.0
django__django-13658	django/django	3.2
django__django-14238	django/django	4.0
django__django-14382	django/django	4.0
django__django-14672	django/django	4.0
django__django-14752	django/django	4.0
django__django-14787	django/django	4.1
django__django-14855	django/django	4.0
django__django-14915	django/django	4.1
django__django-14999	django/django	4.1
django__django-15213	django/django	4.1
django__django-15347	django/django	4.1
django__django-15814	django/django	4.2
django__django-16046	django/django	4.2
django__django-16139	django/django	4.2
django__django-16255	django/django	4.2
django__django-16379	django/django	4.2
django__django-16527	django/django	5.0
django__django-16595	django/django	5.0
django__django-16873	django/django	5.0
matplotlib__matplotlib-23964	matplotlib/matplotlib	3.6
matplotlib__matplotlib-26011	matplotlib/matplotlib	3.7
matplotlib__matplotlib-26020	matplotlib/matplotlib	3.7
mwaskom__seaborn-3010	mwaskom/seaborn	0.12
psf__requests-2317	psf/requests	2.4
psf__requests-2674	psf/requests	2.7
pylint-dev__pylint-5859	pylint-dev/pylint	2.13
pytest-dev__pytest-11143	pytest-dev/pytest	8.0
pytest-dev__pytest-5227	pytest-dev/pytest	4.4
pytest-dev__pytest-5692	pytest-dev/pytest	5.0
scikit-learn__scikit-learn-10297	scikit-learn/scikit-learn	0.20
scikit-learn__scikit-learn-13584	scikit-learn/scikit-learn	0.21
scikit-learn__scikit-learn-14894	scikit-learn/scikit-learn	0.22
scikit-learn__scikit-learn-15535	scikit-learn/scikit-learn	0.22
sphinx-doc__sphinx-8713	sphinx-doc/sphinx	4.0
sympy__sympy-13480	sympy/sympy	1.1
sympy__sympy-13647	sympy/sympy	1.1
sympy__sympy-13971	sympy/sympy	1.1
sympy__sympy-14774	sympy/sympy	1.1
sympy__sympy-16988	sympy/sympy	1.5
sympy__sympy-23262	sympy/sympy	1.11
sympy__sympy-24152	sympy/sympy	1.12
sympy__sympy-24213	sympy/sympy	1.12