Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure in Matlab / Python comparison tests #305

Closed
willrogers opened this issue Sep 24, 2021 · 6 comments
Closed

Test failure in Matlab / Python comparison tests #305

willrogers opened this issue Sep 24, 2021 · 6 comments
Assignees

Comments

@willrogers
Copy link
Contributor

A significant difference in A1 when using fastring, not that I know what A1 is specfically.

cc @MJGaughran

$ python -m pytest test_matlab
============================= test session starts ==============================
platform linux -- Python 3.7.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
system: Linux 4.18.0-305.12.1.el8_4.x86_64
numpy version: 1.21.2
rootdir: /home/hgs15624/code/at/pyat
plugins: lazy-fixture-0.6.3
collected 51 items                                                             

test_matlab/test_cmp_atpass.py ........                                  [ 15%]
test_matlab/test_cmp_avlinopt.py ......                                  [ 27%]
test_matlab/test_cmp_linopt6.py .......                                  [ 41%]
test_matlab/test_cmp_physics.py ........F...                             [ 64%]
test_matlab/test_int.py .                                                [ 66%]
test_matlab/test_orbit_matrices.py .................                     [100%]

=================================== FAILURES ===================================
_____________________________ test_fastring[hmba] ______________________________

engine = <matlab.engine.matlabengine.MatlabEngine object at 0x7f010bfa9358>
lattices = (Lattice([Marker('S28d', Periodicity=32), RFCavity('RFC', 0.0, 187500.0, 352372212.4670127, 31, 6000000000.0, PassMeth...([[0.0,0.0]]), ...}, {'Class': 'Drift', 'FamName': 'DR_02', 'Length': 0.042552, 'PassMethod': 'DriftPass'}, ...], None)

    @pytest.mark.parametrize('lattices',
                             [pytest.lazy_fixture('hmba')])
    def test_fastring(engine, lattices):
        py_lattice, ml_lattice, _ = lattices
        # Python call
        ring, ringrad = physics.fastring.fast_ring(py_lattice)
        # Matlab call
        ring_ml, ringrad_ml = engine.pyproxy('atfastring',
                                             ml_lattice, nargout=2)
        # Comparison
        for r, rml, idq in zip([ring, ringrad], [ring_ml, ringrad_ml], [3, 4]):
            assert_close(r[0].Frequency,    rml[1]['Frequency'], rtol=1.0e-20)
            assert_close(r[0].Voltage,      rml[1]['Voltage'], rtol=1.0e-20)
            assert_close(r[1].I2,           rml[2]['I2'], rtol=1.0e-20)
            assert_close(r[1].Length,       rml[2]['Length'], rtol=1.0e-20)
            assert_close(r[1].M66,          rml[2]['M66'], atol=1.0e-7)
            assert_close(r[1].T1,   numpy.squeeze(rml[2]['T1']),
                         rtol=1.0e-8, atol=1.0e-11)
            assert_close(r[1].T2,   numpy.squeeze(rml[2]['T2']),
                         rtol=1.0e-8, atol=1.0e-11)
>           assert_close(r[idq-1].A1,       rml[idq]['A1'], rtol=0.02)
E           AssertionError: 
E           Not equal to tolerance rtol=0.02, atol=0
E           
E           Mismatched elements: 1 / 1 (100%)
E           Max absolute difference: 92511.75874407
E           Max relative difference: 0.96853074
E            x: array(3005.868695)
E            y: array(95517.627439)

test_matlab/test_cmp_physics.py:121: AssertionError
----------------------------- Captured stdout call -----------------------------
    window + interp. tune:0.38167 (rms:7.27656e-05)
    window + interp. tune:0.145741 (rms:7.81583e-05)
    window + interp. tune:0.381486 (rms:5.15987e-05)
    window + interp. tune:0.145586 (rms:2.57663e-05)
=========================== short test summary info ============================
FAILED test_matlab/test_cmp_physics.py::test_fastring[hmba] - AssertionError: 
======================== 1 failed, 50 passed in 33.11s =========================
@swhite2401
Copy link
Contributor

A1 is a coefficient for amplitude detuning. It is derived from the harmonic analysis of the turn-by-turn motion of the beam at several amplitudes and a polynomial fit of this data.
The algorithms used in this analysis are different in python and matlab and we can expected some differences.

@willrogers
Copy link
Contributor Author

The numbers are completely different: 3005.9 and 95517.6. If it's fine that the numbers are completely different just confirm that and I will remove that check.

@lfarv
Copy link
Contributor

lfarv commented Sep 27, 2021

I noticed this happening after merging #267. I suspect a problem with the new python version, but did not look at it yet…

@swhite2401
Copy link
Contributor

Ok , yes this is big and is most likely a problem, let's leave it open. I ll try to check when I have some time. Most likely not before next week

@swhite2401
Copy link
Contributor

Ok so the issue is the periodicity, in fact there is a factor 32 between the matlab and the python.
From what I can see:
-in python the periodicity is not applied for the fastring -> Q', detuning, etc...
-in matlab it applies nowhere but for the detuning

This is an issue in matlab rather than python I would say, which brings back #169 on the table (same issue different time...), for me this parameter is bringing a lot of issues and very little added value...the only place where I can see it fit is in ringparam/atsummary where it could simply be given as input argument rather than being a lattice property.

@lfarv lfarv mentioned this issue Oct 4, 2021
@lfarv
Copy link
Contributor

lfarv commented Oct 4, 2021

The bug is indeed in Matlab, corrected in #314

@lfarv lfarv closed this as completed Nov 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants