BTD-RZ Add multiple modes

[RevathiJambunathan]

• add a test

To be merged after #3350

input file used for testing on summit :
inputs.txt

[ax3l]

@RevathiJambunathan ready for rebase :)

[EZoni]

Thank you for this PR, @RevathiJambunathan! Sorry, I decided to jump in as a reviewer to maybe help merge this faster. So far I left only a few minor comments on using WarpX::ncomps directly where possible, following the spirit of #2951. Unless we think it's unsafe for some reason.

[lgtm-com]

This pull request introduces 1 alert when merging 0aded06 into 8f9da8c - view on LGTM.com

new alerts:

• 1 for Unused import

[RevathiJambunathan]

The CI test is failing because amrex.fpe_trap_invalid=1 is catching an error which I am able to reproduce locally as well upon adding the same flags.

After running this with DEBUG, Here is the Backtrace.0.0.txt

I am wondering if something about RZ is not quite okay with amrex::get_slice_data?
or it could be that the values in the cell-centered data are junk?
I will print min max values for the multifab and see what I get

In the meanwhile, I also tried running this test with one RZ mode, and I get the following warning

!!! WARNING : [high][Laser] The antenna is completely out of the simulation box
             for laser laser1
!
0::Assertion `msg_priority < abort_priority' failed, file "/home/revanathan/Codes/Warpx_Upstream/Source/ablastr/warn_manager/WarnManager.cpp", line 89, Msg: 
### ERROR   : A warning with priority 'high' has been raised.
 !!!
SIGABRT

[RevathiJambunathan]

Update : I printed the min/max values of the cell-centered multifab being sliced and it all looks okay. nothing suspicious there, definitely no nans or really small/large numbers

[RemiLehe]

Thanks for this PR!
I added a few suggestions. I will simultaneously also have a look at the failing test to try to understand better what is happening there.

[RemiLehe]

Could we replace the whole code above by a loop over components?

Suggested change

	for (int imode = 0; imode < n_rz; ++imode) {
	int offset = 1;
	// number of components for a given mode
	int ncomp = 2;
	if (imode == 0) {
	offset = 0;
	// Mode 0 has only real part, thus ncomp = 1
	ncomp = 1;
	}
	for (int icomp = 0; icomp < ncomp; ++icomp) {
	for (int mode_comp_offset = 0; mode_comp_offset < nr_comp; ++mode_comp_offset) {

[RemiLehe]

If simplify the above loop as suggested, then we can remove these lines.

Suggested change

	// Offset for a given mode, imode, and component, icomp (0-real, 1-imag)
	const int mode_comp_offset = 2*imode + offset*(icomp-1);
	// Offset for a given mode, imode, and component, icomp (0-real, 1-imag)
	const int mode_comp_offset = 2*imode + offset*(icomp-1);

[RevathiJambunathan]

I wanted to keep it this way since it accesses the other fields easily
Or rather it has the same "structure" as for cartesian

                e_lab = gamma_boost * ( arr(i, j, k, 0)
                                        + beta_boost * clight * arr(i, j, k, 4) );
                b_lab = gamma_boost * ( arr(i, j, k, 4)
                                        + beta_boost * inv_clight * arr(i, j, k, 0) );

and for RZ we have

                        e_lab = gamma_boost * ( arr(i, j, k, n_rcomps*0 + mode_comp_offset)
                                                + beta_boost * clight * arr(i, j, k, n_rcomps*4+ mode_comp_offset) );
                        b_lab = gamma_boost * ( arr(i, j, k, n_rcomps*4 + mode_comp_offset)
                                                + beta_boost * inv_clight * arr(i, j, k, n_rcomps*0 + mode_comp_offset) );

[RevathiJambunathan]

But I am open to changing this as well.
Need to think about this one -- I think I can find a way to add in the suggested change in the loop while keeping a similar logic, as long as it does not get too complicated

[RevathiJambunathan]

okay this is done :)
Thank you so much @RemiLehe
Your suggestion is more clean and readable! Thanks!