Momentum from .bp data

[cbontoiu]

Hi,
I have a problem logged for PIConGPU here
ComputationalRadiationPhysics/picongpu#4650

I have a model for which none of ux, uy,uz can be retrieved from the .bp data. The remaining variables, x, y,z and w can be obtained. I used openpmd viewer to read data from the .bp archives. There is nothing wrong with my installation because it can get ux, uy,uz from data saved a few months ago, so therefore with an older dev release of PIConGPU. Something happened in between.

In my speciesDefinition.param file I always have
using DefaultParticleAttributes = MakeSeq_t< position<position_pic>, momentum, weighting, particleId >;

This is the error I get:

uz = ts_particles.get_particle( var_list=['uz'], iteration=ITER[nnRef], species='e', plot=True)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[37], line 1
----> 1 uz = ts_particles.get_particle( var_list=['uz'], iteration=ITER[nnRef], species='e', plot=True)

File ~/anaconda3/lib/python3.10/site-packages/openpmd_viewer/openpmd_timeseries/main.py:274, in OpenPMDTimeSeries.get_particle(self, var_list, species, t, iteration, select, plot, nbins, plot_range, use_field_mesh, histogram_deposition, **kw)
    272 data_list = []
    273 for quantity in var_list:
--> 274     data_list.append( self.data_reader.read_species_data(
    275         iteration, species, quantity, self.extensions))
    276 # Apply selection if needed
    277 if isinstance( select, dict ):

File ~/anaconda3/lib/python3.10/site-packages/openpmd_viewer/openpmd_timeseries/data_reader/data_reader.py:293, in DataReader.read_species_data(self, iteration, species, record_comp, extensions)
    290     return h5py_reader.read_species_data(
    291             filename, iteration, species, record_comp, extensions )
    292 elif self.backend == 'openpmd-api':
--> 293     return io_reader.read_species_data(
    294             self.series, iteration, species, record_comp, extensions )

File ~/anaconda3/lib/python3.10/site-packages/openpmd_viewer/openpmd_timeseries/data_reader/io_reader/particle_reader.py:83, in read_species_data(series, iteration, species_name, component_name, extensions)
     81     if (macro_weighted == 1) and (weighting_power != 0):
     82         w_component = next(species['weighting'].items())[1]
---> 83         w = get_data( w_component )
     84         data *= w ** (-weighting_power)
     86 # - Return positions, with an offset

TypeError: get_data() missing 1 required positional argument: 'record_component'

Here is a snapshot from my data series as a .bp archive
https://drive.google.com/file/d/1Acf8bKL_RQJOvyhViyup6fdKsFCsPih8/view?usp=drive_link

Hopefully, this is an easy fix.
Many thanks, for your help.

[PrometheusPi]

This is an error on the PIConGPU side. PIConGPU did not set the ED-PIC extension correctly, making weighted quantities fail in the openPMD-viewer. (see issue here: ComputationalRadiationPhysics/picongpu#4635 and see fix here: ComputationalRadiationPhysics/picongpu#4636)

@cbontoiu I think, some of your simulation were conducted while the error was still in PIConGPU. If you run the current dev, you should be able to read data with the openPMD-viewer again.
To fix your existing (non working) ADIOS data, you just need to overwrite the openPMDextension attribute in the root path of the file with 1u.

[cbontoiu]

@PrometheusPi
thank you. Indeed my installation is about 2 weeks old.
I will upgrade to the latest dev.

[PrometheusPi]

Strange - the fix was added 3 weeks ago. Did your simulation already contain the fix? If yes, we might look at a different problem.

[cbontoiu]

I can now confirm that the version I have was installed on August 4th following this email

[PrometheusPi]

@pordyna just pointed out, that there might be another bug (in the viewer). We will investigate this

[cbontoiu]

@PrometheusPi Sorry for my late reply. I was in holiday. Thank you for investigating this issue and please let me know when there are things to try.

[PrometheusPi]

@cbontoiu I tested the openPMD-viewer (version '1.7.0', installed today as recommended via conda) with today's PIConGPU dev.
I can create the same error. If I understand it correctly, it works fine with ADIOS openpmd-api, weighting exists and the viewer tries to use it, but internally fails when trying to read the weighting attribute.

Surprising to me, reading the weight directly works and produces meaningful results:

w = ts.get_particle( var_list=['w'], iteration=[0], species='e', plot=False)

Thus, I guess there is an error within the viewer that does not handle weighting correction correctly. I will investigate.

[PrometheusPi]

cc'ing @franzpoeschel

[PrometheusPi]

@cbontoiu I found the bug. I will provide a fix for openPMD-viewer.

[PrometheusPi]

While the fix is not yet mainline, the change you need to do is minimal and can be applied by hand.
see: https://github.com/openPMD/openPMD-viewer/pull/393/files

[cbontoiu]

Hi @PrometheusPi , @RemiLehe , @ax3l

Thanks for the fix. I don't quite get it because I use the openPMDviewer with Conda. It is normally loaded in JupyterLab such that I can add other Python code. For this reason I don't have the file you indicated:

openpmd_viewer/openpmd_timeseries/data_reader/io_reader/particle_reader.py

Neither did https://github.com/openPMD/openPMD-viewer repository release a Conda or Pip version after the fix so I'm stuck with version 1.8, which shows the bug.

conda list
...
openpmd-api               0.15.2                   pypi_0    pypi
openpmd-viewer            1.8.0                    pypi_0    pypi

I'm using picongpu-0.7.0

Any help is appreciated.
Thanks

[cbontoiu]

OK, I managed to edit that file inside anaconda

sudo nano /usr/local/anaconda/lib/python3.11/site-packages/openpmd_viewer/openpmd_timeseries/data_reader/io_reader/particle_reader.py

and the hack works!
It is strange that the file is not listed

[RemiLehe]

Hey, sorry about the delay in releasing this latest version.
I just pushed openPMD-viewer 1.9.0 to PyPI. It should now work for you, after doing:

pip install --upgrade openpmd-viewer