Support parsing QE 6.8 outputs

[qiaojunfeng]

Hi,

This PR adds the XML schema and a test for QE 6.8 output.
Note due to a bug in QE 6.8 output XML, this PR works only after applying this (freshly merged 😅) patch https://gitlab.com/QEF/q-e/-/merge_requests/1531 to QE.

Fix #713

[sphuber]

Thanks @qiaojunfeng . So does that mean that adding this won't help for people running v6.8 and parsing will fail? If that is the case, I think we should add the schema that works with v6.8 and then add this schema once the fix is released in a patch version of QE. Otherwise it seems to make little sense to me.

EDIT: just gave this a try myself to see the behavior. When running develop with v6.8, the parsing simply fails with a 322. When using this branch, the parsing succeeds, but emits the following error log messages:

08/23/2021 08:03:49 PM <24044> aiida.parser.PwParser: [ERROR] 1 XML schema validation error(s) schema: /home/sph/code/aiida/env/dev/aiida-quantumespresso/aiida_quantumespresso/parsers/parse_xml/schemas/qes_210716.xsd:
08/23/2021 08:03:49 PM <24044> aiida.parser.PwParser: [ERROR] failed validating 0 with <function positive_int_validator at 0x7f057756eaf0>:

Reason: attribute calls='0': value must be positive

Instance:

  <partial label="PWSCF" calls="0">
    <cpu>7.964530000000000e-1</cpu>
    <wall>8.695421218872070e-1</wall>
  </partial>

Path: /{http://www.quantum-espresso.org/ns/qes/qes-1.0}espresso/timing_info/partial[1]

I have posted in the linked Gitlab PR to ask whether the fix will be released in a patch version of v6.8. If that is the case, then we can merge this since once people update to the latest patch of v6.8 all will work fine, and for lower patch versions there is just the error message but which can be ignored safely.

[giovannipizzi]

Thanks @qiaojunfeng, and @sphuber for the review! I think since QE will not release a patch version, this will always be in 6.8. There are 2 options:

• have some code to skip that specific validation if the version is 6.8
• just keep as is and merge
  I would vote for the second, because the value is actually wrong, so in a sense it's good to keep the warning.
  So, I would just merge as is.

@sphuber do you agree?

[sphuber]

I think we should get this in somehow just to make 6.8 runnable, but I am not sure of keeping the error. I can already see the questions being asked over and over again by users, and the errors will also get stored in the database taking up space, even though we know it is a dud. I would try to see if we can ignore this particular error because ultimately it is not really important and doesn't matter whatsoever for the user.

[qiaojunfeng]

OK, so I add some code to remove that problematic XML element for QE 6.8 output, so the verdi process report should be clean now.

[mbercx]

Thanks @qiaojunfeng! I did a little test run for a magnetic system, and unfortunately it seems that the output format for the magnetic moments has changed, resulting in the following error:

+-> CRITICAL at 2021-09-10 11:54:54.744964+00:00
 | Traceback (most recent call last):
 |   File "/home/aiida/envs/aiida-lumi/code/aiida-quantumespresso/aiida_quantumespresso/parsers/pw.py", line 324, in parse_stdout
 |     parsed_data, logs = parse_stdout(stdout, parameters, parser_options, parsed_xml)
 |   File "/home/aiida/envs/aiida-lumi/code/aiida-quantumespresso/aiida_quantumespresso/parsers/parse_raw/pw.py", line 640, in parse_stdout
 |     line2 = data_step[count + j]
 | IndexError: list index out of range
+-> ERROR at 2021-09-10 11:54:54.747584+00:00
 | The parser raised an unexpected exception.
+-> WARNING at 2021-09-10 11:54:54.749605+00:00
 | output parser returned exit code<350>: The parser raised an unexpected exception.

This is the format for the magnetic moments in v6.8:

     Magnetic moment per site  (integrated on atomic sphere of radius R)
     atom   1 (R=0.292)  charge=  7.9920  magn= -0.0000
     atom   2 (R=0.292)  charge=  1.2243  magn=  0.0000

Compare that to the following example from v6.7:

     Magnetic moment per site:
     atom:    1    charge:    0.4485    magn:    0.0000    constr:    0.0000
     atom:    2    charge:    0.4485    magn:    0.0000    constr:    0.0000
     atom:    3    charge:    4.1095    magn:   -0.0000    constr:    0.0000
     atom:    4    charge:    4.1095    magn:   -0.0000    constr:    0.0000

This messes up the parsing:

aiida-quantumespresso/aiida_quantumespresso/parsers/parse_raw/pw.py

Lines 641 to 643 in c49def7

	if 'atom:' in line2:
	mag_moments.append(float(line2.split('magn:')[1].split()[0]))
	charges.append(float(line2.split('charge:')[1].split()[0]))

Going to check if the magnetic moments are in the XML now. If not, I have a fix somewhere for this when we were doing some test runs on the QE develop branch, will dig that up.

[mbercx]

No luck, I'm afraid. I can't seem to find the magnetic moments of each site in the XML. Here's the lines I added to fix this after:

aiida-quantumespresso/aiida_quantumespresso/parsers/parse_raw/pw.py

Lines 641 to 643 in c49def7

	if 'atom:' in line2:
	mag_moments.append(float(line2.split('magn:')[1].split()[0]))
	charges.append(float(line2.split('charge:')[1].split()[0]))

                        elif 'atom' in line2:
                            mag_moments.append(float(line2.split('magn=')[1].split()[0]))
                            charges.append(float(line2.split('charge=')[1].split()[0]))

Will try to do this in my branch on thanos and see if the magnetic moments are parsed properly.

[giovannipizzi]

Thanks @mbercx!
It would be great to inform Pietro Delugas

1. about this change
2. about the fact that this information is not in the XML
   so maybe they manage to put this in for the next version(s).

Can you ping him please?

[sphuber]

@mbercx I guess what you are describing is more or less a separate problem is it not? Shall we already merge this, and then I let you open a PR to fix the parsing of magnetic moments from the stdout?

[mbercx]

@sphuber sure, that'd work as well. 👍 Just figured it would fit in the topic of this PR, but I wouldn't want to introduce scope creep for @qiaojunfeng!

@giovannipizzi Sure! Will open a new issue an ping him there.

[sphuber]

Thanks a lot @qiaojunfeng