parse_known_args broken in python 3.12.8+ and 3.13.1+

[cshimmin]

🐛 Bug report

I'm using jsonargparse[signatures] via pytorch lightning, and found today that their CLI tool errors out with the following message:

error: ArgumentParser._parse_known_args() missing 1 required positional argument: 'intermixed'

I traced the issue to a change in the private interface method _parse_known_args from cpython's argparse package. This method is called by jsonargparse here:

jsonargparse/jsonargparse/_core.py

Line 264 in 42c1cae

namespace, args = self._parse_known_args(args, namespace)

however, the upstream implementation of this function now expects a third mandatory positional argument, intermixed:

https://github.com/python/cpython/blob/2dc476bcb9142cd25d7e1d52392b73a3dcdf1756/Lib/argparse.py#L1954

Although this is a breaking change after 15 years of stability, since that is a private method, I suppose the bug report belongs here rather than in cpython.

The relevant upstream issue can be found here: python/cpython/issues/125355
And the particular commits can be found here: python/cpython@da3d81d (python 3.12) and here: python/cpython@1fe63b1 (python 3.13)

To reproduce

Sorry I don't have a minimal standalone example, but it can be easily reproduced via pytorch lightning. Here's a simple main.py:

#!/usr/bin/env python

from lightning.pytorch.cli import LightningCLI


def main():
    cli = LightningCLI(save_config_callback=None)


if __name__ == "__main__":
    main()

pip install lightning==2.4.0
pip install "jsonargparse[signatures]==4.34.1"
python main.py

should exit with the error I described above

Expected behavior

It should not crash

Environment

• jsonargparse version: 4.34.1
• Python version: 3.12.8
• How jsonargparse was installed: pip
• OS: linux

[purnelldj]

I am having the same issue, was able to fix by installing with python 3.12.7

[lantiga]

Thank you @cshimmin, I see that this issue has showed up on CI (at PyTorch Lightning) and it will start impacting users quickly.

@mauvilsa is it possible to prioritize a fix?

[mauvilsa]

@mauvilsa is it possible to prioritize a fix?

Yes, I will fix it as soon as I can. Though I am currently traveling and unable to work on it. Probably I could create a release around Friday.

[lantiga]

Sounds good, thank you!

[lantiga]

FYI in the meantime I merged a change that patches the problem directly in PyTorch Lightning