test_*_code functions in _testcapi/getargs.c have memory leaks

[sobolevn]

Bug report

I don't think it is very important, since this is just a test, but why have it when it is spotted?

1. test_k_code:

   cpython/Modules/_testcapi/getargs.c

   Lines 331 to 398 in 326c6c4

   	/* This function not only tests the 'k' getargs code, but also the
   	PyLong_AsUnsignedLongMask() function. */
   	static PyObject *
   	test_k_code(PyObject *self, PyObject *Py_UNUSED(ignored))
   	{
   	PyObject *tuple, *num;
   	unsigned long value;
   	tuple = PyTuple_New(1);
   	if (tuple == NULL) {
   	return NULL;
   	}
   	/* a number larger than ULONG_MAX even on 64-bit platforms */
   	num = PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16);
   	if (num == NULL) {
   	return NULL;
   	}
   	value = PyLong_AsUnsignedLongMask(num);
   	if (value != ULONG_MAX) {
   	PyErr_SetString(PyExc_AssertionError,
   	"test_k_code: "
   	"PyLong_AsUnsignedLongMask() returned wrong value for long 0xFFF...FFF");
   	return NULL;
   	}
   	PyTuple_SET_ITEM(tuple, 0, num);
   	value = 0;
   	if (!PyArg_ParseTuple(tuple, "k:test_k_code", &value)) {
   	return NULL;
   	}
   	if (value != ULONG_MAX) {
   	PyErr_SetString(PyExc_AssertionError,
   	"test_k_code: k code returned wrong value for long 0xFFF...FFF");
   	return NULL;
   	}
   	Py_DECREF(num);
   	num = PyLong_FromString("-FFFFFFFF000000000000000042", NULL, 16);
   	if (num == NULL) {
   	return NULL;
   	}
   	value = PyLong_AsUnsignedLongMask(num);
   	if (value != (unsigned long)-0x42) {
   	PyErr_SetString(PyExc_AssertionError,
   	"test_k_code: "
   	"PyLong_AsUnsignedLongMask() returned wrong value for long -0xFFF..000042");
   	return NULL;
   	}
   	PyTuple_SET_ITEM(tuple, 0, num);
   	value = 0;
   	if (!PyArg_ParseTuple(tuple, "k:test_k_code", &value)) {
   	return NULL;
   	}
   	if (value != (unsigned long)-0x42) {
   	PyErr_SetString(PyExc_AssertionError,
   	"test_k_code: k code returned wrong value for long -0xFFF..000042");
   	return NULL;
   	}
   	Py_DECREF(tuple);
   	Py_RETURN_NONE;
   	}

On errors tuple is not decrefed.
Also, note these lines:

cpython/Modules/_testcapi/getargs.c

Lines 358 to 374 in 326c6c4

	PyTuple_SET_ITEM(tuple, 0, num);
	value = 0;
	if (!PyArg_ParseTuple(tuple, "k:test_k_code", &value)) {
	return NULL;
	}
	if (value != ULONG_MAX) {
	PyErr_SetString(PyExc_AssertionError,
	"test_k_code: k code returned wrong value for long 0xFFF...FFF");
	return NULL;
	}
	Py_DECREF(num);
	num = PyLong_FromString("-FFFFFFFF000000000000000042", NULL, 16);
	if (num == NULL) {
	return NULL;
	}

Here' we leave a tuple is a semi-broken state. Its 0'th item has a reference count of 0.
We should also recreate a tuple here with the new items. test_L_code also has this problem.

2. cpython/Modules/_testcapi/getargs.c

   Lines 684 to 732 in 326c6c4

   	static PyObject *
   	test_L_code(PyObject *self, PyObject *Py_UNUSED(ignored))
   	{
   	PyObject *tuple, *num;
   	long long value;
   	tuple = PyTuple_New(1);
   	if (tuple == NULL) {
   	return NULL;
   	}
   	num = PyLong_FromLong(42);
   	if (num == NULL) {
   	return NULL;
   	}
   	PyTuple_SET_ITEM(tuple, 0, num);
   	value = -1;
   	if (!PyArg_ParseTuple(tuple, "L:test_L_code", &value)) {
   	return NULL;
   	}
   	if (value != 42) {
   	PyErr_SetString(PyExc_AssertionError,
   	"test_L_code: L code returned wrong value for long 42");
   	return NULL;
   	}
   	Py_DECREF(num);
   	num = PyLong_FromLong(42);
   	if (num == NULL) {
   	return NULL;
   	}
   	PyTuple_SET_ITEM(tuple, 0, num);
   	value = -1;
   	if (!PyArg_ParseTuple(tuple, "L:test_L_code", &value)) {
   	return NULL;
   	}
   	if (value != 42) {
   	PyErr_SetString(PyExc_AssertionError,
   	"test_L_code: L code returned wrong value for int 42");
   	return NULL;
   	}
   	Py_DECREF(tuple);
   	Py_RETURN_NONE;
   	}

On errors tuple is not decrefed. And num is re-assigned without tuple cleanup.

3. cpython/Modules/_testcapi/getargs.c

   Lines 734 to 767 in 326c6c4

   	/* Test the s and z codes for PyArg_ParseTuple.
   	*/
   	static PyObject *
   	test_s_code(PyObject *self, PyObject *Py_UNUSED(ignored))
   	{
   	/* Unicode strings should be accepted */
   	PyObject *tuple = PyTuple_New(1);
   	if (tuple == NULL) {
   	return NULL;
   	}
   	PyObject *obj = PyUnicode_Decode("t\xeate", strlen("t\xeate"),
   	"latin-1", NULL);
   	if (obj == NULL) {
   	return NULL;
   	}
   	PyTuple_SET_ITEM(tuple, 0, obj);
   	/* These two blocks used to raise a TypeError:
   	* "argument must be string without null bytes, not str"
   	*/
   	char *value;
   	if (!PyArg_ParseTuple(tuple, "s:test_s_code1", &value)) {
   	return NULL;
   	}
   	if (!PyArg_ParseTuple(tuple, "z:test_s_code2", &value)) {
   	return NULL;
   	}
   	Py_DECREF(tuple);
   	Py_RETURN_NONE;
   	}

As well, tuple is leaked on errors.

I have a PR ready.

Linked PRs

• gh-110572: Fix leaks in test_*_code in _testcapi/getargs.c #110573
• [3.12] gh-110572: Fix potential leaks in test_*_code in _testcapi/getargs.c (GH-110573) #111161
• gh-110572: Remove test_* from _testcapi/getargs.c #111214

[serhiy-storchaka]

Are these tests still needed?

[sobolevn]

's', 'L', and 'k' are still mentioned in https://docs.python.org/3.12/howto/clinic.html#how-to-use-real-argument-clinic-converters-instead-of-legacy-converters

So, I guess we still need to make sure that they are supported and work correctly.
They are even not deprecated, just called "legacy".

[serhiy-storchaka]

I mean that they are tested in Lib/test/test_capi/test_getargs.py using wrappers like getargs_k(). And #110628 adds tests for functions like PyLong_AsUnsignedLongMask().

[sobolevn]

In this case, yes. We can move some logic from these tests to test_long.py.

Things that right now are not tested in test_long (this list might be not full):

• test_k_code: PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16)
• test_k_code: PyLong_FromString("-FFFFFFFF000000000000000042", NULL, 16)
• test_s_code: getargs_s('t\xeate') (however, 'abc\xe9' is tested)
• test_s_code: getargs_z('t\xeate') (however, 'abc\xe9' is tested)

So, do you agree on this plan?

1. I convert this PR to delete these tests, it will be in a draft state for now
2. We ensure that all checks from test_*_code are in either test_long.py or in test_getargs.py
3. We merge your PR with test_long.py and possible PRs to test_getargs.py
4. We merge my PR from 1. with these tests removed

[serhiy-storchaka]

I merged your PR, and it will be backported to 3.12. In main we should consider removing this tests.

PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16) is just a way to create a long int for the following test of PyLong_AsUnsignedLongMask(). There are already tests for PyLong_AsUnsignedLongMask() for argument outside of 0..ULONG_MAX. There are also similar tests for the "k" format unit. I think that all this can be deleted.

[hugovk]

Thanks for the PRs.