diff --git a/.gitattributes b/.gitattributes index 13289182400109..cb1cf8bcc7c877 100644 --- a/.gitattributes +++ b/.gitattributes @@ -32,6 +32,9 @@ Lib/test/test_importlib/resources/data01/* noeol Lib/test/test_importlib/resources/namespacedata01/* noeol Lib/test/xmltestdata/* noeol +# Shell scripts should have LF even on Windows because of Cygwin +Lib/venv/scripts/common/activate text eol=lf + # CRLF files [attr]dos text eol=crlf diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 60af0519cdb743..9149b38d87601c 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -61,11 +61,7 @@ Python/traceback.c @iritkatriel /Tools/build/parse_html5_entities.py @ezio-melotti # Import (including importlib). -# Ignoring importlib.h so as to not get flagged on -# all pull requests that change the emitted -# bytecode. -**/*import*.c @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw -**/*import*.py @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw +**/*import* @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw **/*importlib/resources/* @jaraco @warsaw @FFY00 **/importlib/metadata/* @jaraco @warsaw diff --git a/Doc/conf.py b/Doc/conf.py index 29fb63cbcc8614..e99b801d0ae87a 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -68,8 +68,10 @@ # Minimum version of sphinx required needs_sphinx = '3.2' +# Ignore any .rst files in the includes/ directory; +# they're embedded in pages but not rendered individually. # Ignore any .rst files in the venv/ directory. -exclude_patterns = ['venv/*', 'README.rst'] +exclude_patterns = ['includes/*.rst', 'venv/*', 'README.rst'] venvdir = os.getenv('VENVDIR') if venvdir is not None: exclude_patterns.append(venvdir + '/*') diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 3d74d550dc345a..53e8cdcae1cd66 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -214,7 +214,7 @@ Glossary A callable is an object that can be called, possibly with a set of arguments (see :term:`argument`), with the following syntax:: - callable(argument1, argument2, ...) + callable(argument1, argument2, argumentN) A :term:`function`, and by extension a :term:`method`, is a callable. An instance of a class that implements the :meth:`~object.__call__` diff --git a/Doc/library/csv.rst b/Doc/library/csv.rst index f1776554d8b9f2..64baa69be4af31 100644 --- a/Doc/library/csv.rst +++ b/Doc/library/csv.rst @@ -327,7 +327,7 @@ The :mod:`csv` module defines the following constants: Instructs :class:`writer` objects to quote all non-numeric fields. - Instructs the reader to convert all non-quoted fields to type *float*. + Instructs :class:`reader` objects to convert all non-quoted fields to type *float*. .. data:: QUOTE_NONE @@ -337,7 +337,25 @@ The :mod:`csv` module defines the following constants: character. If *escapechar* is not set, the writer will raise :exc:`Error` if any characters that require escaping are encountered. - Instructs :class:`reader` to perform no special processing of quote characters. + Instructs :class:`reader` objects to perform no special processing of quote characters. + +.. data:: QUOTE_NOTNULL + + Instructs :class:`writer` objects to quote all fields which are not + ``None``. This is similar to :data:`QUOTE_ALL`, except that if a + field value is ``None`` an empty (unquoted) string is written. + + Instructs :class:`reader` objects to interpret an empty (unquoted) field as None and + to otherwise behave as :data:`QUOTE_ALL`. + +.. data:: QUOTE_STRINGS + + Instructs :class:`writer` objects to always place quotes around fields + which are strings. This is similar to :data:`QUOTE_NONNUMERIC`, except that if a + field value is ``None`` an empty (unquoted) string is written. + + Instructs :class:`reader` objects to interpret an empty (unquoted) string as ``None`` and + to otherwise behave as :data:`QUOTE_NONNUMERIC`. The :mod:`csv` module defines the following exception: diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 8797485cd05d83..7792e598c1155c 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1681,7 +1681,7 @@ are always available. They are listed here in alphabetical order. class C: @staticmethod - def f(arg1, arg2, ...): ... + def f(arg1, arg2, argN): ... The ``@staticmethod`` form is a function :term:`decorator` -- see :ref:`function` for details. diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst index 2daf2422ebd5b4..250246b5cd9adc 100644 --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -253,6 +253,7 @@ otherwise, the context is used to determine what to instantiate. * ``datefmt`` * ``style`` * ``validate`` (since version >=3.8) + * ``defaults`` (since version >=3.12) An optional ``class`` key indicates the name of the formatter's class (as a dotted module and class name). The instantiation @@ -953,16 +954,22 @@ Sections which specify formatter configuration are typified by the following. .. code-block:: ini [formatter_form01] - format=F1 %(asctime)s %(levelname)s %(message)s + format=F1 %(asctime)s %(levelname)s %(message)s %(customfield)s datefmt= style=% validate=True + defaults={'customfield': 'defaultvalue'} class=logging.Formatter The arguments for the formatter configuration are the same as the keys in the dictionary schema :ref:`formatters section `. +The ``defaults`` entry, when :ref:`evaluated ` in the context of +the ``logging`` package's namespace, is a dictionary of default values for +custom formatting fields. If not provided, it defaults to ``None``. + + .. note:: Due to the use of :func:`eval` as described above, there are diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 8454296b815b41..84e309f1bc8326 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -452,7 +452,9 @@ process which created it. importable by the children. This is covered in :ref:`multiprocessing-programming` however it is worth pointing out here. This means that some examples, such as the :class:`multiprocessing.pool.Pool` examples will not work in the - interactive interpreter. For example:: + interactive interpreter. For example: + + .. code-block:: text >>> from multiprocessing import Pool >>> p = Pool(5) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst index d80c5eebbf27a7..5bc48a6d5f77fd 100644 --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -36,73 +36,91 @@ extension interface uses the modules :mod:`bdb` and :mod:`cmd`. Module :mod:`traceback` Standard interface to extract, format and print stack traces of Python programs. -The debugger's prompt is ``(Pdb)``. Typical usage to run a program under control -of the debugger is:: +The typical usage to break into the debugger is to insert:: - >>> import pdb - >>> import mymodule - >>> pdb.run('mymodule.test()') - > (0)?() - (Pdb) continue - > (1)?() + import pdb; pdb.set_trace() + +Or:: + + breakpoint() + +at the location you want to break into the debugger, and then run the program. +You can then step through the code following this statement, and continue +running without the debugger using the :pdbcmd:`continue` command. + +.. versionadded:: 3.7 + The built-in :func:`breakpoint()`, when called with defaults, can be used + instead of ``import pdb; pdb.set_trace()``. + +:: + + def double(x): + breakpoint() + return x * 2 + val = 3 + print(f"{val} * 2 is {double(val)}") + +The debugger's prompt is ``(Pdb)``, which is the indicator that you are in debug mode:: + + > ...(3)double() + -> return x * 2 + (Pdb) p x + 3 (Pdb) continue - NameError: 'spam' - > (1)?() - (Pdb) + 3 * 2 is 6 .. versionchanged:: 3.3 Tab-completion via the :mod:`readline` module is available for commands and command arguments, e.g. the current global and local names are offered as arguments of the ``p`` command. -:file:`pdb.py` can also be invoked as a script to debug other scripts. For + +You can also invoke :mod:`pdb` from the command line to debug other scripts. For example:: python -m pdb myscript.py -When invoked as a script, pdb will automatically enter post-mortem debugging if +When invoked as a module, pdb will automatically enter post-mortem debugging if the program being debugged exits abnormally. After post-mortem debugging (or after normal exit of the program), pdb will restart the program. Automatic restarting preserves pdb's state (such as breakpoints) and in most cases is more useful than quitting the debugger upon program's exit. .. versionadded:: 3.2 - :file:`pdb.py` now accepts a ``-c`` option that executes commands as if given + ``-c`` option is introduced to execute commands as if given in a :file:`.pdbrc` file, see :ref:`debugger-commands`. .. versionadded:: 3.7 - :file:`pdb.py` now accepts a ``-m`` option that execute modules similar to the way + ``-m`` option is introduced to execute modules similar to the way ``python -m`` does. As with a script, the debugger will pause execution just before the first line of the module. +Typical usage to execute a statement under control of the debugger is:: -The typical usage to break into the debugger is to insert:: - - import pdb; pdb.set_trace() - -at the location you want to break into the debugger, and then run the program. -You can then step through the code following this statement, and continue -running without the debugger using the :pdbcmd:`continue` command. - -.. versionadded:: 3.7 - The built-in :func:`breakpoint()`, when called with defaults, can be used - instead of ``import pdb; pdb.set_trace()``. + >>> import pdb + >>> def f(x): + ... print(1 / x) + >>> pdb.run("f(2)") + > (1)() + (Pdb) continue + 0.5 + >>> The typical usage to inspect a crashed program is:: >>> import pdb - >>> import mymodule - >>> mymodule.test() + >>> def f(x): + ... print(1 / x) + ... + >>> f(0) Traceback (most recent call last): File "", line 1, in - File "./mymodule.py", line 4, in test - test2() - File "./mymodule.py", line 3, in test2 - print(spam) - NameError: spam + File "", line 2, in f + ZeroDivisionError: division by zero >>> pdb.pm() - > ./mymodule.py(3)test2() - -> print(spam) + > (2)f() + (Pdb) p x + 0 (Pdb) @@ -275,7 +293,7 @@ can be overridden by the local file. .. pdbcommand:: w(here) - Print a stack trace, with the most recent frame at the bottom. An arrow + Print a stack trace, with the most recent frame at the bottom. An arrow (``>``) indicates the current frame, which determines the context of most commands. .. pdbcommand:: d(own) [count] @@ -315,14 +333,14 @@ can be overridden by the local file. With a space separated list of breakpoint numbers, clear those breakpoints. Without argument, clear all breaks (but first ask confirmation). -.. pdbcommand:: disable [bpnumber ...] +.. pdbcommand:: disable bpnumber [bpnumber ...] Disable the breakpoints given as a space separated list of breakpoint numbers. Disabling a breakpoint means it cannot cause the program to stop execution, but unlike clearing a breakpoint, it remains in the list of breakpoints and can be (re-)enabled. -.. pdbcommand:: enable [bpnumber ...] +.. pdbcommand:: enable bpnumber [bpnumber ...] Enable the breakpoints specified. @@ -442,7 +460,7 @@ can be overridden by the local file. .. pdbcommand:: a(rgs) - Print the argument list of the current function. + Print the arguments of the current function and their current values. .. pdbcommand:: p expression @@ -476,6 +494,50 @@ can be overridden by the local file. Without *expression*, list all display expressions for the current frame. + .. note:: + + Display evaluates *expression* and compares to the result of the previous + evaluation of *expression*, so when the result is mutable, display may not + be able to pick up the changes. + + Example:: + + lst = [] + breakpoint() + pass + lst.append(1) + print(lst) + + Display won't realize ``lst`` has been changed because the result of evaluation + is modified in place by ``lst.append(1)`` before being compared:: + + > example.py(3)() + -> pass + (Pdb) display lst + display lst: [] + (Pdb) n + > example.py(4)() + -> lst.append(1) + (Pdb) n + > example.py(5)() + -> print(lst) + (Pdb) + + You can do some tricks with copy mechanism to make it work:: + + > example.py(3)() + -> pass + (Pdb) display lst[:] + display lst[:]: [] + (Pdb) n + > example.py(4)() + -> lst.append(1) + (Pdb) n + > example.py(5)() + -> print(lst) + display lst[:]: [1] [old: []] + (Pdb) + .. versionadded:: 3.2 .. pdbcommand:: undisplay [expression] @@ -552,7 +614,7 @@ can be overridden by the local file. .. pdbcommand:: retval - Print the return value for the last return of a function. + Print the return value for the last return of the current function. .. rubric:: Footnotes diff --git a/Doc/library/sched.rst b/Doc/library/sched.rst index a051c65b97b05e..04215d31ba10ca 100644 --- a/Doc/library/sched.rst +++ b/Doc/library/sched.rst @@ -36,7 +36,7 @@ scheduler: Example:: >>> import sched, time - >>> s = sched.scheduler(time.time, time.sleep) + >>> s = sched.scheduler(time.monotonic, time.sleep) >>> def print_time(a='default'): ... print("From print_time", time.time(), a) ... diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 00721efd1cf65e..e37d57edce515f 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -220,6 +220,10 @@ always available. .. audit-event:: sys._current_exceptions "" sys._current_exceptions + .. versionchanged:: 3.12 + Each value in the dictionary is now a single exception instance, rather + than a 3-tuple as returned from ``sys.exc_info()``. + .. function:: breakpointhook() This hook function is called by built-in :func:`breakpoint`. By default, diff --git a/Doc/library/types.rst b/Doc/library/types.rst index 747ba58bb225d4..27b9846325914d 100644 --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -75,9 +75,9 @@ Dynamic Type Creation This function looks for items in *bases* that are not instances of :class:`type`, and returns a tuple where each such object that has - an ``__mro_entries__`` method is replaced with an unpacked result of + an :meth:`~object.__mro_entries__` method is replaced with an unpacked result of calling this method. If a *bases* item is an instance of :class:`type`, - or it doesn't have an ``__mro_entries__`` method, then it is included in + or it doesn't have an :meth:`!__mro_entries__` method, then it is included in the return tuple unchanged. .. versionadded:: 3.7 diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index ed9ebaba784bc2..9f91ade35e50dc 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2089,16 +2089,21 @@ Resolving MRO entries .. method:: object.__mro_entries__(self, bases) If a base that appears in a class definition is not an instance of - :class:`type`, then an ``__mro_entries__`` method is searched on the base. - If an ``__mro_entries__`` method is found, the base is substituted with the - result of a call to ``__mro_entries__`` when creating the class. - The method is called with the original bases tuple, and must return a tuple + :class:`type`, then an :meth:`!__mro_entries__` method is searched on the base. + If an :meth:`!__mro_entries__` method is found, the base is substituted with the + result of a call to :meth:`!__mro_entries__` when creating the class. + The method is called with the original bases tuple + passed to the *bases* parameter, and must return a tuple of classes that will be used instead of the base. The returned tuple may be empty: in these cases, the original base is ignored. .. seealso:: - :pep:`560` - Core support for typing module and generic types + :func:`types.resolve_bases` + Dynamically resolve bases that are not instances of :class:`type`. + + :pep:`560` + Core support for typing module and generic types. Determining the appropriate metaclass diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 2a4d070ec057df..b35e8454fa2a1a 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -495,7 +495,8 @@ Miscellaneous options Reserved for various implementation-specific options. CPython currently defines the following possible values: - * ``-X faulthandler`` to enable :mod:`faulthandler`; + * ``-X faulthandler`` to enable :mod:`faulthandler`. + See also :envvar:`PYTHONFAULTHANDLER`. * ``-X showrefcount`` to output the total reference count and number of used memory blocks when the program finishes or after each statement in the interactive interpreter. This only works on :ref:`debug builds @@ -503,8 +504,9 @@ Miscellaneous options * ``-X tracemalloc`` to start tracing Python memory allocations using the :mod:`tracemalloc` module. By default, only the most recent frame is stored in a traceback of a trace. Use ``-X tracemalloc=NFRAME`` to start - tracing with a traceback limit of *NFRAME* frames. See the - :func:`tracemalloc.start` for more information. + tracing with a traceback limit of *NFRAME* frames. + See :func:`tracemalloc.start` and :envvar:`PYTHONTRACEMALLOC` + for more information. * ``-X int_max_str_digits`` configures the :ref:`integer string conversion length limitation `. See also :envvar:`PYTHONINTMAXSTRDIGITS`. @@ -519,6 +521,7 @@ Miscellaneous options * ``-X utf8`` enables the :ref:`Python UTF-8 Mode `. ``-X utf8=0`` explicitly disables :ref:`Python UTF-8 Mode ` (even when it would otherwise activate automatically). + See also :envvar:`PYTHONUTF8`. * ``-X pycache_prefix=PATH`` enables writing ``.pyc`` files to a parallel tree rooted at the given directory instead of to the code tree. See also :envvar:`PYTHONPYCACHEPREFIX`. @@ -861,7 +864,9 @@ conflict. Python memory allocations using the :mod:`tracemalloc` module. The value of the variable is the maximum number of frames stored in a traceback of a trace. For example, ``PYTHONTRACEMALLOC=1`` stores only the most recent - frame. See the :func:`tracemalloc.start` for more information. + frame. + See the :func:`tracemalloc.start` function for more information. + This is equivalent to setting the :option:`-X` ``tracemalloc`` option. .. versionadded:: 3.4 @@ -869,8 +874,8 @@ conflict. .. envvar:: PYTHONPROFILEIMPORTTIME If this environment variable is set to a non-empty string, Python will - show how long each import takes. This is exactly equivalent to setting - ``-X importtime`` on the command line. + show how long each import takes. + This is equivalent to setting the :option:`-X` ``importtime`` option. .. versionadded:: 3.7 @@ -1012,6 +1017,7 @@ conflict. If this environment variable is set to a non-empty string, enable :ref:`Python Development Mode `, introducing additional runtime checks that are too expensive to be enabled by default. + This is equivalent to setting the :option:`-X` ``dev`` option. .. versionadded:: 3.7 diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 651caed864fef7..4165b16ba76441 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -192,6 +192,10 @@ Other Language Changes * :class:`slice` objects are now hashable, allowing them to be used as dict keys and set items. (Contributed by Will Bradshaw and Furkan Onder in :gh:`101264`.) +* Exceptions raised in a typeobject's ``__set_name__`` method are no longer + wrapped by a :exc:`RuntimeError`. Context information is added to the + exception as a :pep:`678` note. (Contributed by Irit Katriel in :gh:`77757`.) + New Modules =========== @@ -249,6 +253,13 @@ asyncio * :func:`asyncio.wait` now accepts generators yielding tasks. (Contributed by Kumar Aditya in :gh:`78530`.) +csv +--- + +* Add :data:`~csv.QUOTE_NOTNULL` and :data:`~csv.QUOTE_STRINGS` flags to + provide finer grained control of ``None`` and empty strings by + :class:`~csv.reader` and :class:`~csv.writer` objects. + inspect ------- @@ -499,6 +510,10 @@ sys :data:`sys.last_type`, :data:`sys.last_value` and :data:`sys.last_traceback`. (Contributed by Irit Katriel in :gh:`102778`.) +* :func:`sys._current_exceptions` now returns a mapping from thread-id to an + exception instance, rather than to a ``(typ, exc, tb)`` tuple. + (Contributed by Irit Katriel in :gh:`103176`.) + Optimizations ============= @@ -940,6 +955,10 @@ Changes in the Python API synchronization is needed, implement locking within the cached property getter function or around multi-threaded access points. +* :func:`sys._current_exceptions` now returns a mapping from thread-id to an + exception instance, rather than to a ``(typ, exc, tb)`` tuple. + (Contributed by Irit Katriel in :gh:`103176`.) + Build Changes ============= diff --git a/Include/cpython/code.h b/Include/cpython/code.h index abcf1250603dfe..6bead361c79245 100644 --- a/Include/cpython/code.h +++ b/Include/cpython/code.h @@ -3,10 +3,22 @@ #ifndef Py_LIMITED_API #ifndef Py_CODE_H #define Py_CODE_H + #ifdef __cplusplus extern "C" { #endif + +/* Count of all "real" monitoring events (not derived from other events) */ +#define PY_MONITORING_UNGROUPED_EVENTS 14 +/* Count of all monitoring events */ +#define PY_MONITORING_EVENTS 16 + +/* Table of which tools are active for each monitored event. */ +typedef struct _Py_Monitors { + uint8_t tools[PY_MONITORING_UNGROUPED_EVENTS]; +} _Py_Monitors; + /* Each instruction in a code object is a fixed-width value, * currently 2 bytes: 1-byte opcode + 1-byte oparg. The EXTENDED_ARG * opcode allows for larger values but the current limit is 3 uses @@ -56,6 +68,35 @@ typedef struct { PyObject *_co_freevars; } _PyCoCached; +/* Ancilliary data structure used for instrumentation. + Line instrumentation creates an array of + these. One entry per code unit.*/ +typedef struct { + uint8_t original_opcode; + int8_t line_delta; +} _PyCoLineInstrumentationData; + +/* Main data structure used for instrumentation. + * This is allocated when needed for instrumentation + */ +typedef struct { + /* Monitoring specific to this code object */ + _Py_Monitors local_monitors; + /* Monitoring that is active on this code object */ + _Py_Monitors active_monitors; + /* The tools that are to be notified for events for the matching code unit */ + uint8_t *tools; + /* Information to support line events */ + _PyCoLineInstrumentationData *lines; + /* The tools that are to be notified for line events for the matching code unit */ + uint8_t *line_tools; + /* Information to support instruction events */ + /* The underlying instructions, which can themselves be instrumented */ + uint8_t *per_instruction_opcodes; + /* The tools that are to be notified for instruction events for the matching code unit */ + uint8_t *per_instruction_tools; +} _PyCoMonitoringData; + // To avoid repeating ourselves in deepfreeze.py, all PyCodeObject members are // defined in this macro: #define _PyCode_DEF(SIZE) { \ @@ -87,7 +128,6 @@ typedef struct { PyObject *co_exceptiontable; /* Byte string encoding exception handling \ table */ \ int co_flags; /* CO_..., see below */ \ - short _co_linearray_entry_size; /* Size of each entry in _co_linearray */ \ \ /* The rest are not so impactful on performance. */ \ int co_argcount; /* #arguments, except *args */ \ @@ -114,8 +154,9 @@ typedef struct { PyObject *co_linetable; /* bytes object that holds location info */ \ PyObject *co_weakreflist; /* to support weakrefs to code objects */ \ _PyCoCached *_co_cached; /* cached co_* attributes */ \ + uint64_t _co_instrumentation_version; /* current instrumentation version */ \ + _PyCoMonitoringData *_co_monitoring; /* Monitoring data */ \ int _co_firsttraceable; /* index of first traceable instruction */ \ - char *_co_linearray; /* array of line offsets */ \ /* Scratch space for extra data relating to the code object. \ Type is a void* to keep the format private in codeobject.c to force \ people to go through the proper APIs. */ \ diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index 3efb241e8237e7..ea6ed8d2bc4a4c 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -58,12 +58,6 @@ typedef int (*Py_tracefunc)(PyObject *, PyFrameObject *, int, PyObject *); #define PyTrace_C_RETURN 6 #define PyTrace_OPCODE 7 - -typedef struct { - PyCodeObject *code; // The code object for the bounds. May be NULL. - PyCodeAddressRange bounds; // Only valid if code != NULL. -} PyTraceInfo; - // Internal structure: you should not use it directly, but use public functions // like PyThreadState_EnterTracing() and PyThreadState_LeaveTracing(). typedef struct _PyCFrame { @@ -77,7 +71,6 @@ typedef struct _PyCFrame { * discipline and make sure that instances of this struct cannot * accessed outside of their lifetime. */ - uint8_t use_tracing; // 0 or 255 (or'ed into opcode, hence 8-bit type) /* Pointer to the currently executing frame (it can be NULL) */ struct _PyInterpreterFrame *current_frame; struct _PyCFrame *previous; @@ -157,7 +150,7 @@ struct _ts { This is to prevent the actual trace/profile code from being recorded in the trace/profile. */ int tracing; - int tracing_what; /* The event currently being traced, if any. */ + int what_event; /* The event currently being monitored, if any. */ /* Pointer to current _PyCFrame in the C stack frame of the currently, * or most recently, executing _PyEval_EvalFrameDefault. */ @@ -228,8 +221,6 @@ struct _ts { /* Unique thread state id. */ uint64_t id; - PyTraceInfo trace_info; - _PyStackChunk *datastack_chunk; PyObject **datastack_top; PyObject **datastack_limit; diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index faf1be585e17ef..d32f37ac44d83c 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -441,32 +441,6 @@ adaptive_counter_backoff(uint16_t counter) { /* Line array cache for tracing */ -extern int _PyCode_CreateLineArray(PyCodeObject *co); - -static inline int -_PyCode_InitLineArray(PyCodeObject *co) -{ - if (co->_co_linearray) { - return 0; - } - return _PyCode_CreateLineArray(co); -} - -static inline int -_PyCode_LineNumberFromArray(PyCodeObject *co, int index) -{ - assert(co->_co_linearray != NULL); - assert(index >= 0); - assert(index < Py_SIZE(co)); - if (co->_co_linearray_entry_size == 2) { - return ((int16_t *)co->_co_linearray)[index]; - } - else { - assert(co->_co_linearray_entry_size == 4); - return ((int32_t *)co->_co_linearray)[index]; - } -} - typedef struct _PyShimCodeDef { const uint8_t *code; int codelen; @@ -500,6 +474,10 @@ extern uint32_t _Py_next_func_version; #define COMPARISON_NOT_EQUALS (COMPARISON_UNORDERED | COMPARISON_LESS_THAN | COMPARISON_GREATER_THAN) +extern int _Py_Instrument(PyCodeObject *co, PyInterpreterState *interp); + +extern int _Py_GetBaseOpcode(PyCodeObject *code, int offset); + #ifdef __cplusplus } diff --git a/Include/internal/pycore_compile.h b/Include/internal/pycore_compile.h index 6a1e02c0b895d7..f85240c48a89b0 100644 --- a/Include/internal/pycore_compile.h +++ b/Include/internal/pycore_compile.h @@ -33,6 +33,45 @@ extern int _PyAST_Optimize( struct _arena *arena, _PyASTOptimizeState *state); + +typedef struct { + int i_opcode; + int i_oparg; + _PyCompilerSrcLocation i_loc; +} _PyCompilerInstruction; + +typedef struct { + _PyCompilerInstruction *s_instrs; + int s_allocated; + int s_used; + + int *s_labelmap; /* label id --> instr offset */ + int s_labelmap_size; + int s_next_free_label; /* next free label id */ +} _PyCompile_InstructionSequence; + +typedef struct { + PyObject *u_name; + PyObject *u_qualname; /* dot-separated qualified name (lazy) */ + + /* The following fields are dicts that map objects to + the index of them in co_XXX. The index is used as + the argument for opcodes that refer to those collections. + */ + PyObject *u_consts; /* all constants */ + PyObject *u_names; /* all names */ + PyObject *u_varnames; /* local variables */ + PyObject *u_cellvars; /* cell variables */ + PyObject *u_freevars; /* free variables */ + + Py_ssize_t u_argcount; /* number of arguments for block */ + Py_ssize_t u_posonlyargcount; /* number of positional only arguments for block */ + Py_ssize_t u_kwonlyargcount; /* number of keyword only arguments for block */ + + int u_firstlineno; /* the first lineno of the block */ +} _PyCompile_CodeUnitMetadata; + + /* Utility for a number of growing arrays used in the compiler */ int _PyCompile_EnsureArrayLargeEnough( int idx, diff --git a/Include/internal/pycore_flowgraph.h b/Include/internal/pycore_flowgraph.h index 7c0b8fe980c3c7..f470dad3aaa459 100644 --- a/Include/internal/pycore_flowgraph.h +++ b/Include/internal/pycore_flowgraph.h @@ -9,6 +9,7 @@ extern "C" { #endif #include "pycore_opcode_utils.h" +#include "pycore_compile.h" static const _PyCompilerSrcLocation NO_LOCATION = {-1, -1, -1, -1}; @@ -33,7 +34,8 @@ typedef struct { typedef struct _PyCfgBasicblock_ { /* Each basicblock in a compilation unit is linked via b_list in the reverse order that the block are allocated. b_list points to the next - block, not to be confused with b_next, which is next by control flow. */ + block in this list, not to be confused with b_next, which is next by + control flow. */ struct _PyCfgBasicblock_ *b_list; /* The label of this block if it is a jump target, -1 otherwise */ _PyCfgJumpTargetLabel b_label; @@ -91,10 +93,9 @@ void _PyCfgBuilder_Fini(_PyCfgBuilder *g); _PyCfgInstruction* _PyCfg_BasicblockLastInstr(const _PyCfgBasicblock *b); int _PyCfg_OptimizeCodeUnit(_PyCfgBuilder *g, PyObject *consts, PyObject *const_cache, - int code_flags, int nlocals, int nparams); + int code_flags, int nlocals, int nparams, int firstlineno); int _PyCfg_Stackdepth(_PyCfgBasicblock *entryblock, int code_flags); void _PyCfg_ConvertExceptionHandlersToNops(_PyCfgBasicblock *entryblock); -int _PyCfg_ResolveLineNumbers(_PyCfgBuilder *g, int firstlineno); int _PyCfg_ResolveJumps(_PyCfgBuilder *g); int _PyCfg_InstrSize(_PyCfgInstruction *instruction); @@ -110,6 +111,10 @@ basicblock_nofallthrough(const _PyCfgBasicblock *b) { #define BB_NO_FALLTHROUGH(B) (basicblock_nofallthrough(B)) #define BB_HAS_FALLTHROUGH(B) (!basicblock_nofallthrough(B)) +PyCodeObject * +_PyAssemble_MakeCodeObject(_PyCompile_CodeUnitMetadata *u, PyObject *const_cache, + PyObject *consts, int maxdepth, _PyCfgBasicblock *entryblock, + int nlocalsplus, int code_flags, PyObject *filename); #ifdef __cplusplus } diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index 5806cf05f174a9..856297aaea8ab4 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -19,6 +19,7 @@ struct _frame { struct _PyInterpreterFrame *f_frame; /* points to the frame data */ PyObject *f_trace; /* Trace function */ int f_lineno; /* Current line number. Only valid if non-zero */ + int f_last_traced_line; /* The last line traced for this frame */ char f_trace_lines; /* Emit per-line trace events? */ char f_trace_opcodes; /* Emit per-opcode trace events? */ char f_fast_as_locals; /* Have the fast locals of this frame been converted to a dict? */ @@ -137,10 +138,16 @@ _PyFrame_GetLocalsArray(_PyInterpreterFrame *frame) return frame->localsplus; } +/* Fetches the stack pointer, and sets stacktop to -1. + Having stacktop <= 0 ensures that invalid + values are not visible to the cycle GC. + We choose -1 rather than 0 to assist debugging. */ static inline PyObject** _PyFrame_GetStackPointer(_PyInterpreterFrame *frame) { - return frame->localsplus+frame->stacktop; + PyObject **sp = frame->localsplus + frame->stacktop; + frame->stacktop = -1; + return sp; } static inline void diff --git a/Include/internal/pycore_instruments.h b/Include/internal/pycore_instruments.h new file mode 100644 index 00000000000000..e94d8755546efd --- /dev/null +++ b/Include/internal/pycore_instruments.h @@ -0,0 +1,107 @@ + +#ifndef Py_INTERNAL_INSTRUMENT_H +#define Py_INTERNAL_INSTRUMENT_H + + +#include "pycore_bitutils.h" // _Py_popcount32 +#include "pycore_frame.h" + +#include "cpython/code.h" + +#ifdef __cplusplus +extern "C" { +#endif + +#define PY_MONITORING_TOOL_IDS 8 + +/* Local events. + * These require bytecode instrumentation */ + +#define PY_MONITORING_EVENT_PY_START 0 +#define PY_MONITORING_EVENT_PY_RESUME 1 +#define PY_MONITORING_EVENT_PY_RETURN 2 +#define PY_MONITORING_EVENT_PY_YIELD 3 +#define PY_MONITORING_EVENT_CALL 4 +#define PY_MONITORING_EVENT_LINE 5 +#define PY_MONITORING_EVENT_INSTRUCTION 6 +#define PY_MONITORING_EVENT_JUMP 7 +#define PY_MONITORING_EVENT_BRANCH 8 +#define PY_MONITORING_EVENT_STOP_ITERATION 9 + +#define PY_MONITORING_INSTRUMENTED_EVENTS 10 + +/* Other events, mainly exceptions */ + +#define PY_MONITORING_EVENT_RAISE 10 +#define PY_MONITORING_EVENT_EXCEPTION_HANDLED 11 +#define PY_MONITORING_EVENT_PY_UNWIND 12 +#define PY_MONITORING_EVENT_PY_THROW 13 + + +/* Ancilliary events */ + +#define PY_MONITORING_EVENT_C_RETURN 14 +#define PY_MONITORING_EVENT_C_RAISE 15 + + +typedef uint32_t _PyMonitoringEventSet; + +/* Tool IDs */ + +/* These are defined in PEP 669 for convenience to avoid clashes */ +#define PY_MONITORING_DEBUGGER_ID 0 +#define PY_MONITORING_COVERAGE_ID 1 +#define PY_MONITORING_PROFILER_ID 2 +#define PY_MONITORING_OPTIMIZER_ID 5 + +/* Internal IDs used to suuport sys.setprofile() and sys.settrace() */ +#define PY_MONITORING_SYS_PROFILE_ID 6 +#define PY_MONITORING_SYS_TRACE_ID 7 + + +PyObject *_PyMonitoring_RegisterCallback(int tool_id, int event_id, PyObject *obj); + +int _PyMonitoring_SetEvents(int tool_id, _PyMonitoringEventSet events); + +extern int +_Py_call_instrumentation(PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr); + +extern int +_Py_call_instrumentation_line(PyThreadState *tstate, _PyInterpreterFrame* frame, + _Py_CODEUNIT *instr); + +extern int +_Py_call_instrumentation_instruction( + PyThreadState *tstate, _PyInterpreterFrame* frame, _Py_CODEUNIT *instr); + +int +_Py_call_instrumentation_jump( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, _Py_CODEUNIT *target); + +extern int +_Py_call_instrumentation_arg(PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *arg); + +extern int +_Py_call_instrumentation_2args(PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *arg0, PyObject *arg1); + +extern void +_Py_call_instrumentation_exc0(PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr); + +extern void +_Py_call_instrumentation_exc2(PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *arg0, PyObject *arg1); + +extern int +_Py_Instrumentation_GetLine(PyCodeObject *code, int index); + +extern PyObject _PyInstrumentation_MISSING; + +#ifdef __cplusplus +} +#endif +#endif /* !Py_INTERNAL_INSTRUMENT_H */ diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index d64a68cd2da54e..86ae3d8dfc1860 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -24,6 +24,7 @@ extern "C" { #include "pycore_genobject.h" // struct _Py_async_gen_state #include "pycore_gc.h" // struct _gc_runtime_state #include "pycore_import.h" // struct _import_state +#include "pycore_instruments.h" // PY_MONITORING_EVENTS #include "pycore_list.h" // struct _Py_list_state #include "pycore_global_objects.h" // struct _Py_interp_static_objects #include "pycore_object_state.h" // struct _py_object_state @@ -37,7 +38,6 @@ struct _Py_long_state { int max_str_digits; }; - /* interpreter state */ /* PyInterpreterState holds the global state for one of the runtime's @@ -49,6 +49,9 @@ struct _is { PyInterpreterState *next; + uint64_t monitoring_version; + uint64_t last_restart_version; + struct pythreads { uint64_t next_unique_id; /* The linked list of threads, newest first. */ @@ -148,6 +151,15 @@ struct _is { struct callable_cache callable_cache; PyCodeObject *interpreter_trampoline; + _Py_Monitors monitors; + bool f_opcode_trace_set; + bool sys_profile_initialized; + bool sys_trace_initialized; + Py_ssize_t sys_profiling_threads; /* Count of threads with c_profilefunc set */ + Py_ssize_t sys_tracing_threads; /* Count of threads with c_tracefunc set */ + PyObject *monitoring_callables[PY_MONITORING_TOOL_IDS][PY_MONITORING_EVENTS]; + PyObject *monitoring_tool_names[PY_MONITORING_TOOL_IDS]; + struct _Py_interp_cached_objects cached_objects; struct _Py_interp_static_objects static_objects; diff --git a/Include/internal/pycore_opcode.h b/Include/internal/pycore_opcode.h index 81ca91465950b7..c039d712dc0ba1 100644 --- a/Include/internal/pycore_opcode.h +++ b/Include/internal/pycore_opcode.h @@ -12,8 +12,6 @@ extern "C" { #include "opcode.h" -extern const uint32_t _PyOpcode_RelativeJump[9]; - extern const uint32_t _PyOpcode_Jump[9]; extern const uint8_t _PyOpcode_Caches[256]; @@ -21,17 +19,6 @@ extern const uint8_t _PyOpcode_Caches[256]; extern const uint8_t _PyOpcode_Deopt[256]; #ifdef NEED_OPCODE_TABLES -const uint32_t _PyOpcode_RelativeJump[9] = { - 0U, - 0U, - 536870912U, - 135020544U, - 4163U, - 0U, - 0U, - 0U, - 48U, -}; const uint32_t _PyOpcode_Jump[9] = { 0U, 0U, @@ -125,6 +112,7 @@ const uint8_t _PyOpcode_Deopt[256] = { [DICT_UPDATE] = DICT_UPDATE, [END_ASYNC_FOR] = END_ASYNC_FOR, [END_FOR] = END_FOR, + [END_SEND] = END_SEND, [EXTENDED_ARG] = EXTENDED_ARG, [FORMAT_VALUE] = FORMAT_VALUE, [FOR_ITER] = FOR_ITER, @@ -140,6 +128,23 @@ const uint8_t _PyOpcode_Deopt[256] = { [GET_YIELD_FROM_ITER] = GET_YIELD_FROM_ITER, [IMPORT_FROM] = IMPORT_FROM, [IMPORT_NAME] = IMPORT_NAME, + [INSTRUMENTED_CALL] = INSTRUMENTED_CALL, + [INSTRUMENTED_CALL_FUNCTION_EX] = INSTRUMENTED_CALL_FUNCTION_EX, + [INSTRUMENTED_END_FOR] = INSTRUMENTED_END_FOR, + [INSTRUMENTED_END_SEND] = INSTRUMENTED_END_SEND, + [INSTRUMENTED_FOR_ITER] = INSTRUMENTED_FOR_ITER, + [INSTRUMENTED_INSTRUCTION] = INSTRUMENTED_INSTRUCTION, + [INSTRUMENTED_JUMP_BACKWARD] = INSTRUMENTED_JUMP_BACKWARD, + [INSTRUMENTED_JUMP_FORWARD] = INSTRUMENTED_JUMP_FORWARD, + [INSTRUMENTED_LINE] = INSTRUMENTED_LINE, + [INSTRUMENTED_POP_JUMP_IF_FALSE] = INSTRUMENTED_POP_JUMP_IF_FALSE, + [INSTRUMENTED_POP_JUMP_IF_NONE] = INSTRUMENTED_POP_JUMP_IF_NONE, + [INSTRUMENTED_POP_JUMP_IF_NOT_NONE] = INSTRUMENTED_POP_JUMP_IF_NOT_NONE, + [INSTRUMENTED_POP_JUMP_IF_TRUE] = INSTRUMENTED_POP_JUMP_IF_TRUE, + [INSTRUMENTED_RESUME] = INSTRUMENTED_RESUME, + [INSTRUMENTED_RETURN_CONST] = INSTRUMENTED_RETURN_CONST, + [INSTRUMENTED_RETURN_VALUE] = INSTRUMENTED_RETURN_VALUE, + [INSTRUMENTED_YIELD_VALUE] = INSTRUMENTED_YIELD_VALUE, [INTERPRETER_EXIT] = INTERPRETER_EXIT, [IS_OP] = IS_OP, [JUMP_BACKWARD] = JUMP_BACKWARD, @@ -192,6 +197,7 @@ const uint8_t _PyOpcode_Deopt[256] = { [PUSH_NULL] = PUSH_NULL, [RAISE_VARARGS] = RAISE_VARARGS, [RERAISE] = RERAISE, + [RESERVED] = RESERVED, [RESUME] = RESUME, [RETURN_CONST] = RETURN_CONST, [RETURN_GENERATOR] = RETURN_GENERATOR, @@ -236,17 +242,19 @@ static const char *const _PyOpcode_OpName[263] = { [PUSH_NULL] = "PUSH_NULL", [INTERPRETER_EXIT] = "INTERPRETER_EXIT", [END_FOR] = "END_FOR", + [END_SEND] = "END_SEND", [BINARY_OP_ADD_FLOAT] = "BINARY_OP_ADD_FLOAT", [BINARY_OP_ADD_INT] = "BINARY_OP_ADD_INT", [BINARY_OP_ADD_UNICODE] = "BINARY_OP_ADD_UNICODE", - [BINARY_OP_INPLACE_ADD_UNICODE] = "BINARY_OP_INPLACE_ADD_UNICODE", [NOP] = "NOP", - [BINARY_OP_MULTIPLY_FLOAT] = "BINARY_OP_MULTIPLY_FLOAT", + [BINARY_OP_INPLACE_ADD_UNICODE] = "BINARY_OP_INPLACE_ADD_UNICODE", [UNARY_NEGATIVE] = "UNARY_NEGATIVE", [UNARY_NOT] = "UNARY_NOT", + [BINARY_OP_MULTIPLY_FLOAT] = "BINARY_OP_MULTIPLY_FLOAT", [BINARY_OP_MULTIPLY_INT] = "BINARY_OP_MULTIPLY_INT", - [BINARY_OP_SUBTRACT_FLOAT] = "BINARY_OP_SUBTRACT_FLOAT", [UNARY_INVERT] = "UNARY_INVERT", + [BINARY_OP_SUBTRACT_FLOAT] = "BINARY_OP_SUBTRACT_FLOAT", + [RESERVED] = "RESERVED", [BINARY_OP_SUBTRACT_INT] = "BINARY_OP_SUBTRACT_INT", [BINARY_SUBSCR_DICT] = "BINARY_SUBSCR_DICT", [BINARY_SUBSCR_GETITEM] = "BINARY_SUBSCR_GETITEM", @@ -254,21 +262,21 @@ static const char *const _PyOpcode_OpName[263] = { [BINARY_SUBSCR_TUPLE_INT] = "BINARY_SUBSCR_TUPLE_INT", [CALL_PY_EXACT_ARGS] = "CALL_PY_EXACT_ARGS", [CALL_PY_WITH_DEFAULTS] = "CALL_PY_WITH_DEFAULTS", - [CALL_BOUND_METHOD_EXACT_ARGS] = "CALL_BOUND_METHOD_EXACT_ARGS", - [CALL_BUILTIN_CLASS] = "CALL_BUILTIN_CLASS", [BINARY_SUBSCR] = "BINARY_SUBSCR", [BINARY_SLICE] = "BINARY_SLICE", [STORE_SLICE] = "STORE_SLICE", - [CALL_BUILTIN_FAST_WITH_KEYWORDS] = "CALL_BUILTIN_FAST_WITH_KEYWORDS", - [CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS] = "CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS", + [CALL_BOUND_METHOD_EXACT_ARGS] = "CALL_BOUND_METHOD_EXACT_ARGS", + [CALL_BUILTIN_CLASS] = "CALL_BUILTIN_CLASS", [GET_LEN] = "GET_LEN", [MATCH_MAPPING] = "MATCH_MAPPING", [MATCH_SEQUENCE] = "MATCH_SEQUENCE", [MATCH_KEYS] = "MATCH_KEYS", - [CALL_NO_KW_BUILTIN_FAST] = "CALL_NO_KW_BUILTIN_FAST", + [CALL_BUILTIN_FAST_WITH_KEYWORDS] = "CALL_BUILTIN_FAST_WITH_KEYWORDS", [PUSH_EXC_INFO] = "PUSH_EXC_INFO", [CHECK_EXC_MATCH] = "CHECK_EXC_MATCH", [CHECK_EG_MATCH] = "CHECK_EG_MATCH", + [CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS] = "CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS", + [CALL_NO_KW_BUILTIN_FAST] = "CALL_NO_KW_BUILTIN_FAST", [CALL_NO_KW_BUILTIN_O] = "CALL_NO_KW_BUILTIN_O", [CALL_NO_KW_ISINSTANCE] = "CALL_NO_KW_ISINSTANCE", [CALL_NO_KW_LEN] = "CALL_NO_KW_LEN", @@ -278,8 +286,6 @@ static const char *const _PyOpcode_OpName[263] = { [CALL_NO_KW_METHOD_DESCRIPTOR_O] = "CALL_NO_KW_METHOD_DESCRIPTOR_O", [CALL_NO_KW_STR_1] = "CALL_NO_KW_STR_1", [CALL_NO_KW_TUPLE_1] = "CALL_NO_KW_TUPLE_1", - [CALL_NO_KW_TYPE_1] = "CALL_NO_KW_TYPE_1", - [COMPARE_OP_FLOAT] = "COMPARE_OP_FLOAT", [WITH_EXCEPT_START] = "WITH_EXCEPT_START", [GET_AITER] = "GET_AITER", [GET_ANEXT] = "GET_ANEXT", @@ -287,39 +293,39 @@ static const char *const _PyOpcode_OpName[263] = { [BEFORE_WITH] = "BEFORE_WITH", [END_ASYNC_FOR] = "END_ASYNC_FOR", [CLEANUP_THROW] = "CLEANUP_THROW", + [CALL_NO_KW_TYPE_1] = "CALL_NO_KW_TYPE_1", + [COMPARE_OP_FLOAT] = "COMPARE_OP_FLOAT", [COMPARE_OP_INT] = "COMPARE_OP_INT", [COMPARE_OP_STR] = "COMPARE_OP_STR", - [FOR_ITER_LIST] = "FOR_ITER_LIST", - [FOR_ITER_TUPLE] = "FOR_ITER_TUPLE", [STORE_SUBSCR] = "STORE_SUBSCR", [DELETE_SUBSCR] = "DELETE_SUBSCR", + [FOR_ITER_LIST] = "FOR_ITER_LIST", + [FOR_ITER_TUPLE] = "FOR_ITER_TUPLE", [FOR_ITER_RANGE] = "FOR_ITER_RANGE", [FOR_ITER_GEN] = "FOR_ITER_GEN", [LOAD_ATTR_CLASS] = "LOAD_ATTR_CLASS", [LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN] = "LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN", - [LOAD_ATTR_INSTANCE_VALUE] = "LOAD_ATTR_INSTANCE_VALUE", - [LOAD_ATTR_MODULE] = "LOAD_ATTR_MODULE", [GET_ITER] = "GET_ITER", [GET_YIELD_FROM_ITER] = "GET_YIELD_FROM_ITER", - [LOAD_ATTR_PROPERTY] = "LOAD_ATTR_PROPERTY", + [LOAD_ATTR_INSTANCE_VALUE] = "LOAD_ATTR_INSTANCE_VALUE", [LOAD_BUILD_CLASS] = "LOAD_BUILD_CLASS", - [LOAD_ATTR_SLOT] = "LOAD_ATTR_SLOT", - [LOAD_ATTR_WITH_HINT] = "LOAD_ATTR_WITH_HINT", + [LOAD_ATTR_MODULE] = "LOAD_ATTR_MODULE", + [LOAD_ATTR_PROPERTY] = "LOAD_ATTR_PROPERTY", [LOAD_ASSERTION_ERROR] = "LOAD_ASSERTION_ERROR", [RETURN_GENERATOR] = "RETURN_GENERATOR", + [LOAD_ATTR_SLOT] = "LOAD_ATTR_SLOT", + [LOAD_ATTR_WITH_HINT] = "LOAD_ATTR_WITH_HINT", [LOAD_ATTR_METHOD_LAZY_DICT] = "LOAD_ATTR_METHOD_LAZY_DICT", [LOAD_ATTR_METHOD_NO_DICT] = "LOAD_ATTR_METHOD_NO_DICT", [LOAD_ATTR_METHOD_WITH_VALUES] = "LOAD_ATTR_METHOD_WITH_VALUES", [LOAD_CONST__LOAD_FAST] = "LOAD_CONST__LOAD_FAST", [LOAD_FAST__LOAD_CONST] = "LOAD_FAST__LOAD_CONST", + [RETURN_VALUE] = "RETURN_VALUE", [LOAD_FAST__LOAD_FAST] = "LOAD_FAST__LOAD_FAST", + [SETUP_ANNOTATIONS] = "SETUP_ANNOTATIONS", [LOAD_GLOBAL_BUILTIN] = "LOAD_GLOBAL_BUILTIN", - [RETURN_VALUE] = "RETURN_VALUE", [LOAD_GLOBAL_MODULE] = "LOAD_GLOBAL_MODULE", - [SETUP_ANNOTATIONS] = "SETUP_ANNOTATIONS", [STORE_ATTR_INSTANCE_VALUE] = "STORE_ATTR_INSTANCE_VALUE", - [STORE_ATTR_SLOT] = "STORE_ATTR_SLOT", - [STORE_ATTR_WITH_HINT] = "STORE_ATTR_WITH_HINT", [POP_EXCEPT] = "POP_EXCEPT", [STORE_NAME] = "STORE_NAME", [DELETE_NAME] = "DELETE_NAME", @@ -342,9 +348,9 @@ static const char *const _PyOpcode_OpName[263] = { [IMPORT_NAME] = "IMPORT_NAME", [IMPORT_FROM] = "IMPORT_FROM", [JUMP_FORWARD] = "JUMP_FORWARD", + [STORE_ATTR_SLOT] = "STORE_ATTR_SLOT", + [STORE_ATTR_WITH_HINT] = "STORE_ATTR_WITH_HINT", [STORE_FAST__LOAD_FAST] = "STORE_FAST__LOAD_FAST", - [STORE_FAST__STORE_FAST] = "STORE_FAST__STORE_FAST", - [STORE_SUBSCR_DICT] = "STORE_SUBSCR_DICT", [POP_JUMP_IF_FALSE] = "POP_JUMP_IF_FALSE", [POP_JUMP_IF_TRUE] = "POP_JUMP_IF_TRUE", [LOAD_GLOBAL] = "LOAD_GLOBAL", @@ -372,9 +378,9 @@ static const char *const _PyOpcode_OpName[263] = { [STORE_DEREF] = "STORE_DEREF", [DELETE_DEREF] = "DELETE_DEREF", [JUMP_BACKWARD] = "JUMP_BACKWARD", - [STORE_SUBSCR_LIST_INT] = "STORE_SUBSCR_LIST_INT", + [STORE_FAST__STORE_FAST] = "STORE_FAST__STORE_FAST", [CALL_FUNCTION_EX] = "CALL_FUNCTION_EX", - [UNPACK_SEQUENCE_LIST] = "UNPACK_SEQUENCE_LIST", + [STORE_SUBSCR_DICT] = "STORE_SUBSCR_DICT", [EXTENDED_ARG] = "EXTENDED_ARG", [LIST_APPEND] = "LIST_APPEND", [SET_ADD] = "SET_ADD", @@ -384,14 +390,14 @@ static const char *const _PyOpcode_OpName[263] = { [YIELD_VALUE] = "YIELD_VALUE", [RESUME] = "RESUME", [MATCH_CLASS] = "MATCH_CLASS", - [UNPACK_SEQUENCE_TUPLE] = "UNPACK_SEQUENCE_TUPLE", - [UNPACK_SEQUENCE_TWO_TUPLE] = "UNPACK_SEQUENCE_TWO_TUPLE", + [STORE_SUBSCR_LIST_INT] = "STORE_SUBSCR_LIST_INT", + [UNPACK_SEQUENCE_LIST] = "UNPACK_SEQUENCE_LIST", [FORMAT_VALUE] = "FORMAT_VALUE", [BUILD_CONST_KEY_MAP] = "BUILD_CONST_KEY_MAP", [BUILD_STRING] = "BUILD_STRING", + [UNPACK_SEQUENCE_TUPLE] = "UNPACK_SEQUENCE_TUPLE", + [UNPACK_SEQUENCE_TWO_TUPLE] = "UNPACK_SEQUENCE_TWO_TUPLE", [SEND_GEN] = "SEND_GEN", - [159] = "<159>", - [160] = "<160>", [161] = "<161>", [LIST_EXTEND] = "LIST_EXTEND", [SET_UPDATE] = "SET_UPDATE", @@ -469,24 +475,24 @@ static const char *const _PyOpcode_OpName[263] = { [235] = "<235>", [236] = "<236>", [237] = "<237>", - [238] = "<238>", - [239] = "<239>", - [240] = "<240>", - [241] = "<241>", - [242] = "<242>", - [243] = "<243>", - [244] = "<244>", - [245] = "<245>", - [246] = "<246>", - [247] = "<247>", - [248] = "<248>", - [249] = "<249>", - [250] = "<250>", - [251] = "<251>", - [252] = "<252>", - [253] = "<253>", - [254] = "<254>", - [DO_TRACING] = "DO_TRACING", + [INSTRUMENTED_POP_JUMP_IF_NONE] = "INSTRUMENTED_POP_JUMP_IF_NONE", + [INSTRUMENTED_POP_JUMP_IF_NOT_NONE] = "INSTRUMENTED_POP_JUMP_IF_NOT_NONE", + [INSTRUMENTED_RESUME] = "INSTRUMENTED_RESUME", + [INSTRUMENTED_CALL] = "INSTRUMENTED_CALL", + [INSTRUMENTED_RETURN_VALUE] = "INSTRUMENTED_RETURN_VALUE", + [INSTRUMENTED_YIELD_VALUE] = "INSTRUMENTED_YIELD_VALUE", + [INSTRUMENTED_CALL_FUNCTION_EX] = "INSTRUMENTED_CALL_FUNCTION_EX", + [INSTRUMENTED_JUMP_FORWARD] = "INSTRUMENTED_JUMP_FORWARD", + [INSTRUMENTED_JUMP_BACKWARD] = "INSTRUMENTED_JUMP_BACKWARD", + [INSTRUMENTED_RETURN_CONST] = "INSTRUMENTED_RETURN_CONST", + [INSTRUMENTED_FOR_ITER] = "INSTRUMENTED_FOR_ITER", + [INSTRUMENTED_POP_JUMP_IF_FALSE] = "INSTRUMENTED_POP_JUMP_IF_FALSE", + [INSTRUMENTED_POP_JUMP_IF_TRUE] = "INSTRUMENTED_POP_JUMP_IF_TRUE", + [INSTRUMENTED_END_FOR] = "INSTRUMENTED_END_FOR", + [INSTRUMENTED_END_SEND] = "INSTRUMENTED_END_SEND", + [INSTRUMENTED_INSTRUCTION] = "INSTRUMENTED_INSTRUCTION", + [INSTRUMENTED_LINE] = "INSTRUMENTED_LINE", + [255] = "<255>", [SETUP_FINALLY] = "SETUP_FINALLY", [SETUP_CLEANUP] = "SETUP_CLEANUP", [SETUP_WITH] = "SETUP_WITH", @@ -498,8 +504,6 @@ static const char *const _PyOpcode_OpName[263] = { #endif #define EXTRA_CASES \ - case 159: \ - case 160: \ case 161: \ case 166: \ case 167: \ @@ -569,23 +573,7 @@ static const char *const _PyOpcode_OpName[263] = { case 235: \ case 236: \ case 237: \ - case 238: \ - case 239: \ - case 240: \ - case 241: \ - case 242: \ - case 243: \ - case 244: \ - case 245: \ - case 246: \ - case 247: \ - case 248: \ - case 249: \ - case 250: \ - case 251: \ - case 252: \ - case 253: \ - case 254: \ + case 255: \ ; #ifdef __cplusplus diff --git a/Include/internal/pycore_opcode_utils.h b/Include/internal/pycore_opcode_utils.h index 96bb4d743bf29f..1d5ff988290bd4 100644 --- a/Include/internal/pycore_opcode_utils.h +++ b/Include/internal/pycore_opcode_utils.h @@ -8,7 +8,7 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -#include "pycore_opcode.h" // _PyOpcode_RelativeJump +#include "pycore_opcode.h" // _PyOpcode_Jump #define MAX_REAL_OPCODE 254 @@ -85,9 +85,6 @@ is_bit_set_in_table(const uint32_t *table, int bitindex) { #undef LOG_BITS_PER_INT #undef MASK_LOW_LOG_BITS -#define IS_RELATIVE_JUMP(opcode) (is_bit_set_in_table(_PyOpcode_RelativeJump, opcode)) - - #ifdef __cplusplus } diff --git a/Include/internal/pycore_pyerrors.h b/Include/internal/pycore_pyerrors.h index 1bb4a9aa103898..4620a269644917 100644 --- a/Include/internal/pycore_pyerrors.h +++ b/Include/internal/pycore_pyerrors.h @@ -109,6 +109,8 @@ extern PyObject* _Py_Offer_Suggestions(PyObject* exception); PyAPI_FUNC(Py_ssize_t) _Py_UTF8_Edit_Cost(PyObject *str_a, PyObject *str_b, Py_ssize_t max_cost); +void _PyErr_FormatNote(const char *format, ...); + #ifdef __cplusplus } #endif diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h index b5408622d9d4b2..6e5f2289cb6b95 100644 --- a/Include/internal/pycore_pystate.h +++ b/Include/internal/pycore_pystate.h @@ -133,16 +133,6 @@ extern void _PyThreadState_BindDetached(PyThreadState *); extern void _PyThreadState_UnbindDetached(PyThreadState *); -static inline void -_PyThreadState_UpdateTracingState(PyThreadState *tstate) -{ - bool use_tracing = - (tstate->tracing == 0) && - (tstate->c_tracefunc != NULL || tstate->c_profilefunc != NULL); - tstate->cframe->use_tracing = (use_tracing ? 255 : 0); -} - - /* Other */ PyAPI_FUNC(PyThreadState *) _PyThreadState_Swap( diff --git a/Include/opcode.h b/Include/opcode.h index 0ff84dc5a551a0..aa8716ef5b4030 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -13,10 +13,12 @@ extern "C" { #define PUSH_NULL 2 #define INTERPRETER_EXIT 3 #define END_FOR 4 +#define END_SEND 5 #define NOP 9 #define UNARY_NEGATIVE 11 #define UNARY_NOT 12 #define UNARY_INVERT 15 +#define RESERVED 17 #define BINARY_SUBSCR 25 #define BINARY_SLICE 26 #define STORE_SLICE 27 @@ -114,6 +116,24 @@ extern "C" { #define KW_NAMES 172 #define CALL_INTRINSIC_1 173 #define CALL_INTRINSIC_2 174 +#define MIN_INSTRUMENTED_OPCODE 238 +#define INSTRUMENTED_POP_JUMP_IF_NONE 238 +#define INSTRUMENTED_POP_JUMP_IF_NOT_NONE 239 +#define INSTRUMENTED_RESUME 240 +#define INSTRUMENTED_CALL 241 +#define INSTRUMENTED_RETURN_VALUE 242 +#define INSTRUMENTED_YIELD_VALUE 243 +#define INSTRUMENTED_CALL_FUNCTION_EX 244 +#define INSTRUMENTED_JUMP_FORWARD 245 +#define INSTRUMENTED_JUMP_BACKWARD 246 +#define INSTRUMENTED_RETURN_CONST 247 +#define INSTRUMENTED_FOR_ITER 248 +#define INSTRUMENTED_POP_JUMP_IF_FALSE 249 +#define INSTRUMENTED_POP_JUMP_IF_TRUE 250 +#define INSTRUMENTED_END_FOR 251 +#define INSTRUMENTED_END_SEND 252 +#define INSTRUMENTED_INSTRUCTION 253 +#define INSTRUMENTED_LINE 254 #define MIN_PSEUDO_OPCODE 256 #define SETUP_FINALLY 256 #define SETUP_CLEANUP 257 @@ -123,69 +143,68 @@ extern "C" { #define JUMP_NO_INTERRUPT 261 #define LOAD_METHOD 262 #define MAX_PSEUDO_OPCODE 262 -#define BINARY_OP_ADD_FLOAT 5 -#define BINARY_OP_ADD_INT 6 -#define BINARY_OP_ADD_UNICODE 7 -#define BINARY_OP_INPLACE_ADD_UNICODE 8 -#define BINARY_OP_MULTIPLY_FLOAT 10 -#define BINARY_OP_MULTIPLY_INT 13 -#define BINARY_OP_SUBTRACT_FLOAT 14 -#define BINARY_OP_SUBTRACT_INT 16 -#define BINARY_SUBSCR_DICT 17 -#define BINARY_SUBSCR_GETITEM 18 -#define BINARY_SUBSCR_LIST_INT 19 -#define BINARY_SUBSCR_TUPLE_INT 20 -#define CALL_PY_EXACT_ARGS 21 -#define CALL_PY_WITH_DEFAULTS 22 -#define CALL_BOUND_METHOD_EXACT_ARGS 23 -#define CALL_BUILTIN_CLASS 24 -#define CALL_BUILTIN_FAST_WITH_KEYWORDS 28 -#define CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS 29 -#define CALL_NO_KW_BUILTIN_FAST 34 -#define CALL_NO_KW_BUILTIN_O 38 -#define CALL_NO_KW_ISINSTANCE 39 -#define CALL_NO_KW_LEN 40 -#define CALL_NO_KW_LIST_APPEND 41 -#define CALL_NO_KW_METHOD_DESCRIPTOR_FAST 42 -#define CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 43 -#define CALL_NO_KW_METHOD_DESCRIPTOR_O 44 -#define CALL_NO_KW_STR_1 45 -#define CALL_NO_KW_TUPLE_1 46 -#define CALL_NO_KW_TYPE_1 47 -#define COMPARE_OP_FLOAT 48 -#define COMPARE_OP_INT 56 -#define COMPARE_OP_STR 57 -#define FOR_ITER_LIST 58 -#define FOR_ITER_TUPLE 59 -#define FOR_ITER_RANGE 62 -#define FOR_ITER_GEN 63 -#define LOAD_ATTR_CLASS 64 -#define LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN 65 -#define LOAD_ATTR_INSTANCE_VALUE 66 -#define LOAD_ATTR_MODULE 67 -#define LOAD_ATTR_PROPERTY 70 -#define LOAD_ATTR_SLOT 72 -#define LOAD_ATTR_WITH_HINT 73 -#define LOAD_ATTR_METHOD_LAZY_DICT 76 -#define LOAD_ATTR_METHOD_NO_DICT 77 -#define LOAD_ATTR_METHOD_WITH_VALUES 78 -#define LOAD_CONST__LOAD_FAST 79 -#define LOAD_FAST__LOAD_CONST 80 -#define LOAD_FAST__LOAD_FAST 81 -#define LOAD_GLOBAL_BUILTIN 82 -#define LOAD_GLOBAL_MODULE 84 -#define STORE_ATTR_INSTANCE_VALUE 86 -#define STORE_ATTR_SLOT 87 -#define STORE_ATTR_WITH_HINT 88 -#define STORE_FAST__LOAD_FAST 111 -#define STORE_FAST__STORE_FAST 112 -#define STORE_SUBSCR_DICT 113 -#define STORE_SUBSCR_LIST_INT 141 -#define UNPACK_SEQUENCE_LIST 143 -#define UNPACK_SEQUENCE_TUPLE 153 -#define UNPACK_SEQUENCE_TWO_TUPLE 154 -#define SEND_GEN 158 -#define DO_TRACING 255 +#define BINARY_OP_ADD_FLOAT 6 +#define BINARY_OP_ADD_INT 7 +#define BINARY_OP_ADD_UNICODE 8 +#define BINARY_OP_INPLACE_ADD_UNICODE 10 +#define BINARY_OP_MULTIPLY_FLOAT 13 +#define BINARY_OP_MULTIPLY_INT 14 +#define BINARY_OP_SUBTRACT_FLOAT 16 +#define BINARY_OP_SUBTRACT_INT 18 +#define BINARY_SUBSCR_DICT 19 +#define BINARY_SUBSCR_GETITEM 20 +#define BINARY_SUBSCR_LIST_INT 21 +#define BINARY_SUBSCR_TUPLE_INT 22 +#define CALL_PY_EXACT_ARGS 23 +#define CALL_PY_WITH_DEFAULTS 24 +#define CALL_BOUND_METHOD_EXACT_ARGS 28 +#define CALL_BUILTIN_CLASS 29 +#define CALL_BUILTIN_FAST_WITH_KEYWORDS 34 +#define CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS 38 +#define CALL_NO_KW_BUILTIN_FAST 39 +#define CALL_NO_KW_BUILTIN_O 40 +#define CALL_NO_KW_ISINSTANCE 41 +#define CALL_NO_KW_LEN 42 +#define CALL_NO_KW_LIST_APPEND 43 +#define CALL_NO_KW_METHOD_DESCRIPTOR_FAST 44 +#define CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 45 +#define CALL_NO_KW_METHOD_DESCRIPTOR_O 46 +#define CALL_NO_KW_STR_1 47 +#define CALL_NO_KW_TUPLE_1 48 +#define CALL_NO_KW_TYPE_1 56 +#define COMPARE_OP_FLOAT 57 +#define COMPARE_OP_INT 58 +#define COMPARE_OP_STR 59 +#define FOR_ITER_LIST 62 +#define FOR_ITER_TUPLE 63 +#define FOR_ITER_RANGE 64 +#define FOR_ITER_GEN 65 +#define LOAD_ATTR_CLASS 66 +#define LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN 67 +#define LOAD_ATTR_INSTANCE_VALUE 70 +#define LOAD_ATTR_MODULE 72 +#define LOAD_ATTR_PROPERTY 73 +#define LOAD_ATTR_SLOT 76 +#define LOAD_ATTR_WITH_HINT 77 +#define LOAD_ATTR_METHOD_LAZY_DICT 78 +#define LOAD_ATTR_METHOD_NO_DICT 79 +#define LOAD_ATTR_METHOD_WITH_VALUES 80 +#define LOAD_CONST__LOAD_FAST 81 +#define LOAD_FAST__LOAD_CONST 82 +#define LOAD_FAST__LOAD_FAST 84 +#define LOAD_GLOBAL_BUILTIN 86 +#define LOAD_GLOBAL_MODULE 87 +#define STORE_ATTR_INSTANCE_VALUE 88 +#define STORE_ATTR_SLOT 111 +#define STORE_ATTR_WITH_HINT 112 +#define STORE_FAST__LOAD_FAST 113 +#define STORE_FAST__STORE_FAST 141 +#define STORE_SUBSCR_DICT 143 +#define STORE_SUBSCR_LIST_INT 153 +#define UNPACK_SEQUENCE_LIST 154 +#define UNPACK_SEQUENCE_TUPLE 158 +#define UNPACK_SEQUENCE_TWO_TUPLE 159 +#define SEND_GEN 160 #define HAS_ARG(op) ((((op) >= HAVE_ARGUMENT) && (!IS_PSEUDO_OPCODE(op)))\ || ((op) == JUMP) \ diff --git a/Lib/abc.py b/Lib/abc.py index 42048ddb855381..f8a4e11ce9c3b1 100644 --- a/Lib/abc.py +++ b/Lib/abc.py @@ -18,7 +18,7 @@ class that has a metaclass derived from ABCMeta cannot be class C(metaclass=ABCMeta): @abstractmethod - def my_abstract_method(self, ...): + def my_abstract_method(self, arg1, arg2, argN): ... """ funcobj.__isabstractmethod__ = True diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py index a5393aad4249c0..03ca2d7e18f6f0 100644 --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -45,6 +45,11 @@ else: _collections_abc.MutableSequence.register(deque) +try: + from _collections import _deque_iterator +except ImportError: + pass + try: from _collections import defaultdict except ImportError: diff --git a/Lib/csv.py b/Lib/csv.py index 4ef8be45ca9e0a..77f30c8d2b1f61 100644 --- a/Lib/csv.py +++ b/Lib/csv.py @@ -9,12 +9,14 @@ unregister_dialect, get_dialect, list_dialects, \ field_size_limit, \ QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC, QUOTE_NONE, \ + QUOTE_STRINGS, QUOTE_NOTNULL, \ __doc__ from _csv import Dialect as _Dialect from io import StringIO __all__ = ["QUOTE_MINIMAL", "QUOTE_ALL", "QUOTE_NONNUMERIC", "QUOTE_NONE", + "QUOTE_STRINGS", "QUOTE_NOTNULL", "Error", "Dialect", "__doc__", "excel", "excel_tab", "field_size_limit", "reader", "writer", "register_dialect", "get_dialect", "list_dialects", "Sniffer", diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py index de6c434450fc82..c0c757d94d8781 100644 --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -439,6 +439,7 @@ def _write_atomic(path, data, mode=0o666): # Python 3.12a7 3523 (Convert COMPARE_AND_BRANCH back to COMPARE_OP) # Python 3.12a7 3524 (Shrink the BINARY_SUBSCR caches) # Python 3.12b1 3525 (Shrink the CALL caches) +# Python 3.12a7 3526 (Add instrumentation support) # Python 3.13 will start with 3550 @@ -455,7 +456,7 @@ def _write_atomic(path, data, mode=0o666): # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3525).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3526).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c diff --git a/Lib/logging/config.py b/Lib/logging/config.py index 7cd16c643e9dad..16c54a6a4f7a2f 100644 --- a/Lib/logging/config.py +++ b/Lib/logging/config.py @@ -114,11 +114,18 @@ def _create_formatters(cp): fs = cp.get(sectname, "format", raw=True, fallback=None) dfs = cp.get(sectname, "datefmt", raw=True, fallback=None) stl = cp.get(sectname, "style", raw=True, fallback='%') + defaults = cp.get(sectname, "defaults", raw=True, fallback=None) + c = logging.Formatter class_name = cp[sectname].get("class") if class_name: c = _resolve(class_name) - f = c(fs, dfs, stl) + + if defaults is not None: + defaults = eval(defaults, vars(logging)) + f = c(fs, dfs, stl, defaults=defaults) + else: + f = c(fs, dfs, stl) formatters[form] = f return formatters @@ -668,18 +675,27 @@ def configure_formatter(self, config): dfmt = config.get('datefmt', None) style = config.get('style', '%') cname = config.get('class', None) + defaults = config.get('defaults', None) if not cname: c = logging.Formatter else: c = _resolve(cname) + kwargs = {} + + # Add defaults only if it exists. + # Prevents TypeError in custom formatter callables that do not + # accept it. + if defaults is not None: + kwargs['defaults'] = defaults + # A TypeError would be raised if "validate" key is passed in with a formatter callable # that does not accept "validate" as a parameter if 'validate' in config: # if user hasn't mentioned it, the default will be fine - result = c(fmt, dfmt, style, config['validate']) + result = c(fmt, dfmt, style, config['validate'], **kwargs) else: - result = c(fmt, dfmt, style) + result = c(fmt, dfmt, style, **kwargs) return result diff --git a/Lib/ntpath.py b/Lib/ntpath.py index 6e2da79c85d3f0..0f3674fe11eecd 100644 --- a/Lib/ntpath.py +++ b/Lib/ntpath.py @@ -142,7 +142,7 @@ def join(path, *paths): result_path = result_path + p_path ## add separator between UNC and non-absolute path if (result_path and not result_root and - result_drive and result_drive[-1:] != colon): + result_drive and result_drive[-1:] not in colon + seps): return result_drive + sep + result_path return result_drive + result_root + result_path except (TypeError, AttributeError, BytesWarning): diff --git a/Lib/opcode.py b/Lib/opcode.py index b62dfa1bcb42c5..dd739e5dd3f6f8 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -83,6 +83,7 @@ def pseudo_op(name, op, real_ops): def_op('INTERPRETER_EXIT', 3) def_op('END_FOR', 4) +def_op('END_SEND', 5) def_op('NOP', 9) @@ -91,6 +92,10 @@ def pseudo_op(name, op, real_ops): def_op('UNARY_INVERT', 15) +# We reserve 17 as it is the initial value for the specializing counter +# This helps us catch cases where we attempt to execute a cache. +def_op('RESERVED', 17) + def_op('BINARY_SUBSCR', 25) def_op('BINARY_SLICE', 26) def_op('STORE_SLICE', 27) @@ -221,6 +226,28 @@ def pseudo_op(name, op, real_ops): def_op('CALL_INTRINSIC_1', 173) def_op('CALL_INTRINSIC_2', 174) +# Instrumented instructions +MIN_INSTRUMENTED_OPCODE = 238 + +def_op('INSTRUMENTED_POP_JUMP_IF_NONE', 238) +def_op('INSTRUMENTED_POP_JUMP_IF_NOT_NONE', 239) +def_op('INSTRUMENTED_RESUME', 240) +def_op('INSTRUMENTED_CALL', 241) +def_op('INSTRUMENTED_RETURN_VALUE', 242) +def_op('INSTRUMENTED_YIELD_VALUE', 243) +def_op('INSTRUMENTED_CALL_FUNCTION_EX', 244) +def_op('INSTRUMENTED_JUMP_FORWARD', 245) +def_op('INSTRUMENTED_JUMP_BACKWARD', 246) +def_op('INSTRUMENTED_RETURN_CONST', 247) +def_op('INSTRUMENTED_FOR_ITER', 248) +def_op('INSTRUMENTED_POP_JUMP_IF_FALSE', 249) +def_op('INSTRUMENTED_POP_JUMP_IF_TRUE', 250) +def_op('INSTRUMENTED_END_FOR', 251) +def_op('INSTRUMENTED_END_SEND', 252) +def_op('INSTRUMENTED_INSTRUCTION', 253) +def_op('INSTRUMENTED_LINE', 254) +# 255 is reserved + hasarg.extend([op for op in opmap.values() if op >= HAVE_ARGUMENT]) MIN_PSEUDO_OPCODE = 256 diff --git a/Lib/pdb.py b/Lib/pdb.py index e03142e9b5d85c..a3553b345a8dd3 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -586,7 +586,7 @@ def _complete_expression(self, text, line, begidx, endidx): # Return true to exit from the command loop def do_commands(self, arg): - """commands [bpnumber] + """(Pdb) commands [bpnumber] (com) ... (com) end (Pdb) @@ -672,6 +672,7 @@ def do_commands(self, arg): def do_break(self, arg, temporary = 0): """b(reak) [ ([filename:]lineno | function) [, condition] ] + Without argument, list all breaks. With a line number argument, set a break at this line in the @@ -780,6 +781,7 @@ def defaultFile(self): def do_tbreak(self, arg): """tbreak [ ([filename:]lineno | function) [, condition] ] + Same arguments as break, but sets a temporary breakpoint: it is automatically deleted when first hit. """ @@ -844,6 +846,7 @@ def checkline(self, filename, lineno): def do_enable(self, arg): """enable bpnumber [bpnumber ...] + Enables the breakpoints given as a space separated list of breakpoint numbers. """ @@ -861,6 +864,7 @@ def do_enable(self, arg): def do_disable(self, arg): """disable bpnumber [bpnumber ...] + Disables the breakpoints given as a space separated list of breakpoint numbers. Disabling a breakpoint means it cannot cause the program to stop execution, but unlike clearing a @@ -881,6 +885,7 @@ def do_disable(self, arg): def do_condition(self, arg): """condition bpnumber [condition] + Set a new condition for the breakpoint, an expression which must evaluate to true before the breakpoint is honored. If condition is absent, any existing condition is removed; i.e., @@ -911,6 +916,7 @@ def do_condition(self, arg): def do_ignore(self, arg): """ignore bpnumber [count] + Set the ignore count for the given breakpoint number. If count is omitted, the ignore count is set to 0. A breakpoint becomes active when the ignore count is zero. When non-zero, @@ -945,7 +951,8 @@ def do_ignore(self, arg): complete_ignore = _complete_bpnumber def do_clear(self, arg): - """cl(ear) filename:lineno\ncl(ear) [bpnumber [bpnumber...]] + """cl(ear) [filename:lineno | bpnumber ...] + With a space separated list of breakpoint numbers, clear those breakpoints. Without argument, clear all breaks (but first ask confirmation). With a filename:lineno argument, @@ -997,6 +1004,7 @@ def do_clear(self, arg): def do_where(self, arg): """w(here) + Print a stack trace, with the most recent frame at the bottom. An arrow indicates the "current frame", which determines the context of most commands. 'bt' is an alias for this command. @@ -1015,6 +1023,7 @@ def _select_frame(self, number): def do_up(self, arg): """u(p) [count] + Move the current frame count (default one) levels up in the stack trace (to an older frame). """ @@ -1035,6 +1044,7 @@ def do_up(self, arg): def do_down(self, arg): """d(own) [count] + Move the current frame count (default one) levels down in the stack trace (to a newer frame). """ @@ -1055,6 +1065,7 @@ def do_down(self, arg): def do_until(self, arg): """unt(il) [lineno] + Without argument, continue execution until the line with a number greater than the current one is reached. With a line number, continue execution until a line with a number greater @@ -1079,6 +1090,7 @@ def do_until(self, arg): def do_step(self, arg): """s(tep) + Execute the current line, stop at the first possible occasion (either in a function that is called or in the current function). @@ -1089,6 +1101,7 @@ def do_step(self, arg): def do_next(self, arg): """n(ext) + Continue execution until the next line in the current function is reached or it returns. """ @@ -1098,6 +1111,7 @@ def do_next(self, arg): def do_run(self, arg): """run [args...] + Restart the debugged python program. If a string is supplied it is split with "shlex", and the result is used as the new sys.argv. History, breakpoints, actions and debugger options @@ -1119,6 +1133,7 @@ def do_run(self, arg): def do_return(self, arg): """r(eturn) + Continue execution until the current function returns. """ self.set_return(self.curframe) @@ -1127,6 +1142,7 @@ def do_return(self, arg): def do_continue(self, arg): """c(ont(inue)) + Continue execution, only stop when a breakpoint is encountered. """ if not self.nosigint: @@ -1145,6 +1161,7 @@ def do_continue(self, arg): def do_jump(self, arg): """j(ump) lineno + Set the next line that will be executed. Only available in the bottom-most frame. This lets you jump back and execute code again, or jump forward to skip code that you don't want @@ -1174,6 +1191,7 @@ def do_jump(self, arg): def do_debug(self, arg): """debug code + Enter a recursive debugger that steps through the code argument (which is an arbitrary expression or statement to be executed in the current environment). @@ -1195,7 +1213,8 @@ def do_debug(self, arg): complete_debug = _complete_expression def do_quit(self, arg): - """q(uit)\nexit + """q(uit) | exit + Quit from the debugger. The program being executed is aborted. """ self._user_requested_quit = True @@ -1207,6 +1226,7 @@ def do_quit(self, arg): def do_EOF(self, arg): """EOF + Handles the receipt of EOF as a command. """ self.message('') @@ -1216,6 +1236,7 @@ def do_EOF(self, arg): def do_args(self, arg): """a(rgs) + Print the argument list of the current function. """ co = self.curframe.f_code @@ -1233,6 +1254,7 @@ def do_args(self, arg): def do_retval(self, arg): """retval + Print the return value for the last return of a function. """ if '__return__' in self.curframe_locals: @@ -1273,12 +1295,14 @@ def _msg_val_func(self, arg, func): def do_p(self, arg): """p expression + Print the value of the expression. """ self._msg_val_func(arg, repr) def do_pp(self, arg): """pp expression + Pretty-print the value of the expression. """ self._msg_val_func(arg, pprint.pformat) @@ -1288,7 +1312,7 @@ def do_pp(self, arg): complete_pp = _complete_expression def do_list(self, arg): - """l(ist) [first [,last] | .] + """l(ist) [first[, last] | .] List source code for the current file. Without arguments, list 11 lines around the current line or continue the previous @@ -1345,7 +1369,8 @@ def do_list(self, arg): do_l = do_list def do_longlist(self, arg): - """longlist | ll + """ll | longlist + List the whole source code for the current function or frame. """ filename = self.curframe.f_code.co_filename @@ -1360,6 +1385,7 @@ def do_longlist(self, arg): def do_source(self, arg): """source expression + Try to get source code for the given object and display it. """ try: @@ -1397,7 +1423,8 @@ def _print_lines(self, lines, start, breaks=(), frame=None): self.message(s + '\t' + line.rstrip()) def do_whatis(self, arg): - """whatis arg + """whatis expression + Print the type of the argument. """ try: @@ -1485,7 +1512,8 @@ def do_interact(self, arg): code.interact("*interactive*", local=ns) def do_alias(self, arg): - """alias [name [command [parameter parameter ...] ]] + """alias [name [command]] + Create an alias called 'name' that executes 'command'. The command must *not* be enclosed in quotes. Replaceable parameters can be indicated by %1, %2, and so on, while %* is @@ -1521,6 +1549,7 @@ def do_alias(self, arg): def do_unalias(self, arg): """unalias name + Delete the specified alias. """ args = arg.split() @@ -1563,6 +1592,7 @@ def print_stack_entry(self, frame_lineno, prompt_prefix=line_prefix): def do_help(self, arg): """h(elp) + Without argument, print the list of available commands. With a command name as argument, print help about that command. "help pdb" shows the full pdb documentation. @@ -1586,12 +1616,13 @@ def do_help(self, arg): if command.__doc__ is None: self.error('No help for %r; __doc__ string missing' % arg) return - self.message(command.__doc__.rstrip()) + self.message(self._help_message_from_doc(command.__doc__)) do_h = do_help def help_exec(self): """(!) statement + Execute the (one-line) statement in the context of the current stack frame. The exclamation point can be omitted unless the first word of the statement resembles a debugger command. To @@ -1672,6 +1703,26 @@ def _getsourcelines(self, obj): lineno = max(1, lineno) return lines, lineno + def _help_message_from_doc(self, doc): + lines = [line.strip() for line in doc.rstrip().splitlines()] + if not lines: + return "No help message found." + if "" in lines: + usage_end = lines.index("") + else: + usage_end = 1 + formatted = [] + indent = " " * len(self.prompt) + for i, line in enumerate(lines): + if i == 0: + prefix = "Usage: " + elif i < usage_end: + prefix = " " + else: + prefix = "" + formatted.append(indent + prefix + line) + return "\n".join(formatted) + # Collect all command help into docstring, if not run with -OO if __doc__ is not None: diff --git a/Lib/shutil.py b/Lib/shutil.py index 8b378645a5a375..95b6c5299cab29 100644 --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -699,7 +699,7 @@ def rmtree(path, ignore_errors=False, onerror=None, *, onexc=None, dir_fd=None): if onerror is not None: warnings.warn("onerror argument is deprecated, use onexc instead", - DeprecationWarning) + DeprecationWarning, stacklevel=2) sys.audit("shutil.rmtree", path, dir_fd) if ignore_errors: diff --git a/Lib/test/test__opcode.py b/Lib/test/test__opcode.py index 31f3c53992db13..7640c6fb57d4f3 100644 --- a/Lib/test/test__opcode.py +++ b/Lib/test/test__opcode.py @@ -20,6 +20,8 @@ def test_stack_effect(self): # All defined opcodes has_arg = dis.hasarg for name, code in filter(lambda item: item[0] not in dis.deoptmap, dis.opmap.items()): + if code >= opcode.MIN_INSTRUMENTED_OPCODE: + continue with self.subTest(opname=name): if code not in has_arg: stack_effect(code) @@ -47,6 +49,8 @@ def test_stack_effect_jump(self): has_exc = dis.hasexc has_jump = dis.hasjabs + dis.hasjrel for name, code in filter(lambda item: item[0] not in dis.deoptmap, dis.opmap.items()): + if code >= opcode.MIN_INSTRUMENTED_OPCODE: + continue with self.subTest(opname=name): if code not in has_arg: common = stack_effect(code) diff --git a/Lib/test/test_bdb.py b/Lib/test/test_bdb.py index 042c2daea7f797..fc4b8094316332 100644 --- a/Lib/test/test_bdb.py +++ b/Lib/test/test_bdb.py @@ -433,8 +433,9 @@ def __exit__(self, type_=None, value=None, traceback=None): not_empty = '' if self.tracer.set_list: not_empty += 'All paired tuples have not been processed, ' - not_empty += ('the last one was number %d' % + not_empty += ('the last one was number %d\n' % self.tracer.expect_set_no) + not_empty += repr(self.tracer.set_list) # Make a BdbNotExpectedError a unittest failure. if type_ is not None and issubclass(BdbNotExpectedError, type_): diff --git a/Lib/test/test_code.py b/Lib/test/test_code.py index 7543c9ab342119..ecb3525a928468 100644 --- a/Lib/test/test_code.py +++ b/Lib/test/test_code.py @@ -349,14 +349,14 @@ def test_invalid_bytecode(self): def foo(): pass - # assert that opcode 238 is invalid - self.assertEqual(opname[238], '<238>') + # assert that opcode 229 is invalid + self.assertEqual(opname[229], '<229>') - # change first opcode to 0xee (=238) + # change first opcode to 0xeb (=229) foo.__code__ = foo.__code__.replace( - co_code=b'\xee' + foo.__code__.co_code[1:]) + co_code=b'\xe5' + foo.__code__.co_code[1:]) - msg = f"unknown opcode 238" + msg = f"unknown opcode 229" with self.assertRaisesRegex(SystemError, msg): foo() diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py index 8289ddb1c3a54f..8fb97bc0c1a1a7 100644 --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -187,6 +187,10 @@ def test_write_quoting(self): quoting = csv.QUOTE_ALL) self._write_test(['a\nb',1], '"a\nb","1"', quoting = csv.QUOTE_ALL) + self._write_test(['a','',None,1], '"a","",,1', + quoting = csv.QUOTE_STRINGS) + self._write_test(['a','',None,1], '"a","",,"1"', + quoting = csv.QUOTE_NOTNULL) def test_write_escape(self): self._write_test(['a',1,'p,q'], 'a,1,"p,q"', diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 4a2144743f6567..0a60a979614d52 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -479,8 +479,7 @@ async def _asyncwith(c): YIELD_VALUE 2 RESUME 3 JUMP_BACKWARD_NO_INTERRUPT 5 (to 14) - >> SWAP 2 - POP_TOP + >> END_SEND POP_TOP %3d LOAD_CONST 1 (1) @@ -492,11 +491,11 @@ async def _asyncwith(c): CALL 2 GET_AWAITABLE 2 LOAD_CONST 0 (None) - >> SEND 3 (to 62) + >> SEND 3 (to 60) YIELD_VALUE 2 RESUME 3 - JUMP_BACKWARD_NO_INTERRUPT 5 (to 52) - >> POP_TOP + JUMP_BACKWARD_NO_INTERRUPT 5 (to 50) + >> END_SEND POP_TOP %3d LOAD_CONST 2 (2) @@ -504,21 +503,20 @@ async def _asyncwith(c): RETURN_CONST 0 (None) %3d >> CLEANUP_THROW - JUMP_BACKWARD 26 (to 24) + JUMP_BACKWARD 25 (to 24) >> CLEANUP_THROW - JUMP_BACKWARD 9 (to 62) + JUMP_BACKWARD 9 (to 60) >> PUSH_EXC_INFO WITH_EXCEPT_START GET_AWAITABLE 2 LOAD_CONST 0 (None) - >> SEND 4 (to 100) + >> SEND 4 (to 98) YIELD_VALUE 3 RESUME 3 - JUMP_BACKWARD_NO_INTERRUPT 5 (to 88) + JUMP_BACKWARD_NO_INTERRUPT 5 (to 86) >> CLEANUP_THROW - >> SWAP 2 - POP_TOP - POP_JUMP_IF_TRUE 1 (to 108) + >> END_SEND + POP_JUMP_IF_TRUE 1 (to 104) RERAISE 2 >> POP_TOP POP_EXCEPT @@ -878,9 +876,9 @@ def test_boundaries(self): def test_widths(self): long_opcodes = set(['JUMP_BACKWARD_NO_INTERRUPT', - ]) + 'INSTRUMENTED_CALL_FUNCTION_EX']) for opcode, opname in enumerate(dis.opname): - if opname in long_opcodes: + if opname in long_opcodes or opname.startswith("INSTRUMENTED"): continue with self.subTest(opname=opname): width = dis._OPNAME_WIDTH diff --git a/Lib/test/test_exception_group.py b/Lib/test/test_exception_group.py index b11524e778e665..fa159a76ec1aff 100644 --- a/Lib/test/test_exception_group.py +++ b/Lib/test/test_exception_group.py @@ -102,6 +102,20 @@ class MyEG(BaseExceptionGroup, ValueError): with self.assertRaisesRegex(TypeError, msg): MyEG("eg", [ValueError(12), KeyboardInterrupt(42)]) + def test_EG_and_specific_subclass_can_wrap_any_nonbase_exception(self): + class MyEG(ExceptionGroup, ValueError): + pass + + # The restriction is specific to Exception, not "the other base class" + MyEG("eg", [ValueError(12), Exception()]) + + def test_BEG_and_specific_subclass_can_wrap_any_nonbase_exception(self): + class MyEG(BaseExceptionGroup, ValueError): + pass + + # The restriction is specific to Exception, not "the other base class" + MyEG("eg", [ValueError(12), Exception()]) + def test_BEG_subclass_wraps_anything(self): class MyBEG(BaseExceptionGroup): diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index 57db96d37ee369..af286052a7d560 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -2980,7 +2980,7 @@ class MyClass(metaclass=MyMeta): def test_reuse_different_names(self): """Disallow this case because decorated function a would not be cached.""" - with self.assertRaises(RuntimeError) as ctx: + with self.assertRaises(TypeError) as ctx: class ReusedCachedProperty: @py_functools.cached_property def a(self): @@ -2989,7 +2989,7 @@ def a(self): b = a self.assertEqual( - str(ctx.exception.__context__), + str(ctx.exception), str(TypeError("Cannot assign the same cached_property to two different names ('a' and 'b').")) ) diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index c6de34e9dbdc8f..9176d8eeb56d01 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -1524,6 +1524,32 @@ class ConfigFileTest(BaseTest): kwargs={{"encoding": "utf-8"}} """ + + config9 = """ + [loggers] + keys=root + + [handlers] + keys=hand1 + + [formatters] + keys=form1 + + [logger_root] + level=WARNING + handlers=hand1 + + [handler_hand1] + class=StreamHandler + level=NOTSET + formatter=form1 + args=(sys.stdout,) + + [formatter_form1] + format=%(message)s ++ %(customfield)s + defaults={"customfield": "defaultvalue"} + """ + disable_test = """ [loggers] keys=root @@ -1687,6 +1713,16 @@ def test_config8_ok(self): handler = logging.root.handlers[0] self.addCleanup(closeFileHandler, handler, fn) + def test_config9_ok(self): + self.apply_config(self.config9) + formatter = logging.root.handlers[0].formatter + result = formatter.format(logging.makeLogRecord({'msg': 'test'})) + self.assertEqual(result, 'test ++ defaultvalue') + result = formatter.format(logging.makeLogRecord( + {'msg': 'test', 'customfield': "customvalue"})) + self.assertEqual(result, 'test ++ customvalue') + + def test_logger_disabling(self): self.apply_config(self.disable_test) logger = logging.getLogger('some_pristine_logger') @@ -2909,6 +2945,30 @@ class ConfigDictTest(BaseTest): }, } + # config0 but with default values for formatter. Skipped 15, it is defined + # in the test code. + config16 = { + 'version': 1, + 'formatters': { + 'form1' : { + 'format' : '%(message)s ++ %(customfield)s', + 'defaults': {"customfield": "defaultvalue"} + }, + }, + 'handlers' : { + 'hand1' : { + 'class' : 'logging.StreamHandler', + 'formatter' : 'form1', + 'level' : 'NOTSET', + 'stream' : 'ext://sys.stdout', + }, + }, + 'root' : { + 'level' : 'WARNING', + 'handlers' : ['hand1'], + }, + } + bad_format = { "version": 1, "formatters": { @@ -3021,7 +3081,7 @@ class ConfigDictTest(BaseTest): } } - # Configuration with custom function and 'validate' set to False + # Configuration with custom function, 'validate' set to False and no defaults custom_formatter_with_function = { 'version': 1, 'formatters': { @@ -3048,6 +3108,33 @@ class ConfigDictTest(BaseTest): } } + # Configuration with custom function, and defaults + custom_formatter_with_defaults = { + 'version': 1, + 'formatters': { + 'form1': { + '()': formatFunc, + 'format': '%(levelname)s:%(name)s:%(message)s:%(customfield)s', + 'defaults': {"customfield": "myvalue"} + }, + }, + 'handlers' : { + 'hand1' : { + 'class': 'logging.StreamHandler', + 'formatter': 'form1', + 'level': 'NOTSET', + 'stream': 'ext://sys.stdout', + }, + }, + "loggers": { + "my_test_logger_custom_formatter": { + "level": "DEBUG", + "handlers": ["hand1"], + "propagate": "true" + } + } + } + config_queue_handler = { 'version': 1, 'handlers' : { @@ -3349,6 +3436,22 @@ def test_config15_ok(self): handler = logging.root.handlers[0] self.addCleanup(closeFileHandler, handler, fn) + def test_config16_ok(self): + self.apply_config(self.config16) + h = logging._handlers['hand1'] + + # Custom value + result = h.formatter.format(logging.makeLogRecord( + {'msg': 'Hello', 'customfield': 'customvalue'})) + self.assertEqual(result, 'Hello ++ customvalue') + + # Default value + result = h.formatter.format(logging.makeLogRecord( + {'msg': 'Hello'})) + self.assertEqual(result, 'Hello ++ defaultvalue') + + + def setup_via_listener(self, text, verify=None): text = text.encode("utf-8") # Ask for a randomly assigned port (by using port 0) @@ -3516,6 +3619,9 @@ def test_custom_formatter_class_with_validate3(self): def test_custom_formatter_function_with_validate(self): self.assertRaises(ValueError, self.apply_config, self.custom_formatter_with_function) + def test_custom_formatter_function_with_defaults(self): + self.assertRaises(ValueError, self.apply_config, self.custom_formatter_with_defaults) + def test_baseconfig(self): d = { 'atuple': (1, 2, 3), diff --git a/Lib/test/test_monitoring.py b/Lib/test/test_monitoring.py new file mode 100644 index 00000000000000..4aad3da61b6bee --- /dev/null +++ b/Lib/test/test_monitoring.py @@ -0,0 +1,1044 @@ +"""Test suite for the sys.monitoring.""" + +import collections +import functools +import operator +import sys +import types +import unittest + + +PAIR = (0,1) + +def f1(): + pass + +def f2(): + len([]) + sys.getsizeof(0) + +def floop(): + for item in PAIR: + pass + +def gen(): + yield + yield + +def g1(): + for _ in gen(): + pass + +TEST_TOOL = 2 +TEST_TOOL2 = 3 +TEST_TOOL3 = 4 + +class MonitoringBasicTest(unittest.TestCase): + + def test_has_objects(self): + m = sys.monitoring + m.events + m.use_tool_id + m.free_tool_id + m.get_tool + m.get_events + m.set_events + m.get_local_events + m.set_local_events + m.register_callback + m.restart_events + m.DISABLE + m.MISSING + m.events.NO_EVENTS + + def test_tool(self): + sys.monitoring.use_tool_id(TEST_TOOL, "MonitoringTest.Tool") + self.assertEqual(sys.monitoring.get_tool(TEST_TOOL), "MonitoringTest.Tool") + sys.monitoring.set_events(TEST_TOOL, 15) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL), 15) + sys.monitoring.set_events(TEST_TOOL, 0) + with self.assertRaises(ValueError): + sys.monitoring.set_events(TEST_TOOL, sys.monitoring.events.C_RETURN) + with self.assertRaises(ValueError): + sys.monitoring.set_events(TEST_TOOL, sys.monitoring.events.C_RAISE) + sys.monitoring.free_tool_id(TEST_TOOL) + self.assertEqual(sys.monitoring.get_tool(TEST_TOOL), None) + with self.assertRaises(ValueError): + sys.monitoring.set_events(TEST_TOOL, sys.monitoring.events.CALL) + + +class MonitoringTestBase: + + def setUp(self): + # Check that a previous test hasn't left monitoring on. + for tool in range(6): + self.assertEqual(sys.monitoring.get_events(tool), 0) + self.assertIs(sys.monitoring.get_tool(TEST_TOOL), None) + self.assertIs(sys.monitoring.get_tool(TEST_TOOL2), None) + self.assertIs(sys.monitoring.get_tool(TEST_TOOL3), None) + sys.monitoring.use_tool_id(TEST_TOOL, "test " + self.__class__.__name__) + sys.monitoring.use_tool_id(TEST_TOOL2, "test2 " + self.__class__.__name__) + sys.monitoring.use_tool_id(TEST_TOOL3, "test3 " + self.__class__.__name__) + + def tearDown(self): + # Check that test hasn't left monitoring on. + for tool in range(6): + self.assertEqual(sys.monitoring.get_events(tool), 0) + sys.monitoring.free_tool_id(TEST_TOOL) + sys.monitoring.free_tool_id(TEST_TOOL2) + sys.monitoring.free_tool_id(TEST_TOOL3) + + +class MonitoringCountTest(MonitoringTestBase, unittest.TestCase): + + def check_event_count(self, func, event, expected): + + class Counter: + def __init__(self): + self.count = 0 + def __call__(self, *args): + self.count += 1 + + counter = Counter() + sys.monitoring.register_callback(TEST_TOOL, event, counter) + if event == E.C_RETURN or event == E.C_RAISE: + sys.monitoring.set_events(TEST_TOOL, E.CALL) + else: + sys.monitoring.set_events(TEST_TOOL, event) + self.assertEqual(counter.count, 0) + counter.count = 0 + func() + self.assertEqual(counter.count, expected) + prev = sys.monitoring.register_callback(TEST_TOOL, event, None) + counter.count = 0 + func() + self.assertEqual(counter.count, 0) + self.assertEqual(prev, counter) + sys.monitoring.set_events(TEST_TOOL, 0) + + def test_start_count(self): + self.check_event_count(f1, E.PY_START, 1) + + def test_resume_count(self): + self.check_event_count(g1, E.PY_RESUME, 2) + + def test_return_count(self): + self.check_event_count(f1, E.PY_RETURN, 1) + + def test_call_count(self): + self.check_event_count(f2, E.CALL, 3) + + def test_c_return_count(self): + self.check_event_count(f2, E.C_RETURN, 2) + + +E = sys.monitoring.events + +SIMPLE_EVENTS = [ + (E.PY_START, "start"), + (E.PY_RESUME, "resume"), + (E.PY_RETURN, "return"), + (E.PY_YIELD, "yield"), + (E.JUMP, "jump"), + (E.BRANCH, "branch"), + (E.RAISE, "raise"), + (E.PY_UNWIND, "unwind"), + (E.EXCEPTION_HANDLED, "exception_handled"), + (E.C_RAISE, "c_raise"), + (E.C_RETURN, "c_return"), +] + +SIMPLE_EVENT_SET = functools.reduce(operator.or_, [ev for (ev, _) in SIMPLE_EVENTS], 0) | E.CALL + + +def just_pass(): + pass + +just_pass.events = [ + "py_call", + "start", + "return", +] + +def just_raise(): + raise Exception + +just_raise.events = [ + 'py_call', + "start", + "raise", + "unwind", +] + +def just_call(): + len([]) + +just_call.events = [ + 'py_call', + "start", + "c_call", + "c_return", + "return", +] + +def caught(): + try: + 1/0 + except Exception: + pass + +caught.events = [ + 'py_call', + "start", + "raise", + "exception_handled", + "branch", + "return", +] + +def nested_call(): + just_pass() + +nested_call.events = [ + "py_call", + "start", + "py_call", + "start", + "return", + "return", +] + +PY_CALLABLES = (types.FunctionType, types.MethodType) + +class MonitoringEventsBase(MonitoringTestBase): + + def gather_events(self, func): + events = [] + for event, event_name in SIMPLE_EVENTS: + def record(*args, event_name=event_name): + events.append(event_name) + sys.monitoring.register_callback(TEST_TOOL, event, record) + def record_call(code, offset, obj, arg): + if isinstance(obj, PY_CALLABLES): + events.append("py_call") + else: + events.append("c_call") + sys.monitoring.register_callback(TEST_TOOL, E.CALL, record_call) + sys.monitoring.set_events(TEST_TOOL, SIMPLE_EVENT_SET) + events = [] + try: + func() + except: + pass + sys.monitoring.set_events(TEST_TOOL, 0) + #Remove the final event, the call to `sys.monitoring.set_events` + events = events[:-1] + return events + + def check_events(self, func, expected=None): + events = self.gather_events(func) + if expected is None: + expected = func.events + self.assertEqual(events, expected) + + +class MonitoringEventsTest(MonitoringEventsBase, unittest.TestCase): + + def test_just_pass(self): + self.check_events(just_pass) + + def test_just_raise(self): + try: + self.check_events(just_raise) + except Exception: + pass + self.assertEqual(sys.monitoring.get_events(TEST_TOOL), 0) + + def test_just_call(self): + self.check_events(just_call) + + def test_caught(self): + self.check_events(caught) + + def test_nested_call(self): + self.check_events(nested_call) + +UP_EVENTS = (E.C_RETURN, E.C_RAISE, E.PY_RETURN, E.PY_UNWIND, E.PY_YIELD) +DOWN_EVENTS = (E.PY_START, E.PY_RESUME) + +from test.profilee import testfunc + +class SimulateProfileTest(MonitoringEventsBase, unittest.TestCase): + + def test_balanced(self): + events = self.gather_events(testfunc) + c = collections.Counter(events) + self.assertEqual(c["c_call"], c["c_return"]) + self.assertEqual(c["start"], c["return"] + c["unwind"]) + self.assertEqual(c["raise"], c["exception_handled"] + c["unwind"]) + + def test_frame_stack(self): + self.maxDiff = None + stack = [] + errors = [] + seen = set() + def up(*args): + frame = sys._getframe(1) + if not stack: + errors.append("empty") + else: + expected = stack.pop() + if frame != expected: + errors.append(f" Popping {frame} expected {expected}") + def down(*args): + frame = sys._getframe(1) + stack.append(frame) + seen.add(frame.f_code) + def call(code, offset, callable, arg): + if not isinstance(callable, PY_CALLABLES): + stack.append(sys._getframe(1)) + for event in UP_EVENTS: + sys.monitoring.register_callback(TEST_TOOL, event, up) + for event in DOWN_EVENTS: + sys.monitoring.register_callback(TEST_TOOL, event, down) + sys.monitoring.register_callback(TEST_TOOL, E.CALL, call) + sys.monitoring.set_events(TEST_TOOL, SIMPLE_EVENT_SET) + testfunc() + sys.monitoring.set_events(TEST_TOOL, 0) + self.assertEqual(errors, []) + self.assertEqual(stack, [sys._getframe()]) + self.assertEqual(len(seen), 9) + + +class CounterWithDisable: + + def __init__(self): + self.disable = False + self.count = 0 + + def __call__(self, *args): + self.count += 1 + if self.disable: + return sys.monitoring.DISABLE + + +class RecorderWithDisable: + + def __init__(self, events): + self.disable = False + self.events = events + + def __call__(self, code, event): + self.events.append(event) + if self.disable: + return sys.monitoring.DISABLE + + +class MontoringDisableAndRestartTest(MonitoringTestBase, unittest.TestCase): + + def test_disable(self): + try: + counter = CounterWithDisable() + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, counter) + sys.monitoring.set_events(TEST_TOOL, E.PY_START) + self.assertEqual(counter.count, 0) + counter.count = 0 + f1() + self.assertEqual(counter.count, 1) + counter.disable = True + counter.count = 0 + f1() + self.assertEqual(counter.count, 1) + counter.count = 0 + f1() + self.assertEqual(counter.count, 0) + sys.monitoring.set_events(TEST_TOOL, 0) + finally: + sys.monitoring.restart_events() + + def test_restart(self): + try: + counter = CounterWithDisable() + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, counter) + sys.monitoring.set_events(TEST_TOOL, E.PY_START) + counter.disable = True + f1() + counter.count = 0 + f1() + self.assertEqual(counter.count, 0) + sys.monitoring.restart_events() + counter.count = 0 + f1() + self.assertEqual(counter.count, 1) + sys.monitoring.set_events(TEST_TOOL, 0) + finally: + sys.monitoring.restart_events() + + +class MultipleMonitorsTest(MonitoringTestBase, unittest.TestCase): + + def test_two_same(self): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + counter1 = CounterWithDisable() + counter2 = CounterWithDisable() + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, counter1) + sys.monitoring.register_callback(TEST_TOOL2, E.PY_START, counter2) + sys.monitoring.set_events(TEST_TOOL, E.PY_START) + sys.monitoring.set_events(TEST_TOOL2, E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL), E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL2), E.PY_START) + self.assertEqual(sys.monitoring._all_events(), {'PY_START': (1 << TEST_TOOL) | (1 << TEST_TOOL2)}) + counter1.count = 0 + counter2.count = 0 + f1() + count1 = counter1.count + count2 = counter2.count + self.assertEqual((count1, count2), (1, 1)) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.set_events(TEST_TOOL2, 0) + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, None) + sys.monitoring.register_callback(TEST_TOOL2, E.PY_START, None) + self.assertEqual(sys.monitoring._all_events(), {}) + + def test_three_same(self): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + counter1 = CounterWithDisable() + counter2 = CounterWithDisable() + counter3 = CounterWithDisable() + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, counter1) + sys.monitoring.register_callback(TEST_TOOL2, E.PY_START, counter2) + sys.monitoring.register_callback(TEST_TOOL3, E.PY_START, counter3) + sys.monitoring.set_events(TEST_TOOL, E.PY_START) + sys.monitoring.set_events(TEST_TOOL2, E.PY_START) + sys.monitoring.set_events(TEST_TOOL3, E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL), E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL2), E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL3), E.PY_START) + self.assertEqual(sys.monitoring._all_events(), {'PY_START': (1 << TEST_TOOL) | (1 << TEST_TOOL2) | (1 << TEST_TOOL3)}) + counter1.count = 0 + counter2.count = 0 + counter3.count = 0 + f1() + count1 = counter1.count + count2 = counter2.count + count3 = counter3.count + self.assertEqual((count1, count2, count3), (1, 1, 1)) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.set_events(TEST_TOOL2, 0) + sys.monitoring.set_events(TEST_TOOL3, 0) + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, None) + sys.monitoring.register_callback(TEST_TOOL2, E.PY_START, None) + sys.monitoring.register_callback(TEST_TOOL3, E.PY_START, None) + self.assertEqual(sys.monitoring._all_events(), {}) + + def test_two_different(self): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + counter1 = CounterWithDisable() + counter2 = CounterWithDisable() + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, counter1) + sys.monitoring.register_callback(TEST_TOOL2, E.PY_RETURN, counter2) + sys.monitoring.set_events(TEST_TOOL, E.PY_START) + sys.monitoring.set_events(TEST_TOOL2, E.PY_RETURN) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL), E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL2), E.PY_RETURN) + self.assertEqual(sys.monitoring._all_events(), {'PY_START': 1 << TEST_TOOL, 'PY_RETURN': 1 << TEST_TOOL2}) + counter1.count = 0 + counter2.count = 0 + f1() + count1 = counter1.count + count2 = counter2.count + self.assertEqual((count1, count2), (1, 1)) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.set_events(TEST_TOOL2, 0) + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, None) + sys.monitoring.register_callback(TEST_TOOL2, E.PY_RETURN, None) + self.assertEqual(sys.monitoring._all_events(), {}) + + def test_two_with_disable(self): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + counter1 = CounterWithDisable() + counter2 = CounterWithDisable() + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, counter1) + sys.monitoring.register_callback(TEST_TOOL2, E.PY_START, counter2) + sys.monitoring.set_events(TEST_TOOL, E.PY_START) + sys.monitoring.set_events(TEST_TOOL2, E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL), E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL2), E.PY_START) + self.assertEqual(sys.monitoring._all_events(), {'PY_START': (1 << TEST_TOOL) | (1 << TEST_TOOL2)}) + counter1.count = 0 + counter2.count = 0 + counter1.disable = True + f1() + count1 = counter1.count + count2 = counter2.count + self.assertEqual((count1, count2), (1, 1)) + counter1.count = 0 + counter2.count = 0 + f1() + count1 = counter1.count + count2 = counter2.count + self.assertEqual((count1, count2), (0, 1)) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.set_events(TEST_TOOL2, 0) + sys.monitoring.register_callback(TEST_TOOL, E.PY_START, None) + sys.monitoring.register_callback(TEST_TOOL2, E.PY_START, None) + self.assertEqual(sys.monitoring._all_events(), {}) + sys.monitoring.restart_events() + +class LineMonitoringTest(MonitoringTestBase, unittest.TestCase): + + def test_lines_single(self): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + events = [] + recorder = RecorderWithDisable(events) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, recorder) + sys.monitoring.set_events(TEST_TOOL, E.LINE) + f1() + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, None) + start = LineMonitoringTest.test_lines_single.__code__.co_firstlineno + self.assertEqual(events, [start+7, 14, start+8]) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, None) + self.assertEqual(sys.monitoring._all_events(), {}) + sys.monitoring.restart_events() + + def test_lines_loop(self): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + events = [] + recorder = RecorderWithDisable(events) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, recorder) + sys.monitoring.set_events(TEST_TOOL, E.LINE) + floop() + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, None) + start = LineMonitoringTest.test_lines_loop.__code__.co_firstlineno + self.assertEqual(events, [start+7, 21, 22, 22, 21, start+8]) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, None) + self.assertEqual(sys.monitoring._all_events(), {}) + sys.monitoring.restart_events() + + def test_lines_two(self): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + events = [] + recorder = RecorderWithDisable(events) + events2 = [] + recorder2 = RecorderWithDisable(events2) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, recorder) + sys.monitoring.register_callback(TEST_TOOL2, E.LINE, recorder2) + sys.monitoring.set_events(TEST_TOOL, E.LINE); sys.monitoring.set_events(TEST_TOOL2, E.LINE) + f1() + sys.monitoring.set_events(TEST_TOOL, 0); sys.monitoring.set_events(TEST_TOOL2, 0) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, None) + sys.monitoring.register_callback(TEST_TOOL2, E.LINE, None) + start = LineMonitoringTest.test_lines_two.__code__.co_firstlineno + expected = [start+10, 14, start+11] + self.assertEqual(events, expected) + self.assertEqual(events2, expected) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.set_events(TEST_TOOL2, 0) + sys.monitoring.register_callback(TEST_TOOL, E.LINE, None) + sys.monitoring.register_callback(TEST_TOOL2, E.LINE, None) + self.assertEqual(sys.monitoring._all_events(), {}) + sys.monitoring.restart_events() + + def check_lines(self, func, expected, tool=TEST_TOOL): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + events = [] + recorder = RecorderWithDisable(events) + sys.monitoring.register_callback(tool, E.LINE, recorder) + sys.monitoring.set_events(tool, E.LINE) + func() + sys.monitoring.set_events(tool, 0) + sys.monitoring.register_callback(tool, E.LINE, None) + lines = [ line - func.__code__.co_firstlineno for line in events[1:-1] ] + self.assertEqual(lines, expected) + finally: + sys.monitoring.set_events(tool, 0) + + + def test_linear(self): + + def func(): + line = 1 + line = 2 + line = 3 + line = 4 + line = 5 + + self.check_lines(func, [1,2,3,4,5]) + + def test_branch(self): + def func(): + if "true".startswith("t"): + line = 2 + line = 3 + else: + line = 5 + line = 6 + + self.check_lines(func, [1,2,3,6]) + + def test_try_except(self): + + def func1(): + try: + line = 2 + line = 3 + except: + line = 5 + line = 6 + + self.check_lines(func1, [1,2,3,6]) + + def func2(): + try: + line = 2 + raise 3 + except: + line = 5 + line = 6 + + self.check_lines(func2, [1,2,3,4,5,6]) + + +class ExceptionRecorder: + + event_type = E.RAISE + + def __init__(self, events): + self.events = events + + def __call__(self, code, offset, exc): + self.events.append(("raise", type(exc))) + +class CheckEvents(MonitoringTestBase, unittest.TestCase): + + def check_events(self, func, expected, tool=TEST_TOOL, recorders=(ExceptionRecorder,)): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + event_list = [] + all_events = 0 + for recorder in recorders: + ev = recorder.event_type + sys.monitoring.register_callback(tool, ev, recorder(event_list)) + all_events |= ev + sys.monitoring.set_events(tool, all_events) + func() + sys.monitoring.set_events(tool, 0) + for recorder in recorders: + sys.monitoring.register_callback(tool, recorder.event_type, None) + self.assertEqual(event_list, expected) + finally: + sys.monitoring.set_events(tool, 0) + for recorder in recorders: + sys.monitoring.register_callback(tool, recorder.event_type, None) + +class StopiterationRecorder(ExceptionRecorder): + + event_type = E.STOP_ITERATION + +class ExceptionMontoringTest(CheckEvents): + + recorder = ExceptionRecorder + + def test_simple_try_except(self): + + def func1(): + try: + line = 2 + raise KeyError + except: + line = 5 + line = 6 + + self.check_events(func1, [("raise", KeyError)]) + + def gen(): + yield 1 + return 2 + + def implicit_stop_iteration(): + for _ in gen(): + pass + + self.check_events(implicit_stop_iteration, [("raise", StopIteration)], recorders=(StopiterationRecorder,)) + +class LineRecorder: + + event_type = E.LINE + + + def __init__(self, events): + self.events = events + + def __call__(self, code, line): + self.events.append(("line", code.co_name, line - code.co_firstlineno)) + +class CallRecorder: + + event_type = E.CALL + + def __init__(self, events): + self.events = events + + def __call__(self, code, offset, func, arg): + self.events.append(("call", func.__name__, arg)) + +class CEventRecorder: + + def __init__(self, events): + self.events = events + + def __call__(self, code, offset, func, arg): + self.events.append((self.event_name, func.__name__, arg)) + +class CReturnRecorder(CEventRecorder): + + event_type = E.C_RETURN + event_name = "C return" + +class CRaiseRecorder(CEventRecorder): + + event_type = E.C_RAISE + event_name = "C raise" + +MANY_RECORDERS = ExceptionRecorder, CallRecorder, LineRecorder, CReturnRecorder, CRaiseRecorder + +class TestManyEvents(CheckEvents): + + def test_simple(self): + + def func1(): + line1 = 1 + line2 = 2 + line3 = 3 + + self.check_events(func1, recorders = MANY_RECORDERS, expected = [ + ('line', 'check_events', 10), + ('call', 'func1', sys.monitoring.MISSING), + ('line', 'func1', 1), + ('line', 'func1', 2), + ('line', 'func1', 3), + ('line', 'check_events', 11), + ('call', 'set_events', 2)]) + + def test_c_call(self): + + def func2(): + line1 = 1 + [].append(2) + line3 = 3 + + self.check_events(func2, recorders = MANY_RECORDERS, expected = [ + ('line', 'check_events', 10), + ('call', 'func2', sys.monitoring.MISSING), + ('line', 'func2', 1), + ('line', 'func2', 2), + ('call', 'append', [2]), + ('C return', 'append', [2]), + ('line', 'func2', 3), + ('line', 'check_events', 11), + ('call', 'set_events', 2)]) + + def test_try_except(self): + + def func3(): + try: + line = 2 + raise KeyError + except: + line = 5 + line = 6 + + self.check_events(func3, recorders = MANY_RECORDERS, expected = [ + ('line', 'check_events', 10), + ('call', 'func3', sys.monitoring.MISSING), + ('line', 'func3', 1), + ('line', 'func3', 2), + ('line', 'func3', 3), + ('raise', KeyError), + ('line', 'func3', 4), + ('line', 'func3', 5), + ('line', 'func3', 6), + ('line', 'check_events', 11), + ('call', 'set_events', 2)]) + +class InstructionRecorder: + + event_type = E.INSTRUCTION + + def __init__(self, events): + self.events = events + + def __call__(self, code, offset): + # Filter out instructions in check_events to lower noise + if code.co_name != "check_events": + self.events.append(("instruction", code.co_name, offset)) + + +LINE_AND_INSTRUCTION_RECORDERS = InstructionRecorder, LineRecorder + +class TestLineAndInstructionEvents(CheckEvents): + maxDiff = None + + def test_simple(self): + + def func1(): + line1 = 1 + line2 = 2 + line3 = 3 + + self.check_events(func1, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ + ('line', 'check_events', 10), + ('line', 'func1', 1), + ('instruction', 'func1', 2), + ('instruction', 'func1', 4), + ('line', 'func1', 2), + ('instruction', 'func1', 6), + ('instruction', 'func1', 8), + ('line', 'func1', 3), + ('instruction', 'func1', 10), + ('instruction', 'func1', 12), + ('instruction', 'func1', 14), + ('line', 'check_events', 11)]) + + def test_c_call(self): + + def func2(): + line1 = 1 + [].append(2) + line3 = 3 + + self.check_events(func2, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ + ('line', 'check_events', 10), + ('line', 'func2', 1), + ('instruction', 'func2', 2), + ('instruction', 'func2', 4), + ('line', 'func2', 2), + ('instruction', 'func2', 6), + ('instruction', 'func2', 8), + ('instruction', 'func2', 28), + ('instruction', 'func2', 30), + ('instruction', 'func2', 38), + ('line', 'func2', 3), + ('instruction', 'func2', 40), + ('instruction', 'func2', 42), + ('instruction', 'func2', 44), + ('line', 'check_events', 11)]) + + def test_try_except(self): + + def func3(): + try: + line = 2 + raise KeyError + except: + line = 5 + line = 6 + + self.check_events(func3, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ + ('line', 'check_events', 10), + ('line', 'func3', 1), + ('instruction', 'func3', 2), + ('line', 'func3', 2), + ('instruction', 'func3', 4), + ('instruction', 'func3', 6), + ('line', 'func3', 3), + ('instruction', 'func3', 8), + ('instruction', 'func3', 18), + ('instruction', 'func3', 20), + ('line', 'func3', 4), + ('instruction', 'func3', 22), + ('line', 'func3', 5), + ('instruction', 'func3', 24), + ('instruction', 'func3', 26), + ('instruction', 'func3', 28), + ('line', 'func3', 6), + ('instruction', 'func3', 30), + ('instruction', 'func3', 32), + ('instruction', 'func3', 34), + ('line', 'check_events', 11)]) + +class TestInstallIncrementallly(MonitoringTestBase, unittest.TestCase): + + def check_events(self, func, must_include, tool=TEST_TOOL, recorders=(ExceptionRecorder,)): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + event_list = [] + all_events = 0 + for recorder in recorders: + all_events |= recorder.event_type + sys.monitoring.set_events(tool, all_events) + for recorder in recorders: + sys.monitoring.register_callback(tool, recorder.event_type, recorder(event_list)) + func() + sys.monitoring.set_events(tool, 0) + for recorder in recorders: + sys.monitoring.register_callback(tool, recorder.event_type, None) + for line in must_include: + self.assertIn(line, event_list) + finally: + sys.monitoring.set_events(tool, 0) + for recorder in recorders: + sys.monitoring.register_callback(tool, recorder.event_type, None) + + @staticmethod + def func1(): + line1 = 1 + + MUST_INCLUDE_LI = [ + ('instruction', 'func1', 2), + ('line', 'func1', 1), + ('instruction', 'func1', 4), + ('instruction', 'func1', 6)] + + def test_line_then_instruction(self): + recorders = [ LineRecorder, InstructionRecorder ] + self.check_events(self.func1, + recorders = recorders, must_include = self.EXPECTED_LI) + + def test_instruction_then_line(self): + recorders = [ InstructionRecorder, LineRecorderLowNoise ] + self.check_events(self.func1, + recorders = recorders, must_include = self.EXPECTED_LI) + + @staticmethod + def func2(): + len(()) + + MUST_INCLUDE_CI = [ + ('instruction', 'func2', 2), + ('call', 'func2', sys.monitoring.MISSING), + ('call', 'len', ()), + ('instruction', 'func2', 12), + ('instruction', 'func2', 14)] + + + + def test_line_then_instruction(self): + recorders = [ CallRecorder, InstructionRecorder ] + self.check_events(self.func2, + recorders = recorders, must_include = self.MUST_INCLUDE_CI) + + def test_instruction_then_line(self): + recorders = [ InstructionRecorder, CallRecorder ] + self.check_events(self.func2, + recorders = recorders, must_include = self.MUST_INCLUDE_CI) + +class TestLocalEvents(MonitoringTestBase, unittest.TestCase): + + def check_events(self, func, expected, tool=TEST_TOOL, recorders=(ExceptionRecorder,)): + try: + self.assertEqual(sys.monitoring._all_events(), {}) + event_list = [] + all_events = 0 + for recorder in recorders: + ev = recorder.event_type + sys.monitoring.register_callback(tool, ev, recorder(event_list)) + all_events |= ev + sys.monitoring.set_local_events(tool, func.__code__, all_events) + func() + sys.monitoring.set_local_events(tool, func.__code__, 0) + for recorder in recorders: + sys.monitoring.register_callback(tool, recorder.event_type, None) + self.assertEqual(event_list, expected) + finally: + sys.monitoring.set_local_events(tool, func.__code__, 0) + for recorder in recorders: + sys.monitoring.register_callback(tool, recorder.event_type, None) + + + def test_simple(self): + + def func1(): + line1 = 1 + line2 = 2 + line3 = 3 + + self.check_events(func1, recorders = MANY_RECORDERS, expected = [ + ('line', 'func1', 1), + ('line', 'func1', 2), + ('line', 'func1', 3)]) + + def test_c_call(self): + + def func2(): + line1 = 1 + [].append(2) + line3 = 3 + + self.check_events(func2, recorders = MANY_RECORDERS, expected = [ + ('line', 'func2', 1), + ('line', 'func2', 2), + ('call', 'append', [2]), + ('C return', 'append', [2]), + ('line', 'func2', 3)]) + + def test_try_except(self): + + def func3(): + try: + line = 2 + raise KeyError + except: + line = 5 + line = 6 + + self.check_events(func3, recorders = MANY_RECORDERS, expected = [ + ('line', 'func3', 1), + ('line', 'func3', 2), + ('line', 'func3', 3), + ('raise', KeyError), + ('line', 'func3', 4), + ('line', 'func3', 5), + ('line', 'func3', 6)]) + + +class TestSetGetEvents(MonitoringTestBase, unittest.TestCase): + + def test_global(self): + sys.monitoring.set_events(TEST_TOOL, E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL), E.PY_START) + sys.monitoring.set_events(TEST_TOOL2, E.PY_START) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL2), E.PY_START) + sys.monitoring.set_events(TEST_TOOL, 0) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL), 0) + sys.monitoring.set_events(TEST_TOOL2,0) + self.assertEqual(sys.monitoring.get_events(TEST_TOOL2), 0) + + def test_local(self): + code = f1.__code__ + sys.monitoring.set_local_events(TEST_TOOL, code, E.PY_START) + self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL, code), E.PY_START) + sys.monitoring.set_local_events(TEST_TOOL2, code, E.PY_START) + self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL2, code), E.PY_START) + sys.monitoring.set_local_events(TEST_TOOL, code, 0) + self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL, code), 0) + sys.monitoring.set_local_events(TEST_TOOL2, code, 0) + self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL2, code), 0) + +class TestUninitialized(unittest.TestCase, MonitoringTestBase): + + @staticmethod + def f(): + pass + + def test_get_local_events_uninitialized(self): + self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL, self.f.__code__), 0) diff --git a/Lib/test/test_ntpath.py b/Lib/test/test_ntpath.py index 4e755d15403916..42b9587ca18107 100644 --- a/Lib/test/test_ntpath.py +++ b/Lib/test/test_ntpath.py @@ -300,6 +300,11 @@ def test_join(self): tester("ntpath.join('//computer/share', 'a', 'b')", '//computer/share\\a\\b') tester("ntpath.join('//computer/share', 'a/b')", '//computer/share\\a/b') + tester("ntpath.join('\\\\', 'computer')", '\\\\computer') + tester("ntpath.join('\\\\computer\\', 'share')", '\\\\computer\\share') + tester("ntpath.join('\\\\computer\\share\\', 'a')", '\\\\computer\\share\\a') + tester("ntpath.join('\\\\computer\\share\\a\\', 'b')", '\\\\computer\\share\\a\\b') + def test_normpath(self): tester("ntpath.normpath('A//////././//.//B')", r'A\B') tester("ntpath.normpath('A/./B')", r'A\B') diff --git a/Lib/test/test_subclassinit.py b/Lib/test/test_subclassinit.py index 0ad7d17fbd4ddd..310473a4a2fe58 100644 --- a/Lib/test/test_subclassinit.py +++ b/Lib/test/test_subclassinit.py @@ -134,30 +134,28 @@ class Descriptor: def __set_name__(self, owner, name): 1/0 - with self.assertRaises(RuntimeError) as cm: + with self.assertRaises(ZeroDivisionError) as cm: class NotGoingToWork: attr = Descriptor() - exc = cm.exception - self.assertRegex(str(exc), r'\bNotGoingToWork\b') - self.assertRegex(str(exc), r'\battr\b') - self.assertRegex(str(exc), r'\bDescriptor\b') - self.assertIsInstance(exc.__cause__, ZeroDivisionError) + notes = cm.exception.__notes__ + self.assertRegex(str(notes), r'\bNotGoingToWork\b') + self.assertRegex(str(notes), r'\battr\b') + self.assertRegex(str(notes), r'\bDescriptor\b') def test_set_name_wrong(self): class Descriptor: def __set_name__(self): pass - with self.assertRaises(RuntimeError) as cm: + with self.assertRaises(TypeError) as cm: class NotGoingToWork: attr = Descriptor() - exc = cm.exception - self.assertRegex(str(exc), r'\bNotGoingToWork\b') - self.assertRegex(str(exc), r'\battr\b') - self.assertRegex(str(exc), r'\bDescriptor\b') - self.assertIsInstance(exc.__cause__, TypeError) + notes = cm.exception.__notes__ + self.assertRegex(str(notes), r'\bNotGoingToWork\b') + self.assertRegex(str(notes), r'\battr\b') + self.assertRegex(str(notes), r'\bDescriptor\b') def test_set_name_lookup(self): resolved = [] diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index d7456d6d9480b2..1aebe1b111f2e9 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -532,13 +532,13 @@ def g456(): main_id = threading.get_ident() self.assertIn(main_id, d) self.assertIn(thread_id, d) - self.assertEqual((None, None, None), d.pop(main_id)) + self.assertEqual(None, d.pop(main_id)) # Verify that the captured thread frame is blocked in g456, called # from f123. This is a little tricky, since various bits of # threading.py are also in the thread's call stack. - exc_type, exc_value, exc_tb = d.pop(thread_id) - stack = traceback.extract_stack(exc_tb.tb_frame) + exc_value = d.pop(thread_id) + stack = traceback.extract_stack(exc_value.__traceback__.tb_frame) for i, (filename, lineno, funcname, sourceline) in enumerate(stack): if funcname == "f123": break @@ -1445,7 +1445,7 @@ class C(object): pass def func(): return sys._getframe() x = func() - check(x, size('3Pi3c7P2ic??2P')) + check(x, size('3Pii3c7P2ic??2P')) # function def func(): pass check(func, size('14Pi')) diff --git a/Lib/test/test_sys_settrace.py b/Lib/test/test_sys_settrace.py index 4907c930e143d5..980321e169b9e5 100644 --- a/Lib/test/test_sys_settrace.py +++ b/Lib/test/test_sys_settrace.py @@ -2808,5 +2808,65 @@ def foo(*args): sys.settrace(sys.gettrace()) +class TestLinesAfterTraceStarted(TraceTestCase): + + def test_events(self): + tracer = Tracer() + sys._getframe().f_trace = tracer.trace + sys.settrace(tracer.trace) + line = 4 + line = 5 + sys.settrace(None) + self.compare_events( + TestLinesAfterTraceStarted.test_events.__code__.co_firstlineno, + tracer.events, [ + (4, 'line'), + (5, 'line'), + (6, 'line')]) + + +class TestSetLocalTrace(TraceTestCase): + + def test_with_branches(self): + + def tracefunc(frame, event, arg): + if frame.f_code.co_name == "func": + frame.f_trace = tracefunc + line = frame.f_lineno - frame.f_code.co_firstlineno + events.append((line, event)) + return tracefunc + + def func(arg = 1): + N = 1 + if arg >= 2: + not_reached = 3 + else: + reached = 5 + if arg >= 3: + not_reached = 7 + else: + reached = 9 + the_end = 10 + + EXPECTED_EVENTS = [ + (0, 'call'), + (1, 'line'), + (2, 'line'), + (5, 'line'), + (6, 'line'), + (9, 'line'), + (10, 'line'), + (10, 'return'), + ] + + events = [] + sys.settrace(tracefunc) + sys._getframe().f_trace = tracefunc + func() + self.assertEqual(events, EXPECTED_EVENTS) + sys.settrace(None) + + + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py index 4e18dfc23c40c2..23328431a7aad8 100644 --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -611,6 +611,21 @@ def test_zippath_from_non_installed_posix(self): out, err = check_output(cmd) self.assertTrue(zip_landmark.encode() in out) + def test_activate_shell_script_has_no_dos_newlines(self): + """ + Test that the `activate` shell script contains no CR LF. + This is relevant for Cygwin, as the Windows build might have + converted line endings accidentally. + """ + venv_dir = pathlib.Path(self.env_dir) + rmtree(venv_dir) + [[scripts_dir], *_] = self.ENV_SUBDIRS + script_path = venv_dir / scripts_dir / "activate" + venv.create(venv_dir) + with open(script_path, 'rb') as script: + for line in script: + self.assertFalse(line.endswith(b'\r\n'), line) + @requireVenvCreate class EnsurePipTest(BaseTest): """Test venv module installation of pip.""" diff --git a/Lib/venv/scripts/common/activate b/Lib/venv/scripts/common/activate index 6fbc2b8801da04..cb898b39670c47 100644 --- a/Lib/venv/scripts/common/activate +++ b/Lib/venv/scripts/common/activate @@ -38,8 +38,15 @@ deactivate () { # unset irrelevant variables deactivate nondestructive -VIRTUAL_ENV="__VENV_DIR__" -export VIRTUAL_ENV +# on Windows, a path can contain colons and backslashes and has to be converted: +if [ "$OSTYPE" = "cygwin" ] || [ "$OSTYPE" = "msys" ] ; then + # transform D:\path\to\venv to /d/path/to/venv on MSYS + # and to /cygdrive/d/path/to/venv on Cygwin + export VIRTUAL_ENV=$(cygpath "__VENV_DIR__") +else + # use the path as-is + export VIRTUAL_ENV="__VENV_DIR__" +fi _OLD_VIRTUAL_PATH="$PATH" PATH="$VIRTUAL_ENV/__VENV_BIN_NAME__:$PATH" diff --git a/Makefile.pre.in b/Makefile.pre.in index fefa5d4e8147bf..afd503ef126339 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -369,17 +369,18 @@ PYTHON_OBJS= \ Python/Python-ast.o \ Python/Python-tokenize.o \ Python/asdl.o \ + Python/assemble.o \ Python/ast.o \ Python/ast_opt.o \ Python/ast_unparse.o \ Python/bltinmodule.o \ Python/ceval.o \ - Python/flowgraph.o \ Python/codecs.o \ Python/compile.o \ Python/context.o \ Python/dynamic_annotations.o \ Python/errors.o \ + Python/flowgraph.o \ Python/frame.o \ Python/frozenmain.o \ Python/future.o \ @@ -394,7 +395,9 @@ PYTHON_OBJS= \ Python/import.o \ Python/importdl.o \ Python/initconfig.o \ + Python/instrumentation.o \ Python/intrinsics.o \ + Python/legacy_tracing.o \ Python/marshal.o \ Python/modsupport.o \ Python/mysnprintf.o \ diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-03-31-17-24-03.gh-issue-103082.isRUcV.rst b/Misc/NEWS.d/next/Core and Builtins/2023-03-31-17-24-03.gh-issue-103082.isRUcV.rst new file mode 100644 index 00000000000000..631ef4c7890450 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-03-31-17-24-03.gh-issue-103082.isRUcV.rst @@ -0,0 +1 @@ +Implement :pep:`669` Low Impact Monitoring for CPython. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-04-09-22-21-57.gh-issue-77757._Ow-u2.rst b/Misc/NEWS.d/next/Core and Builtins/2023-04-09-22-21-57.gh-issue-77757._Ow-u2.rst new file mode 100644 index 00000000000000..85c8ecf7de8d1b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-04-09-22-21-57.gh-issue-77757._Ow-u2.rst @@ -0,0 +1,3 @@ +Exceptions raised in a typeobject's ``__set_name__`` method are no longer +wrapped by a :exc:`RuntimeError`. Context information is added to the +exception as a :pep:`678` note. diff --git a/Misc/NEWS.d/next/Library/2021-11-07-15-31-25.bpo-23041.564i32.rst b/Misc/NEWS.d/next/Library/2021-11-07-15-31-25.bpo-23041.564i32.rst new file mode 100644 index 00000000000000..53c32d397b206b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-11-07-15-31-25.bpo-23041.564i32.rst @@ -0,0 +1,2 @@ +Add :data:`~csv.QUOTE_STRINGS` and :data:`~csv.QUOTE_NOTNULL` to the suite +of :mod:`csv` module quoting styles. diff --git a/Misc/NEWS.d/next/Library/2023-03-31-01-13-00.gh-issue-103143.6eMluy.rst b/Misc/NEWS.d/next/Library/2023-03-31-01-13-00.gh-issue-103143.6eMluy.rst new file mode 100644 index 00000000000000..32bd62d27c7c6d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-03-31-01-13-00.gh-issue-103143.6eMluy.rst @@ -0,0 +1 @@ +Polish the help messages and docstrings of :mod:`pdb`. diff --git a/Misc/NEWS.d/next/Library/2023-04-01-23-01-31.gh-issue-103176.FBsdxa.rst b/Misc/NEWS.d/next/Library/2023-04-01-23-01-31.gh-issue-103176.FBsdxa.rst new file mode 100644 index 00000000000000..b89f9bae595457 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-04-01-23-01-31.gh-issue-103176.FBsdxa.rst @@ -0,0 +1,2 @@ +:func:`sys._current_exceptions` now returns a mapping from thread-id to an +exception instance, rather than to a ``(typ, exc, tb)`` tuple. diff --git a/Misc/NEWS.d/next/Library/2023-04-03-21-08-53.gh-issue-103220.OW_Bj5.rst b/Misc/NEWS.d/next/Library/2023-04-03-21-08-53.gh-issue-103220.OW_Bj5.rst new file mode 100644 index 00000000000000..9cf26c26873b2a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-04-03-21-08-53.gh-issue-103220.OW_Bj5.rst @@ -0,0 +1,2 @@ +Fix issue where :func:`os.path.join` added a slash when joining onto an +incomplete UNC drive with a trailing slash on Windows. diff --git a/Misc/NEWS.d/next/Library/2023-04-04-21-44-25.gh-issue-103092.Dz0_Xn.rst b/Misc/NEWS.d/next/Library/2023-04-04-21-44-25.gh-issue-103092.Dz0_Xn.rst new file mode 100644 index 00000000000000..7bd191e3c22b2b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-04-04-21-44-25.gh-issue-103092.Dz0_Xn.rst @@ -0,0 +1 @@ +Adapt the :mod:`msvcrt` extension module to :pep:`687`. diff --git a/Misc/NEWS.d/next/Library/2023-04-08-01-33-12.gh-issue-103357.vjin28.rst b/Misc/NEWS.d/next/Library/2023-04-08-01-33-12.gh-issue-103357.vjin28.rst new file mode 100644 index 00000000000000..83dce56ed0b7c5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-04-08-01-33-12.gh-issue-103357.vjin28.rst @@ -0,0 +1,3 @@ +Added support for :class:`logging.Formatter` ``defaults`` parameter to +:func:`logging.config.dictConfig` and :func:`logging.config.fileConfig`. +Patch by Bar Harel. diff --git a/Misc/NEWS.d/next/Library/2023-04-09-06-59-36.gh-issue-103092.vskbro.rst b/Misc/NEWS.d/next/Library/2023-04-09-06-59-36.gh-issue-103092.vskbro.rst new file mode 100644 index 00000000000000..6977c1489a29cb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-04-09-06-59-36.gh-issue-103092.vskbro.rst @@ -0,0 +1 @@ +Isolate :mod:`!_collections` (apply :pep:`687`). Patch by Erlend E. Aasland. diff --git a/Misc/NEWS.d/next/Windows/2023-04-11-09-22-22.gh-issue-103088.6AJEuR.rst b/Misc/NEWS.d/next/Windows/2023-04-11-09-22-22.gh-issue-103088.6AJEuR.rst new file mode 100644 index 00000000000000..f9f5343f4210dc --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2023-04-11-09-22-22.gh-issue-103088.6AJEuR.rst @@ -0,0 +1 @@ +Fixes venvs not working in bash on Windows across different disks diff --git a/Misc/NEWS.d/next/Windows/2023-04-12-10-49-21.gh-issue-103088.Yjj-qJ.rst b/Misc/NEWS.d/next/Windows/2023-04-12-10-49-21.gh-issue-103088.Yjj-qJ.rst new file mode 100644 index 00000000000000..1fee99da240378 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2023-04-12-10-49-21.gh-issue-103088.Yjj-qJ.rst @@ -0,0 +1 @@ +Fix virtual environment :file:`activate` script having incorrect line endings for Cygwin. diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index 9d8aef1e677157..a9b1425177c3d7 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -1,17 +1,56 @@ #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_long.h" // _PyLong_GetZero() +#include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_typeobject.h" // _PyType_GetModuleState() #include "structmember.h" // PyMemberDef #include +typedef struct { + PyTypeObject *deque_type; + PyTypeObject *defdict_type; + PyTypeObject *dequeiter_type; + PyTypeObject *dequereviter_type; + PyTypeObject *tuplegetter_type; +} collections_state; + +static inline collections_state * +get_module_state(PyObject *mod) +{ + void *state = _PyModule_GetState(mod); + assert(state != NULL); + return (collections_state *)state; +} + +static inline collections_state * +get_module_state_by_cls(PyTypeObject *cls) +{ + void *state = _PyType_GetModuleState(cls); + assert(state != NULL); + return (collections_state *)state; +} + +static struct PyModuleDef _collectionsmodule; + +static inline collections_state * +find_module_state_by_def(PyTypeObject *type) +{ + PyObject *mod = PyType_GetModuleByDef(type, &_collectionsmodule); + assert(mod != NULL); + return get_module_state(mod); +} + /*[clinic input] module _collections -class _tuplegetter "_tuplegetterobject *" "&tuplegetter_type" +class _tuplegetter "_tuplegetterobject *" "clinic_state()->tuplegetter_type" [clinic start generated code]*/ -/*[clinic end generated code: output=da39a3ee5e6b4b0d input=a8ece4ccad7e30ac]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=7356042a89862e0e]*/ -static PyTypeObject tuplegetter_type; +/* We can safely assume type to be the defining class, + * since tuplegetter is not a base type */ +#define clinic_state() (get_module_state_by_cls(type)) #include "clinic/_collectionsmodule.c.h" +#undef clinic_state /* collections module implementation of a deque() datatype Written and maintained by Raymond D. Hettinger @@ -94,8 +133,6 @@ typedef struct { PyObject *weakreflist; } dequeobject; -static PyTypeObject deque_type; - /* For debug builds, add error checking to track the endpoints * in the chain of links. The goal is to make sure that link * assignments only take place at endpoints so that links already @@ -484,11 +521,13 @@ deque_copy(PyObject *deque, PyObject *Py_UNUSED(ignored)) { PyObject *result; dequeobject *old_deque = (dequeobject *)deque; - if (Py_IS_TYPE(deque, &deque_type)) { + collections_state *state = find_module_state_by_def(Py_TYPE(deque)); + if (Py_IS_TYPE(deque, state->deque_type)) { dequeobject *new_deque; PyObject *rv; - new_deque = (dequeobject *)deque_new(&deque_type, (PyObject *)NULL, (PyObject *)NULL); + new_deque = (dequeobject *)deque_new(state->deque_type, + (PyObject *)NULL, (PyObject *)NULL); if (new_deque == NULL) return NULL; new_deque->maxlen = old_deque->maxlen; @@ -511,7 +550,7 @@ deque_copy(PyObject *deque, PyObject *Py_UNUSED(ignored)) else result = PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", deque, old_deque->maxlen, NULL); - if (result != NULL && !PyObject_TypeCheck(result, &deque_type)) { + if (result != NULL && !PyObject_TypeCheck(result, state->deque_type)) { PyErr_Format(PyExc_TypeError, "%.200s() must return a deque, not %.200s", Py_TYPE(deque)->tp_name, Py_TYPE(result)->tp_name); @@ -529,7 +568,8 @@ deque_concat(dequeobject *deque, PyObject *other) PyObject *new_deque, *result; int rv; - rv = PyObject_IsInstance(other, (PyObject *)&deque_type); + collections_state *state = find_module_state_by_def(Py_TYPE(deque)); + rv = PyObject_IsInstance(other, (PyObject *)state->deque_type); if (rv <= 0) { if (rv == 0) { PyErr_Format(PyExc_TypeError, @@ -1288,6 +1328,7 @@ deque_ass_item(dequeobject *deque, Py_ssize_t i, PyObject *v) static void deque_dealloc(dequeobject *deque) { + PyTypeObject *tp = Py_TYPE(deque); Py_ssize_t i; PyObject_GC_UnTrack(deque); @@ -1303,12 +1344,15 @@ deque_dealloc(dequeobject *deque) for (i=0 ; i < deque->numfreeblocks ; i++) { PyMem_Free(deque->freeblocks[i]); } - Py_TYPE(deque)->tp_free(deque); + tp->tp_free(deque); + Py_DECREF(tp); } static int deque_traverse(dequeobject *deque, visitproc visit, void *arg) { + Py_VISIT(Py_TYPE(deque)); + block *b; PyObject *item; Py_ssize_t index; @@ -1393,8 +1437,9 @@ deque_richcompare(PyObject *v, PyObject *w, int op) Py_ssize_t vs, ws; int b, cmp=-1; - if (!PyObject_TypeCheck(v, &deque_type) || - !PyObject_TypeCheck(w, &deque_type)) { + collections_state *state = find_module_state_by_def(Py_TYPE(v)); + if (!PyObject_TypeCheck(v, state->deque_type) || + !PyObject_TypeCheck(w, state->deque_type)) { Py_RETURN_NOTIMPLEMENTED; } @@ -1537,19 +1582,6 @@ static PyGetSetDef deque_getset[] = { {0} }; -static PySequenceMethods deque_as_sequence = { - (lenfunc)deque_len, /* sq_length */ - (binaryfunc)deque_concat, /* sq_concat */ - (ssizeargfunc)deque_repeat, /* sq_repeat */ - (ssizeargfunc)deque_item, /* sq_item */ - 0, /* sq_slice */ - (ssizeobjargproc)deque_ass_item, /* sq_ass_item */ - 0, /* sq_ass_slice */ - (objobjproc)deque_contains, /* sq_contains */ - (binaryfunc)deque_inplace_concat, /* sq_inplace_concat */ - (ssizeargfunc)deque_inplace_repeat, /* sq_inplace_repeat */ -}; - static PyObject *deque_iter(dequeobject *deque); static PyObject *deque_reviter(dequeobject *deque, PyObject *Py_UNUSED(ignored)); PyDoc_STRVAR(reversed_doc, @@ -1597,54 +1629,53 @@ static PyMethodDef deque_methods[] = { {NULL, NULL} /* sentinel */ }; +static PyMemberDef deque_members[] = { + {"__weaklistoffset__", T_PYSSIZET, offsetof(dequeobject, weakreflist), READONLY}, + {NULL}, +}; + PyDoc_STRVAR(deque_doc, "deque([iterable[, maxlen]]) --> deque object\n\ \n\ A list-like sequence optimized for data accesses near its endpoints."); -static PyTypeObject deque_type = { - PyVarObject_HEAD_INIT(NULL, 0) - "collections.deque", /* tp_name */ - sizeof(dequeobject), /* tp_basicsize */ - 0, /* tp_itemsize */ - /* methods */ - (destructor)deque_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - deque_repr, /* tp_repr */ - 0, /* tp_as_number */ - &deque_as_sequence, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - PyObject_HashNotImplemented, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | - Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_SEQUENCE, - /* tp_flags */ - deque_doc, /* tp_doc */ - (traverseproc)deque_traverse, /* tp_traverse */ - (inquiry)deque_clear, /* tp_clear */ - (richcmpfunc)deque_richcompare, /* tp_richcompare */ - offsetof(dequeobject, weakreflist), /* tp_weaklistoffset*/ - (getiterfunc)deque_iter, /* tp_iter */ - 0, /* tp_iternext */ - deque_methods, /* tp_methods */ - 0, /* tp_members */ - deque_getset, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - (initproc)deque_init, /* tp_init */ - PyType_GenericAlloc, /* tp_alloc */ - deque_new, /* tp_new */ - PyObject_GC_Del, /* tp_free */ +static PyType_Slot deque_slots[] = { + {Py_tp_dealloc, deque_dealloc}, + {Py_tp_repr, deque_repr}, + {Py_tp_hash, PyObject_HashNotImplemented}, + {Py_tp_getattro, PyObject_GenericGetAttr}, + {Py_tp_doc, (void *)deque_doc}, + {Py_tp_traverse, deque_traverse}, + {Py_tp_clear, deque_clear}, + {Py_tp_richcompare, deque_richcompare}, + {Py_tp_iter, deque_iter}, + {Py_tp_getset, deque_getset}, + {Py_tp_init, deque_init}, + {Py_tp_alloc, PyType_GenericAlloc}, + {Py_tp_new, deque_new}, + {Py_tp_free, PyObject_GC_Del}, + {Py_tp_methods, deque_methods}, + {Py_tp_members, deque_members}, + + // Sequence protocol + {Py_sq_length, deque_len}, + {Py_sq_concat, deque_concat}, + {Py_sq_repeat, deque_repeat}, + {Py_sq_item, deque_item}, + {Py_sq_ass_item, deque_ass_item}, + {Py_sq_contains, deque_contains}, + {Py_sq_inplace_concat, deque_inplace_concat}, + {Py_sq_inplace_repeat, deque_inplace_repeat}, + {0, NULL}, +}; + +static PyType_Spec deque_spec = { + .name = "collections.deque", + .basicsize = sizeof(dequeobject), + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | + Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_SEQUENCE | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = deque_slots, }; /*********************** Deque Iterator **************************/ @@ -1658,14 +1689,13 @@ typedef struct { Py_ssize_t counter; /* number of items remaining for iteration */ } dequeiterobject; -static PyTypeObject dequeiter_type; - static PyObject * deque_iter(dequeobject *deque) { dequeiterobject *it; - it = PyObject_GC_New(dequeiterobject, &dequeiter_type); + collections_state *state = find_module_state_by_def(Py_TYPE(deque)); + it = PyObject_GC_New(dequeiterobject, state->dequeiter_type); if (it == NULL) return NULL; it->b = deque->leftblock; @@ -1680,17 +1710,27 @@ deque_iter(dequeobject *deque) static int dequeiter_traverse(dequeiterobject *dio, visitproc visit, void *arg) { + Py_VISIT(Py_TYPE(dio)); Py_VISIT(dio->deque); return 0; } +static int +dequeiter_clear(dequeiterobject *dio) +{ + Py_CLEAR(dio->deque); + return 0; +} + static void dequeiter_dealloc(dequeiterobject *dio) { /* bpo-31095: UnTrack is needed before calling any callbacks */ + PyTypeObject *tp = Py_TYPE(dio); PyObject_GC_UnTrack(dio); - Py_XDECREF(dio->deque); + (void)dequeiter_clear(dio); PyObject_GC_Del(dio); + Py_DECREF(tp); } static PyObject * @@ -1726,9 +1766,10 @@ dequeiter_new(PyTypeObject *type, PyObject *args, PyObject *kwds) Py_ssize_t i, index=0; PyObject *deque; dequeiterobject *it; - if (!PyArg_ParseTuple(args, "O!|n", &deque_type, &deque, &index)) + collections_state *state = get_module_state_by_cls(type); + if (!PyArg_ParseTuple(args, "O!|n", state->deque_type, &deque, &index)) return NULL; - assert(type == &dequeiter_type); + assert(type == state->dequeiter_type); it = (dequeiterobject*)deque_iter((dequeobject *)deque); if (!it) @@ -1769,59 +1810,35 @@ static PyMethodDef dequeiter_methods[] = { {NULL, NULL} /* sentinel */ }; -static PyTypeObject dequeiter_type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_collections._deque_iterator", /* tp_name */ - sizeof(dequeiterobject), /* tp_basicsize */ - 0, /* tp_itemsize */ - /* methods */ - (destructor)dequeiter_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - 0, /* tp_doc */ - (traverseproc)dequeiter_traverse, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - PyObject_SelfIter, /* tp_iter */ - (iternextfunc)dequeiter_next, /* tp_iternext */ - dequeiter_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - dequeiter_new, /* tp_new */ - 0, +static PyType_Slot dequeiter_slots[] = { + {Py_tp_dealloc, dequeiter_dealloc}, + {Py_tp_getattro, PyObject_GenericGetAttr}, + {Py_tp_traverse, dequeiter_traverse}, + {Py_tp_clear, dequeiter_clear}, + {Py_tp_iter, PyObject_SelfIter}, + {Py_tp_iternext, dequeiter_next}, + {Py_tp_methods, dequeiter_methods}, + {Py_tp_new, dequeiter_new}, + {0, NULL}, }; -/*********************** Deque Reverse Iterator **************************/ +static PyType_Spec dequeiter_spec = { + .name = "collections._deque_iterator", + .basicsize = sizeof(dequeiterobject), + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = dequeiter_slots, +}; -static PyTypeObject dequereviter_type; +/*********************** Deque Reverse Iterator **************************/ static PyObject * deque_reviter(dequeobject *deque, PyObject *Py_UNUSED(ignored)) { dequeiterobject *it; + collections_state *state = find_module_state_by_def(Py_TYPE(deque)); - it = PyObject_GC_New(dequeiterobject, &dequereviter_type); + it = PyObject_GC_New(dequeiterobject, state->dequereviter_type); if (it == NULL) return NULL; it->b = deque->rightblock; @@ -1866,9 +1883,10 @@ dequereviter_new(PyTypeObject *type, PyObject *args, PyObject *kwds) Py_ssize_t i, index=0; PyObject *deque; dequeiterobject *it; - if (!PyArg_ParseTuple(args, "O!|n", &deque_type, &deque, &index)) + collections_state *state = get_module_state_by_cls(type); + if (!PyArg_ParseTuple(args, "O!|n", state->deque_type, &deque, &index)) return NULL; - assert(type == &dequereviter_type); + assert(type == state->dequereviter_type); it = (dequeiterobject*)deque_reviter((dequeobject *)deque, NULL); if (!it) @@ -1889,47 +1907,24 @@ dequereviter_new(PyTypeObject *type, PyObject *args, PyObject *kwds) return (PyObject*)it; } -static PyTypeObject dequereviter_type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_collections._deque_reverse_iterator", /* tp_name */ - sizeof(dequeiterobject), /* tp_basicsize */ - 0, /* tp_itemsize */ - /* methods */ - (destructor)dequeiter_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - 0, /* tp_doc */ - (traverseproc)dequeiter_traverse, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - PyObject_SelfIter, /* tp_iter */ - (iternextfunc)dequereviter_next, /* tp_iternext */ - dequeiter_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - dequereviter_new, /* tp_new */ - 0, +static PyType_Slot dequereviter_slots[] = { + {Py_tp_dealloc, dequeiter_dealloc}, + {Py_tp_getattro, PyObject_GenericGetAttr}, + {Py_tp_traverse, dequeiter_traverse}, + {Py_tp_clear, dequeiter_clear}, + {Py_tp_iter, PyObject_SelfIter}, + {Py_tp_iternext, dequereviter_next}, + {Py_tp_methods, dequeiter_methods}, + {Py_tp_new, dequereviter_new}, + {0, NULL}, +}; + +static PyType_Spec dequereviter_spec = { + .name = "collections._deque_reverse_iterator", + .basicsize = sizeof(dequeiterobject), + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = dequereviter_slots, }; /* defaultdict type *********************************************************/ @@ -1939,8 +1934,6 @@ typedef struct { PyObject *default_factory; } defdictobject; -static PyTypeObject defdict_type; /* Forward */ - PyDoc_STRVAR(defdict_missing_doc, "__missing__(key) # Called by __getitem__ for missing key; pseudo-code:\n\ if self.default_factory is None: raise KeyError((key,))\n\ @@ -2071,9 +2064,11 @@ static void defdict_dealloc(defdictobject *dd) { /* bpo-31095: UnTrack is needed before calling any callbacks */ + PyTypeObject *tp = Py_TYPE(dd); PyObject_GC_UnTrack(dd); Py_CLEAR(dd->default_factory); PyDict_Type.tp_dealloc((PyObject *)dd); + Py_DECREF(tp); } static PyObject * @@ -2117,11 +2112,24 @@ static PyObject* defdict_or(PyObject* left, PyObject* right) { PyObject *self, *other; - if (PyObject_TypeCheck(left, &defdict_type)) { + + // Find module state + PyTypeObject *tp = Py_TYPE(left); + PyObject *mod = PyType_GetModuleByDef(tp, &_collectionsmodule); + if (mod == NULL) { + PyErr_Clear(); + tp = Py_TYPE(right); + mod = PyType_GetModuleByDef(tp, &_collectionsmodule); + } + assert(mod != NULL); + collections_state *state = get_module_state(mod); + + if (PyObject_TypeCheck(left, state->defdict_type)) { self = left; other = right; } else { + assert(PyObject_TypeCheck(right, state->defdict_type)); self = right; other = left; } @@ -2141,13 +2149,10 @@ defdict_or(PyObject* left, PyObject* right) return new; } -static PyNumberMethods defdict_as_number = { - .nb_or = defdict_or, -}; - static int defdict_traverse(PyObject *self, visitproc visit, void *arg) { + Py_VISIT(Py_TYPE(self)); Py_VISIT(((defdictobject *)self)->default_factory); return PyDict_Type.tp_traverse(self, visit, arg); } @@ -2203,48 +2208,28 @@ passed to the dict constructor, including keyword arguments.\n\ /* See comment in xxsubtype.c */ #define DEFERRED_ADDRESS(ADDR) 0 -static PyTypeObject defdict_type = { - PyVarObject_HEAD_INIT(DEFERRED_ADDRESS(&PyType_Type), 0) - "collections.defaultdict", /* tp_name */ - sizeof(defdictobject), /* tp_basicsize */ - 0, /* tp_itemsize */ - /* methods */ - (destructor)defdict_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - (reprfunc)defdict_repr, /* tp_repr */ - &defdict_as_number, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - /* tp_flags */ - defdict_doc, /* tp_doc */ - defdict_traverse, /* tp_traverse */ - (inquiry)defdict_tp_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset*/ - 0, /* tp_iter */ - 0, /* tp_iternext */ - defdict_methods, /* tp_methods */ - defdict_members, /* tp_members */ - 0, /* tp_getset */ - DEFERRED_ADDRESS(&PyDict_Type), /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - defdict_init, /* tp_init */ - PyType_GenericAlloc, /* tp_alloc */ - 0, /* tp_new */ - PyObject_GC_Del, /* tp_free */ +static PyType_Slot defdict_slots[] = { + {Py_tp_dealloc, defdict_dealloc}, + {Py_tp_repr, defdict_repr}, + {Py_nb_or, defdict_or}, + {Py_tp_getattro, PyObject_GenericGetAttr}, + {Py_tp_doc, (void *)defdict_doc}, + {Py_tp_traverse, defdict_traverse}, + {Py_tp_clear, defdict_tp_clear}, + {Py_tp_methods, defdict_methods}, + {Py_tp_members, defdict_members}, + {Py_tp_init, defdict_init}, + {Py_tp_alloc, PyType_GenericAlloc}, + {Py_tp_free, PyObject_GC_Del}, + {0, NULL}, +}; + +static PyType_Spec defdict_spec = { + .name = "collections.defaultdict", + .basicsize = sizeof(defdictobject), + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = defdict_slots, }; /* helper function for Counter *********************************************/ @@ -2442,6 +2427,7 @@ static int tuplegetter_traverse(PyObject *self, visitproc visit, void *arg) { _tuplegetterobject *tuplegetter = (_tuplegetterobject *)self; + Py_VISIT(Py_TYPE(tuplegetter)); Py_VISIT(tuplegetter->doc); return 0; } @@ -2457,9 +2443,11 @@ tuplegetter_clear(PyObject *self) static void tuplegetter_dealloc(_tuplegetterobject *self) { + PyTypeObject *tp = Py_TYPE(self); PyObject_GC_UnTrack(self); tuplegetter_clear((PyObject*)self); - Py_TYPE(self)->tp_free((PyObject*)self); + tp->tp_free((PyObject*)self); + Py_DECREF(tp); } static PyObject* @@ -2487,52 +2475,60 @@ static PyMethodDef tuplegetter_methods[] = { {NULL}, }; -static PyTypeObject tuplegetter_type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_collections._tuplegetter", /* tp_name */ - sizeof(_tuplegetterobject), /* tp_basicsize */ - 0, /* tp_itemsize */ - /* methods */ - (destructor)tuplegetter_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - (reprfunc)tuplegetter_repr, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - 0, /* tp_doc */ - (traverseproc)tuplegetter_traverse, /* tp_traverse */ - (inquiry)tuplegetter_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - tuplegetter_methods, /* tp_methods */ - tuplegetter_members, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - tuplegetter_descr_get, /* tp_descr_get */ - tuplegetter_descr_set, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - tuplegetter_new, /* tp_new */ - 0, +static PyType_Slot tuplegetter_slots[] = { + {Py_tp_dealloc, tuplegetter_dealloc}, + {Py_tp_repr, tuplegetter_repr}, + {Py_tp_traverse, tuplegetter_traverse}, + {Py_tp_clear, tuplegetter_clear}, + {Py_tp_methods, tuplegetter_methods}, + {Py_tp_members, tuplegetter_members}, + {Py_tp_descr_get, tuplegetter_descr_get}, + {Py_tp_descr_set, tuplegetter_descr_set}, + {Py_tp_new, tuplegetter_new}, + {0, NULL}, +}; + +static PyType_Spec tuplegetter_spec = { + .name = "collections._tuplegetter", + .basicsize = sizeof(_tuplegetterobject), + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = tuplegetter_slots, }; /* module level code ********************************************************/ +static int +collections_traverse(PyObject *mod, visitproc visit, void *arg) +{ + collections_state *state = get_module_state(mod); + Py_VISIT(state->deque_type); + Py_VISIT(state->defdict_type); + Py_VISIT(state->dequeiter_type); + Py_VISIT(state->dequereviter_type); + Py_VISIT(state->tuplegetter_type); + return 0; +} + +static int +collections_clear(PyObject *mod) +{ + collections_state *state = get_module_state(mod); + Py_CLEAR(state->deque_type); + Py_CLEAR(state->defdict_type); + Py_CLEAR(state->dequeiter_type); + Py_CLEAR(state->dequereviter_type); + Py_CLEAR(state->tuplegetter_type); + return 0; +} + +static void +collections_free(void *module) +{ + collections_clear((PyObject *)module); +} + PyDoc_STRVAR(collections_doc, "High performance data structures.\n\ - deque: ordered collection accessible from endpoints only\n\ @@ -2544,43 +2540,50 @@ static struct PyMethodDef collections_methods[] = { {NULL, NULL} /* sentinel */ }; +#define ADD_TYPE(MOD, SPEC, TYPE, BASE) do { \ + TYPE = (PyTypeObject *)PyType_FromMetaclass(NULL, MOD, SPEC, \ + (PyObject *)BASE); \ + if (TYPE == NULL) { \ + return -1; \ + } \ + if (PyModule_AddType(MOD, TYPE) < 0) { \ + return -1; \ + } \ +} while (0) + static int collections_exec(PyObject *module) { - PyTypeObject *typelist[] = { - &deque_type, - &defdict_type, - &PyODict_Type, - &dequeiter_type, - &dequereviter_type, - &tuplegetter_type - }; - - defdict_type.tp_base = &PyDict_Type; - - for (size_t i = 0; i < Py_ARRAY_LENGTH(typelist); i++) { - if (PyModule_AddType(module, typelist[i]) < 0) { - return -1; - } + collections_state *state = get_module_state(module); + ADD_TYPE(module, &deque_spec, state->deque_type, NULL); + ADD_TYPE(module, &defdict_spec, state->defdict_type, &PyDict_Type); + ADD_TYPE(module, &dequeiter_spec, state->dequeiter_type, NULL); + ADD_TYPE(module, &dequereviter_spec, state->dequereviter_type, NULL); + ADD_TYPE(module, &tuplegetter_spec, state->tuplegetter_type, NULL); + + if (PyModule_AddType(module, &PyODict_Type) < 0) { + return -1; } return 0; } +#undef ADD_TYPE + static struct PyModuleDef_Slot collections_slots[] = { {Py_mod_exec, collections_exec}, {0, NULL} }; static struct PyModuleDef _collectionsmodule = { - PyModuleDef_HEAD_INIT, - "_collections", - collections_doc, - 0, - collections_methods, - collections_slots, - NULL, - NULL, - NULL + .m_base = PyModuleDef_HEAD_INIT, + .m_name = "_collections", + .m_doc = collections_doc, + .m_size = sizeof(collections_state), + .m_methods = collections_methods, + .m_slots = collections_slots, + .m_traverse = collections_traverse, + .m_clear = collections_clear, + .m_free = collections_free, }; PyMODINIT_FUNC diff --git a/Modules/_csv.c b/Modules/_csv.c index bd337084dbff81..2217cc2ca7a775 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -82,7 +82,8 @@ typedef enum { } ParserState; typedef enum { - QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC, QUOTE_NONE + QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC, QUOTE_NONE, + QUOTE_STRINGS, QUOTE_NOTNULL } QuoteStyle; typedef struct { @@ -95,6 +96,8 @@ static const StyleDesc quote_styles[] = { { QUOTE_ALL, "QUOTE_ALL" }, { QUOTE_NONNUMERIC, "QUOTE_NONNUMERIC" }, { QUOTE_NONE, "QUOTE_NONE" }, + { QUOTE_STRINGS, "QUOTE_STRINGS" }, + { QUOTE_NOTNULL, "QUOTE_NOTNULL" }, { 0 } }; @@ -1264,6 +1267,12 @@ csv_writerow(WriterObj *self, PyObject *seq) case QUOTE_ALL: quoted = 1; break; + case QUOTE_STRINGS: + quoted = PyUnicode_Check(field); + break; + case QUOTE_NOTNULL: + quoted = field != Py_None; + break; default: quoted = 0; break; @@ -1659,6 +1668,11 @@ PyDoc_STRVAR(csv_module_doc, " csv.QUOTE_NONNUMERIC means that quotes are always placed around\n" " fields which do not parse as integers or floating point\n" " numbers.\n" +" csv.QUOTE_STRINGS means that quotes are always placed around\n" +" fields which are strings. Note that the Python value None\n" +" is not a string.\n" +" csv.QUOTE_NOTNULL means that quotes are only placed around fields\n" +" that are not the Python value None.\n" " csv.QUOTE_NONE means that quotes are never placed around fields.\n" " * escapechar - specifies a one-character string used to escape\n" " the delimiter when quoting is set to QUOTE_NONE.\n" diff --git a/Modules/clinic/_collectionsmodule.c.h b/Modules/clinic/_collectionsmodule.c.h index 8ea0255b061070..3882d069216e28 100644 --- a/Modules/clinic/_collectionsmodule.c.h +++ b/Modules/clinic/_collectionsmodule.c.h @@ -46,7 +46,7 @@ static PyObject * tuplegetter_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - PyTypeObject *base_tp = &tuplegetter_type; + PyTypeObject *base_tp = clinic_state()->tuplegetter_type; Py_ssize_t index; PyObject *doc; @@ -75,4 +75,4 @@ tuplegetter_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) exit: return return_value; } -/*[clinic end generated code: output=91a0f221c7b1f96c input=a9049054013a1b77]*/ +/*[clinic end generated code: output=00e516317d2b8bed input=a9049054013a1b77]*/ diff --git a/Objects/codeobject.c b/Objects/codeobject.c index 755d0b85e7cf30..9b54c610581174 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -431,13 +431,13 @@ init_code(PyCodeObject *co, struct _PyCodeConstructor *con) if (_Py_next_func_version != 0) { _Py_next_func_version++; } + co->_co_monitoring = NULL; + co->_co_instrumentation_version = 0; /* not set */ co->co_weakreflist = NULL; co->co_extra = NULL; co->_co_cached = NULL; - co->_co_linearray_entry_size = 0; - co->_co_linearray = NULL; memcpy(_PyCode_CODE(co), PyBytes_AS_STRING(con->code), PyBytes_GET_SIZE(con->code)); int entry_point = 0; @@ -816,54 +816,6 @@ PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) * source location tracking (co_lines/co_positions) ******************/ -/* Use co_linetable to compute the line number from a bytecode index, addrq. See - lnotab_notes.txt for the details of the lnotab representation. -*/ - -int -_PyCode_CreateLineArray(PyCodeObject *co) -{ - assert(co->_co_linearray == NULL); - PyCodeAddressRange bounds; - int size; - int max_line = 0; - _PyCode_InitAddressRange(co, &bounds); - while(_PyLineTable_NextAddressRange(&bounds)) { - if (bounds.ar_line > max_line) { - max_line = bounds.ar_line; - } - } - if (max_line < (1 << 15)) { - size = 2; - } - else { - size = 4; - } - co->_co_linearray = PyMem_Malloc(Py_SIZE(co)*size); - if (co->_co_linearray == NULL) { - PyErr_NoMemory(); - return -1; - } - co->_co_linearray_entry_size = size; - _PyCode_InitAddressRange(co, &bounds); - while(_PyLineTable_NextAddressRange(&bounds)) { - int start = bounds.ar_start / sizeof(_Py_CODEUNIT); - int end = bounds.ar_end / sizeof(_Py_CODEUNIT); - for (int index = start; index < end; index++) { - assert(index < (int)Py_SIZE(co)); - if (size == 2) { - assert(((int16_t)bounds.ar_line) == bounds.ar_line); - ((int16_t *)co->_co_linearray)[index] = bounds.ar_line; - } - else { - assert(size == 4); - ((int32_t *)co->_co_linearray)[index] = bounds.ar_line; - } - } - } - return 0; -} - int PyCode_Addr2Line(PyCodeObject *co, int addrq) { @@ -871,9 +823,6 @@ PyCode_Addr2Line(PyCodeObject *co, int addrq) return co->co_firstlineno; } assert(addrq >= 0 && addrq < _PyCode_NBYTES(co)); - if (co->_co_linearray) { - return _PyCode_LineNumberFromArray(co, addrq / sizeof(_Py_CODEUNIT)); - } PyCodeAddressRange bounds; _PyCode_InitAddressRange(co, &bounds); return _PyCode_CheckLineNumber(addrq, &bounds); @@ -1531,17 +1480,17 @@ PyCode_GetFreevars(PyCodeObject *code) } static void -deopt_code(_Py_CODEUNIT *instructions, Py_ssize_t len) +deopt_code(PyCodeObject *code, _Py_CODEUNIT *instructions) { + Py_ssize_t len = Py_SIZE(code); for (int i = 0; i < len; i++) { - _Py_CODEUNIT instruction = instructions[i]; - int opcode = _PyOpcode_Deopt[instruction.op.code]; + int opcode = _Py_GetBaseOpcode(code, i); int caches = _PyOpcode_Caches[opcode]; instructions[i].op.code = opcode; - while (caches--) { - instructions[++i].op.code = CACHE; - instructions[i].op.arg = 0; + for (int j = 1; j <= caches; j++) { + instructions[i+j].cache = 0; } + i += caches; } } @@ -1559,7 +1508,7 @@ _PyCode_GetCode(PyCodeObject *co) if (code == NULL) { return NULL; } - deopt_code((_Py_CODEUNIT *)PyBytes_AS_STRING(code), Py_SIZE(co)); + deopt_code(co, (_Py_CODEUNIT *)PyBytes_AS_STRING(code)); assert(co->_co_cached->_co_code == NULL); co->_co_cached->_co_code = Py_NewRef(code); return code; @@ -1693,6 +1642,30 @@ code_new_impl(PyTypeObject *type, int argcount, int posonlyargcount, return co; } +static void +free_monitoring_data(_PyCoMonitoringData *data) +{ + if (data == NULL) { + return; + } + if (data->tools) { + PyMem_Free(data->tools); + } + if (data->lines) { + PyMem_Free(data->lines); + } + if (data->line_tools) { + PyMem_Free(data->line_tools); + } + if (data->per_instruction_opcodes) { + PyMem_Free(data->per_instruction_opcodes); + } + if (data->per_instruction_tools) { + PyMem_Free(data->per_instruction_tools); + } + PyMem_Free(data); +} + static void code_dealloc(PyCodeObject *co) { @@ -1739,9 +1712,7 @@ code_dealloc(PyCodeObject *co) if (co->co_weakreflist != NULL) { PyObject_ClearWeakRefs((PyObject*)co); } - if (co->_co_linearray) { - PyMem_Free(co->_co_linearray); - } + free_monitoring_data(co->_co_monitoring); PyObject_Free(co); } @@ -1885,7 +1856,7 @@ code_hash(PyCodeObject *co) SCRAMBLE_IN(co->co_firstlineno); SCRAMBLE_IN(Py_SIZE(co)); for (int i = 0; i < Py_SIZE(co); i++) { - int deop = _PyOpcode_Deopt[_PyCode_CODE(co)[i].op.code]; + int deop = _Py_GetBaseOpcode(co, i); SCRAMBLE_IN(deop); SCRAMBLE_IN(_PyCode_CODE(co)[i].op.arg); i += _PyOpcode_Caches[deop]; @@ -2314,7 +2285,7 @@ _PyCode_ConstantKey(PyObject *op) void _PyStaticCode_Fini(PyCodeObject *co) { - deopt_code(_PyCode_CODE(co), Py_SIZE(co)); + deopt_code(co, _PyCode_CODE(co)); PyMem_Free(co->co_extra); if (co->_co_cached != NULL) { Py_CLEAR(co->_co_cached->_co_code); @@ -2329,10 +2300,8 @@ _PyStaticCode_Fini(PyCodeObject *co) PyObject_ClearWeakRefs((PyObject *)co); co->co_weakreflist = NULL; } - if (co->_co_linearray) { - PyMem_Free(co->_co_linearray); - co->_co_linearray = NULL; - } + free_monitoring_data(co->_co_monitoring); + co->_co_monitoring = NULL; } int diff --git a/Objects/frameobject.c b/Objects/frameobject.c index 63590d58809e3e..ef0070199ab2c0 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -17,7 +17,6 @@ static PyMemberDef frame_memberlist[] = { {"f_trace_lines", T_BOOL, OFF(f_trace_lines), 0}, - {"f_trace_opcodes", T_BOOL, OFF(f_trace_opcodes), 0}, {NULL} /* Sentinel */ }; @@ -104,24 +103,29 @@ frame_getback(PyFrameObject *f, void *closure) return res; } -// Given the index of the effective opcode, scan back to construct the oparg -// with EXTENDED_ARG. This only works correctly with *unquickened* code, -// obtained via a call to _PyCode_GetCode! -static unsigned int -get_arg(const _Py_CODEUNIT *codestr, Py_ssize_t i) +static PyObject * +frame_gettrace_opcodes(PyFrameObject *f, void *closure) { - _Py_CODEUNIT word; - unsigned int oparg = codestr[i].op.arg; - if (i >= 1 && (word = codestr[i-1]).op.code == EXTENDED_ARG) { - oparg |= word.op.arg << 8; - if (i >= 2 && (word = codestr[i-2]).op.code == EXTENDED_ARG) { - oparg |= word.op.arg << 16; - if (i >= 3 && (word = codestr[i-3]).op.code == EXTENDED_ARG) { - oparg |= word.op.arg << 24; - } - } + PyObject *result = f->f_trace_opcodes ? Py_True : Py_False; + return Py_NewRef(result); +} + +static int +frame_settrace_opcodes(PyFrameObject *f, PyObject* value, void *Py_UNUSED(ignored)) +{ + if (!PyBool_Check(value)) { + PyErr_SetString(PyExc_TypeError, + "attribute value type must be bool"); + return -1; + } + if (value == Py_True) { + f->f_trace_opcodes = 1; + _PyInterpreterState_GET()->f_opcode_trace_set = true; } - return oparg; + else { + f->f_trace_opcodes = 0; + } + return 0; } /* Model the evaluation stack, to determine which jumps @@ -299,46 +303,52 @@ mark_stacks(PyCodeObject *code_obj, int len) while (todo) { todo = 0; /* Scan instructions */ - for (i = 0; i < len; i++) { + for (i = 0; i < len;) { int64_t next_stack = stacks[i]; + opcode = _Py_GetBaseOpcode(code_obj, i); + int oparg = 0; + while (opcode == EXTENDED_ARG) { + oparg = (oparg << 8) | code[i].op.arg; + i++; + opcode = _Py_GetBaseOpcode(code_obj, i); + stacks[i] = next_stack; + } + int next_i = i + _PyOpcode_Caches[opcode] + 1; if (next_stack == UNINITIALIZED) { + i = next_i; continue; } - opcode = code[i].op.code; + oparg = (oparg << 8) | code[i].op.arg; switch (opcode) { case POP_JUMP_IF_FALSE: case POP_JUMP_IF_TRUE: { int64_t target_stack; - int j = get_arg(code, i); - j += i + 1; + int j = next_i + oparg; assert(j < len); - if (stacks[j] == UNINITIALIZED && j < i) { - todo = 1; - } next_stack = pop_value(next_stack); target_stack = next_stack; assert(stacks[j] == UNINITIALIZED || stacks[j] == target_stack); stacks[j] = target_stack; - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; } case SEND: - j = get_arg(code, i) + i + INLINE_CACHE_ENTRIES_SEND + 1; + j = oparg + i + INLINE_CACHE_ENTRIES_SEND + 1; assert(j < len); assert(stacks[j] == UNINITIALIZED || stacks[j] == next_stack); stacks[j] = next_stack; - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; case JUMP_FORWARD: - j = get_arg(code, i) + i + 1; + j = oparg + i + 1; assert(j < len); assert(stacks[j] == UNINITIALIZED || stacks[j] == next_stack); stacks[j] = next_stack; break; case JUMP_BACKWARD: case JUMP_BACKWARD_NO_INTERRUPT: - j = i + 1 - get_arg(code, i); + j = i + 1 - oparg; assert(j >= 0); assert(j < len); if (stacks[j] == UNINITIALIZED && j < i) { @@ -350,13 +360,13 @@ mark_stacks(PyCodeObject *code_obj, int len) case GET_ITER: case GET_AITER: next_stack = push_value(pop_value(next_stack), Iterator); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; case FOR_ITER: { int64_t target_stack = push_value(next_stack, Object); - stacks[i+1] = target_stack; - j = get_arg(code, i) + 1 + INLINE_CACHE_ENTRIES_FOR_ITER + i; + stacks[next_i] = target_stack; + j = oparg + 1 + INLINE_CACHE_ENTRIES_FOR_ITER + i; assert(j < len); assert(stacks[j] == UNINITIALIZED || stacks[j] == target_stack); stacks[j] = target_stack; @@ -364,16 +374,16 @@ mark_stacks(PyCodeObject *code_obj, int len) } case END_ASYNC_FOR: next_stack = pop_value(pop_value(next_stack)); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; case PUSH_EXC_INFO: next_stack = push_value(next_stack, Except); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; case POP_EXCEPT: assert(top_of_stack(next_stack) == Except); next_stack = pop_value(next_stack); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; case RETURN_VALUE: assert(pop_value(next_stack) == EMPTY_STACK); @@ -389,57 +399,62 @@ mark_stacks(PyCodeObject *code_obj, int len) break; case PUSH_NULL: next_stack = push_value(next_stack, Null); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; case LOAD_GLOBAL: { - int j = get_arg(code, i); + int j = oparg; if (j & 1) { next_stack = push_value(next_stack, Null); } next_stack = push_value(next_stack, Object); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; } case LOAD_ATTR: { assert(top_of_stack(next_stack) == Object); - int j = get_arg(code, i); + int j = oparg; if (j & 1) { next_stack = pop_value(next_stack); next_stack = push_value(next_stack, Null); next_stack = push_value(next_stack, Object); } - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; } case CALL: { - int args = get_arg(code, i); + int args = oparg; for (int j = 0; j < args+2; j++) { next_stack = pop_value(next_stack); } next_stack = push_value(next_stack, Object); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; } case SWAP: { - int n = get_arg(code, i); + int n = oparg; next_stack = stack_swap(next_stack, n); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; } case COPY: { - int n = get_arg(code, i); + int n = oparg; next_stack = push_value(next_stack, peek(next_stack, n)); - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; break; } + case CACHE: + case RESERVED: + { + assert(0); + } default: { - int delta = PyCompile_OpcodeStackEffect(opcode, get_arg(code, i)); + int delta = PyCompile_OpcodeStackEffect(opcode, oparg); assert(delta != PY_INVALID_STACK_EFFECT); while (delta < 0) { next_stack = pop_value(next_stack); @@ -449,9 +464,10 @@ mark_stacks(PyCodeObject *code_obj, int len) next_stack = push_value(next_stack, Object); delta--; } - stacks[i+1] = next_stack; + stacks[next_i] = next_stack; } } + i = next_i; } /* Scan exception table */ unsigned char *start = (unsigned char *)PyBytes_AS_STRING(code_obj->co_exceptiontable); @@ -646,31 +662,43 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno, void *Py_UNUSED(ignore * In addition, jumps are forbidden when not tracing, * as this is a debugging feature. */ - switch(PyThreadState_GET()->tracing_what) { - case PyTrace_EXCEPTION: - PyErr_SetString(PyExc_ValueError, - "can only jump from a 'line' trace event"); - return -1; - case PyTrace_CALL: + int what_event = PyThreadState_GET()->what_event; + if (what_event < 0) { + PyErr_Format(PyExc_ValueError, + "f_lineno can only be set in a trace function"); + return -1; + } + switch (what_event) { + case PY_MONITORING_EVENT_PY_RESUME: + case PY_MONITORING_EVENT_JUMP: + case PY_MONITORING_EVENT_BRANCH: + case PY_MONITORING_EVENT_LINE: + case PY_MONITORING_EVENT_PY_YIELD: + /* Setting f_lineno is allowed for the above events */ + break; + case PY_MONITORING_EVENT_PY_START: PyErr_Format(PyExc_ValueError, "can't jump from the 'call' trace event of a new frame"); return -1; - case PyTrace_LINE: - break; - case PyTrace_RETURN: - if (state == FRAME_SUSPENDED) { - break; - } - /* fall through */ - default: + case PY_MONITORING_EVENT_CALL: + case PY_MONITORING_EVENT_C_RETURN: PyErr_SetString(PyExc_ValueError, + "can't jump during a call"); + return -1; + case PY_MONITORING_EVENT_PY_RETURN: + case PY_MONITORING_EVENT_PY_UNWIND: + case PY_MONITORING_EVENT_PY_THROW: + case PY_MONITORING_EVENT_RAISE: + case PY_MONITORING_EVENT_C_RAISE: + case PY_MONITORING_EVENT_INSTRUCTION: + case PY_MONITORING_EVENT_EXCEPTION_HANDLED: + PyErr_Format(PyExc_ValueError, "can only jump from a 'line' trace event"); return -1; - } - if (!f->f_trace) { - PyErr_Format(PyExc_ValueError, - "f_lineno can only be set by a trace function"); - return -1; + default: + PyErr_SetString(PyExc_SystemError, + "unexpected event type"); + return -1; } int new_lineno; @@ -803,6 +831,7 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno, void *Py_UNUSED(ignore start_stack = pop_value(start_stack); } /* Finally set the new lasti and return OK. */ + f->f_last_traced_line = new_lineno; f->f_lineno = 0; f->f_frame->prev_instr = _PyCode_CODE(f->f_frame->f_code) + best_addr; return 0; @@ -823,7 +852,10 @@ frame_settrace(PyFrameObject *f, PyObject* v, void *closure) if (v == Py_None) { v = NULL; } - Py_XSETREF(f->f_trace, Py_XNewRef(v)); + if (v != f->f_trace) { + Py_XSETREF(f->f_trace, Py_XNewRef(v)); + f->f_last_traced_line = -1; + } return 0; } @@ -838,6 +870,7 @@ static PyGetSetDef frame_getsetlist[] = { {"f_globals", (getter)frame_getglobals, NULL, NULL}, {"f_builtins", (getter)frame_getbuiltins, NULL, NULL}, {"f_code", (getter)frame_getcode, NULL, NULL}, + {"f_trace_opcodes", (getter)frame_gettrace_opcodes, (setter)frame_settrace_opcodes, NULL}, {0} }; @@ -1023,6 +1056,7 @@ _PyFrame_New_NoTrack(PyCodeObject *code) f->f_trace_opcodes = 0; f->f_fast_as_locals = 0; f->f_lineno = 0; + f->f_last_traced_line = -1; return f; } diff --git a/Objects/funcobject.c b/Objects/funcobject.c index ce5d7bda32c032..78c1144afca2eb 100644 --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -942,7 +942,7 @@ functools_wraps(PyObject *wrapper, PyObject *wrapped) class C: @classmethod - def f(cls, arg1, arg2, ...): + def f(cls, arg1, arg2, argN): ... It can be called either on the class (e.g. C.f()) or on an instance @@ -1066,7 +1066,7 @@ To declare a class method, use this idiom:\n\ \n\ class C:\n\ @classmethod\n\ - def f(cls, arg1, arg2, ...):\n\ + def f(cls, arg1, arg2, argN):\n\ ...\n\ \n\ It can be called either on the class (e.g. C.f()) or on an instance\n\ @@ -1138,7 +1138,7 @@ PyClassMethod_New(PyObject *callable) class C: @staticmethod - def f(arg1, arg2, ...): + def f(arg1, arg2, argN): ... It can be called either on the class (e.g. C.f()) or on an instance @@ -1260,7 +1260,7 @@ To declare a static method, use this idiom:\n\ \n\ class C:\n\ @staticmethod\n\ - def f(arg1, arg2, ...):\n\ + def f(arg1, arg2, argN):\n\ ...\n\ \n\ It can be called either on the class (e.g. C.f()) or on an instance\n\ diff --git a/Objects/object.c b/Objects/object.c index 71f098eed37f51..56747fa193e178 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -1972,6 +1972,7 @@ extern PyTypeObject _Py_GenericAliasIterType; extern PyTypeObject _PyMemoryIter_Type; extern PyTypeObject _PyLineIterator; extern PyTypeObject _PyPositionsIterator; +extern PyTypeObject _PyLegacyEventHandler_Type; static PyTypeObject* static_types[] = { // The two most important base types: must be initialized first and @@ -2069,6 +2070,7 @@ static PyTypeObject* static_types[] = { &_PyHamt_BitmapNode_Type, &_PyHamt_CollisionNode_Type, &_PyHamt_Type, + &_PyLegacyEventHandler_Type, &_PyInterpreterID_Type, &_PyLineIterator, &_PyManagedBuffer_Type, diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 995547e7915f77..9ea458f30394e3 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -9137,13 +9137,15 @@ type_new_set_names(PyTypeObject *type) Py_DECREF(set_name); if (res == NULL) { - _PyErr_FormatFromCause(PyExc_RuntimeError, + _PyErr_FormatNote( "Error calling __set_name__ on '%.100s' instance %R " "in '%.100s'", Py_TYPE(value)->tp_name, key, type->tp_name); goto error; } - Py_DECREF(res); + else { + Py_DECREF(res); + } } Py_DECREF(names_to_set); diff --git a/PC/msvcrtmodule.c b/PC/msvcrtmodule.c index 6e8b423c3839a9..de9a88946aff3e 100644 --- a/PC/msvcrtmodule.c +++ b/PC/msvcrtmodule.c @@ -564,19 +564,6 @@ static struct PyMethodDef msvcrt_functions[] = { {NULL, NULL} }; - -static struct PyModuleDef msvcrtmodule = { - PyModuleDef_HEAD_INIT, - "msvcrt", - NULL, - -1, - msvcrt_functions, - NULL, - NULL, - NULL, - NULL -}; - static void insertint(PyObject *d, char *name, int value) { @@ -605,14 +592,10 @@ insertptr(PyObject *d, char *name, void *value) } } -PyMODINIT_FUNC -PyInit_msvcrt(void) +static int +exec_module(PyObject* m) { int st; - PyObject *m = PyModule_Create(&msvcrtmodule); - if (m == NULL) { - return NULL; - } PyObject *d = PyModule_GetDict(m); // Borrowed ref. /* constants for the locking() function's mode argument */ @@ -645,21 +628,21 @@ PyInit_msvcrt(void) st = PyModule_AddStringConstant(m, "VC_ASSEMBLY_PUBLICKEYTOKEN", _VC_ASSEMBLY_PUBLICKEYTOKEN); if (st < 0) { - goto error; + return -1; } #endif #ifdef _CRT_ASSEMBLY_VERSION st = PyModule_AddStringConstant(m, "CRT_ASSEMBLY_VERSION", _CRT_ASSEMBLY_VERSION); if (st < 0) { - goto error; + return -1; } #endif #ifdef __LIBRARIES_ASSEMBLY_NAME_PREFIX st = PyModule_AddStringConstant(m, "LIBRARIES_ASSEMBLY_NAME_PREFIX", __LIBRARIES_ASSEMBLY_NAME_PREFIX); if (st < 0) { - goto error; + return -1; } #endif @@ -671,20 +654,34 @@ PyInit_msvcrt(void) _VC_CRT_BUILD_VERSION, _VC_CRT_RBUILD_VERSION); if (version == NULL) { - goto error; + return -1; } st = PyModule_AddObjectRef(m, "CRT_ASSEMBLY_VERSION", version); Py_DECREF(version); if (st < 0) { - goto error; + return -1; } #endif /* make compiler warning quiet if st is unused */ (void)st; - return m; + return 0; +} + +static PyModuleDef_Slot msvcrt_slots[] = { + {Py_mod_exec, exec_module}, + {0, NULL} +}; -error: - Py_DECREF(m); - return NULL; +static struct PyModuleDef msvcrtmodule = { + .m_base = PyModuleDef_HEAD_INIT, + .m_name = "msvcrt", + .m_methods = msvcrt_functions, + .m_slots = msvcrt_slots, +}; + +PyMODINIT_FUNC +PyInit_msvcrt(void) +{ + return PyModuleDef_Init(&msvcrtmodule); } diff --git a/PCbuild/_freeze_module.vcxproj b/PCbuild/_freeze_module.vcxproj index 28eced6d4b2862..d897925f58c0de 100644 --- a/PCbuild/_freeze_module.vcxproj +++ b/PCbuild/_freeze_module.vcxproj @@ -177,13 +177,13 @@ + - @@ -192,6 +192,7 @@ + @@ -208,6 +209,8 @@ + + diff --git a/PCbuild/_freeze_module.vcxproj.filters b/PCbuild/_freeze_module.vcxproj.filters index e4faa89bb831d9..176935a63c4852 100644 --- a/PCbuild/_freeze_module.vcxproj.filters +++ b/PCbuild/_freeze_module.vcxproj.filters @@ -28,6 +28,9 @@ Source Files + + Source Files + Source Files @@ -76,9 +79,6 @@ Source Files - - Source Files - Source Files @@ -142,6 +142,9 @@ Source Files + + Source Files + Source Files @@ -211,6 +214,12 @@ Source Files + + Source Files + + + Source Files + Source Files diff --git a/PCbuild/find_python.bat b/PCbuild/find_python.bat index 11d6cba7a172c9..7af5503d80a0fc 100644 --- a/PCbuild/find_python.bat +++ b/PCbuild/find_python.bat @@ -42,7 +42,7 @@ @if NOT "%HOST_PYTHON%"=="" @%HOST_PYTHON% -Ec "import sys; assert sys.version_info[:2] >= (3, 9)" >nul 2>nul && (set PYTHON="%HOST_PYTHON%") && (set _Py_Python_Source=found as HOST_PYTHON) && goto :found @rem If py.exe finds a recent enough version, use that one -@for %%p in (3.10 3.9) do @py -%%p -EV >nul 2>&1 && (set PYTHON=py -%%p) && (set _Py_Python_Source=found %%p with py.exe) && goto :found +@for %%p in (3.11 3.10 3.9) do @py -%%p -EV >nul 2>&1 && (set PYTHON=py -%%p) && (set _Py_Python_Source=found %%p with py.exe) && goto :found @if NOT exist "%_Py_EXTERNALS_DIR%" mkdir "%_Py_EXTERNALS_DIR%" @set _Py_NUGET=%NUGET% diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 29f32db579fa40..8aafcb786a6064 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -500,13 +500,13 @@ + - @@ -514,6 +514,7 @@ + @@ -531,6 +532,8 @@ + + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 6a622fd93930ad..07476f30833372 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -1094,6 +1094,9 @@ Python + + Python + Python @@ -1109,9 +1112,6 @@ Python - - Python - Python @@ -1130,6 +1130,9 @@ Python + + Python + Python @@ -1175,6 +1178,12 @@ Source Files + + Source Files + + + Source Files + Python diff --git a/Programs/_freeze_module.c b/Programs/_freeze_module.c index 90fc2dc6e87da8..e55f1d56745c4d 100644 --- a/Programs/_freeze_module.c +++ b/Programs/_freeze_module.c @@ -1,6 +1,5 @@ /* This is built as a stand-alone executable by the Makefile, and helps turn - modules into frozen modules (like Lib/importlib/_bootstrap.py - into Python/importlib.h). + modules into frozen modules. This is used directly by Tools/build/freeze_modules.py, and indirectly by "make regen-frozen". diff --git a/Python/assemble.c b/Python/assemble.c new file mode 100644 index 00000000000000..e5a361b230cf1c --- /dev/null +++ b/Python/assemble.c @@ -0,0 +1,602 @@ +#include + +#include "Python.h" +#include "pycore_flowgraph.h" +#include "pycore_compile.h" +#include "pycore_pymem.h" // _PyMem_IsPtrFreed() +#include "pycore_code.h" // write_location_entry_start() + + +#define DEFAULT_CODE_SIZE 128 +#define DEFAULT_LNOTAB_SIZE 16 +#define DEFAULT_CNOTAB_SIZE 32 + +#undef SUCCESS +#undef ERROR +#define SUCCESS 0 +#define ERROR -1 + +#define RETURN_IF_ERROR(X) \ + if ((X) == -1) { \ + return ERROR; \ + } + +typedef _PyCompilerSrcLocation location; +typedef _PyCfgInstruction cfg_instr; +typedef _PyCfgBasicblock basicblock; + +static inline bool +same_location(location a, location b) +{ + return a.lineno == b.lineno && + a.end_lineno == b.end_lineno && + a.col_offset == b.col_offset && + a.end_col_offset == b.end_col_offset; +} + +struct assembler { + PyObject *a_bytecode; /* bytes containing bytecode */ + int a_offset; /* offset into bytecode */ + PyObject *a_except_table; /* bytes containing exception table */ + int a_except_table_off; /* offset into exception table */ + /* Location Info */ + int a_lineno; /* lineno of last emitted instruction */ + PyObject* a_linetable; /* bytes containing location info */ + int a_location_off; /* offset of last written location info frame */ +}; + +static int +assemble_init(struct assembler *a, int firstlineno) +{ + memset(a, 0, sizeof(struct assembler)); + a->a_lineno = firstlineno; + a->a_linetable = NULL; + a->a_location_off = 0; + a->a_except_table = NULL; + a->a_bytecode = PyBytes_FromStringAndSize(NULL, DEFAULT_CODE_SIZE); + if (a->a_bytecode == NULL) { + goto error; + } + a->a_linetable = PyBytes_FromStringAndSize(NULL, DEFAULT_CNOTAB_SIZE); + if (a->a_linetable == NULL) { + goto error; + } + a->a_except_table = PyBytes_FromStringAndSize(NULL, DEFAULT_LNOTAB_SIZE); + if (a->a_except_table == NULL) { + goto error; + } + return SUCCESS; +error: + Py_XDECREF(a->a_bytecode); + Py_XDECREF(a->a_linetable); + Py_XDECREF(a->a_except_table); + return ERROR; +} + +static void +assemble_free(struct assembler *a) +{ + Py_XDECREF(a->a_bytecode); + Py_XDECREF(a->a_linetable); + Py_XDECREF(a->a_except_table); +} + +static inline void +write_except_byte(struct assembler *a, int byte) { + unsigned char *p = (unsigned char *) PyBytes_AS_STRING(a->a_except_table); + p[a->a_except_table_off++] = byte; +} + +#define CONTINUATION_BIT 64 + +static void +assemble_emit_exception_table_item(struct assembler *a, int value, int msb) +{ + assert ((msb | 128) == 128); + assert(value >= 0 && value < (1 << 30)); + if (value >= 1 << 24) { + write_except_byte(a, (value >> 24) | CONTINUATION_BIT | msb); + msb = 0; + } + if (value >= 1 << 18) { + write_except_byte(a, ((value >> 18)&0x3f) | CONTINUATION_BIT | msb); + msb = 0; + } + if (value >= 1 << 12) { + write_except_byte(a, ((value >> 12)&0x3f) | CONTINUATION_BIT | msb); + msb = 0; + } + if (value >= 1 << 6) { + write_except_byte(a, ((value >> 6)&0x3f) | CONTINUATION_BIT | msb); + msb = 0; + } + write_except_byte(a, (value&0x3f) | msb); +} + +/* See Objects/exception_handling_notes.txt for details of layout */ +#define MAX_SIZE_OF_ENTRY 20 + +static int +assemble_emit_exception_table_entry(struct assembler *a, int start, int end, basicblock *handler) +{ + Py_ssize_t len = PyBytes_GET_SIZE(a->a_except_table); + if (a->a_except_table_off + MAX_SIZE_OF_ENTRY >= len) { + RETURN_IF_ERROR(_PyBytes_Resize(&a->a_except_table, len * 2)); + } + int size = end-start; + assert(end > start); + int target = handler->b_offset; + int depth = handler->b_startdepth - 1; + if (handler->b_preserve_lasti) { + depth -= 1; + } + assert(depth >= 0); + int depth_lasti = (depth<<1) | handler->b_preserve_lasti; + assemble_emit_exception_table_item(a, start, (1<<7)); + assemble_emit_exception_table_item(a, size, 0); + assemble_emit_exception_table_item(a, target, 0); + assemble_emit_exception_table_item(a, depth_lasti, 0); + return SUCCESS; +} + +static int +assemble_exception_table(struct assembler *a, basicblock *entryblock) +{ + basicblock *b; + int ioffset = 0; + basicblock *handler = NULL; + int start = -1; + for (b = entryblock; b != NULL; b = b->b_next) { + ioffset = b->b_offset; + for (int i = 0; i < b->b_iused; i++) { + cfg_instr *instr = &b->b_instr[i]; + if (instr->i_except != handler) { + if (handler != NULL) { + RETURN_IF_ERROR( + assemble_emit_exception_table_entry(a, start, ioffset, handler)); + } + start = ioffset; + handler = instr->i_except; + } + ioffset += _PyCfg_InstrSize(instr); + } + } + if (handler != NULL) { + RETURN_IF_ERROR(assemble_emit_exception_table_entry(a, start, ioffset, handler)); + } + return SUCCESS; +} + + +/* Code location emitting code. See locations.md for a description of the format. */ + +#define MSB 0x80 + +static void +write_location_byte(struct assembler* a, int val) +{ + PyBytes_AS_STRING(a->a_linetable)[a->a_location_off] = val&255; + a->a_location_off++; +} + + +static uint8_t * +location_pointer(struct assembler* a) +{ + return (uint8_t *)PyBytes_AS_STRING(a->a_linetable) + + a->a_location_off; +} + +static void +write_location_first_byte(struct assembler* a, int code, int length) +{ + a->a_location_off += write_location_entry_start( + location_pointer(a), code, length); +} + +static void +write_location_varint(struct assembler* a, unsigned int val) +{ + uint8_t *ptr = location_pointer(a); + a->a_location_off += write_varint(ptr, val); +} + + +static void +write_location_signed_varint(struct assembler* a, int val) +{ + uint8_t *ptr = location_pointer(a); + a->a_location_off += write_signed_varint(ptr, val); +} + +static void +write_location_info_short_form(struct assembler* a, int length, int column, int end_column) +{ + assert(length > 0 && length <= 8); + int column_low_bits = column & 7; + int column_group = column >> 3; + assert(column < 80); + assert(end_column >= column); + assert(end_column - column < 16); + write_location_first_byte(a, PY_CODE_LOCATION_INFO_SHORT0 + column_group, length); + write_location_byte(a, (column_low_bits << 4) | (end_column - column)); +} + +static void +write_location_info_oneline_form(struct assembler* a, int length, int line_delta, int column, int end_column) +{ + assert(length > 0 && length <= 8); + assert(line_delta >= 0 && line_delta < 3); + assert(column < 128); + assert(end_column < 128); + write_location_first_byte(a, PY_CODE_LOCATION_INFO_ONE_LINE0 + line_delta, length); + write_location_byte(a, column); + write_location_byte(a, end_column); +} + +static void +write_location_info_long_form(struct assembler* a, location loc, int length) +{ + assert(length > 0 && length <= 8); + write_location_first_byte(a, PY_CODE_LOCATION_INFO_LONG, length); + write_location_signed_varint(a, loc.lineno - a->a_lineno); + assert(loc.end_lineno >= loc.lineno); + write_location_varint(a, loc.end_lineno - loc.lineno); + write_location_varint(a, loc.col_offset + 1); + write_location_varint(a, loc.end_col_offset + 1); +} + +static void +write_location_info_none(struct assembler* a, int length) +{ + write_location_first_byte(a, PY_CODE_LOCATION_INFO_NONE, length); +} + +static void +write_location_info_no_column(struct assembler* a, int length, int line_delta) +{ + write_location_first_byte(a, PY_CODE_LOCATION_INFO_NO_COLUMNS, length); + write_location_signed_varint(a, line_delta); +} + +#define THEORETICAL_MAX_ENTRY_SIZE 25 /* 1 + 6 + 6 + 6 + 6 */ + + +static int +write_location_info_entry(struct assembler* a, location loc, int isize) +{ + Py_ssize_t len = PyBytes_GET_SIZE(a->a_linetable); + if (a->a_location_off + THEORETICAL_MAX_ENTRY_SIZE >= len) { + assert(len > THEORETICAL_MAX_ENTRY_SIZE); + RETURN_IF_ERROR(_PyBytes_Resize(&a->a_linetable, len*2)); + } + if (loc.lineno < 0) { + write_location_info_none(a, isize); + return SUCCESS; + } + int line_delta = loc.lineno - a->a_lineno; + int column = loc.col_offset; + int end_column = loc.end_col_offset; + assert(column >= -1); + assert(end_column >= -1); + if (column < 0 || end_column < 0) { + if (loc.end_lineno == loc.lineno || loc.end_lineno == -1) { + write_location_info_no_column(a, isize, line_delta); + a->a_lineno = loc.lineno; + return SUCCESS; + } + } + else if (loc.end_lineno == loc.lineno) { + if (line_delta == 0 && column < 80 && end_column - column < 16 && end_column >= column) { + write_location_info_short_form(a, isize, column, end_column); + return SUCCESS; + } + if (line_delta >= 0 && line_delta < 3 && column < 128 && end_column < 128) { + write_location_info_oneline_form(a, isize, line_delta, column, end_column); + a->a_lineno = loc.lineno; + return SUCCESS; + } + } + write_location_info_long_form(a, loc, isize); + a->a_lineno = loc.lineno; + return SUCCESS; +} + +static int +assemble_emit_location(struct assembler* a, location loc, int isize) +{ + if (isize == 0) { + return SUCCESS; + } + while (isize > 8) { + RETURN_IF_ERROR(write_location_info_entry(a, loc, 8)); + isize -= 8; + } + return write_location_info_entry(a, loc, isize); +} + +static int +assemble_location_info(struct assembler *a, basicblock *entryblock, int firstlineno) +{ + a->a_lineno = firstlineno; + location loc = NO_LOCATION; + int size = 0; + for (basicblock *b = entryblock; b != NULL; b = b->b_next) { + for (int j = 0; j < b->b_iused; j++) { + if (!same_location(loc, b->b_instr[j].i_loc)) { + RETURN_IF_ERROR(assemble_emit_location(a, loc, size)); + loc = b->b_instr[j].i_loc; + size = 0; + } + size += _PyCfg_InstrSize(&b->b_instr[j]); + } + } + RETURN_IF_ERROR(assemble_emit_location(a, loc, size)); + return SUCCESS; +} + +static void +write_instr(_Py_CODEUNIT *codestr, cfg_instr *instruction, int ilen) +{ + int opcode = instruction->i_opcode; + assert(!IS_PSEUDO_OPCODE(opcode)); + int oparg = instruction->i_oparg; + assert(HAS_ARG(opcode) || oparg == 0); + int caches = _PyOpcode_Caches[opcode]; + switch (ilen - caches) { + case 4: + codestr->op.code = EXTENDED_ARG; + codestr->op.arg = (oparg >> 24) & 0xFF; + codestr++; + /* fall through */ + case 3: + codestr->op.code = EXTENDED_ARG; + codestr->op.arg = (oparg >> 16) & 0xFF; + codestr++; + /* fall through */ + case 2: + codestr->op.code = EXTENDED_ARG; + codestr->op.arg = (oparg >> 8) & 0xFF; + codestr++; + /* fall through */ + case 1: + codestr->op.code = opcode; + codestr->op.arg = oparg & 0xFF; + codestr++; + break; + default: + Py_UNREACHABLE(); + } + while (caches--) { + codestr->op.code = CACHE; + codestr->op.arg = 0; + codestr++; + } +} + +/* assemble_emit_instr() + Extend the bytecode with a new instruction. + Update lnotab if necessary. +*/ + +static int +assemble_emit_instr(struct assembler *a, cfg_instr *i) +{ + Py_ssize_t len = PyBytes_GET_SIZE(a->a_bytecode); + _Py_CODEUNIT *code; + + int size = _PyCfg_InstrSize(i); + if (a->a_offset + size >= len / (int)sizeof(_Py_CODEUNIT)) { + if (len > PY_SSIZE_T_MAX / 2) { + return ERROR; + } + RETURN_IF_ERROR(_PyBytes_Resize(&a->a_bytecode, len * 2)); + } + code = (_Py_CODEUNIT *)PyBytes_AS_STRING(a->a_bytecode) + a->a_offset; + a->a_offset += size; + write_instr(code, i, size); + return SUCCESS; +} + +static int +assemble_emit(struct assembler *a, basicblock *entryblock, int first_lineno, + PyObject *const_cache) +{ + RETURN_IF_ERROR(assemble_init(a, first_lineno)); + + for (basicblock *b = entryblock; b != NULL; b = b->b_next) { + for (int j = 0; j < b->b_iused; j++) { + RETURN_IF_ERROR(assemble_emit_instr(a, &b->b_instr[j])); + } + } + + RETURN_IF_ERROR(assemble_location_info(a, entryblock, a->a_lineno)); + + RETURN_IF_ERROR(assemble_exception_table(a, entryblock)); + + RETURN_IF_ERROR(_PyBytes_Resize(&a->a_except_table, a->a_except_table_off)); + RETURN_IF_ERROR(_PyCompile_ConstCacheMergeOne(const_cache, &a->a_except_table)); + + RETURN_IF_ERROR(_PyBytes_Resize(&a->a_linetable, a->a_location_off)); + RETURN_IF_ERROR(_PyCompile_ConstCacheMergeOne(const_cache, &a->a_linetable)); + + RETURN_IF_ERROR(_PyBytes_Resize(&a->a_bytecode, a->a_offset * sizeof(_Py_CODEUNIT))); + RETURN_IF_ERROR(_PyCompile_ConstCacheMergeOne(const_cache, &a->a_bytecode)); + return SUCCESS; +} + +static PyObject * +dict_keys_inorder(PyObject *dict, Py_ssize_t offset) +{ + PyObject *tuple, *k, *v; + Py_ssize_t i, pos = 0, size = PyDict_GET_SIZE(dict); + + tuple = PyTuple_New(size); + if (tuple == NULL) + return NULL; + while (PyDict_Next(dict, &pos, &k, &v)) { + i = PyLong_AS_LONG(v); + assert((i - offset) < size); + assert((i - offset) >= 0); + PyTuple_SET_ITEM(tuple, i - offset, Py_NewRef(k)); + } + return tuple; +} + +// This is in codeobject.c. +extern void _Py_set_localsplus_info(int, PyObject *, unsigned char, + PyObject *, PyObject *); + +static void +compute_localsplus_info(_PyCompile_CodeUnitMetadata *umd, int nlocalsplus, + PyObject *names, PyObject *kinds) +{ + PyObject *k, *v; + Py_ssize_t pos = 0; + while (PyDict_Next(umd->u_varnames, &pos, &k, &v)) { + int offset = (int)PyLong_AS_LONG(v); + assert(offset >= 0); + assert(offset < nlocalsplus); + // For now we do not distinguish arg kinds. + _PyLocals_Kind kind = CO_FAST_LOCAL; + if (PyDict_GetItem(umd->u_cellvars, k) != NULL) { + kind |= CO_FAST_CELL; + } + _Py_set_localsplus_info(offset, k, kind, names, kinds); + } + int nlocals = (int)PyDict_GET_SIZE(umd->u_varnames); + + // This counter mirrors the fix done in fix_cell_offsets(). + int numdropped = 0; + pos = 0; + while (PyDict_Next(umd->u_cellvars, &pos, &k, &v)) { + if (PyDict_GetItem(umd->u_varnames, k) != NULL) { + // Skip cells that are already covered by locals. + numdropped += 1; + continue; + } + int offset = (int)PyLong_AS_LONG(v); + assert(offset >= 0); + offset += nlocals - numdropped; + assert(offset < nlocalsplus); + _Py_set_localsplus_info(offset, k, CO_FAST_CELL, names, kinds); + } + + pos = 0; + while (PyDict_Next(umd->u_freevars, &pos, &k, &v)) { + int offset = (int)PyLong_AS_LONG(v); + assert(offset >= 0); + offset += nlocals - numdropped; + assert(offset < nlocalsplus); + _Py_set_localsplus_info(offset, k, CO_FAST_FREE, names, kinds); + } +} + +static PyCodeObject * +makecode(_PyCompile_CodeUnitMetadata *umd, struct assembler *a, PyObject *const_cache, + PyObject *constslist, int maxdepth, int nlocalsplus, int code_flags, + PyObject *filename) +{ + PyCodeObject *co = NULL; + PyObject *names = NULL; + PyObject *consts = NULL; + PyObject *localsplusnames = NULL; + PyObject *localspluskinds = NULL; + names = dict_keys_inorder(umd->u_names, 0); + if (!names) { + goto error; + } + if (_PyCompile_ConstCacheMergeOne(const_cache, &names) < 0) { + goto error; + } + + consts = PyList_AsTuple(constslist); /* PyCode_New requires a tuple */ + if (consts == NULL) { + goto error; + } + if (_PyCompile_ConstCacheMergeOne(const_cache, &consts) < 0) { + goto error; + } + + assert(umd->u_posonlyargcount < INT_MAX); + assert(umd->u_argcount < INT_MAX); + assert(umd->u_kwonlyargcount < INT_MAX); + int posonlyargcount = (int)umd->u_posonlyargcount; + int posorkwargcount = (int)umd->u_argcount; + assert(INT_MAX - posonlyargcount - posorkwargcount > 0); + int kwonlyargcount = (int)umd->u_kwonlyargcount; + + localsplusnames = PyTuple_New(nlocalsplus); + if (localsplusnames == NULL) { + goto error; + } + localspluskinds = PyBytes_FromStringAndSize(NULL, nlocalsplus); + if (localspluskinds == NULL) { + goto error; + } + compute_localsplus_info(umd, nlocalsplus, localsplusnames, localspluskinds); + + struct _PyCodeConstructor con = { + .filename = filename, + .name = umd->u_name, + .qualname = umd->u_qualname ? umd->u_qualname : umd->u_name, + .flags = code_flags, + + .code = a->a_bytecode, + .firstlineno = umd->u_firstlineno, + .linetable = a->a_linetable, + + .consts = consts, + .names = names, + + .localsplusnames = localsplusnames, + .localspluskinds = localspluskinds, + + .argcount = posonlyargcount + posorkwargcount, + .posonlyargcount = posonlyargcount, + .kwonlyargcount = kwonlyargcount, + + .stacksize = maxdepth, + + .exceptiontable = a->a_except_table, + }; + + if (_PyCode_Validate(&con) < 0) { + goto error; + } + + if (_PyCompile_ConstCacheMergeOne(const_cache, &localsplusnames) < 0) { + goto error; + } + con.localsplusnames = localsplusnames; + + co = _PyCode_New(&con); + if (co == NULL) { + goto error; + } + +error: + Py_XDECREF(names); + Py_XDECREF(consts); + Py_XDECREF(localsplusnames); + Py_XDECREF(localspluskinds); + return co; +} + + +PyCodeObject * +_PyAssemble_MakeCodeObject(_PyCompile_CodeUnitMetadata *umd, PyObject *const_cache, + PyObject *consts, int maxdepth, basicblock *entryblock, + int nlocalsplus, int code_flags, PyObject *filename) +{ + PyCodeObject *co = NULL; + + struct assembler a; + int res = assemble_emit(&a, entryblock, umd->u_firstlineno, const_cache); + if (res == SUCCESS) { + co = makecode(umd, &a, const_cache, consts, maxdepth, nlocalsplus, + code_flags, filename); + } + assemble_free(&a); + return co; +} diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 72f85cc92b0c92..5c6398aba198f9 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -14,6 +14,7 @@ #include "pycore_function.h" #include "pycore_intrinsics.h" #include "pycore_long.h" // _PyLong_GetZero() +#include "pycore_instruments.h" #include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_moduleobject.h" // PyModuleObject #include "pycore_opcode.h" // EXTRA_CASES @@ -134,11 +135,45 @@ dummy_func( inst(RESUME, (--)) { assert(tstate->cframe == &cframe); assert(frame == cframe.current_frame); - if (_Py_atomic_load_relaxed_int32(eval_breaker) && oparg < 2) { + /* Possibly combine this with eval breaker */ + if (frame->f_code->_co_instrumentation_version != tstate->interp->monitoring_version) { + int err = _Py_Instrument(frame->f_code, tstate->interp); + ERROR_IF(err, error); + next_instr--; + } + else if (_Py_atomic_load_relaxed_int32(eval_breaker) && oparg < 2) { goto handle_eval_breaker; } } + inst(INSTRUMENTED_RESUME, (--)) { + /* Possible performance enhancement: + * We need to check the eval breaker anyway, can we + * combine the instrument verison check and the eval breaker test? + */ + if (frame->f_code->_co_instrumentation_version != tstate->interp->monitoring_version) { + if (_Py_Instrument(frame->f_code, tstate->interp)) { + goto error; + } + next_instr--; + } + else { + _PyFrame_SetStackPointer(frame, stack_pointer); + int err = _Py_call_instrumentation( + tstate, oparg > 0, frame, next_instr-1); + stack_pointer = _PyFrame_GetStackPointer(frame); + ERROR_IF(err, error); + if (frame->prev_instr != next_instr-1) { + /* Instrumentation has jumped */ + next_instr = frame->prev_instr; + DISPATCH(); + } + if (_Py_atomic_load_relaxed_int32(eval_breaker) && oparg < 2) { + goto handle_eval_breaker; + } + } + } + inst(LOAD_CLOSURE, (-- value)) { /* We keep LOAD_CLOSURE so that the bytecode stays more readable. */ value = GETLOCAL(oparg); @@ -183,6 +218,34 @@ dummy_func( macro(END_FOR) = POP_TOP + POP_TOP; + inst(INSTRUMENTED_END_FOR, (receiver, value --)) { + /* Need to create a fake StopIteration error here, + * to conform to PEP 380 */ + if (PyGen_Check(receiver)) { + PyErr_SetObject(PyExc_StopIteration, value); + if (monitor_stop_iteration(tstate, frame, next_instr-1)) { + goto error; + } + PyErr_SetRaisedException(NULL); + } + DECREF_INPUTS(); + } + + inst(END_SEND, (receiver, value -- value)) { + Py_DECREF(receiver); + } + + inst(INSTRUMENTED_END_SEND, (receiver, value -- value)) { + if (PyGen_Check(receiver) || PyCoro_CheckExact(receiver)) { + PyErr_SetObject(PyExc_StopIteration, value); + if (monitor_stop_iteration(tstate, frame, next_instr-1)) { + goto error; + } + PyErr_SetRaisedException(NULL); + } + Py_DECREF(receiver); + } + inst(UNARY_NEGATIVE, (value -- res)) { res = PyNumber_Negative(value); DECREF_INPUTS(); @@ -222,7 +285,6 @@ dummy_func( inst(BINARY_OP_MULTIPLY_INT, (unused/1, left, right -- prod)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -233,7 +295,6 @@ dummy_func( } inst(BINARY_OP_MULTIPLY_FLOAT, (unused/1, left, right -- prod)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -243,7 +304,6 @@ dummy_func( } inst(BINARY_OP_SUBTRACT_INT, (unused/1, left, right -- sub)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -254,7 +314,6 @@ dummy_func( } inst(BINARY_OP_SUBTRACT_FLOAT, (unused/1, left, right -- sub)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -263,7 +322,6 @@ dummy_func( } inst(BINARY_OP_ADD_UNICODE, (unused/1, left, right -- res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(Py_TYPE(right) != Py_TYPE(left), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -280,7 +338,6 @@ dummy_func( // specializations, but there is no output. // At the end we just skip over the STORE_FAST. inst(BINARY_OP_INPLACE_ADD_UNICODE, (left, right --)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(Py_TYPE(right) != Py_TYPE(left), BINARY_OP); _Py_CODEUNIT true_next = next_instr[INLINE_CACHE_ENTRIES_BINARY_OP]; @@ -310,7 +367,6 @@ dummy_func( } inst(BINARY_OP_ADD_FLOAT, (unused/1, left, right -- sum)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(Py_TYPE(right) != Py_TYPE(left), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -320,7 +376,6 @@ dummy_func( } inst(BINARY_OP_ADD_INT, (unused/1, left, right -- sum)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(Py_TYPE(right) != Py_TYPE(left), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -342,7 +397,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PyBinarySubscrCache *cache = (_PyBinarySubscrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_BinarySubscr(container, sub, next_instr); DISPATCH_SAME_OPARG(); @@ -386,7 +440,6 @@ dummy_func( } inst(BINARY_SUBSCR_LIST_INT, (unused/1, list, sub -- res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), BINARY_SUBSCR); @@ -403,7 +456,6 @@ dummy_func( } inst(BINARY_SUBSCR_TUPLE_INT, (unused/1, tuple, sub -- res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyTuple_CheckExact(tuple), BINARY_SUBSCR); @@ -420,7 +472,6 @@ dummy_func( } inst(BINARY_SUBSCR_DICT, (unused/1, dict, sub -- res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyDict_CheckExact(dict), BINARY_SUBSCR); STAT_INC(BINARY_SUBSCR, hit); res = PyDict_GetItemWithError(dict, sub); @@ -479,7 +530,6 @@ dummy_func( inst(STORE_SUBSCR, (counter/1, v, container, sub -- )) { #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_StoreSubscr(container, sub, next_instr); DISPATCH_SAME_OPARG(); @@ -497,7 +547,6 @@ dummy_func( } inst(STORE_SUBSCR_LIST_INT, (unused/1, value, list, sub -- )) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyLong_CheckExact(sub), STORE_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), STORE_SUBSCR); @@ -517,7 +566,6 @@ dummy_func( } inst(STORE_SUBSCR_DICT, (unused/1, value, dict, sub -- )) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyDict_CheckExact(dict), STORE_SUBSCR); STAT_INC(STORE_SUBSCR, hit); int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub, value); @@ -573,7 +621,6 @@ dummy_func( assert(EMPTY()); /* Restore previous cframe and return. */ tstate->cframe = cframe.previous; - tstate->cframe->use_tracing = cframe.use_tracing; assert(tstate->cframe->current_frame == frame->previous); assert(!_PyErr_Occurred(tstate)); _Py_LeaveRecursiveCallTstate(tstate); @@ -584,8 +631,24 @@ dummy_func( STACK_SHRINK(1); assert(EMPTY()); _PyFrame_SetStackPointer(frame, stack_pointer); - TRACE_FUNCTION_EXIT(); - DTRACE_FUNCTION_EXIT(); + _Py_LeaveRecursiveCallPy(tstate); + assert(frame != &entry_frame); + // GH-99729: We need to unlink the frame *before* clearing it: + _PyInterpreterFrame *dying = frame; + frame = cframe.current_frame = dying->previous; + _PyEvalFrameClearAndPop(tstate, dying); + _PyFrame_StackPush(frame, retval); + goto resume_frame; + } + + inst(INSTRUMENTED_RETURN_VALUE, (retval --)) { + int err = _Py_call_instrumentation_arg( + tstate, PY_MONITORING_EVENT_PY_RETURN, + frame, next_instr-1, retval); + if (err) goto error; + STACK_SHRINK(1); + assert(EMPTY()); + _PyFrame_SetStackPointer(frame, stack_pointer); _Py_LeaveRecursiveCallPy(tstate); assert(frame != &entry_frame); // GH-99729: We need to unlink the frame *before* clearing it: @@ -601,8 +664,25 @@ dummy_func( Py_INCREF(retval); assert(EMPTY()); _PyFrame_SetStackPointer(frame, stack_pointer); - TRACE_FUNCTION_EXIT(); - DTRACE_FUNCTION_EXIT(); + _Py_LeaveRecursiveCallPy(tstate); + assert(frame != &entry_frame); + // GH-99729: We need to unlink the frame *before* clearing it: + _PyInterpreterFrame *dying = frame; + frame = cframe.current_frame = dying->previous; + _PyEvalFrameClearAndPop(tstate, dying); + _PyFrame_StackPush(frame, retval); + goto resume_frame; + } + + inst(INSTRUMENTED_RETURN_CONST, (--)) { + PyObject *retval = GETITEM(frame->f_code->co_consts, oparg); + int err = _Py_call_instrumentation_arg( + tstate, PY_MONITORING_EVENT_PY_RETURN, + frame, next_instr-1, retval); + if (err) goto error; + Py_INCREF(retval); + assert(EMPTY()); + _PyFrame_SetStackPointer(frame, stack_pointer); _Py_LeaveRecursiveCallPy(tstate); assert(frame != &entry_frame); // GH-99729: We need to unlink the frame *before* clearing it: @@ -730,7 +810,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PySendCache *cache = (_PySendCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_Send(receiver, next_instr); DISPATCH_SAME_OPARG(); @@ -739,6 +818,20 @@ dummy_func( DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ assert(frame != &entry_frame); + if ((Py_TYPE(receiver) == &PyGen_Type || + Py_TYPE(receiver) == &PyCoro_Type) && ((PyGenObject *)receiver)->gi_frame_state < FRAME_EXECUTING) + { + PyGenObject *gen = (PyGenObject *)receiver; + _PyInterpreterFrame *gen_frame = (_PyInterpreterFrame *)gen->gi_iframe; + frame->yield_offset = oparg; + STACK_SHRINK(1); + _PyFrame_StackPush(gen_frame, v); + gen->gi_frame_state = FRAME_EXECUTING; + gen->gi_exc_state.previous_item = tstate->exc_info; + tstate->exc_info = &gen->gi_exc_state; + JUMPBY(INLINE_CACHE_ENTRIES_SEND + oparg); + DISPATCH_INLINED(gen_frame); + } if (Py_IsNone(v) && PyIter_Check(receiver)) { retval = Py_TYPE(receiver)->tp_iternext(receiver); } @@ -746,26 +839,22 @@ dummy_func( retval = PyObject_CallMethodOneArg(receiver, &_Py_ID(send), v); } if (retval == NULL) { - if (tstate->c_tracefunc != NULL - && _PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) - call_exc_trace(tstate->c_tracefunc, tstate->c_traceobj, tstate, frame); + if (_PyErr_ExceptionMatches(tstate, PyExc_StopIteration) + ) { + monitor_raise(tstate, frame, next_instr-1); + } if (_PyGen_FetchStopIterationValue(&retval) == 0) { assert(retval != NULL); JUMPBY(oparg); } else { - assert(retval == NULL); goto error; } } - else { - assert(retval != NULL); - } Py_DECREF(v); } inst(SEND_GEN, (unused/1, receiver, v -- receiver)) { - assert(cframe.use_tracing == 0); PyGenObject *gen = (PyGenObject *)receiver; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type && Py_TYPE(gen) != &PyCoro_Type, SEND); @@ -782,6 +871,26 @@ dummy_func( DISPATCH_INLINED(gen_frame); } + inst(INSTRUMENTED_YIELD_VALUE, (retval -- unused)) { + assert(frame != &entry_frame); + PyGenObject *gen = _PyFrame_GetGenerator(frame); + gen->gi_frame_state = FRAME_SUSPENDED; + _PyFrame_SetStackPointer(frame, stack_pointer - 1); + int err = _Py_call_instrumentation_arg( + tstate, PY_MONITORING_EVENT_PY_YIELD, + frame, next_instr-1, retval); + if (err) goto error; + tstate->exc_info = gen->gi_exc_state.previous_item; + gen->gi_exc_state.previous_item = NULL; + _Py_LeaveRecursiveCallPy(tstate); + _PyInterpreterFrame *gen_frame = frame; + frame = cframe.current_frame = frame->previous; + gen_frame->previous = NULL; + frame->prev_instr -= frame->yield_offset; + _PyFrame_StackPush(frame, retval); + goto resume_frame; + } + inst(YIELD_VALUE, (retval -- unused)) { // NOTE: It's important that YIELD_VALUE never raises an exception! // The compiler treats any exception raised here as a failed close() @@ -790,8 +899,6 @@ dummy_func( PyGenObject *gen = _PyFrame_GetGenerator(frame); gen->gi_frame_state = FRAME_SUSPENDED; _PyFrame_SetStackPointer(frame, stack_pointer - 1); - TRACE_FUNCTION_EXIT(); - DTRACE_FUNCTION_EXIT(); tstate->exc_info = gen->gi_exc_state.previous_item; gen->gi_exc_state.previous_item = NULL; _Py_LeaveRecursiveCallPy(tstate); @@ -930,7 +1037,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_UnpackSequence(seq, next_instr, oparg); DISPATCH_SAME_OPARG(); @@ -994,7 +1100,6 @@ dummy_func( inst(STORE_ATTR, (counter/1, unused/3, v, owner --)) { #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { - assert(cframe.use_tracing == 0); PyObject *name = GETITEM(frame->f_code->co_names, oparg); next_instr--; _Py_Specialize_StoreAttr(owner, next_instr, name); @@ -1111,7 +1216,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); PyObject *name = GETITEM(frame->f_code->co_names, oparg>>1); next_instr--; _Py_Specialize_LoadGlobal(GLOBALS(), BUILTINS(), next_instr, name); @@ -1163,7 +1267,6 @@ dummy_func( } inst(LOAD_GLOBAL_MODULE, (unused/1, index/1, version/1, unused/1 -- null if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); @@ -1177,11 +1280,11 @@ dummy_func( } inst(LOAD_GLOBAL_BUILTIN, (unused/1, index/1, mod_version/1, bltn_version/1 -- null if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); DEOPT_IF(!PyDict_CheckExact(BUILTINS()), LOAD_GLOBAL); PyDictObject *mdict = (PyDictObject *)GLOBALS(); PyDictObject *bdict = (PyDictObject *)BUILTINS(); + assert(opcode == LOAD_GLOBAL_BUILTIN); DEOPT_IF(mdict->ma_keys->dk_version != mod_version, LOAD_GLOBAL); DEOPT_IF(bdict->ma_keys->dk_version != bltn_version, LOAD_GLOBAL); assert(DK_IS_UNICODE(bdict->ma_keys)); @@ -1465,7 +1568,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); PyObject *name = GETITEM(frame->f_code->co_names, oparg>>1); next_instr--; _Py_Specialize_LoadAttr(owner, next_instr, name); @@ -1511,7 +1613,6 @@ dummy_func( } inst(LOAD_ATTR_INSTANCE_VALUE, (unused/1, type_version/2, index/1, unused/5, owner -- res2 if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -1528,7 +1629,6 @@ dummy_func( } inst(LOAD_ATTR_MODULE, (unused/1, type_version/2, index/1, unused/5, owner -- res2 if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; assert(dict != NULL); @@ -1545,7 +1645,6 @@ dummy_func( } inst(LOAD_ATTR_WITH_HINT, (unused/1, type_version/2, index/1, unused/5, owner -- res2 if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -1576,7 +1675,6 @@ dummy_func( } inst(LOAD_ATTR_SLOT, (unused/1, type_version/2, index/1, unused/5, owner -- res2 if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -1590,7 +1688,6 @@ dummy_func( } inst(LOAD_ATTR_CLASS, (unused/1, type_version/2, unused/2, descr/4, cls -- res2 if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyType_Check(cls), LOAD_ATTR); DEOPT_IF(((PyTypeObject *)cls)->tp_version_tag != type_version, @@ -1606,7 +1703,6 @@ dummy_func( } inst(LOAD_ATTR_PROPERTY, (unused/1, type_version/2, func_version/2, fget/4, owner -- unused if (oparg & 1), unused)) { - assert(cframe.use_tracing == 0); DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); @@ -1632,7 +1728,6 @@ dummy_func( } inst(LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN, (unused/1, type_version/2, func_version/2, getattribute/4, owner -- unused if (oparg & 1), unused)) { - assert(cframe.use_tracing == 0); DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); DEOPT_IF(cls->tp_version_tag != type_version, LOAD_ATTR); @@ -1660,7 +1755,6 @@ dummy_func( } inst(STORE_ATTR_INSTANCE_VALUE, (unused/1, type_version/2, index/1, value, owner --)) { - assert(cframe.use_tracing == 0); PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -1681,7 +1775,6 @@ dummy_func( } inst(STORE_ATTR_WITH_HINT, (unused/1, type_version/2, hint/1, value, owner --)) { - assert(cframe.use_tracing == 0); PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -1723,7 +1816,6 @@ dummy_func( } inst(STORE_ATTR_SLOT, (unused/1, type_version/2, index/1, value, owner --)) { - assert(cframe.use_tracing == 0); PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -1746,7 +1838,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_CompareOp(left, right, next_instr, oparg); DISPATCH_SAME_OPARG(); @@ -1761,7 +1852,6 @@ dummy_func( } inst(COMPARE_OP_FLOAT, (unused/1, left, right -- res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1777,7 +1867,6 @@ dummy_func( // Similar to COMPARE_OP_FLOAT inst(COMPARE_OP_INT, (unused/1, left, right -- res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -1797,7 +1886,6 @@ dummy_func( // Similar to COMPARE_OP_FLOAT, but for ==, != only inst(COMPARE_OP_STR, (unused/1, left, right -- res)) { - assert(cframe.use_tracing == 0); DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2044,7 +2132,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_ForIter(iter, next_instr, oparg); DISPATCH_SAME_OPARG(); @@ -2059,13 +2146,12 @@ dummy_func( if (!_PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) { goto error; } - else if (tstate->c_tracefunc != NULL) { - call_exc_trace(tstate->c_tracefunc, tstate->c_traceobj, tstate, frame); - } + monitor_raise(tstate, frame, next_instr-1); _PyErr_Clear(tstate); } /* iterator ended normally */ - assert(next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == END_FOR); + assert(next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == END_FOR || + next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == INSTRUMENTED_END_FOR); Py_DECREF(iter); STACK_SHRINK(1); /* Jump forward oparg, then skip following END_FOR instruction */ @@ -2075,8 +2161,35 @@ dummy_func( // Common case: no jump, leave it to the code generator } + inst(INSTRUMENTED_FOR_ITER, ( -- )) { + _Py_CODEUNIT *here = next_instr-1; + _Py_CODEUNIT *target; + PyObject *iter = TOP(); + PyObject *next = (*Py_TYPE(iter)->tp_iternext)(iter); + if (next != NULL) { + PUSH(next); + target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER; + } + else { + if (_PyErr_Occurred(tstate)) { + if (!_PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) { + goto error; + } + monitor_raise(tstate, frame, here); + _PyErr_Clear(tstate); + } + /* iterator ended normally */ + assert(next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == END_FOR || + next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == INSTRUMENTED_END_FOR); + STACK_SHRINK(1); + Py_DECREF(iter); + /* Skip END_FOR */ + target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER + oparg + 1; + } + INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); + } + inst(FOR_ITER_LIST, (unused/1, iter -- iter, next)) { - assert(cframe.use_tracing == 0); DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -2099,7 +2212,6 @@ dummy_func( } inst(FOR_ITER_TUPLE, (unused/1, iter -- iter, next)) { - assert(cframe.use_tracing == 0); _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -2122,7 +2234,6 @@ dummy_func( } inst(FOR_ITER_RANGE, (unused/1, iter -- iter, next)) { - assert(cframe.use_tracing == 0); _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -2143,7 +2254,6 @@ dummy_func( } inst(FOR_ITER_GEN, (unused/1, iter -- iter, unused)) { - assert(cframe.use_tracing == 0); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); DEOPT_IF(gen->gi_frame_state >= FRAME_EXECUTING, FOR_ITER); @@ -2155,7 +2265,8 @@ dummy_func( gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; JUMPBY(INLINE_CACHE_ENTRIES_FOR_ITER + oparg); - assert(next_instr->op.code == END_FOR); + assert(next_instr->op.code == END_FOR || + next_instr->op.code == INSTRUMENTED_END_FOR); DISPATCH_INLINED(gen_frame); } @@ -2264,7 +2375,6 @@ dummy_func( inst(LOAD_ATTR_METHOD_WITH_VALUES, (unused/1, type_version/2, keys_version/2, descr/4, self -- res2 if (oparg & 1), res)) { /* Cached method object */ - assert(cframe.use_tracing == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -2283,7 +2393,6 @@ dummy_func( } inst(LOAD_ATTR_METHOD_NO_DICT, (unused/1, type_version/2, unused/2, descr/4, self -- res2 if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); assert(self_cls->tp_dictoffset == 0); @@ -2296,7 +2405,6 @@ dummy_func( } inst(LOAD_ATTR_METHOD_LAZY_DICT, (unused/1, type_version/2, unused/2, descr/4, self -- res2 if (oparg & 1), res)) { - assert(cframe.use_tracing == 0); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); Py_ssize_t dictoffset = self_cls->tp_dictoffset; @@ -2318,6 +2426,21 @@ dummy_func( kwnames = GETITEM(frame->f_code->co_consts, oparg); } + inst(INSTRUMENTED_CALL, ( -- )) { + int is_meth = PEEK(oparg+2) != NULL; + int total_args = oparg + is_meth; + PyObject *function = PEEK(total_args + 1); + PyObject *arg = total_args == 0 ? + &_PyInstrumentation_MISSING : PEEK(total_args); + int err = _Py_call_instrumentation_2args( + tstate, PY_MONITORING_EVENT_CALL, + frame, next_instr-1, function, arg); + ERROR_IF(err, error); + _PyCallCache *cache = (_PyCallCache *)next_instr; + INCREMENT_ADAPTIVE_COUNTER(cache->counter); + GO_TO_INSTRUCTION(CALL); + } + // Cache layout: counter/1, func_version/2 // Neither CALL_INTRINSIC_1/2 nor CALL_FUNCTION_EX are members! family(call, INLINE_CACHE_ENTRIES_CALL) = { @@ -2359,7 +2482,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PyCallCache *cache = (_PyCallCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_Call(callable, next_instr, total_args, kwnames); DISPATCH_SAME_OPARG(); @@ -2402,16 +2524,26 @@ dummy_func( DISPATCH_INLINED(new_frame); } /* Callable is not a normal Python function */ - if (cframe.use_tracing) { - res = trace_call_function( - tstate, callable, args, - positional_args, kwnames); - } - else { - res = PyObject_Vectorcall( - callable, args, - positional_args | PY_VECTORCALL_ARGUMENTS_OFFSET, - kwnames); + res = PyObject_Vectorcall( + callable, args, + positional_args | PY_VECTORCALL_ARGUMENTS_OFFSET, + kwnames); + if (opcode == INSTRUMENTED_CALL) { + PyObject *arg = total_args == 0 ? + &_PyInstrumentation_MISSING : PEEK(total_args); + if (res == NULL) { + _Py_call_instrumentation_exc2( + tstate, PY_MONITORING_EVENT_C_RAISE, + frame, next_instr-1, callable, arg); + } + else { + int err = _Py_call_instrumentation_2args( + tstate, PY_MONITORING_EVENT_C_RETURN, + frame, next_instr-1, callable, arg); + if (err < 0) { + Py_CLEAR(res); + } + } } kwnames = NULL; assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); @@ -2504,7 +2636,6 @@ dummy_func( inst(CALL_NO_KW_TYPE_1, (unused/1, unused/2, null, callable, args[oparg] -- res)) { assert(kwnames == NULL); - assert(cframe.use_tracing == 0); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); PyObject *obj = args[0]; @@ -2517,7 +2648,6 @@ dummy_func( inst(CALL_NO_KW_STR_1, (unused/1, unused/2, null, callable, args[oparg] -- res)) { assert(kwnames == NULL); - assert(cframe.use_tracing == 0); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); DEOPT_IF(callable != (PyObject *)&PyUnicode_Type, CALL); @@ -2570,7 +2700,6 @@ dummy_func( } inst(CALL_NO_KW_BUILTIN_O, (unused/1, unused/2, method, callable, args[oparg] -- res)) { - assert(cframe.use_tracing == 0); /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -2602,7 +2731,6 @@ dummy_func( } inst(CALL_NO_KW_BUILTIN_FAST, (unused/1, unused/2, method, callable, args[oparg] -- res)) { - assert(cframe.use_tracing == 0); /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -2638,7 +2766,6 @@ dummy_func( } inst(CALL_BUILTIN_FAST_WITH_KEYWORDS, (unused/1, unused/2, method, callable, args[oparg] -- res)) { - assert(cframe.use_tracing == 0); /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -2674,7 +2801,6 @@ dummy_func( } inst(CALL_NO_KW_LEN, (unused/1, unused/2, method, callable, args[oparg] -- res)) { - assert(cframe.use_tracing == 0); assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -2702,7 +2828,6 @@ dummy_func( } inst(CALL_NO_KW_ISINSTANCE, (unused/1, unused/2, method, callable, args[oparg] -- res)) { - assert(cframe.use_tracing == 0); assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -2733,7 +2858,6 @@ dummy_func( // This is secretly a super-instruction inst(CALL_NO_KW_LIST_APPEND, (unused/1, unused/2, method, self, args[oparg] -- unused)) { - assert(cframe.use_tracing == 0); assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -2882,12 +3006,14 @@ dummy_func( CHECK_EVAL_BREAKER(); } + inst(INSTRUMENTED_CALL_FUNCTION_EX, ( -- )) { + GO_TO_INSTRUCTION(CALL_FUNCTION_EX); + } + inst(CALL_FUNCTION_EX, (unused, func, callargs, kwargs if (oparg & 1) -- result)) { - if (oparg & 1) { - // DICT_MERGE is called before this opcode if there are kwargs. - // It converts all dict subtypes in kwargs into regular dicts. - assert(PyDict_CheckExact(kwargs)); - } + // DICT_MERGE is called before this opcode if there are kwargs. + // It converts all dict subtypes in kwargs into regular dicts. + assert(kwargs == NULL || PyDict_CheckExact(kwargs)); if (!PyTuple_CheckExact(callargs)) { if (check_args_iterable(tstate, func, callargs) < 0) { goto error; @@ -2899,10 +3025,35 @@ dummy_func( Py_SETREF(callargs, tuple); } assert(PyTuple_CheckExact(callargs)); - - result = do_call_core(tstate, func, callargs, kwargs, cframe.use_tracing); + EVAL_CALL_STAT_INC_IF_FUNCTION(EVAL_CALL_FUNCTION_EX, func); + if (opcode == INSTRUMENTED_CALL_FUNCTION_EX && + !PyFunction_Check(func) && !PyMethod_Check(func) + ) { + PyObject *arg = PyTuple_GET_SIZE(callargs) > 0 ? + PyTuple_GET_ITEM(callargs, 0) : Py_None; + int err = _Py_call_instrumentation_2args( + tstate, PY_MONITORING_EVENT_CALL, + frame, next_instr-1, func, arg); + if (err) goto error; + result = PyObject_Call(func, callargs, kwargs); + if (result == NULL) { + _Py_call_instrumentation_exc2( + tstate, PY_MONITORING_EVENT_C_RAISE, + frame, next_instr-1, func, arg); + } + else { + int err = _Py_call_instrumentation_2args( + tstate, PY_MONITORING_EVENT_C_RETURN, + frame, next_instr-1, func, arg); + if (err < 0) { + Py_CLEAR(result); + } + } + } + else { + result = PyObject_Call(func, callargs, kwargs); + } DECREF_INPUTS(); - assert(PEEK(3 + (oparg & 1)) == NULL); ERROR_IF(result == NULL, error); CHECK_EVAL_BREAKER(); @@ -3018,7 +3169,6 @@ dummy_func( #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, &GETLOCAL(0)); DISPATCH_SAME_OPARG(); @@ -3039,9 +3189,105 @@ dummy_func( assert(oparg >= 2); } - inst(EXTENDED_ARG, (--)) { + inst(INSTRUMENTED_LINE, ( -- )) { + _Py_CODEUNIT *here = next_instr-1; + _PyFrame_SetStackPointer(frame, stack_pointer); + int original_opcode = _Py_call_instrumentation_line( + tstate, frame, here); + stack_pointer = _PyFrame_GetStackPointer(frame); + if (original_opcode < 0) { + next_instr = here+1; + goto error; + } + next_instr = frame->prev_instr; + if (next_instr != here) { + DISPATCH(); + } + if (_PyOpcode_Caches[original_opcode]) { + _PyBinaryOpCache *cache = (_PyBinaryOpCache *)(next_instr+1); + INCREMENT_ADAPTIVE_COUNTER(cache->counter); + } + opcode = original_opcode; + DISPATCH_GOTO(); + } + + inst(INSTRUMENTED_INSTRUCTION, ( -- )) { + int next_opcode = _Py_call_instrumentation_instruction( + tstate, frame, next_instr-1); + ERROR_IF(next_opcode < 0, error); + next_instr--; + if (_PyOpcode_Caches[next_opcode]) { + _PyBinaryOpCache *cache = (_PyBinaryOpCache *)(next_instr+1); + INCREMENT_ADAPTIVE_COUNTER(cache->counter); + } + assert(next_opcode > 0 && next_opcode < 256); + opcode = next_opcode; + DISPATCH_GOTO(); + } + + inst(INSTRUMENTED_JUMP_FORWARD, ( -- )) { + INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); + } + + inst(INSTRUMENTED_JUMP_BACKWARD, ( -- )) { + INSTRUMENTED_JUMP(next_instr-1, next_instr-oparg, PY_MONITORING_EVENT_JUMP); + CHECK_EVAL_BREAKER(); + } + + inst(INSTRUMENTED_POP_JUMP_IF_TRUE, ( -- )) { + PyObject *cond = POP(); + int err = PyObject_IsTrue(cond); + Py_DECREF(cond); + ERROR_IF(err < 0, error); + _Py_CODEUNIT *here = next_instr-1; + assert(err == 0 || err == 1); + int offset = err*oparg; + INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); + } + + inst(INSTRUMENTED_POP_JUMP_IF_FALSE, ( -- )) { + PyObject *cond = POP(); + int err = PyObject_IsTrue(cond); + Py_DECREF(cond); + ERROR_IF(err < 0, error); + _Py_CODEUNIT *here = next_instr-1; + assert(err == 0 || err == 1); + int offset = (1-err)*oparg; + INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); + } + + inst(INSTRUMENTED_POP_JUMP_IF_NONE, ( -- )) { + PyObject *value = POP(); + _Py_CODEUNIT *here = next_instr-1; + int offset; + if (Py_IsNone(value)) { + _Py_DECREF_NO_DEALLOC(value); + offset = oparg; + } + else { + Py_DECREF(value); + offset = 0; + } + INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); + } + + inst(INSTRUMENTED_POP_JUMP_IF_NOT_NONE, ( -- )) { + PyObject *value = POP(); + _Py_CODEUNIT *here = next_instr-1; + int offset; + if (Py_IsNone(value)) { + _Py_DECREF_NO_DEALLOC(value); + offset = 0; + } + else { + Py_DECREF(value); + offset = oparg; + } + INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); + } + + inst(EXTENDED_ARG, ( -- )) { assert(oparg); - assert(cframe.use_tracing == 0); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); @@ -3049,6 +3295,12 @@ dummy_func( } inst(CACHE, (--)) { + assert(0 && "Executing a cache."); + Py_UNREACHABLE(); + } + + inst(RESERVED, (--)) { + assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); } diff --git a/Python/ceval.c b/Python/ceval.c index 7d60cf987e9c47..a38c9ec9ad5f9c 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -10,6 +10,7 @@ #include "pycore_function.h" #include "pycore_intrinsics.h" #include "pycore_long.h" // _PyLong_GetZero() +#include "pycore_instruments.h" #include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_moduleobject.h" // PyModuleObject #include "pycore_opcode.h" // EXTRA_CASES @@ -92,13 +93,6 @@ #define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) _Py_atomic_load_relaxed(ATOMIC_VAL) #endif -/* Forward declarations */ -static PyObject *trace_call_function( - PyThreadState *tstate, PyObject *callable, PyObject **stack, - Py_ssize_t oparg, PyObject *kwnames); -static PyObject * do_call_core( - PyThreadState *tstate, PyObject *func, - PyObject *callargs, PyObject *kwdict, int use_tracing); #ifdef LLTRACE static void @@ -179,19 +173,22 @@ lltrace_resume_frame(_PyInterpreterFrame *frame) PyErr_SetRaisedException(exc); } #endif -static int call_trace(Py_tracefunc, PyObject *, - PyThreadState *, _PyInterpreterFrame *, - int, PyObject *); -static int call_trace_protected(Py_tracefunc, PyObject *, - PyThreadState *, _PyInterpreterFrame *, - int, PyObject *); -static void call_exc_trace(Py_tracefunc, PyObject *, - PyThreadState *, _PyInterpreterFrame *); -static int maybe_call_line_trace(Py_tracefunc, PyObject *, - PyThreadState *, _PyInterpreterFrame *, int); -static void maybe_dtrace_line(_PyInterpreterFrame *, PyTraceInfo *, int); -static void dtrace_function_entry(_PyInterpreterFrame *); -static void dtrace_function_return(_PyInterpreterFrame *); + +static void monitor_raise(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr); +static int monitor_stop_iteration(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr); +static void monitor_unwind(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr); +static void monitor_handled(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr, PyObject *exc); +static void monitor_throw(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr); static PyObject * import_name(PyThreadState *, _PyInterpreterFrame *, PyObject *, PyObject *, PyObject *); @@ -217,21 +214,6 @@ _PyEvalFrameClearAndPop(PyThreadState *tstate, _PyInterpreterFrame *frame); "cannot access free variable '%s' where it is not associated with a" \ " value in enclosing scope" -#ifndef NDEBUG -/* Ensure that tstate is valid: sanity check for PyEval_AcquireThread() and - PyEval_RestoreThread(). Detect if tstate memory was freed. It can happen - when a thread continues to run after Python finalization, especially - daemon threads. */ -static int -is_tstate_valid(PyThreadState *tstate) -{ - assert(!_PyMem_IsPtrFreed(tstate)); - assert(!_PyMem_IsPtrFreed(tstate->interp)); - return 1; -} -#endif - - #ifdef HAVE_ERRNO_H #include #endif @@ -596,63 +578,6 @@ PyEval_EvalFrameEx(PyFrameObject *f, int throwflag) #include "ceval_macros.h" -static int -trace_function_entry(PyThreadState *tstate, _PyInterpreterFrame *frame) -{ - if (tstate->c_tracefunc != NULL) { - /* tstate->c_tracefunc, if defined, is a - function that will be called on *every* entry - to a code block. Its return value, if not - None, is a function that will be called at - the start of each executed line of code. - (Actually, the function must return itself - in order to continue tracing.) The trace - functions are called with three arguments: - a pointer to the current frame, a string - indicating why the function is called, and - an argument which depends on the situation. - The global trace function is also called - whenever an exception is detected. */ - if (call_trace_protected(tstate->c_tracefunc, - tstate->c_traceobj, - tstate, frame, - PyTrace_CALL, Py_None)) { - /* Trace function raised an error */ - return -1; - } - } - if (tstate->c_profilefunc != NULL) { - /* Similar for c_profilefunc, except it needn't - return itself and isn't called for "line" events */ - if (call_trace_protected(tstate->c_profilefunc, - tstate->c_profileobj, - tstate, frame, - PyTrace_CALL, Py_None)) { - /* Profile function raised an error */ - return -1; - } - } - return 0; -} - -static int -trace_function_exit(PyThreadState *tstate, _PyInterpreterFrame *frame, PyObject *retval) -{ - if (tstate->c_tracefunc) { - if (call_trace_protected(tstate->c_tracefunc, tstate->c_traceobj, - tstate, frame, PyTrace_RETURN, retval)) { - return -1; - } - } - if (tstate->c_profilefunc) { - if (call_trace_protected(tstate->c_profilefunc, tstate->c_profileobj, - tstate, frame, PyTrace_RETURN, retval)) { - return -1; - } - } - return 0; -} - int _Py_CheckRecursiveCallPy( PyThreadState *tstate) @@ -730,7 +655,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int * strict stack discipline must be maintained. */ _PyCFrame *prev_cframe = tstate->cframe; - cframe.use_tracing = prev_cframe->use_tracing; cframe.previous = prev_cframe; tstate->cframe = &cframe; @@ -765,8 +689,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int if (_Py_EnterRecursivePy(tstate)) { goto exit_unwind; } - TRACE_FUNCTION_THROW_ENTRY(); - DTRACE_FUNCTION_ENTRY(); + /* Because this avoids the RESUME, + * we need to update instrumentation */ + _Py_Instrument(frame->f_code, tstate->interp); + monitor_throw(tstate, frame, frame->prev_instr); + /* TO DO -- Monitor throw entry. */ goto resume_with_error; } @@ -781,15 +708,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int assert(_PyInterpreterFrame_LASTI(frame) >= -1); \ /* Jump back to the last instruction executed... */ \ next_instr = frame->prev_instr + 1; \ - stack_pointer = _PyFrame_GetStackPointer(frame); \ - /* Set stackdepth to -1. \ - Update when returning or calling trace function. \ - Having stackdepth <= 0 ensures that invalid \ - values are not visible to the cycle GC. \ - We choose -1 rather than 0 to assist debugging. \ - */ \ - frame->stacktop = -1; - + stack_pointer = _PyFrame_GetStackPointer(frame); start_frame: if (_Py_EnterRecursivePy(tstate)) { @@ -845,91 +764,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int #include "generated_cases.c.h" -#if USE_COMPUTED_GOTOS - TARGET_DO_TRACING: -#else - case DO_TRACING: -#endif - { - assert(cframe.use_tracing); - assert(tstate->tracing == 0); - if (INSTR_OFFSET() >= frame->f_code->_co_firsttraceable) { - int instr_prev = _PyInterpreterFrame_LASTI(frame); - frame->prev_instr = next_instr; - NEXTOPARG(); - // No _PyOpcode_Deopt here, since RESUME has no optimized forms: - if (opcode == RESUME) { - if (oparg < 2) { - CHECK_EVAL_BREAKER(); - } - /* Call tracing */ - TRACE_FUNCTION_ENTRY(); - DTRACE_FUNCTION_ENTRY(); - } - else { - /* line-by-line tracing support */ - if (PyDTrace_LINE_ENABLED()) { - maybe_dtrace_line(frame, &tstate->trace_info, instr_prev); - } - - if (cframe.use_tracing && - tstate->c_tracefunc != NULL && !tstate->tracing) { - int err; - /* see maybe_call_line_trace() - for expository comments */ - _PyFrame_SetStackPointer(frame, stack_pointer); - - err = maybe_call_line_trace(tstate->c_tracefunc, - tstate->c_traceobj, - tstate, frame, instr_prev); - // Reload possibly changed frame fields: - stack_pointer = _PyFrame_GetStackPointer(frame); - frame->stacktop = -1; - // next_instr is only reloaded if tracing *does not* raise. - // This is consistent with the behavior of older Python - // versions. If a trace function sets a new f_lineno and - // *then* raises, we use the *old* location when searching - // for an exception handler, displaying the traceback, and - // so on: - if (err) { - // next_instr wasn't incremented at the start of this - // instruction. Increment it before handling the error, - // so that it looks the same as a "normal" instruction: - next_instr++; - goto error; - } - // Reload next_instr. Don't increment it, though, since - // we're going to re-dispatch to the "true" instruction now: - next_instr = frame->prev_instr; - } - } - } - NEXTOPARG(); - PRE_DISPATCH_GOTO(); - // No _PyOpcode_Deopt here, since EXTENDED_ARG has no optimized forms: - while (opcode == EXTENDED_ARG) { - // CPython hasn't ever traced the instruction after an EXTENDED_ARG. - // Inline the EXTENDED_ARG here, so we can avoid branching there: - INSTRUCTION_START(EXTENDED_ARG); - opcode = next_instr->op.code; - oparg = oparg << 8 | next_instr->op.arg; - // Make sure the next instruction isn't a RESUME, since that needs - // to trace properly (and shouldn't have an EXTENDED_ARG, anyways): - assert(opcode != RESUME); - PRE_DISPATCH_GOTO(); - } - opcode = _PyOpcode_Deopt[opcode]; - if (_PyOpcode_Caches[opcode]) { - uint16_t *counter = &next_instr[1].cache; - // The instruction is going to decrement the counter, so we need to - // increment it here to make sure it doesn't try to specialize: - if (!ADAPTIVE_COUNTER_IS_MAX(*counter)) { - INCREMENT_ADAPTIVE_COUNTER(*counter); - } - } - DISPATCH_GOTO(); - } - #if USE_COMPUTED_GOTOS _unknown_opcode: #else @@ -988,12 +822,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int PyTraceBack_Here(f); } } - - if (tstate->c_tracefunc != NULL) { - /* Make sure state is set to FRAME_UNWINDING for tracing */ - call_exc_trace(tstate->c_tracefunc, tstate->c_traceobj, - tstate, frame); - } + monitor_raise(tstate, frame, next_instr-1); exception_unwind: { @@ -1012,8 +841,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int } assert(STACK_LEVEL() == 0); _PyFrame_SetStackPointer(frame, stack_pointer); - TRACE_FUNCTION_UNWIND(); - DTRACE_FUNCTION_EXIT(); + monitor_unwind(tstate, frame, next_instr-1); goto exit_unwind; } @@ -1036,8 +864,10 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int available to the handler, so a program can emulate the Python main loop. */ - PUSH(_PyErr_GetRaisedException(tstate)); + PyObject *exc = _PyErr_GetRaisedException(tstate); + PUSH(exc); JUMPTO(handler); + monitor_handled(tstate, frame, next_instr, exc); /* Resume normal execution */ DISPATCH(); } @@ -1054,7 +884,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int if (frame == &entry_frame) { /* Restore previous cframe and exit */ tstate->cframe = cframe.previous; - tstate->cframe->use_tracing = cframe.use_tracing; assert(tstate->cframe->current_frame == frame->previous); _Py_LeaveRecursiveCallTstate(tstate); return NULL; @@ -2020,105 +1849,108 @@ unpack_iterable(PyThreadState *tstate, PyObject *v, return 0; } -static void -call_exc_trace(Py_tracefunc func, PyObject *self, - PyThreadState *tstate, - _PyInterpreterFrame *f) +static int +do_monitor_exc(PyThreadState *tstate, _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr, int event) { - PyObject *exc = _PyErr_GetRaisedException(tstate); - assert(exc && PyExceptionInstance_Check(exc)); - PyObject *type = PyExceptionInstance_Class(exc); - PyObject *traceback = PyException_GetTraceback(exc); - if (traceback == NULL) { - traceback = Py_NewRef(Py_None); + assert(event < PY_MONITORING_UNGROUPED_EVENTS); + PyObject *exc = PyErr_GetRaisedException(); + assert(exc != NULL); + int err = _Py_call_instrumentation_arg(tstate, event, frame, instr, exc); + if (err == 0) { + PyErr_SetRaisedException(exc); } - PyObject *arg = PyTuple_Pack(3, type, exc, traceback); - Py_XDECREF(traceback); - - if (arg == NULL) { - _PyErr_SetRaisedException(tstate, exc); - return; + else { + Py_DECREF(exc); } - int err = call_trace(func, self, tstate, f, PyTrace_EXCEPTION, arg); - Py_DECREF(arg); - if (err == 0) { - _PyErr_SetRaisedException(tstate, exc); + return err; +} + +static inline int +no_tools_for_event(PyThreadState *tstate, _PyInterpreterFrame *frame, int event) +{ + _PyCoMonitoringData *data = frame->f_code->_co_monitoring; + if (data) { + if (data->active_monitors.tools[event] == 0) { + return 1; + } } else { - Py_XDECREF(exc); + if (tstate->interp->monitors.tools[event] == 0) { + return 1; + } + } + return 0; +} + +static void +monitor_raise(PyThreadState *tstate, _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr) +{ + if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_RAISE)) { + return; } + do_monitor_exc(tstate, frame, instr, PY_MONITORING_EVENT_RAISE); } static int -call_trace_protected(Py_tracefunc func, PyObject *obj, - PyThreadState *tstate, _PyInterpreterFrame *frame, - int what, PyObject *arg) +monitor_stop_iteration(PyThreadState *tstate, _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr) { - PyObject *exc = _PyErr_GetRaisedException(tstate); - int err = call_trace(func, obj, tstate, frame, what, arg); - if (err == 0) - { - _PyErr_SetRaisedException(tstate, exc); + if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_STOP_ITERATION)) { return 0; } - else { - Py_XDECREF(exc); - return -1; + return do_monitor_exc(tstate, frame, instr, PY_MONITORING_EVENT_STOP_ITERATION); +} + +static void +monitor_unwind(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr) +{ + if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_PY_UNWIND)) { + return; } + _Py_call_instrumentation_exc0(tstate, PY_MONITORING_EVENT_PY_UNWIND, frame, instr); } + static void -initialize_trace_info(PyTraceInfo *trace_info, _PyInterpreterFrame *frame) +monitor_handled(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr, PyObject *exc) { - PyCodeObject *code = frame->f_code; - if (trace_info->code != code) { - trace_info->code = code; - _PyCode_InitAddressRange(code, &trace_info->bounds); + if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_EXCEPTION_HANDLED)) { + return; } + _Py_call_instrumentation_arg(tstate, PY_MONITORING_EVENT_EXCEPTION_HANDLED, frame, instr, exc); +} + +static void +monitor_throw(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr) +{ + if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_PY_THROW)) { + return; + } + _Py_call_instrumentation_exc0(tstate, PY_MONITORING_EVENT_PY_THROW, frame, instr); } void PyThreadState_EnterTracing(PyThreadState *tstate) { + assert(tstate->tracing >= 0); tstate->tracing++; - tstate->cframe->use_tracing = 0; } void PyThreadState_LeaveTracing(PyThreadState *tstate) { - assert(tstate->tracing > 0 && tstate->cframe->use_tracing == 0); + assert(tstate->tracing > 0); tstate->tracing--; - _PyThreadState_UpdateTracingState(tstate); } -static int -call_trace(Py_tracefunc func, PyObject *obj, - PyThreadState *tstate, _PyInterpreterFrame *frame, - int what, PyObject *arg) -{ - int result; - if (tstate->tracing) { - return 0; - } - PyFrameObject *f = _PyFrame_GetFrameObject(frame); - if (f == NULL) { - return -1; - } - int old_what = tstate->tracing_what; - tstate->tracing_what = what; - PyThreadState_EnterTracing(tstate); - assert(_PyInterpreterFrame_LASTI(frame) >= 0); - if (_PyCode_InitLineArray(frame->f_code)) { - return -1; - } - f->f_lineno = _PyCode_LineNumberFromArray(frame->f_code, _PyInterpreterFrame_LASTI(frame)); - result = func(obj, f, what, arg); - f->f_lineno = 0; - PyThreadState_LeaveTracing(tstate); - tstate->tracing_what = old_what; - return result; -} PyObject* _PyEval_CallTracing(PyObject *func, PyObject *args) @@ -2126,7 +1958,6 @@ _PyEval_CallTracing(PyObject *func, PyObject *args) // Save and disable tracing PyThreadState *tstate = _PyThreadState_GET(); int save_tracing = tstate->tracing; - int save_use_tracing = tstate->cframe->use_tracing; tstate->tracing = 0; // Call the tracing function @@ -2134,81 +1965,9 @@ _PyEval_CallTracing(PyObject *func, PyObject *args) // Restore tracing tstate->tracing = save_tracing; - tstate->cframe->use_tracing = save_use_tracing; return result; } -/* See Objects/lnotab_notes.txt for a description of how tracing works. */ -static int -maybe_call_line_trace(Py_tracefunc func, PyObject *obj, - PyThreadState *tstate, _PyInterpreterFrame *frame, int instr_prev) -{ - int result = 0; - - /* If the last instruction falls at the start of a line or if it - represents a jump backwards, update the frame's line number and - then call the trace function if we're tracing source lines. - */ - if (_PyCode_InitLineArray(frame->f_code)) { - return -1; - } - int lastline; - if (instr_prev <= frame->f_code->_co_firsttraceable) { - lastline = -1; - } - else { - lastline = _PyCode_LineNumberFromArray(frame->f_code, instr_prev); - } - int line = _PyCode_LineNumberFromArray(frame->f_code, _PyInterpreterFrame_LASTI(frame)); - PyFrameObject *f = _PyFrame_GetFrameObject(frame); - if (f == NULL) { - return -1; - } - if (line != -1 && f->f_trace_lines) { - /* Trace backward edges (except in 'yield from') or if line number has changed */ - int trace = line != lastline || - (_PyInterpreterFrame_LASTI(frame) < instr_prev && - // SEND has no quickened forms, so no need to use _PyOpcode_Deopt - // here: - frame->prev_instr->op.code != SEND); - if (trace) { - result = call_trace(func, obj, tstate, frame, PyTrace_LINE, Py_None); - } - } - /* Always emit an opcode event if we're tracing all opcodes. */ - if (f->f_trace_opcodes && result == 0) { - result = call_trace(func, obj, tstate, frame, PyTrace_OPCODE, Py_None); - } - return result; -} - -int -_PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) -{ - assert(is_tstate_valid(tstate)); - /* The caller must hold the GIL */ - assert(PyGILState_Check()); - - /* Call _PySys_Audit() in the context of the current thread state, - even if tstate is not the current thread state. */ - PyThreadState *current_tstate = _PyThreadState_GET(); - if (_PySys_Audit(current_tstate, "sys.setprofile", NULL) < 0) { - return -1; - } - - tstate->c_profilefunc = func; - PyObject *old_profileobj = tstate->c_profileobj; - tstate->c_profileobj = Py_XNewRef(arg); - /* Flag that tracing or profiling is turned on */ - _PyThreadState_UpdateTracingState(tstate); - - // gh-98257: Only call Py_XDECREF() once the new profile function is fully - // set, so it's safe to call sys.setprofile() again (reentrant call). - Py_XDECREF(old_profileobj); - - return 0; -} - void PyEval_SetProfile(Py_tracefunc func, PyObject *arg) { @@ -2240,33 +1999,6 @@ PyEval_SetProfileAllThreads(Py_tracefunc func, PyObject *arg) } } -int -_PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) -{ - assert(is_tstate_valid(tstate)); - /* The caller must hold the GIL */ - assert(PyGILState_Check()); - - /* Call _PySys_Audit() in the context of the current thread state, - even if tstate is not the current thread state. */ - PyThreadState *current_tstate = _PyThreadState_GET(); - if (_PySys_Audit(current_tstate, "sys.settrace", NULL) < 0) { - return -1; - } - - tstate->c_tracefunc = func; - PyObject *old_traceobj = tstate->c_traceobj; - tstate->c_traceobj = Py_XNewRef(arg); - /* Flag that tracing or profiling is turned on */ - _PyThreadState_UpdateTracingState(tstate); - - // gh-98257: Only call Py_XDECREF() once the new trace function is fully - // set, so it's safe to call sys.settrace() again (reentrant call). - Py_XDECREF(old_traceobj); - - return 0; -} - void PyEval_SetTrace(Py_tracefunc func, PyObject *arg) { @@ -2492,114 +2224,6 @@ PyEval_GetFuncDesc(PyObject *func) return " object"; } -#define C_TRACE(x, call) \ -if (use_tracing && tstate->c_profilefunc) { \ - if (call_trace(tstate->c_profilefunc, tstate->c_profileobj, \ - tstate, tstate->cframe->current_frame, \ - PyTrace_C_CALL, func)) { \ - x = NULL; \ - } \ - else { \ - x = call; \ - if (tstate->c_profilefunc != NULL) { \ - if (x == NULL) { \ - call_trace_protected(tstate->c_profilefunc, \ - tstate->c_profileobj, \ - tstate, tstate->cframe->current_frame, \ - PyTrace_C_EXCEPTION, func); \ - /* XXX should pass (type, value, tb) */ \ - } else { \ - if (call_trace(tstate->c_profilefunc, \ - tstate->c_profileobj, \ - tstate, tstate->cframe->current_frame, \ - PyTrace_C_RETURN, func)) { \ - Py_DECREF(x); \ - x = NULL; \ - } \ - } \ - } \ - } \ -} else { \ - x = call; \ - } - - -static PyObject * -trace_call_function(PyThreadState *tstate, - PyObject *func, - PyObject **args, Py_ssize_t nargs, - PyObject *kwnames) -{ - int use_tracing = 1; - PyObject *x; - if (PyCFunction_CheckExact(func) || PyCMethod_CheckExact(func)) { - C_TRACE(x, PyObject_Vectorcall(func, args, nargs, kwnames)); - return x; - } - else if (Py_IS_TYPE(func, &PyMethodDescr_Type) && nargs > 0) { - /* We need to create a temporary bound method as argument - for profiling. - - If nargs == 0, then this cannot work because we have no - "self". In any case, the call itself would raise - TypeError (foo needs an argument), so we just skip - profiling. */ - PyObject *self = args[0]; - func = Py_TYPE(func)->tp_descr_get(func, self, (PyObject*)Py_TYPE(self)); - if (func == NULL) { - return NULL; - } - C_TRACE(x, PyObject_Vectorcall(func, - args+1, nargs-1, - kwnames)); - Py_DECREF(func); - return x; - } - return PyObject_Vectorcall(func, args, nargs | PY_VECTORCALL_ARGUMENTS_OFFSET, kwnames); -} - -static PyObject * -do_call_core(PyThreadState *tstate, - PyObject *func, - PyObject *callargs, - PyObject *kwdict, - int use_tracing - ) -{ - PyObject *result; - if (PyCFunction_CheckExact(func) || PyCMethod_CheckExact(func)) { - C_TRACE(result, PyObject_Call(func, callargs, kwdict)); - return result; - } - else if (Py_IS_TYPE(func, &PyMethodDescr_Type)) { - Py_ssize_t nargs = PyTuple_GET_SIZE(callargs); - if (nargs > 0 && use_tracing) { - /* We need to create a temporary bound method as argument - for profiling. - - If nargs == 0, then this cannot work because we have no - "self". In any case, the call itself would raise - TypeError (foo needs an argument), so we just skip - profiling. */ - PyObject *self = PyTuple_GET_ITEM(callargs, 0); - func = Py_TYPE(func)->tp_descr_get(func, self, (PyObject*)Py_TYPE(self)); - if (func == NULL) { - return NULL; - } - - C_TRACE(result, _PyObject_FastCallDictTstate( - tstate, func, - &_PyTuple_ITEMS(callargs)[1], - nargs - 1, - kwdict)); - Py_DECREF(func); - return result; - } - } - EVAL_CALL_STAT_INC_IF_FUNCTION(EVAL_CALL_FUNCTION_EX, func); - return PyObject_Call(func, callargs, kwdict); -} - /* Extract a slice index from a PyLong or an object with the nb_index slot defined, and store in *pi. Silently reduce values larger than PY_SSIZE_T_MAX to PY_SSIZE_T_MAX, @@ -2973,69 +2597,6 @@ PyUnstable_Eval_RequestCodeExtraIndex(freefunc free) return new_index; } -static void -dtrace_function_entry(_PyInterpreterFrame *frame) -{ - const char *filename; - const char *funcname; - int lineno; - - PyCodeObject *code = frame->f_code; - filename = PyUnicode_AsUTF8(code->co_filename); - funcname = PyUnicode_AsUTF8(code->co_name); - lineno = _PyInterpreterFrame_GetLine(frame); - - PyDTrace_FUNCTION_ENTRY(filename, funcname, lineno); -} - -static void -dtrace_function_return(_PyInterpreterFrame *frame) -{ - const char *filename; - const char *funcname; - int lineno; - - PyCodeObject *code = frame->f_code; - filename = PyUnicode_AsUTF8(code->co_filename); - funcname = PyUnicode_AsUTF8(code->co_name); - lineno = _PyInterpreterFrame_GetLine(frame); - - PyDTrace_FUNCTION_RETURN(filename, funcname, lineno); -} - -/* DTrace equivalent of maybe_call_line_trace. */ -static void -maybe_dtrace_line(_PyInterpreterFrame *frame, - PyTraceInfo *trace_info, - int instr_prev) -{ - const char *co_filename, *co_name; - - /* If the last instruction executed isn't in the current - instruction window, reset the window. - */ - initialize_trace_info(trace_info, frame); - int lastline = _PyCode_CheckLineNumber(instr_prev*sizeof(_Py_CODEUNIT), &trace_info->bounds); - int addr = _PyInterpreterFrame_LASTI(frame) * sizeof(_Py_CODEUNIT); - int line = _PyCode_CheckLineNumber(addr, &trace_info->bounds); - if (line != -1) { - /* Trace backward edges or first instruction of a new line */ - if (_PyInterpreterFrame_LASTI(frame) < instr_prev || - (line != lastline && addr == trace_info->bounds.ar_start)) - { - co_filename = PyUnicode_AsUTF8(frame->f_code->co_filename); - if (!co_filename) { - co_filename = "?"; - } - co_name = PyUnicode_AsUTF8(frame->f_code->co_name); - if (!co_name) { - co_name = "?"; - } - PyDTrace_LINE(co_filename, co_name, line); - } - } -} - /* Implement Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() as functions for the limited API. */ diff --git a/Python/ceval_macros.h b/Python/ceval_macros.h index c2257515a30599..485771ac65a767 100644 --- a/Python/ceval_macros.h +++ b/Python/ceval_macros.h @@ -93,8 +93,6 @@ { \ NEXTOPARG(); \ PRE_DISPATCH_GOTO(); \ - assert(cframe.use_tracing == 0 || cframe.use_tracing == 255); \ - opcode |= cframe.use_tracing OR_DTRACE_LINE; \ DISPATCH_GOTO(); \ } @@ -102,7 +100,6 @@ { \ opcode = next_instr->op.code; \ PRE_DISPATCH_GOTO(); \ - opcode |= cframe.use_tracing OR_DTRACE_LINE; \ DISPATCH_GOTO(); \ } @@ -183,7 +180,7 @@ GETITEM(PyObject *v, Py_ssize_t i) { #define PREDICT(next_op) \ do { \ _Py_CODEUNIT word = *next_instr; \ - opcode = word.op.code | cframe.use_tracing OR_DTRACE_LINE; \ + opcode = word.op.code; \ if (opcode == next_op) { \ oparg = word.op.arg; \ INSTRUCTION_START(next_op); \ @@ -283,47 +280,6 @@ GETITEM(PyObject *v, Py_ssize_t i) { #define BUILTINS() frame->f_builtins #define LOCALS() frame->f_locals -/* Shared opcode macros */ - -#define TRACE_FUNCTION_EXIT() \ - if (cframe.use_tracing) { \ - if (trace_function_exit(tstate, frame, retval)) { \ - Py_DECREF(retval); \ - goto exit_unwind; \ - } \ - } - -#define DTRACE_FUNCTION_EXIT() \ - if (PyDTrace_FUNCTION_RETURN_ENABLED()) { \ - dtrace_function_return(frame); \ - } - -#define TRACE_FUNCTION_UNWIND() \ - if (cframe.use_tracing) { \ - /* Since we are already unwinding, \ - * we don't care if this raises */ \ - trace_function_exit(tstate, frame, NULL); \ - } - -#define TRACE_FUNCTION_ENTRY() \ - if (cframe.use_tracing) { \ - _PyFrame_SetStackPointer(frame, stack_pointer); \ - int err = trace_function_entry(tstate, frame); \ - stack_pointer = _PyFrame_GetStackPointer(frame); \ - frame->stacktop = -1; \ - if (err) { \ - goto error; \ - } \ - } - -#define TRACE_FUNCTION_THROW_ENTRY() \ - if (cframe.use_tracing) { \ - assert(frame->stacktop >= 0); \ - if (trace_function_entry(tstate, frame)) { \ - goto exit_unwind; \ - } \ - } - #define DTRACE_FUNCTION_ENTRY() \ if (PyDTrace_FUNCTION_ENTRY_ENABLED()) { \ dtrace_function_entry(frame); \ @@ -371,3 +327,18 @@ do { \ _Py_DECREF_NO_DEALLOC(right); \ } \ } while (0) + +// If a trace function sets a new f_lineno and +// *then* raises, we use the destination when searching +// for an exception handler, displaying the traceback, and so on +#define INSTRUMENTED_JUMP(src, dest, event) \ +do { \ + _PyFrame_SetStackPointer(frame, stack_pointer); \ + int err = _Py_call_instrumentation_jump(tstate, event, frame, src, dest); \ + stack_pointer = _PyFrame_GetStackPointer(frame); \ + if (err) { \ + next_instr = (dest)+1; \ + goto error; \ + } \ + next_instr = frame->prev_instr; \ +} while (0); diff --git a/Python/clinic/instrumentation.c.h b/Python/clinic/instrumentation.c.h new file mode 100644 index 00000000000000..cf3984ca24bbfe --- /dev/null +++ b/Python/clinic/instrumentation.c.h @@ -0,0 +1,311 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +#if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) +# include "pycore_gc.h" // PyGC_Head +# include "pycore_runtime.h" // _Py_ID() +#endif + + +PyDoc_STRVAR(monitoring_use_tool_id__doc__, +"use_tool_id($module, tool_id, name, /)\n" +"--\n" +"\n"); + +#define MONITORING_USE_TOOL_ID_METHODDEF \ + {"use_tool_id", _PyCFunction_CAST(monitoring_use_tool_id), METH_FASTCALL, monitoring_use_tool_id__doc__}, + +static PyObject * +monitoring_use_tool_id_impl(PyObject *module, int tool_id, PyObject *name); + +static PyObject * +monitoring_use_tool_id(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + int tool_id; + PyObject *name; + + if (!_PyArg_CheckPositional("use_tool_id", nargs, 2, 2)) { + goto exit; + } + tool_id = _PyLong_AsInt(args[0]); + if (tool_id == -1 && PyErr_Occurred()) { + goto exit; + } + name = args[1]; + return_value = monitoring_use_tool_id_impl(module, tool_id, name); + +exit: + return return_value; +} + +PyDoc_STRVAR(monitoring_free_tool_id__doc__, +"free_tool_id($module, tool_id, /)\n" +"--\n" +"\n"); + +#define MONITORING_FREE_TOOL_ID_METHODDEF \ + {"free_tool_id", (PyCFunction)monitoring_free_tool_id, METH_O, monitoring_free_tool_id__doc__}, + +static PyObject * +monitoring_free_tool_id_impl(PyObject *module, int tool_id); + +static PyObject * +monitoring_free_tool_id(PyObject *module, PyObject *arg) +{ + PyObject *return_value = NULL; + int tool_id; + + tool_id = _PyLong_AsInt(arg); + if (tool_id == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = monitoring_free_tool_id_impl(module, tool_id); + +exit: + return return_value; +} + +PyDoc_STRVAR(monitoring_get_tool__doc__, +"get_tool($module, tool_id, /)\n" +"--\n" +"\n"); + +#define MONITORING_GET_TOOL_METHODDEF \ + {"get_tool", (PyCFunction)monitoring_get_tool, METH_O, monitoring_get_tool__doc__}, + +static PyObject * +monitoring_get_tool_impl(PyObject *module, int tool_id); + +static PyObject * +monitoring_get_tool(PyObject *module, PyObject *arg) +{ + PyObject *return_value = NULL; + int tool_id; + + tool_id = _PyLong_AsInt(arg); + if (tool_id == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = monitoring_get_tool_impl(module, tool_id); + +exit: + return return_value; +} + +PyDoc_STRVAR(monitoring_register_callback__doc__, +"register_callback($module, tool_id, event, func, /)\n" +"--\n" +"\n"); + +#define MONITORING_REGISTER_CALLBACK_METHODDEF \ + {"register_callback", _PyCFunction_CAST(monitoring_register_callback), METH_FASTCALL, monitoring_register_callback__doc__}, + +static PyObject * +monitoring_register_callback_impl(PyObject *module, int tool_id, int event, + PyObject *func); + +static PyObject * +monitoring_register_callback(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + int tool_id; + int event; + PyObject *func; + + if (!_PyArg_CheckPositional("register_callback", nargs, 3, 3)) { + goto exit; + } + tool_id = _PyLong_AsInt(args[0]); + if (tool_id == -1 && PyErr_Occurred()) { + goto exit; + } + event = _PyLong_AsInt(args[1]); + if (event == -1 && PyErr_Occurred()) { + goto exit; + } + func = args[2]; + return_value = monitoring_register_callback_impl(module, tool_id, event, func); + +exit: + return return_value; +} + +PyDoc_STRVAR(monitoring_get_events__doc__, +"get_events($module, tool_id, /)\n" +"--\n" +"\n"); + +#define MONITORING_GET_EVENTS_METHODDEF \ + {"get_events", (PyCFunction)monitoring_get_events, METH_O, monitoring_get_events__doc__}, + +static int +monitoring_get_events_impl(PyObject *module, int tool_id); + +static PyObject * +monitoring_get_events(PyObject *module, PyObject *arg) +{ + PyObject *return_value = NULL; + int tool_id; + int _return_value; + + tool_id = _PyLong_AsInt(arg); + if (tool_id == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = monitoring_get_events_impl(module, tool_id); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyLong_FromLong((long)_return_value); + +exit: + return return_value; +} + +PyDoc_STRVAR(monitoring_set_events__doc__, +"set_events($module, tool_id, event_set, /)\n" +"--\n" +"\n"); + +#define MONITORING_SET_EVENTS_METHODDEF \ + {"set_events", _PyCFunction_CAST(monitoring_set_events), METH_FASTCALL, monitoring_set_events__doc__}, + +static PyObject * +monitoring_set_events_impl(PyObject *module, int tool_id, int event_set); + +static PyObject * +monitoring_set_events(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + int tool_id; + int event_set; + + if (!_PyArg_CheckPositional("set_events", nargs, 2, 2)) { + goto exit; + } + tool_id = _PyLong_AsInt(args[0]); + if (tool_id == -1 && PyErr_Occurred()) { + goto exit; + } + event_set = _PyLong_AsInt(args[1]); + if (event_set == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = monitoring_set_events_impl(module, tool_id, event_set); + +exit: + return return_value; +} + +PyDoc_STRVAR(monitoring_get_local_events__doc__, +"get_local_events($module, tool_id, code, /)\n" +"--\n" +"\n"); + +#define MONITORING_GET_LOCAL_EVENTS_METHODDEF \ + {"get_local_events", _PyCFunction_CAST(monitoring_get_local_events), METH_FASTCALL, monitoring_get_local_events__doc__}, + +static int +monitoring_get_local_events_impl(PyObject *module, int tool_id, + PyObject *code); + +static PyObject * +monitoring_get_local_events(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + int tool_id; + PyObject *code; + int _return_value; + + if (!_PyArg_CheckPositional("get_local_events", nargs, 2, 2)) { + goto exit; + } + tool_id = _PyLong_AsInt(args[0]); + if (tool_id == -1 && PyErr_Occurred()) { + goto exit; + } + code = args[1]; + _return_value = monitoring_get_local_events_impl(module, tool_id, code); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyLong_FromLong((long)_return_value); + +exit: + return return_value; +} + +PyDoc_STRVAR(monitoring_set_local_events__doc__, +"set_local_events($module, tool_id, code, event_set, /)\n" +"--\n" +"\n"); + +#define MONITORING_SET_LOCAL_EVENTS_METHODDEF \ + {"set_local_events", _PyCFunction_CAST(monitoring_set_local_events), METH_FASTCALL, monitoring_set_local_events__doc__}, + +static PyObject * +monitoring_set_local_events_impl(PyObject *module, int tool_id, + PyObject *code, int event_set); + +static PyObject * +monitoring_set_local_events(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + int tool_id; + PyObject *code; + int event_set; + + if (!_PyArg_CheckPositional("set_local_events", nargs, 3, 3)) { + goto exit; + } + tool_id = _PyLong_AsInt(args[0]); + if (tool_id == -1 && PyErr_Occurred()) { + goto exit; + } + code = args[1]; + event_set = _PyLong_AsInt(args[2]); + if (event_set == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = monitoring_set_local_events_impl(module, tool_id, code, event_set); + +exit: + return return_value; +} + +PyDoc_STRVAR(monitoring_restart_events__doc__, +"restart_events($module, /)\n" +"--\n" +"\n"); + +#define MONITORING_RESTART_EVENTS_METHODDEF \ + {"restart_events", (PyCFunction)monitoring_restart_events, METH_NOARGS, monitoring_restart_events__doc__}, + +static PyObject * +monitoring_restart_events_impl(PyObject *module); + +static PyObject * +monitoring_restart_events(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return monitoring_restart_events_impl(module); +} + +PyDoc_STRVAR(monitoring__all_events__doc__, +"_all_events($module, /)\n" +"--\n" +"\n"); + +#define MONITORING__ALL_EVENTS_METHODDEF \ + {"_all_events", (PyCFunction)monitoring__all_events, METH_NOARGS, monitoring__all_events__doc__}, + +static PyObject * +monitoring__all_events_impl(PyObject *module); + +static PyObject * +monitoring__all_events(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return monitoring__all_events_impl(module); +} +/*[clinic end generated code: output=11cc0803875b3ffa input=a9049054013a1b77]*/ diff --git a/Python/codecs.c b/Python/codecs.c index 9d800f9856c2f7..1983f56ba204c1 100644 --- a/Python/codecs.c +++ b/Python/codecs.c @@ -11,6 +11,7 @@ Copyright (c) Corporation for National Research Initiatives. #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_interp.h" // PyInterpreterState.codec_search_path +#include "pycore_pyerrors.h" // _PyErr_FormatNote() #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_ucnhash.h" // _PyUnicode_Name_CAPI #include @@ -382,29 +383,6 @@ PyObject *PyCodec_StreamWriter(const char *encoding, return codec_getstreamcodec(encoding, stream, errors, 3); } -static void -add_note_to_codec_error(const char *operation, - const char *encoding) -{ - PyObject *exc = PyErr_GetRaisedException(); - if (exc == NULL) { - return; - } - PyObject *note = PyUnicode_FromFormat("%s with '%s' codec failed", - operation, encoding); - if (note == NULL) { - _PyErr_ChainExceptions1(exc); - return; - } - int res = _PyException_AddNote(exc, note); - Py_DECREF(note); - if (res < 0) { - _PyErr_ChainExceptions1(exc); - return; - } - PyErr_SetRaisedException(exc); -} - /* Encode an object (e.g. a Unicode object) using the given encoding and return the resulting encoded object (usually a Python string). @@ -425,7 +403,7 @@ _PyCodec_EncodeInternal(PyObject *object, result = PyObject_Call(encoder, args, NULL); if (result == NULL) { - add_note_to_codec_error("encoding", encoding); + _PyErr_FormatNote("%s with '%s' codec failed", "encoding", encoding); goto onError; } @@ -470,7 +448,7 @@ _PyCodec_DecodeInternal(PyObject *object, result = PyObject_Call(decoder, args, NULL); if (result == NULL) { - add_note_to_codec_error("decoding", encoding); + _PyErr_FormatNote("%s with '%s' codec failed", "decoding", encoding); goto onError; } if (!PyTuple_Check(result) || diff --git a/Python/compile.c b/Python/compile.c index 0572ac373ada78..c53a64f73fc74a 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -6,10 +6,10 @@ * object: * 1. Checks for future statements. See future.c * 2. Builds a symbol table. See symtable.c. - * 3. Generate code for basic blocks. See compiler_mod() in this file. - * 4. Assemble the basic blocks into final code. See assemble() in - * this file. - * 5. Optimize the byte code (peephole optimizations). + * 3. Generate an instruction sequence. See compiler_mod() in this file. + * 4. Generate a control flow graph and run optimizations on it. See flowgraph.c. + * 5. Assemble the basic blocks into final code. See optimize_and_assemble() in + * this file, and assembler.c. * * Note that compiler_mod() suggests module, but the module ast type * (mod_ty) has cases for expressions and interactive statements. @@ -101,15 +101,6 @@ location_is_after(location loc1, location loc2) { (loc1.col_offset > loc2.end_col_offset)); } -static inline bool -same_location(location a, location b) -{ - return a.lineno == b.lineno && - a.end_lineno == b.end_lineno && - a.col_offset == b.col_offset && - a.end_col_offset == b.end_col_offset; -} - #define LOC(x) SRC_LOCATION_FROM_AST(x) typedef _PyCfgJumpTargetLabel jump_target_label; @@ -129,46 +120,6 @@ static jump_target_label NO_LABEL = {-1}; RETURN_IF_ERROR(instr_sequence_use_label(INSTR_SEQUENCE(C), (LBL).id)) -static void -write_instr(_Py_CODEUNIT *codestr, cfg_instr *instruction, int ilen) -{ - int opcode = instruction->i_opcode; - assert(!IS_PSEUDO_OPCODE(opcode)); - int oparg = instruction->i_oparg; - assert(HAS_ARG(opcode) || oparg == 0); - int caches = _PyOpcode_Caches[opcode]; - switch (ilen - caches) { - case 4: - codestr->op.code = EXTENDED_ARG; - codestr->op.arg = (oparg >> 24) & 0xFF; - codestr++; - /* fall through */ - case 3: - codestr->op.code = EXTENDED_ARG; - codestr->op.arg = (oparg >> 16) & 0xFF; - codestr++; - /* fall through */ - case 2: - codestr->op.code = EXTENDED_ARG; - codestr->op.arg = (oparg >> 8) & 0xFF; - codestr++; - /* fall through */ - case 1: - codestr->op.code = opcode; - codestr->op.arg = oparg & 0xFF; - codestr++; - break; - default: - Py_UNREACHABLE(); - } - while (caches--) { - codestr->op.code = CACHE; - codestr->op.arg = 0; - codestr++; - } -} - - /* fblockinfo tracks the current frame block. A frame block is used to handle loops, try/except, and try/finally. @@ -198,21 +149,8 @@ enum { COMPILER_SCOPE_COMPREHENSION, }; -typedef struct { - int i_opcode; - int i_oparg; - location i_loc; -} instruction; - -typedef struct instr_sequence_ { - instruction *s_instrs; - int s_allocated; - int s_used; - - int *s_labelmap; /* label id --> instr offset */ - int s_labelmap_size; - int s_next_free_label; /* next free label id */ -} instr_sequence; +typedef _PyCompilerInstruction instruction; +typedef _PyCompile_InstructionSequence instr_sequence; #define INITIAL_INSTR_SEQUENCE_SIZE 100 #define INITIAL_INSTR_SEQUENCE_LABELS_MAP_SIZE 10 @@ -314,7 +252,6 @@ static int instr_sequence_addop(instr_sequence *seq, int opcode, int oparg, location loc) { assert(IS_WITHIN_OPCODE_RANGE(opcode)); - assert(!IS_ASSEMBLER_OPCODE(opcode)); assert(HAS_ARG(opcode) || HAS_TARGET(opcode) || oparg == 0); assert(0 <= oparg && oparg < (1 << 30)); @@ -432,32 +369,17 @@ instr_sequence_to_cfg(instr_sequence *seq, cfg_builder *g) { struct compiler_unit { PySTEntryObject *u_ste; - PyObject *u_name; - PyObject *u_qualname; /* dot-separated qualified name (lazy) */ int u_scope_type; - /* The following fields are dicts that map objects to - the index of them in co_XXX. The index is used as - the argument for opcodes that refer to those collections. - */ - PyObject *u_consts; /* all constants */ - PyObject *u_names; /* all names */ - PyObject *u_varnames; /* local variables */ - PyObject *u_cellvars; /* cell variables */ - PyObject *u_freevars; /* free variables */ PyObject *u_private; /* for private name mangling */ - Py_ssize_t u_argcount; /* number of arguments for block */ - Py_ssize_t u_posonlyargcount; /* number of positional only arguments for block */ - Py_ssize_t u_kwonlyargcount; /* number of keyword only arguments for block */ - instr_sequence u_instr_sequence; /* codegen output */ int u_nfblocks; struct fblockinfo u_fblock[CO_MAXBLOCKS]; - int u_firstlineno; /* the first lineno of the block */ + _PyCompile_CodeUnitMetadata u_metadata; }; /* This struct captures the global state of a compilation. @@ -566,7 +488,7 @@ static int compiler_match(struct compiler *, stmt_ty); static int compiler_pattern_subpattern(struct compiler *, pattern_ty, pattern_context *); -static PyCodeObject *assemble(struct compiler *, int addNone); +static PyCodeObject *optimize_and_assemble(struct compiler *, int addNone); #define CAPSULE_NAME "compile.c compiler unit" @@ -750,13 +672,13 @@ compiler_unit_free(struct compiler_unit *u) { instr_sequence_fini(&u->u_instr_sequence); Py_CLEAR(u->u_ste); - Py_CLEAR(u->u_name); - Py_CLEAR(u->u_qualname); - Py_CLEAR(u->u_consts); - Py_CLEAR(u->u_names); - Py_CLEAR(u->u_varnames); - Py_CLEAR(u->u_freevars); - Py_CLEAR(u->u_cellvars); + Py_CLEAR(u->u_metadata.u_name); + Py_CLEAR(u->u_metadata.u_qualname); + Py_CLEAR(u->u_metadata.u_consts); + Py_CLEAR(u->u_metadata.u_names); + Py_CLEAR(u->u_metadata.u_varnames); + Py_CLEAR(u->u_metadata.u_freevars); + Py_CLEAR(u->u_metadata.u_cellvars); Py_CLEAR(u->u_private); PyObject_Free(u); } @@ -783,8 +705,8 @@ compiler_set_qualname(struct compiler *c) if (u->u_scope_type == COMPILER_SCOPE_FUNCTION || u->u_scope_type == COMPILER_SCOPE_ASYNC_FUNCTION || u->u_scope_type == COMPILER_SCOPE_CLASS) { - assert(u->u_name); - mangled = _Py_Mangle(parent->u_private, u->u_name); + assert(u->u_metadata.u_name); + mangled = _Py_Mangle(parent->u_private, u->u_metadata.u_name); if (!mangled) { return ERROR; } @@ -802,14 +724,14 @@ compiler_set_qualname(struct compiler *c) || parent->u_scope_type == COMPILER_SCOPE_LAMBDA) { _Py_DECLARE_STR(dot_locals, "."); - base = PyUnicode_Concat(parent->u_qualname, + base = PyUnicode_Concat(parent->u_metadata.u_qualname, &_Py_STR(dot_locals)); if (base == NULL) { return ERROR; } } else { - base = Py_NewRef(parent->u_qualname); + base = Py_NewRef(parent->u_metadata.u_qualname); } } } @@ -821,15 +743,15 @@ compiler_set_qualname(struct compiler *c) if (name == NULL) { return ERROR; } - PyUnicode_Append(&name, u->u_name); + PyUnicode_Append(&name, u->u_metadata.u_name); if (name == NULL) { return ERROR; } } else { - name = Py_NewRef(u->u_name); + name = Py_NewRef(u->u_metadata.u_name); } - u->u_qualname = name; + u->u_metadata.u_qualname = name; return SUCCESS; } @@ -930,6 +852,7 @@ static int codegen_addop_noarg(instr_sequence *seq, int opcode, location loc) { assert(!HAS_ARG(opcode)); + assert(!IS_ASSEMBLER_OPCODE(opcode)); return instr_sequence_addop(seq, opcode, 0, loc); } @@ -1077,7 +1000,7 @@ compiler_add_const(PyObject *const_cache, struct compiler_unit *u, PyObject *o) return ERROR; } - Py_ssize_t arg = dict_add_o(u->u_consts, key); + Py_ssize_t arg = dict_add_o(u->u_metadata.u_consts, key); Py_DECREF(key); return arg; } @@ -1140,6 +1063,7 @@ codegen_addop_i(instr_sequence *seq, int opcode, Py_ssize_t oparg, location loc) EXTENDED_ARG is used for 16, 24, and 32-bit arguments. */ int oparg_ = Py_SAFE_DOWNCAST(oparg, Py_ssize_t, int); + assert(!IS_ASSEMBLER_OPCODE(opcode)); return instr_sequence_addop(seq, opcode, oparg_, loc); } @@ -1149,6 +1073,7 @@ codegen_addop_j(instr_sequence *seq, location loc, { assert(IS_LABEL(target)); assert(IS_JUMP_OPCODE(opcode) || IS_BLOCK_PUSH_OPCODE(opcode)); + assert(!IS_ASSEMBLER_OPCODE(opcode)); return instr_sequence_addop(seq, opcode, target.id, loc); } @@ -1180,7 +1105,7 @@ codegen_addop_j(instr_sequence *seq, location loc, #define ADDOP_N(C, LOC, OP, O, TYPE) { \ assert(!HAS_CONST(OP)); /* use ADDOP_LOAD_CONST_NEW */ \ - if (compiler_addop_o((C)->u, (LOC), (OP), (C)->u->u_ ## TYPE, (O)) < 0) { \ + if (compiler_addop_o((C)->u, (LOC), (OP), (C)->u->u_metadata.u_ ## TYPE, (O)) < 0) { \ Py_DECREF((O)); \ return ERROR; \ } \ @@ -1188,7 +1113,7 @@ codegen_addop_j(instr_sequence *seq, location loc, } #define ADDOP_NAME(C, LOC, OP, O, TYPE) \ - RETURN_IF_ERROR(compiler_addop_name((C)->u, (LOC), (OP), (C)->u->u_ ## TYPE, (O))) + RETURN_IF_ERROR(compiler_addop_name((C)->u, (LOC), (OP), (C)->u->u_metadata.u_ ## TYPE, (O))) #define ADDOP_I(C, LOC, OP, O) \ RETURN_IF_ERROR(codegen_addop_i(INSTR_SEQUENCE(C), (OP), (O), (LOC))) @@ -1265,18 +1190,18 @@ compiler_enter_scope(struct compiler *c, identifier name, return ERROR; } u->u_scope_type = scope_type; - u->u_argcount = 0; - u->u_posonlyargcount = 0; - u->u_kwonlyargcount = 0; + u->u_metadata.u_argcount = 0; + u->u_metadata.u_posonlyargcount = 0; + u->u_metadata.u_kwonlyargcount = 0; u->u_ste = PySymtable_Lookup(c->c_st, key); if (!u->u_ste) { compiler_unit_free(u); return ERROR; } - u->u_name = Py_NewRef(name); - u->u_varnames = list2dict(u->u_ste->ste_varnames); - u->u_cellvars = dictbytype(u->u_ste->ste_symbols, CELL, 0, 0); - if (!u->u_varnames || !u->u_cellvars) { + u->u_metadata.u_name = Py_NewRef(name); + u->u_metadata.u_varnames = list2dict(u->u_ste->ste_varnames); + u->u_metadata.u_cellvars = dictbytype(u->u_ste->ste_symbols, CELL, 0, 0); + if (!u->u_metadata.u_varnames || !u->u_metadata.u_cellvars) { compiler_unit_free(u); return ERROR; } @@ -1284,8 +1209,8 @@ compiler_enter_scope(struct compiler *c, identifier name, /* Cook up an implicit __class__ cell. */ int res; assert(u->u_scope_type == COMPILER_SCOPE_CLASS); - assert(PyDict_GET_SIZE(u->u_cellvars) == 0); - res = PyDict_SetItem(u->u_cellvars, &_Py_ID(__class__), + assert(PyDict_GET_SIZE(u->u_metadata.u_cellvars) == 0); + res = PyDict_SetItem(u->u_metadata.u_cellvars, &_Py_ID(__class__), _PyLong_GetZero()); if (res < 0) { compiler_unit_free(u); @@ -1293,22 +1218,22 @@ compiler_enter_scope(struct compiler *c, identifier name, } } - u->u_freevars = dictbytype(u->u_ste->ste_symbols, FREE, DEF_FREE_CLASS, - PyDict_GET_SIZE(u->u_cellvars)); - if (!u->u_freevars) { + u->u_metadata.u_freevars = dictbytype(u->u_ste->ste_symbols, FREE, DEF_FREE_CLASS, + PyDict_GET_SIZE(u->u_metadata.u_cellvars)); + if (!u->u_metadata.u_freevars) { compiler_unit_free(u); return ERROR; } u->u_nfblocks = 0; - u->u_firstlineno = lineno; - u->u_consts = PyDict_New(); - if (!u->u_consts) { + u->u_metadata.u_firstlineno = lineno; + u->u_metadata.u_consts = PyDict_New(); + if (!u->u_metadata.u_consts) { compiler_unit_free(u); return ERROR; } - u->u_names = PyDict_New(); - if (!u->u_names) { + u->u_metadata.u_names = PyDict_New(); + if (!u->u_metadata.u_names) { compiler_unit_free(u); return ERROR; } @@ -1502,8 +1427,7 @@ compiler_add_yield_from(struct compiler *c, location loc, int await) ADDOP(c, loc, CLEANUP_THROW); USE_LABEL(c, exit); - ADDOP_I(c, loc, SWAP, 2); - ADDOP(c, loc, POP_TOP); + ADDOP(c, loc, END_SEND); return SUCCESS; } @@ -1667,7 +1591,7 @@ compiler_body(struct compiler *c, location loc, asdl_stmt_seq *stmts) /* Set current line number to the line number of first statement. This way line number for SETUP_ANNOTATIONS will always coincide with the line number of first "real" statement in module. - If body is empty, then lineno will be set later in assemble. */ + If body is empty, then lineno will be set later in optimize_and_assemble. */ if (c->u->u_scope_type == COMPILER_SCOPE_MODULE && asdl_seq_LEN(stmts)) { st = (stmt_ty)asdl_seq_GET(stmts, 0); loc = LOC(st); @@ -1738,7 +1662,7 @@ compiler_mod(struct compiler *c, mod_ty mod) if (compiler_codegen(c, mod) < 0) { return NULL; } - PyCodeObject *co = assemble(c, addNone); + PyCodeObject *co = optimize_and_assemble(c, addNone); compiler_exit_scope(c); return co; } @@ -1762,8 +1686,8 @@ get_ref_type(struct compiler *c, PyObject *name) "unknown scope in unit %S (%R); " "symbols: %R; locals: %R; globals: %R", name, - c->u->u_name, c->u->u_ste->ste_id, - c->u->u_ste->ste_symbols, c->u->u_varnames, c->u->u_names); + c->u->u_metadata.u_name, c->u->u_ste->ste_id, + c->u->u_ste->ste_symbols, c->u->u_metadata.u_varnames, c->u->u_metadata.u_names); return ERROR; } return scope; @@ -1803,10 +1727,10 @@ compiler_make_closure(struct compiler *c, location loc, } int arg; if (reftype == CELL) { - arg = compiler_lookup_arg(c->u->u_cellvars, name); + arg = compiler_lookup_arg(c->u->u_metadata.u_cellvars, name); } else { - arg = compiler_lookup_arg(c->u->u_freevars, name); + arg = compiler_lookup_arg(c->u->u_metadata.u_freevars, name); } if (arg == -1) { PyObject *freevars = _PyCode_GetFreevars(co); @@ -1818,7 +1742,7 @@ compiler_make_closure(struct compiler *c, location loc, "freevars of code %S: %R", name, reftype, - c->u->u_name, + c->u->u_metadata.u_name, co->co_name, freevars); Py_DECREF(freevars); @@ -2187,9 +2111,9 @@ compiler_function(struct compiler *c, stmt_ty s, int is_async) return ERROR; } - c->u->u_argcount = asdl_seq_LEN(args->args); - c->u->u_posonlyargcount = asdl_seq_LEN(args->posonlyargs); - c->u->u_kwonlyargcount = asdl_seq_LEN(args->kwonlyargs); + c->u->u_metadata.u_argcount = asdl_seq_LEN(args->args); + c->u->u_metadata.u_posonlyargcount = asdl_seq_LEN(args->posonlyargs); + c->u->u_metadata.u_kwonlyargcount = asdl_seq_LEN(args->kwonlyargs); for (i = docstring ? 1 : 0; i < asdl_seq_LEN(body); i++) { VISIT_IN_SCOPE(c, stmt, (stmt_ty)asdl_seq_GET(body, i)); } @@ -2199,7 +2123,7 @@ compiler_function(struct compiler *c, stmt_ty s, int is_async) return ERROR; } } - co = assemble(c, 1); + co = optimize_and_assemble(c, 1); compiler_exit_scope(c); if (co == NULL) { Py_XDECREF(co); @@ -2259,8 +2183,8 @@ compiler_class(struct compiler *c, stmt_ty s) compiler_exit_scope(c); return ERROR; } - assert(c->u->u_qualname); - ADDOP_LOAD_CONST(c, loc, c->u->u_qualname); + assert(c->u->u_metadata.u_qualname); + ADDOP_LOAD_CONST(c, loc, c->u->u_metadata.u_qualname); if (compiler_nameop(c, loc, &_Py_ID(__qualname__), Store) < 0) { compiler_exit_scope(c); return ERROR; @@ -2274,7 +2198,7 @@ compiler_class(struct compiler *c, stmt_ty s) /* Return __classcell__ if it is referenced, otherwise return None */ if (c->u->u_ste->ste_needs_class_closure) { /* Store __classcell__ into class namespace & return it */ - i = compiler_lookup_arg(c->u->u_cellvars, &_Py_ID(__class__)); + i = compiler_lookup_arg(c->u->u_metadata.u_cellvars, &_Py_ID(__class__)); if (i < 0) { compiler_exit_scope(c); return ERROR; @@ -2289,12 +2213,12 @@ compiler_class(struct compiler *c, stmt_ty s) } else { /* No methods referenced __class__, so just return None */ - assert(PyDict_GET_SIZE(c->u->u_cellvars) == 0); + assert(PyDict_GET_SIZE(c->u->u_metadata.u_cellvars) == 0); ADDOP_LOAD_CONST(c, NO_LOCATION, Py_None); } ADDOP_IN_SCOPE(c, NO_LOCATION, RETURN_VALUE); /* create the code object */ - co = assemble(c, 1); + co = optimize_and_assemble(c, 1); } /* leave the new scope */ compiler_exit_scope(c); @@ -2565,17 +2489,17 @@ compiler_lambda(struct compiler *c, expr_ty e) docstring. */ RETURN_IF_ERROR(compiler_add_const(c->c_const_cache, c->u, Py_None)); - c->u->u_argcount = asdl_seq_LEN(args->args); - c->u->u_posonlyargcount = asdl_seq_LEN(args->posonlyargs); - c->u->u_kwonlyargcount = asdl_seq_LEN(args->kwonlyargs); + c->u->u_metadata.u_argcount = asdl_seq_LEN(args->args); + c->u->u_metadata.u_posonlyargcount = asdl_seq_LEN(args->posonlyargs); + c->u->u_metadata.u_kwonlyargcount = asdl_seq_LEN(args->kwonlyargs); VISIT_IN_SCOPE(c, expr, e->v.Lambda.body); if (c->u->u_ste->ste_generator) { - co = assemble(c, 0); + co = optimize_and_assemble(c, 0); } else { location loc = LOCATION(e->lineno, e->lineno, 0, 0); ADDOP_IN_SCOPE(c, loc, RETURN_VALUE); - co = assemble(c, 1); + co = optimize_and_assemble(c, 1); } compiler_exit_scope(c); if (co == NULL) { @@ -3721,7 +3645,7 @@ compiler_nameop(struct compiler *c, location loc, Py_ssize_t arg; enum { OP_FAST, OP_GLOBAL, OP_DEREF, OP_NAME } optype; - PyObject *dict = c->u->u_names; + PyObject *dict = c->u->u_metadata.u_names; PyObject *mangled; assert(!_PyUnicode_EqualToASCIIString(name, "None") && @@ -3742,11 +3666,11 @@ compiler_nameop(struct compiler *c, location loc, scope = _PyST_GetScope(c->u->u_ste, mangled); switch (scope) { case FREE: - dict = c->u->u_freevars; + dict = c->u->u_metadata.u_freevars; optype = OP_DEREF; break; case CELL: - dict = c->u->u_cellvars; + dict = c->u->u_metadata.u_cellvars; optype = OP_DEREF; break; case LOCAL: @@ -4712,7 +4636,7 @@ compiler_sync_comprehension_generator(struct compiler *c, location loc, if (gen_index == 0) { /* Receive outermost iter as an implicit argument */ - c->u->u_argcount = 1; + c->u->u_metadata.u_argcount = 1; ADDOP_I(c, loc, LOAD_FAST, 0); } else { @@ -4825,7 +4749,7 @@ compiler_async_comprehension_generator(struct compiler *c, location loc, if (gen_index == 0) { /* Receive outermost iter as an implicit argument */ - c->u->u_argcount = 1; + c->u->u_metadata.u_argcount = 1; ADDOP_I(c, loc, LOAD_FAST, 0); } else { @@ -4973,7 +4897,7 @@ compiler_comprehension(struct compiler *c, expr_ty e, int type, } } - co = assemble(c, 1); + co = optimize_and_assemble(c, 1); compiler_exit_scope(c); if (is_top_level_await && is_async_generator){ c->u->u_ste->ste_coroutine = 1; @@ -6596,378 +6520,6 @@ compiler_match(struct compiler *c, stmt_ty s) #undef WILDCARD_CHECK #undef WILDCARD_STAR_CHECK - -/* End of the compiler section, beginning of the assembler section */ - - -struct assembler { - PyObject *a_bytecode; /* bytes containing bytecode */ - int a_offset; /* offset into bytecode */ - PyObject *a_except_table; /* bytes containing exception table */ - int a_except_table_off; /* offset into exception table */ - /* Location Info */ - int a_lineno; /* lineno of last emitted instruction */ - PyObject* a_linetable; /* bytes containing location info */ - int a_location_off; /* offset of last written location info frame */ -}; - -static int -assemble_init(struct assembler *a, int firstlineno) -{ - memset(a, 0, sizeof(struct assembler)); - a->a_lineno = firstlineno; - a->a_linetable = NULL; - a->a_location_off = 0; - a->a_except_table = NULL; - a->a_bytecode = PyBytes_FromStringAndSize(NULL, DEFAULT_CODE_SIZE); - if (a->a_bytecode == NULL) { - goto error; - } - a->a_linetable = PyBytes_FromStringAndSize(NULL, DEFAULT_CNOTAB_SIZE); - if (a->a_linetable == NULL) { - goto error; - } - a->a_except_table = PyBytes_FromStringAndSize(NULL, DEFAULT_LNOTAB_SIZE); - if (a->a_except_table == NULL) { - goto error; - } - return SUCCESS; -error: - Py_XDECREF(a->a_bytecode); - Py_XDECREF(a->a_linetable); - Py_XDECREF(a->a_except_table); - return ERROR; -} - -static void -assemble_free(struct assembler *a) -{ - Py_XDECREF(a->a_bytecode); - Py_XDECREF(a->a_linetable); - Py_XDECREF(a->a_except_table); -} - -static inline void -write_except_byte(struct assembler *a, int byte) { - unsigned char *p = (unsigned char *) PyBytes_AS_STRING(a->a_except_table); - p[a->a_except_table_off++] = byte; -} - -#define CONTINUATION_BIT 64 - -static void -assemble_emit_exception_table_item(struct assembler *a, int value, int msb) -{ - assert ((msb | 128) == 128); - assert(value >= 0 && value < (1 << 30)); - if (value >= 1 << 24) { - write_except_byte(a, (value >> 24) | CONTINUATION_BIT | msb); - msb = 0; - } - if (value >= 1 << 18) { - write_except_byte(a, ((value >> 18)&0x3f) | CONTINUATION_BIT | msb); - msb = 0; - } - if (value >= 1 << 12) { - write_except_byte(a, ((value >> 12)&0x3f) | CONTINUATION_BIT | msb); - msb = 0; - } - if (value >= 1 << 6) { - write_except_byte(a, ((value >> 6)&0x3f) | CONTINUATION_BIT | msb); - msb = 0; - } - write_except_byte(a, (value&0x3f) | msb); -} - -/* See Objects/exception_handling_notes.txt for details of layout */ -#define MAX_SIZE_OF_ENTRY 20 - -static int -assemble_emit_exception_table_entry(struct assembler *a, int start, int end, basicblock *handler) -{ - Py_ssize_t len = PyBytes_GET_SIZE(a->a_except_table); - if (a->a_except_table_off + MAX_SIZE_OF_ENTRY >= len) { - RETURN_IF_ERROR(_PyBytes_Resize(&a->a_except_table, len * 2)); - } - int size = end-start; - assert(end > start); - int target = handler->b_offset; - int depth = handler->b_startdepth - 1; - if (handler->b_preserve_lasti) { - depth -= 1; - } - assert(depth >= 0); - int depth_lasti = (depth<<1) | handler->b_preserve_lasti; - assemble_emit_exception_table_item(a, start, (1<<7)); - assemble_emit_exception_table_item(a, size, 0); - assemble_emit_exception_table_item(a, target, 0); - assemble_emit_exception_table_item(a, depth_lasti, 0); - return SUCCESS; -} - -static int -assemble_exception_table(struct assembler *a, basicblock *entryblock) -{ - basicblock *b; - int ioffset = 0; - basicblock *handler = NULL; - int start = -1; - for (b = entryblock; b != NULL; b = b->b_next) { - ioffset = b->b_offset; - for (int i = 0; i < b->b_iused; i++) { - cfg_instr *instr = &b->b_instr[i]; - if (instr->i_except != handler) { - if (handler != NULL) { - RETURN_IF_ERROR( - assemble_emit_exception_table_entry(a, start, ioffset, handler)); - } - start = ioffset; - handler = instr->i_except; - } - ioffset += _PyCfg_InstrSize(instr); - } - } - if (handler != NULL) { - RETURN_IF_ERROR(assemble_emit_exception_table_entry(a, start, ioffset, handler)); - } - return SUCCESS; -} - -/* Code location emitting code. See locations.md for a description of the format. */ - -#define MSB 0x80 - -static void -write_location_byte(struct assembler* a, int val) -{ - PyBytes_AS_STRING(a->a_linetable)[a->a_location_off] = val&255; - a->a_location_off++; -} - - -static uint8_t * -location_pointer(struct assembler* a) -{ - return (uint8_t *)PyBytes_AS_STRING(a->a_linetable) + - a->a_location_off; -} - -static void -write_location_first_byte(struct assembler* a, int code, int length) -{ - a->a_location_off += write_location_entry_start( - location_pointer(a), code, length); -} - -static void -write_location_varint(struct assembler* a, unsigned int val) -{ - uint8_t *ptr = location_pointer(a); - a->a_location_off += write_varint(ptr, val); -} - - -static void -write_location_signed_varint(struct assembler* a, int val) -{ - uint8_t *ptr = location_pointer(a); - a->a_location_off += write_signed_varint(ptr, val); -} - -static void -write_location_info_short_form(struct assembler* a, int length, int column, int end_column) -{ - assert(length > 0 && length <= 8); - int column_low_bits = column & 7; - int column_group = column >> 3; - assert(column < 80); - assert(end_column >= column); - assert(end_column - column < 16); - write_location_first_byte(a, PY_CODE_LOCATION_INFO_SHORT0 + column_group, length); - write_location_byte(a, (column_low_bits << 4) | (end_column - column)); -} - -static void -write_location_info_oneline_form(struct assembler* a, int length, int line_delta, int column, int end_column) -{ - assert(length > 0 && length <= 8); - assert(line_delta >= 0 && line_delta < 3); - assert(column < 128); - assert(end_column < 128); - write_location_first_byte(a, PY_CODE_LOCATION_INFO_ONE_LINE0 + line_delta, length); - write_location_byte(a, column); - write_location_byte(a, end_column); -} - -static void -write_location_info_long_form(struct assembler* a, location loc, int length) -{ - assert(length > 0 && length <= 8); - write_location_first_byte(a, PY_CODE_LOCATION_INFO_LONG, length); - write_location_signed_varint(a, loc.lineno - a->a_lineno); - assert(loc.end_lineno >= loc.lineno); - write_location_varint(a, loc.end_lineno - loc.lineno); - write_location_varint(a, loc.col_offset + 1); - write_location_varint(a, loc.end_col_offset + 1); -} - -static void -write_location_info_none(struct assembler* a, int length) -{ - write_location_first_byte(a, PY_CODE_LOCATION_INFO_NONE, length); -} - -static void -write_location_info_no_column(struct assembler* a, int length, int line_delta) -{ - write_location_first_byte(a, PY_CODE_LOCATION_INFO_NO_COLUMNS, length); - write_location_signed_varint(a, line_delta); -} - -#define THEORETICAL_MAX_ENTRY_SIZE 25 /* 1 + 6 + 6 + 6 + 6 */ - -static int -write_location_info_entry(struct assembler* a, location loc, int isize) -{ - Py_ssize_t len = PyBytes_GET_SIZE(a->a_linetable); - if (a->a_location_off + THEORETICAL_MAX_ENTRY_SIZE >= len) { - assert(len > THEORETICAL_MAX_ENTRY_SIZE); - RETURN_IF_ERROR(_PyBytes_Resize(&a->a_linetable, len*2)); - } - if (loc.lineno < 0) { - write_location_info_none(a, isize); - return SUCCESS; - } - int line_delta = loc.lineno - a->a_lineno; - int column = loc.col_offset; - int end_column = loc.end_col_offset; - assert(column >= -1); - assert(end_column >= -1); - if (column < 0 || end_column < 0) { - if (loc.end_lineno == loc.lineno || loc.end_lineno == -1) { - write_location_info_no_column(a, isize, line_delta); - a->a_lineno = loc.lineno; - return SUCCESS; - } - } - else if (loc.end_lineno == loc.lineno) { - if (line_delta == 0 && column < 80 && end_column - column < 16 && end_column >= column) { - write_location_info_short_form(a, isize, column, end_column); - return SUCCESS; - } - if (line_delta >= 0 && line_delta < 3 && column < 128 && end_column < 128) { - write_location_info_oneline_form(a, isize, line_delta, column, end_column); - a->a_lineno = loc.lineno; - return SUCCESS; - } - } - write_location_info_long_form(a, loc, isize); - a->a_lineno = loc.lineno; - return SUCCESS; -} - -static int -assemble_emit_location(struct assembler* a, location loc, int isize) -{ - if (isize == 0) { - return SUCCESS; - } - while (isize > 8) { - RETURN_IF_ERROR(write_location_info_entry(a, loc, 8)); - isize -= 8; - } - return write_location_info_entry(a, loc, isize); -} - -static int -assemble_location_info(struct assembler *a, basicblock *entryblock, int firstlineno) -{ - a->a_lineno = firstlineno; - location loc = NO_LOCATION; - int size = 0; - for (basicblock *b = entryblock; b != NULL; b = b->b_next) { - for (int j = 0; j < b->b_iused; j++) { - if (!same_location(loc, b->b_instr[j].i_loc)) { - RETURN_IF_ERROR(assemble_emit_location(a, loc, size)); - loc = b->b_instr[j].i_loc; - size = 0; - } - size += _PyCfg_InstrSize(&b->b_instr[j]); - } - } - RETURN_IF_ERROR(assemble_emit_location(a, loc, size)); - return SUCCESS; -} - -/* assemble_emit_instr() - Extend the bytecode with a new instruction. - Update lnotab if necessary. -*/ - -static int -assemble_emit_instr(struct assembler *a, cfg_instr *i) -{ - Py_ssize_t len = PyBytes_GET_SIZE(a->a_bytecode); - _Py_CODEUNIT *code; - - int size = _PyCfg_InstrSize(i); - if (a->a_offset + size >= len / (int)sizeof(_Py_CODEUNIT)) { - if (len > PY_SSIZE_T_MAX / 2) { - return ERROR; - } - RETURN_IF_ERROR(_PyBytes_Resize(&a->a_bytecode, len * 2)); - } - code = (_Py_CODEUNIT *)PyBytes_AS_STRING(a->a_bytecode) + a->a_offset; - a->a_offset += size; - write_instr(code, i, size); - return SUCCESS; -} - -static int -assemble_emit(struct assembler *a, basicblock *entryblock, int first_lineno, - PyObject *const_cache) -{ - RETURN_IF_ERROR(assemble_init(a, first_lineno)); - - for (basicblock *b = entryblock; b != NULL; b = b->b_next) { - for (int j = 0; j < b->b_iused; j++) { - RETURN_IF_ERROR(assemble_emit_instr(a, &b->b_instr[j])); - } - } - - RETURN_IF_ERROR(assemble_location_info(a, entryblock, a->a_lineno)); - - RETURN_IF_ERROR(assemble_exception_table(a, entryblock)); - - RETURN_IF_ERROR(_PyBytes_Resize(&a->a_except_table, a->a_except_table_off)); - RETURN_IF_ERROR(_PyCompile_ConstCacheMergeOne(const_cache, &a->a_except_table)); - - RETURN_IF_ERROR(_PyBytes_Resize(&a->a_linetable, a->a_location_off)); - RETURN_IF_ERROR(_PyCompile_ConstCacheMergeOne(const_cache, &a->a_linetable)); - - RETURN_IF_ERROR(_PyBytes_Resize(&a->a_bytecode, a->a_offset * sizeof(_Py_CODEUNIT))); - RETURN_IF_ERROR(_PyCompile_ConstCacheMergeOne(const_cache, &a->a_bytecode)); - return SUCCESS; -} - -static PyObject * -dict_keys_inorder(PyObject *dict, Py_ssize_t offset) -{ - PyObject *tuple, *k, *v; - Py_ssize_t i, pos = 0, size = PyDict_GET_SIZE(dict); - - tuple = PyTuple_New(size); - if (tuple == NULL) - return NULL; - while (PyDict_Next(dict, &pos, &k, &v)) { - i = PyLong_AS_LONG(v); - assert((i - offset) < size); - assert((i - offset) >= 0); - PyTuple_SET_ITEM(tuple, i - offset, Py_NewRef(k)); - } - return tuple; -} - static PyObject * consts_dict_keys_inorder(PyObject *dict) { @@ -7055,153 +6607,13 @@ _PyCompile_ConstCacheMergeOne(PyObject *const_cache, PyObject **obj) return SUCCESS; } -// This is in codeobject.c. -extern void _Py_set_localsplus_info(int, PyObject *, unsigned char, - PyObject *, PyObject *); - -static void -compute_localsplus_info(struct compiler_unit *u, int nlocalsplus, - PyObject *names, PyObject *kinds) -{ - PyObject *k, *v; - Py_ssize_t pos = 0; - while (PyDict_Next(u->u_varnames, &pos, &k, &v)) { - int offset = (int)PyLong_AS_LONG(v); - assert(offset >= 0); - assert(offset < nlocalsplus); - // For now we do not distinguish arg kinds. - _PyLocals_Kind kind = CO_FAST_LOCAL; - if (PyDict_GetItem(u->u_cellvars, k) != NULL) { - kind |= CO_FAST_CELL; - } - _Py_set_localsplus_info(offset, k, kind, names, kinds); - } - int nlocals = (int)PyDict_GET_SIZE(u->u_varnames); - - // This counter mirrors the fix done in fix_cell_offsets(). - int numdropped = 0; - pos = 0; - while (PyDict_Next(u->u_cellvars, &pos, &k, &v)) { - if (PyDict_GetItem(u->u_varnames, k) != NULL) { - // Skip cells that are already covered by locals. - numdropped += 1; - continue; - } - int offset = (int)PyLong_AS_LONG(v); - assert(offset >= 0); - offset += nlocals - numdropped; - assert(offset < nlocalsplus); - _Py_set_localsplus_info(offset, k, CO_FAST_CELL, names, kinds); - } - - pos = 0; - while (PyDict_Next(u->u_freevars, &pos, &k, &v)) { - int offset = (int)PyLong_AS_LONG(v); - assert(offset >= 0); - offset += nlocals - numdropped; - assert(offset < nlocalsplus); - _Py_set_localsplus_info(offset, k, CO_FAST_FREE, names, kinds); - } -} - -static PyCodeObject * -makecode(struct compiler_unit *u, struct assembler *a, PyObject *const_cache, - PyObject *constslist, int maxdepth, int nlocalsplus, int code_flags, - PyObject *filename) -{ - PyCodeObject *co = NULL; - PyObject *names = NULL; - PyObject *consts = NULL; - PyObject *localsplusnames = NULL; - PyObject *localspluskinds = NULL; - names = dict_keys_inorder(u->u_names, 0); - if (!names) { - goto error; - } - if (_PyCompile_ConstCacheMergeOne(const_cache, &names) < 0) { - goto error; - } - - consts = PyList_AsTuple(constslist); /* PyCode_New requires a tuple */ - if (consts == NULL) { - goto error; - } - if (_PyCompile_ConstCacheMergeOne(const_cache, &consts) < 0) { - goto error; - } - - assert(u->u_posonlyargcount < INT_MAX); - assert(u->u_argcount < INT_MAX); - assert(u->u_kwonlyargcount < INT_MAX); - int posonlyargcount = (int)u->u_posonlyargcount; - int posorkwargcount = (int)u->u_argcount; - assert(INT_MAX - posonlyargcount - posorkwargcount > 0); - int kwonlyargcount = (int)u->u_kwonlyargcount; - - localsplusnames = PyTuple_New(nlocalsplus); - if (localsplusnames == NULL) { - goto error; - } - localspluskinds = PyBytes_FromStringAndSize(NULL, nlocalsplus); - if (localspluskinds == NULL) { - goto error; - } - compute_localsplus_info(u, nlocalsplus, localsplusnames, localspluskinds); - - struct _PyCodeConstructor con = { - .filename = filename, - .name = u->u_name, - .qualname = u->u_qualname ? u->u_qualname : u->u_name, - .flags = code_flags, - - .code = a->a_bytecode, - .firstlineno = u->u_firstlineno, - .linetable = a->a_linetable, - - .consts = consts, - .names = names, - - .localsplusnames = localsplusnames, - .localspluskinds = localspluskinds, - - .argcount = posonlyargcount + posorkwargcount, - .posonlyargcount = posonlyargcount, - .kwonlyargcount = kwonlyargcount, - - .stacksize = maxdepth, - - .exceptiontable = a->a_except_table, - }; - - if (_PyCode_Validate(&con) < 0) { - goto error; - } - - if (_PyCompile_ConstCacheMergeOne(const_cache, &localsplusnames) < 0) { - goto error; - } - con.localsplusnames = localsplusnames; - - co = _PyCode_New(&con); - if (co == NULL) { - goto error; - } - - error: - Py_XDECREF(names); - Py_XDECREF(consts); - Py_XDECREF(localsplusnames); - Py_XDECREF(localspluskinds); - return co; -} - static int * build_cellfixedoffsets(struct compiler_unit *u) { - int nlocals = (int)PyDict_GET_SIZE(u->u_varnames); - int ncellvars = (int)PyDict_GET_SIZE(u->u_cellvars); - int nfreevars = (int)PyDict_GET_SIZE(u->u_freevars); + int nlocals = (int)PyDict_GET_SIZE(u->u_metadata.u_varnames); + int ncellvars = (int)PyDict_GET_SIZE(u->u_metadata.u_cellvars); + int nfreevars = (int)PyDict_GET_SIZE(u->u_metadata.u_freevars); int noffsets = ncellvars + nfreevars; int *fixed = PyMem_New(int, noffsets); @@ -7215,8 +6627,8 @@ build_cellfixedoffsets(struct compiler_unit *u) PyObject *varname, *cellindex; Py_ssize_t pos = 0; - while (PyDict_Next(u->u_cellvars, &pos, &varname, &cellindex)) { - PyObject *varindex = PyDict_GetItem(u->u_varnames, varname); + while (PyDict_Next(u->u_metadata.u_cellvars, &pos, &varname, &cellindex)) { + PyObject *varindex = PyDict_GetItem(u->u_metadata.u_varnames, varname); if (varindex != NULL) { assert(PyLong_AS_LONG(cellindex) < INT_MAX); assert(PyLong_AS_LONG(varindex) < INT_MAX); @@ -7233,14 +6645,14 @@ static int insert_prefix_instructions(struct compiler_unit *u, basicblock *entryblock, int *fixed, int nfreevars, int code_flags) { - assert(u->u_firstlineno > 0); + assert(u->u_metadata.u_firstlineno > 0); /* Add the generator prefix instructions. */ if (code_flags & (CO_GENERATOR | CO_COROUTINE | CO_ASYNC_GENERATOR)) { cfg_instr make_gen = { .i_opcode = RETURN_GENERATOR, .i_oparg = 0, - .i_loc = LOCATION(u->u_firstlineno, u->u_firstlineno, -1, -1), + .i_loc = LOCATION(u->u_metadata.u_firstlineno, u->u_metadata.u_firstlineno, -1, -1), .i_target = NULL, }; RETURN_IF_ERROR(_PyBasicblock_InsertInstruction(entryblock, 0, &make_gen)); @@ -7254,12 +6666,12 @@ insert_prefix_instructions(struct compiler_unit *u, basicblock *entryblock, } /* Set up cells for any variable that escapes, to be put in a closure. */ - const int ncellvars = (int)PyDict_GET_SIZE(u->u_cellvars); + const int ncellvars = (int)PyDict_GET_SIZE(u->u_metadata.u_cellvars); if (ncellvars) { - // u->u_cellvars has the cells out of order so we sort them + // u->u_metadata.u_cellvars has the cells out of order so we sort them // before adding the MAKE_CELL instructions. Note that we // adjust for arg cells, which come first. - const int nvars = ncellvars + (int)PyDict_GET_SIZE(u->u_varnames); + const int nvars = ncellvars + (int)PyDict_GET_SIZE(u->u_metadata.u_varnames); int *sorted = PyMem_RawCalloc(nvars, sizeof(int)); if (sorted == NULL) { PyErr_NoMemory(); @@ -7302,9 +6714,9 @@ insert_prefix_instructions(struct compiler_unit *u, basicblock *entryblock, static int fix_cell_offsets(struct compiler_unit *u, basicblock *entryblock, int *fixedmap) { - int nlocals = (int)PyDict_GET_SIZE(u->u_varnames); - int ncellvars = (int)PyDict_GET_SIZE(u->u_cellvars); - int nfreevars = (int)PyDict_GET_SIZE(u->u_freevars); + int nlocals = (int)PyDict_GET_SIZE(u->u_metadata.u_varnames); + int ncellvars = (int)PyDict_GET_SIZE(u->u_metadata.u_cellvars); + int nfreevars = (int)PyDict_GET_SIZE(u->u_metadata.u_freevars); int noffsets = ncellvars + nfreevars; // First deal with duplicates (arg cells). @@ -7348,12 +6760,12 @@ fix_cell_offsets(struct compiler_unit *u, basicblock *entryblock, int *fixedmap) static int prepare_localsplus(struct compiler_unit* u, cfg_builder *g, int code_flags) { - assert(PyDict_GET_SIZE(u->u_varnames) < INT_MAX); - assert(PyDict_GET_SIZE(u->u_cellvars) < INT_MAX); - assert(PyDict_GET_SIZE(u->u_freevars) < INT_MAX); - int nlocals = (int)PyDict_GET_SIZE(u->u_varnames); - int ncellvars = (int)PyDict_GET_SIZE(u->u_cellvars); - int nfreevars = (int)PyDict_GET_SIZE(u->u_freevars); + assert(PyDict_GET_SIZE(u->u_metadata.u_varnames) < INT_MAX); + assert(PyDict_GET_SIZE(u->u_metadata.u_cellvars) < INT_MAX); + assert(PyDict_GET_SIZE(u->u_metadata.u_freevars) < INT_MAX); + int nlocals = (int)PyDict_GET_SIZE(u->u_metadata.u_varnames); + int ncellvars = (int)PyDict_GET_SIZE(u->u_metadata.u_cellvars); + int nfreevars = (int)PyDict_GET_SIZE(u->u_metadata.u_freevars); assert(INT_MAX - nlocals - ncellvars > 0); assert(INT_MAX - nlocals - ncellvars - nfreevars > 0); int nlocalsplus = nlocals + ncellvars + nfreevars; @@ -7393,13 +6805,17 @@ add_return_at_end(struct compiler *c, int addNone) return SUCCESS; } +static int cfg_to_instr_sequence(cfg_builder *g, instr_sequence *seq); static PyCodeObject * -assemble_code_unit(struct compiler_unit *u, PyObject *const_cache, +optimize_and_assemble_code_unit(struct compiler_unit *u, PyObject *const_cache, int code_flags, PyObject *filename) { + instr_sequence optimized_instrs; + memset(&optimized_instrs, 0, sizeof(instr_sequence)); + PyCodeObject *co = NULL; - PyObject *consts = consts_dict_keys_inorder(u->u_consts); + PyObject *consts = consts_dict_keys_inorder(u->u_metadata.u_consts); if (consts == NULL) { goto error; } @@ -7408,25 +6824,18 @@ assemble_code_unit(struct compiler_unit *u, PyObject *const_cache, goto error; } int nparams = (int)PyList_GET_SIZE(u->u_ste->ste_varnames); - int nlocals = (int)PyDict_GET_SIZE(u->u_varnames); - if (_PyCfg_OptimizeCodeUnit(&g, consts, const_cache, code_flags, nlocals, nparams) < 0) { + int nlocals = (int)PyDict_GET_SIZE(u->u_metadata.u_varnames); + assert(u->u_metadata.u_firstlineno); + if (_PyCfg_OptimizeCodeUnit(&g, consts, const_cache, code_flags, nlocals, + nparams, u->u_metadata.u_firstlineno) < 0) { goto error; } - /** Assembly **/ - /* Set firstlineno if it wasn't explicitly set. */ - if (!u->u_firstlineno) { - if (g.g_entryblock->b_instr && g.g_entryblock->b_instr->i_loc.lineno) { - u->u_firstlineno = g.g_entryblock->b_instr->i_loc.lineno; - } - else { - u->u_firstlineno = 1; - } - } - if (_PyCfg_ResolveLineNumbers(&g, u->u_firstlineno) < 0) { + if (cfg_to_instr_sequence(&g, &optimized_instrs) < 0) { goto error; } + /** Assembly **/ int nlocalsplus = prepare_localsplus(u, &g, code_flags); if (nlocalsplus < 0) { goto error; @@ -7436,7 +6845,6 @@ assemble_code_unit(struct compiler_unit *u, PyObject *const_cache, if (maxdepth < 0) { goto error; } - /* TO DO -- For 3.12, make sure that `maxdepth <= MAX_ALLOWED_STACK_USE` */ _PyCfg_ConvertExceptionHandlersToNops(g.g_entryblock); @@ -7445,25 +6853,26 @@ assemble_code_unit(struct compiler_unit *u, PyObject *const_cache, if (_PyCfg_ResolveJumps(&g) < 0) { goto error; } + if (cfg_to_instr_sequence(&g, &optimized_instrs) < 0) { + goto error; + } + /* Can't modify the bytecode after computing jump offsets. */ - struct assembler a; - int res = assemble_emit(&a, g.g_entryblock, u->u_firstlineno, const_cache); - if (res == SUCCESS) { - co = makecode(u, &a, const_cache, consts, maxdepth, nlocalsplus, - code_flags, filename); - } - assemble_free(&a); + co = _PyAssemble_MakeCodeObject(&u->u_metadata, const_cache, consts, + maxdepth, g.g_entryblock, nlocalsplus, + code_flags, filename); error: Py_XDECREF(consts); + instr_sequence_fini(&optimized_instrs); _PyCfgBuilder_Fini(&g); return co; } static PyCodeObject * -assemble(struct compiler *c, int addNone) +optimize_and_assemble(struct compiler *c, int addNone) { struct compiler_unit *u = c->u; PyObject *const_cache = c->c_const_cache; @@ -7478,9 +6887,32 @@ assemble(struct compiler *c, int addNone) return NULL; } - return assemble_code_unit(u, const_cache, code_flags, filename); + return optimize_and_assemble_code_unit(u, const_cache, code_flags, filename); } +static int +cfg_to_instr_sequence(cfg_builder *g, instr_sequence *seq) +{ + int lbl = 0; + for (basicblock *b = g->g_entryblock; b != NULL; b = b->b_next) { + b->b_label = (jump_target_label){lbl}; + lbl += b->b_iused; + } + for (basicblock *b = g->g_entryblock; b != NULL; b = b->b_next) { + RETURN_IF_ERROR(instr_sequence_use_label(seq, b->b_label.id)); + for (int i = 0; i < b->b_iused; i++) { + cfg_instr *instr = &b->b_instr[i]; + int arg = HAS_TARGET(instr->i_opcode) ? + instr->i_target->b_label.id : + instr->i_oparg; + RETURN_IF_ERROR( + instr_sequence_addop(seq, instr->i_opcode, arg, instr->i_loc)); + } + } + return SUCCESS; +} + + /* Access to compiler optimizations for unit tests. * * _PyCompile_CodeGen takes and AST, applies code-gen and @@ -7496,7 +6928,7 @@ assemble(struct compiler *c, int addNone) */ static int -instructions_to_cfg(PyObject *instructions, cfg_builder *g) +instructions_to_instr_sequence(PyObject *instructions, instr_sequence *seq) { assert(PyList_Check(instructions)); @@ -7530,8 +6962,7 @@ instructions_to_cfg(PyObject *instructions, cfg_builder *g) for (int i = 0; i < num_insts; i++) { if (is_target[i]) { - jump_target_label lbl = {i}; - RETURN_IF_ERROR(_PyCfgBuilder_UseLabel(g, lbl)); + RETURN_IF_ERROR(instr_sequence_use_label(seq, i)); } PyObject *item = PyList_GET_ITEM(instructions, i); if (!PyTuple_Check(item) || PyTuple_GET_SIZE(item) != 6) { @@ -7569,11 +7000,10 @@ instructions_to_cfg(PyObject *instructions, cfg_builder *g) if (PyErr_Occurred()) { goto error; } - RETURN_IF_ERROR(_PyCfgBuilder_Addop(g, opcode, oparg, loc)); + RETURN_IF_ERROR(instr_sequence_addop(seq, opcode, oparg, loc)); } - cfg_instr *last = _PyCfg_BasicblockLastInstr(g->g_curblock); - if (last && !IS_TERMINATOR_OPCODE(last->i_opcode)) { - RETURN_IF_ERROR(_PyCfgBuilder_Addop(g, RETURN_VALUE, 0, NO_LOCATION)); + if (seq->s_used && !IS_TERMINATOR_OPCODE(seq->s_instrs[seq->s_used-1].i_opcode)) { + RETURN_IF_ERROR(instr_sequence_addop(seq, RETURN_VALUE, 0, NO_LOCATION)); } PyMem_Free(is_target); return SUCCESS; @@ -7582,8 +7012,23 @@ instructions_to_cfg(PyObject *instructions, cfg_builder *g) return ERROR; } +static int +instructions_to_cfg(PyObject *instructions, cfg_builder *g) +{ + instr_sequence seq; + memset(&seq, 0, sizeof(instr_sequence)); + + RETURN_IF_ERROR( + instructions_to_instr_sequence(instructions, &seq)); + + RETURN_IF_ERROR(instr_sequence_to_cfg(&seq, g)); + instr_sequence_fini(&seq); + return SUCCESS; +} + static PyObject * -instr_sequence_to_instructions(instr_sequence *seq) { +instr_sequence_to_instructions(instr_sequence *seq) +{ PyObject *instructions = PyList_New(0); if (instructions == NULL) { return NULL; @@ -7706,15 +7151,12 @@ _PyCompile_OptimizeCfg(PyObject *instructions, PyObject *consts) } cfg_builder g; - memset(&g, 0, sizeof(cfg_builder)); - if (_PyCfgBuilder_Init(&g) < 0) { - goto error; - } if (instructions_to_cfg(instructions, &g) < 0) { goto error; } - int code_flags = 0, nlocals = 0, nparams = 0; - if (_PyCfg_OptimizeCodeUnit(&g, consts, const_cache, code_flags, nlocals, nparams) < 0) { + int code_flags = 0, nlocals = 0, nparams = 0, firstlineno = 1; + if (_PyCfg_OptimizeCodeUnit(&g, consts, const_cache, code_flags, nlocals, + nparams, firstlineno) < 0) { goto error; } res = cfg_to_instructions(&g); diff --git a/Python/errors.c b/Python/errors.c index ab14770c6e810c..0ff6a0d5985f0f 100644 --- a/Python/errors.c +++ b/Python/errors.c @@ -1200,6 +1200,33 @@ PyErr_Format(PyObject *exception, const char *format, ...) } +/* Adds a note to the current exception (if any) */ +void +_PyErr_FormatNote(const char *format, ...) +{ + PyObject *exc = PyErr_GetRaisedException(); + if (exc == NULL) { + return; + } + va_list vargs; + va_start(vargs, format); + PyObject *note = PyUnicode_FromFormatV(format, vargs); + va_end(vargs); + if (note == NULL) { + goto error; + } + int res = _PyException_AddNote(exc, note); + Py_DECREF(note); + if (res < 0) { + goto error; + } + PyErr_SetRaisedException(exc); + return; +error: + _PyErr_ChainExceptions1(exc); +} + + PyObject * PyErr_NewException(const char *name, PyObject *base, PyObject *dict) { diff --git a/Python/flowgraph.c b/Python/flowgraph.c index cecddbd39c94b3..67cc5c5e88be10 100644 --- a/Python/flowgraph.c +++ b/Python/flowgraph.c @@ -42,12 +42,6 @@ is_block_push(cfg_instr *i) return IS_BLOCK_PUSH_OPCODE(i->i_opcode); } -static inline int -is_relative_jump(cfg_instr *i) -{ - return IS_RELATIVE_JUMP(i->i_opcode); -} - static inline int is_jump(cfg_instr *i) { @@ -199,8 +193,7 @@ blocksize(basicblock *b) static void dump_instr(cfg_instr *i) { - const char *jrel = (is_relative_jump(i)) ? "jrel " : ""; - const char *jabs = (is_jump(i) && !is_relative_jump(i))? "jabs " : ""; + const char *jump = is_jump(i) ? "jump " : ""; char arg[128]; @@ -211,8 +204,8 @@ dump_instr(cfg_instr *i) if (HAS_TARGET(i->i_opcode)) { sprintf(arg, "target: %p [%d] ", i->i_target, i->i_oparg); } - fprintf(stderr, "line: %d, opcode: %d %s%s%s\n", - i->i_loc.lineno, i->i_opcode, arg, jabs, jrel); + fprintf(stderr, "line: %d, opcode: %d %s%s\n", + i->i_loc.lineno, i->i_opcode, arg, jump); } static inline int @@ -500,25 +493,20 @@ resolve_jump_offsets(basicblock *entryblock) for (int i = 0; i < b->b_iused; i++) { cfg_instr *instr = &b->b_instr[i]; int isize = _PyCfg_InstrSize(instr); - /* Relative jumps are computed relative to - the instruction pointer after fetching - the jump instruction. - */ + /* jump offsets are computed relative to + * the instruction pointer after fetching + * the jump instruction. + */ bsize += isize; if (is_jump(instr)) { instr->i_oparg = instr->i_target->b_offset; - if (is_relative_jump(instr)) { - if (instr->i_oparg < bsize) { - assert(IS_BACKWARDS_JUMP_OPCODE(instr->i_opcode)); - instr->i_oparg = bsize - instr->i_oparg; - } - else { - assert(!IS_BACKWARDS_JUMP_OPCODE(instr->i_opcode)); - instr->i_oparg -= bsize; - } + if (instr->i_oparg < bsize) { + assert(IS_BACKWARDS_JUMP_OPCODE(instr->i_opcode)); + instr->i_oparg = bsize - instr->i_oparg; } else { assert(!IS_BACKWARDS_JUMP_OPCODE(instr->i_opcode)); + instr->i_oparg -= bsize; } if (_PyCfg_InstrSize(instr) != isize) { extended_arg_recompile = 1; @@ -1978,28 +1966,6 @@ push_cold_blocks_to_end(cfg_builder *g, int code_flags) { return SUCCESS; } -int -_PyCfg_OptimizeCodeUnit(cfg_builder *g, PyObject *consts, PyObject *const_cache, - int code_flags, int nlocals, int nparams) -{ - assert(cfg_builder_check(g)); - /** Preprocessing **/ - /* Map labels to targets and mark exception handlers */ - RETURN_IF_ERROR(translate_jump_labels_to_targets(g->g_entryblock)); - RETURN_IF_ERROR(mark_except_handlers(g->g_entryblock)); - RETURN_IF_ERROR(label_exception_targets(g->g_entryblock)); - - /** Optimization **/ - RETURN_IF_ERROR(optimize_cfg(g, consts, const_cache)); - RETURN_IF_ERROR(remove_unused_consts(g->g_entryblock, consts)); - RETURN_IF_ERROR( - add_checks_for_loads_of_uninitialized_variables( - g->g_entryblock, nlocals, nparams)); - - RETURN_IF_ERROR(push_cold_blocks_to_end(g, code_flags)); - return SUCCESS; -} - void _PyCfg_ConvertExceptionHandlersToNops(basicblock *entryblock) { @@ -2149,8 +2115,8 @@ guarantee_lineno_for_exits(basicblock *entryblock, int firstlineno) { } } -int -_PyCfg_ResolveLineNumbers(cfg_builder *g, int firstlineno) +static int +resolve_line_numbers(cfg_builder *g, int firstlineno) { RETURN_IF_ERROR(duplicate_exits_without_lineno(g)); propagate_line_numbers(g->g_entryblock); @@ -2158,3 +2124,25 @@ _PyCfg_ResolveLineNumbers(cfg_builder *g, int firstlineno) return SUCCESS; } +int +_PyCfg_OptimizeCodeUnit(cfg_builder *g, PyObject *consts, PyObject *const_cache, + int code_flags, int nlocals, int nparams, int firstlineno) +{ + assert(cfg_builder_check(g)); + /** Preprocessing **/ + /* Map labels to targets and mark exception handlers */ + RETURN_IF_ERROR(translate_jump_labels_to_targets(g->g_entryblock)); + RETURN_IF_ERROR(mark_except_handlers(g->g_entryblock)); + RETURN_IF_ERROR(label_exception_targets(g->g_entryblock)); + + /** Optimization **/ + RETURN_IF_ERROR(optimize_cfg(g, consts, const_cache)); + RETURN_IF_ERROR(remove_unused_consts(g->g_entryblock, consts)); + RETURN_IF_ERROR( + add_checks_for_loads_of_uninitialized_variables( + g->g_entryblock, nlocals, nparams)); + + RETURN_IF_ERROR(push_cold_blocks_to_end(g, code_flags)); + RETURN_IF_ERROR(resolve_line_numbers(g, firstlineno)); + return SUCCESS; +} diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 420d136bb98158..4d52e95dc04a27 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -8,24 +8,61 @@ } TARGET(RESUME) { - #line 135 "Python/bytecodes.c" + #line 136 "Python/bytecodes.c" assert(tstate->cframe == &cframe); assert(frame == cframe.current_frame); - if (_Py_atomic_load_relaxed_int32(eval_breaker) && oparg < 2) { + /* Possibly combine this with eval breaker */ + if (frame->f_code->_co_instrumentation_version != tstate->interp->monitoring_version) { + int err = _Py_Instrument(frame->f_code, tstate->interp); + if (err) goto error; + next_instr--; + } + else if (_Py_atomic_load_relaxed_int32(eval_breaker) && oparg < 2) { goto handle_eval_breaker; } - #line 18 "Python/generated_cases.c.h" + #line 24 "Python/generated_cases.c.h" + DISPATCH(); + } + + TARGET(INSTRUMENTED_RESUME) { + #line 150 "Python/bytecodes.c" + /* Possible performance enhancement: + * We need to check the eval breaker anyway, can we + * combine the instrument verison check and the eval breaker test? + */ + if (frame->f_code->_co_instrumentation_version != tstate->interp->monitoring_version) { + if (_Py_Instrument(frame->f_code, tstate->interp)) { + goto error; + } + next_instr--; + } + else { + _PyFrame_SetStackPointer(frame, stack_pointer); + int err = _Py_call_instrumentation( + tstate, oparg > 0, frame, next_instr-1); + stack_pointer = _PyFrame_GetStackPointer(frame); + if (err) goto error; + if (frame->prev_instr != next_instr-1) { + /* Instrumentation has jumped */ + next_instr = frame->prev_instr; + DISPATCH(); + } + if (_Py_atomic_load_relaxed_int32(eval_breaker) && oparg < 2) { + goto handle_eval_breaker; + } + } + #line 55 "Python/generated_cases.c.h" DISPATCH(); } TARGET(LOAD_CLOSURE) { PyObject *value; - #line 143 "Python/bytecodes.c" + #line 178 "Python/bytecodes.c" /* We keep LOAD_CLOSURE so that the bytecode stays more readable. */ value = GETLOCAL(oparg); if (value == NULL) goto unbound_local_error; Py_INCREF(value); - #line 29 "Python/generated_cases.c.h" + #line 66 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -33,11 +70,11 @@ TARGET(LOAD_FAST_CHECK) { PyObject *value; - #line 150 "Python/bytecodes.c" + #line 185 "Python/bytecodes.c" value = GETLOCAL(oparg); if (value == NULL) goto unbound_local_error; Py_INCREF(value); - #line 41 "Python/generated_cases.c.h" + #line 78 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -45,11 +82,11 @@ TARGET(LOAD_FAST) { PyObject *value; - #line 156 "Python/bytecodes.c" + #line 191 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 53 "Python/generated_cases.c.h" + #line 90 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -58,10 +95,10 @@ TARGET(LOAD_CONST) { PREDICTED(LOAD_CONST); PyObject *value; - #line 162 "Python/bytecodes.c" + #line 197 "Python/bytecodes.c" value = GETITEM(frame->f_code->co_consts, oparg); Py_INCREF(value); - #line 65 "Python/generated_cases.c.h" + #line 102 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -69,9 +106,9 @@ TARGET(STORE_FAST) { PyObject *value = stack_pointer[-1]; - #line 167 "Python/bytecodes.c" + #line 202 "Python/bytecodes.c" SETLOCAL(oparg, value); - #line 75 "Python/generated_cases.c.h" + #line 112 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -81,21 +118,21 @@ PyObject *_tmp_2; { PyObject *value; - #line 156 "Python/bytecodes.c" + #line 191 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 89 "Python/generated_cases.c.h" + #line 126 "Python/generated_cases.c.h" _tmp_2 = value; } oparg = (next_instr++)->op.arg; { PyObject *value; - #line 156 "Python/bytecodes.c" + #line 191 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 99 "Python/generated_cases.c.h" + #line 136 "Python/generated_cases.c.h" _tmp_1 = value; } STACK_GROW(2); @@ -109,20 +146,20 @@ PyObject *_tmp_2; { PyObject *value; - #line 156 "Python/bytecodes.c" + #line 191 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 117 "Python/generated_cases.c.h" + #line 154 "Python/generated_cases.c.h" _tmp_2 = value; } oparg = (next_instr++)->op.arg; { PyObject *value; - #line 162 "Python/bytecodes.c" + #line 197 "Python/bytecodes.c" value = GETITEM(frame->f_code->co_consts, oparg); Py_INCREF(value); - #line 126 "Python/generated_cases.c.h" + #line 163 "Python/generated_cases.c.h" _tmp_1 = value; } STACK_GROW(2); @@ -135,18 +172,18 @@ PyObject *_tmp_1 = stack_pointer[-1]; { PyObject *value = _tmp_1; - #line 167 "Python/bytecodes.c" + #line 202 "Python/bytecodes.c" SETLOCAL(oparg, value); - #line 141 "Python/generated_cases.c.h" + #line 178 "Python/generated_cases.c.h" } oparg = (next_instr++)->op.arg; { PyObject *value; - #line 156 "Python/bytecodes.c" + #line 191 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 150 "Python/generated_cases.c.h" + #line 187 "Python/generated_cases.c.h" _tmp_1 = value; } stack_pointer[-1] = _tmp_1; @@ -158,16 +195,16 @@ PyObject *_tmp_2 = stack_pointer[-2]; { PyObject *value = _tmp_1; - #line 167 "Python/bytecodes.c" + #line 202 "Python/bytecodes.c" SETLOCAL(oparg, value); - #line 164 "Python/generated_cases.c.h" + #line 201 "Python/generated_cases.c.h" } oparg = (next_instr++)->op.arg; { PyObject *value = _tmp_2; - #line 167 "Python/bytecodes.c" + #line 202 "Python/bytecodes.c" SETLOCAL(oparg, value); - #line 171 "Python/generated_cases.c.h" + #line 208 "Python/generated_cases.c.h" } STACK_SHRINK(2); DISPATCH(); @@ -178,20 +215,20 @@ PyObject *_tmp_2; { PyObject *value; - #line 162 "Python/bytecodes.c" + #line 197 "Python/bytecodes.c" value = GETITEM(frame->f_code->co_consts, oparg); Py_INCREF(value); - #line 185 "Python/generated_cases.c.h" + #line 222 "Python/generated_cases.c.h" _tmp_2 = value; } oparg = (next_instr++)->op.arg; { PyObject *value; - #line 156 "Python/bytecodes.c" + #line 191 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 195 "Python/generated_cases.c.h" + #line 232 "Python/generated_cases.c.h" _tmp_1 = value; } STACK_GROW(2); @@ -202,8 +239,8 @@ TARGET(POP_TOP) { PyObject *value = stack_pointer[-1]; - #line 177 "Python/bytecodes.c" - #line 207 "Python/generated_cases.c.h" + #line 212 "Python/bytecodes.c" + #line 244 "Python/generated_cases.c.h" Py_DECREF(value); STACK_SHRINK(1); DISPATCH(); @@ -211,9 +248,9 @@ TARGET(PUSH_NULL) { PyObject *res; - #line 181 "Python/bytecodes.c" + #line 216 "Python/bytecodes.c" res = NULL; - #line 217 "Python/generated_cases.c.h" + #line 254 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -224,30 +261,79 @@ PyObject *_tmp_2 = stack_pointer[-2]; { PyObject *value = _tmp_1; - #line 177 "Python/bytecodes.c" - #line 229 "Python/generated_cases.c.h" + #line 212 "Python/bytecodes.c" + #line 266 "Python/generated_cases.c.h" Py_DECREF(value); } { PyObject *value = _tmp_2; - #line 177 "Python/bytecodes.c" - #line 235 "Python/generated_cases.c.h" + #line 212 "Python/bytecodes.c" + #line 272 "Python/generated_cases.c.h" Py_DECREF(value); } STACK_SHRINK(2); DISPATCH(); } + TARGET(INSTRUMENTED_END_FOR) { + PyObject *value = stack_pointer[-1]; + PyObject *receiver = stack_pointer[-2]; + #line 222 "Python/bytecodes.c" + /* Need to create a fake StopIteration error here, + * to conform to PEP 380 */ + if (PyGen_Check(receiver)) { + PyErr_SetObject(PyExc_StopIteration, value); + if (monitor_stop_iteration(tstate, frame, next_instr-1)) { + goto error; + } + PyErr_SetRaisedException(NULL); + } + #line 292 "Python/generated_cases.c.h" + Py_DECREF(receiver); + Py_DECREF(value); + STACK_SHRINK(2); + DISPATCH(); + } + + TARGET(END_SEND) { + PyObject *value = stack_pointer[-1]; + PyObject *receiver = stack_pointer[-2]; + #line 235 "Python/bytecodes.c" + Py_DECREF(receiver); + #line 304 "Python/generated_cases.c.h" + STACK_SHRINK(1); + stack_pointer[-1] = value; + DISPATCH(); + } + + TARGET(INSTRUMENTED_END_SEND) { + PyObject *value = stack_pointer[-1]; + PyObject *receiver = stack_pointer[-2]; + #line 239 "Python/bytecodes.c" + if (PyGen_Check(receiver) || PyCoro_CheckExact(receiver)) { + PyErr_SetObject(PyExc_StopIteration, value); + if (monitor_stop_iteration(tstate, frame, next_instr-1)) { + goto error; + } + PyErr_SetRaisedException(NULL); + } + Py_DECREF(receiver); + #line 322 "Python/generated_cases.c.h" + STACK_SHRINK(1); + stack_pointer[-1] = value; + DISPATCH(); + } + TARGET(UNARY_NEGATIVE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 187 "Python/bytecodes.c" + #line 250 "Python/bytecodes.c" res = PyNumber_Negative(value); - #line 247 "Python/generated_cases.c.h" + #line 333 "Python/generated_cases.c.h" Py_DECREF(value); - #line 189 "Python/bytecodes.c" + #line 252 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 251 "Python/generated_cases.c.h" + #line 337 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -255,11 +341,11 @@ TARGET(UNARY_NOT) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 193 "Python/bytecodes.c" + #line 256 "Python/bytecodes.c" int err = PyObject_IsTrue(value); - #line 261 "Python/generated_cases.c.h" + #line 347 "Python/generated_cases.c.h" Py_DECREF(value); - #line 195 "Python/bytecodes.c" + #line 258 "Python/bytecodes.c" if (err < 0) goto pop_1_error; if (err == 0) { res = Py_True; @@ -268,7 +354,7 @@ res = Py_False; } Py_INCREF(res); - #line 272 "Python/generated_cases.c.h" + #line 358 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -276,13 +362,13 @@ TARGET(UNARY_INVERT) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 206 "Python/bytecodes.c" + #line 269 "Python/bytecodes.c" res = PyNumber_Invert(value); - #line 282 "Python/generated_cases.c.h" + #line 368 "Python/generated_cases.c.h" Py_DECREF(value); - #line 208 "Python/bytecodes.c" + #line 271 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 286 "Python/generated_cases.c.h" + #line 372 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -291,8 +377,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *prod; - #line 225 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 288 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -300,7 +385,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (prod == NULL) goto pop_2_error; - #line 304 "Python/generated_cases.c.h" + #line 389 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = prod; next_instr += 1; @@ -311,15 +396,14 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *prod; - #line 236 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 298 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); STAT_INC(BINARY_OP, hit); double dprod = ((PyFloatObject *)left)->ob_fval * ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dprod, prod); - #line 323 "Python/generated_cases.c.h" + #line 407 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = prod; next_instr += 1; @@ -330,8 +414,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *sub; - #line 246 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 307 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -339,7 +422,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (sub == NULL) goto pop_2_error; - #line 343 "Python/generated_cases.c.h" + #line 426 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = sub; next_instr += 1; @@ -350,14 +433,13 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *sub; - #line 257 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 317 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); STAT_INC(BINARY_OP, hit); double dsub = ((PyFloatObject *)left)->ob_fval - ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dsub, sub); - #line 361 "Python/generated_cases.c.h" + #line 443 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = sub; next_instr += 1; @@ -368,8 +450,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 266 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 325 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(Py_TYPE(right) != Py_TYPE(left), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -377,7 +458,7 @@ _Py_DECREF_SPECIALIZED(left, _PyUnicode_ExactDealloc); _Py_DECREF_SPECIALIZED(right, _PyUnicode_ExactDealloc); if (res == NULL) goto pop_2_error; - #line 381 "Python/generated_cases.c.h" + #line 462 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -387,8 +468,7 @@ TARGET(BINARY_OP_INPLACE_ADD_UNICODE) { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; - #line 283 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 341 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(Py_TYPE(right) != Py_TYPE(left), BINARY_OP); _Py_CODEUNIT true_next = next_instr[INLINE_CACHE_ENTRIES_BINARY_OP]; @@ -415,7 +495,7 @@ if (*target_local == NULL) goto pop_2_error; // The STORE_FAST is already done. JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP + 1); - #line 419 "Python/generated_cases.c.h" + #line 499 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } @@ -424,15 +504,14 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *sum; - #line 313 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 370 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(Py_TYPE(right) != Py_TYPE(left), BINARY_OP); STAT_INC(BINARY_OP, hit); double dsum = ((PyFloatObject *)left)->ob_fval + ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dsum, sum); - #line 436 "Python/generated_cases.c.h" + #line 515 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = sum; next_instr += 1; @@ -443,8 +522,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *sum; - #line 323 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 379 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(Py_TYPE(right) != Py_TYPE(left), BINARY_OP); STAT_INC(BINARY_OP, hit); @@ -452,7 +530,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (sum == NULL) goto pop_2_error; - #line 456 "Python/generated_cases.c.h" + #line 534 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = sum; next_instr += 1; @@ -465,11 +543,10 @@ PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; PyObject *res; - #line 342 "Python/bytecodes.c" + #line 397 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinarySubscrCache *cache = (_PyBinarySubscrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_BinarySubscr(container, sub, next_instr); DISPATCH_SAME_OPARG(); @@ -478,12 +555,12 @@ DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ res = PyObject_GetItem(container, sub); - #line 482 "Python/generated_cases.c.h" + #line 559 "Python/generated_cases.c.h" Py_DECREF(container); Py_DECREF(sub); - #line 355 "Python/bytecodes.c" + #line 409 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 487 "Python/generated_cases.c.h" + #line 564 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -495,7 +572,7 @@ PyObject *start = stack_pointer[-2]; PyObject *container = stack_pointer[-3]; PyObject *res; - #line 359 "Python/bytecodes.c" + #line 413 "Python/bytecodes.c" PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop); // Can't use ERROR_IF() here, because we haven't // DECREF'ed container yet, and we still own slice. @@ -508,7 +585,7 @@ } Py_DECREF(container); if (res == NULL) goto pop_3_error; - #line 512 "Python/generated_cases.c.h" + #line 589 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = res; DISPATCH(); @@ -519,7 +596,7 @@ PyObject *start = stack_pointer[-2]; PyObject *container = stack_pointer[-3]; PyObject *v = stack_pointer[-4]; - #line 374 "Python/bytecodes.c" + #line 428 "Python/bytecodes.c" PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop); int err; if (slice == NULL) { @@ -532,7 +609,7 @@ Py_DECREF(v); Py_DECREF(container); if (err) goto pop_4_error; - #line 536 "Python/generated_cases.c.h" + #line 613 "Python/generated_cases.c.h" STACK_SHRINK(4); DISPATCH(); } @@ -541,8 +618,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; PyObject *res; - #line 389 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 443 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), BINARY_SUBSCR); @@ -556,7 +632,7 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 560 "Python/generated_cases.c.h" + #line 636 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -567,8 +643,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *tuple = stack_pointer[-2]; PyObject *res; - #line 406 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 459 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyTuple_CheckExact(tuple), BINARY_SUBSCR); @@ -582,7 +657,7 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(tuple); - #line 586 "Python/generated_cases.c.h" + #line 661 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -593,8 +668,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *dict = stack_pointer[-2]; PyObject *res; - #line 423 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 475 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(dict), BINARY_SUBSCR); STAT_INC(BINARY_SUBSCR, hit); res = PyDict_GetItemWithError(dict, sub); @@ -602,14 +676,14 @@ if (!_PyErr_Occurred(tstate)) { _PyErr_SetKeyError(sub); } - #line 606 "Python/generated_cases.c.h" + #line 680 "Python/generated_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); - #line 432 "Python/bytecodes.c" + #line 483 "Python/bytecodes.c" if (true) goto pop_2_error; } Py_INCREF(res); // Do this before DECREF'ing dict, sub - #line 613 "Python/generated_cases.c.h" + #line 687 "Python/generated_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); STACK_SHRINK(1); @@ -621,7 +695,7 @@ TARGET(BINARY_SUBSCR_GETITEM) { PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; - #line 439 "Python/bytecodes.c" + #line 490 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(container); DEOPT_IF(!PyType_HasFeature(tp, Py_TPFLAGS_HEAPTYPE), BINARY_SUBSCR); PyHeapTypeObject *ht = (PyHeapTypeObject *)tp; @@ -642,15 +716,15 @@ new_frame->localsplus[1] = sub; JUMPBY(INLINE_CACHE_ENTRIES_BINARY_SUBSCR); DISPATCH_INLINED(new_frame); - #line 646 "Python/generated_cases.c.h" + #line 720 "Python/generated_cases.c.h" } TARGET(LIST_APPEND) { PyObject *v = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 462 "Python/bytecodes.c" + #line 513 "Python/bytecodes.c" if (_PyList_AppendTakeRef((PyListObject *)list, v) < 0) goto pop_1_error; - #line 654 "Python/generated_cases.c.h" + #line 728 "Python/generated_cases.c.h" STACK_SHRINK(1); PREDICT(JUMP_BACKWARD); DISPATCH(); @@ -659,13 +733,13 @@ TARGET(SET_ADD) { PyObject *v = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 467 "Python/bytecodes.c" + #line 518 "Python/bytecodes.c" int err = PySet_Add(set, v); - #line 665 "Python/generated_cases.c.h" + #line 739 "Python/generated_cases.c.h" Py_DECREF(v); - #line 469 "Python/bytecodes.c" + #line 520 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 669 "Python/generated_cases.c.h" + #line 743 "Python/generated_cases.c.h" STACK_SHRINK(1); PREDICT(JUMP_BACKWARD); DISPATCH(); @@ -678,10 +752,9 @@ PyObject *container = stack_pointer[-2]; PyObject *v = stack_pointer[-3]; uint16_t counter = read_u16(&next_instr[0].cache); - #line 480 "Python/bytecodes.c" + #line 531 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_StoreSubscr(container, sub, next_instr); DISPATCH_SAME_OPARG(); @@ -694,13 +767,13 @@ #endif /* ENABLE_SPECIALIZATION */ /* container[sub] = v */ int err = PyObject_SetItem(container, sub, v); - #line 698 "Python/generated_cases.c.h" + #line 771 "Python/generated_cases.c.h" Py_DECREF(v); Py_DECREF(container); Py_DECREF(sub); - #line 496 "Python/bytecodes.c" + #line 546 "Python/bytecodes.c" if (err) goto pop_3_error; - #line 704 "Python/generated_cases.c.h" + #line 777 "Python/generated_cases.c.h" STACK_SHRINK(3); next_instr += 1; DISPATCH(); @@ -710,8 +783,7 @@ PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; PyObject *value = stack_pointer[-3]; - #line 500 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 550 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), STORE_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), STORE_SUBSCR); @@ -728,7 +800,7 @@ Py_DECREF(old_value); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 732 "Python/generated_cases.c.h" + #line 804 "Python/generated_cases.c.h" STACK_SHRINK(3); next_instr += 1; DISPATCH(); @@ -738,14 +810,13 @@ PyObject *sub = stack_pointer[-1]; PyObject *dict = stack_pointer[-2]; PyObject *value = stack_pointer[-3]; - #line 520 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 569 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(dict), STORE_SUBSCR); STAT_INC(STORE_SUBSCR, hit); int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub, value); Py_DECREF(dict); if (err) goto pop_3_error; - #line 749 "Python/generated_cases.c.h" + #line 820 "Python/generated_cases.c.h" STACK_SHRINK(3); next_instr += 1; DISPATCH(); @@ -754,15 +825,15 @@ TARGET(DELETE_SUBSCR) { PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; - #line 529 "Python/bytecodes.c" + #line 577 "Python/bytecodes.c" /* del container[sub] */ int err = PyObject_DelItem(container, sub); - #line 761 "Python/generated_cases.c.h" + #line 832 "Python/generated_cases.c.h" Py_DECREF(container); Py_DECREF(sub); - #line 532 "Python/bytecodes.c" + #line 580 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 766 "Python/generated_cases.c.h" + #line 837 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } @@ -770,14 +841,14 @@ TARGET(CALL_INTRINSIC_1) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 536 "Python/bytecodes.c" + #line 584 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_1); res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); - #line 777 "Python/generated_cases.c.h" + #line 848 "Python/generated_cases.c.h" Py_DECREF(value); - #line 539 "Python/bytecodes.c" + #line 587 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 781 "Python/generated_cases.c.h" + #line 852 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -786,15 +857,15 @@ PyObject *value1 = stack_pointer[-1]; PyObject *value2 = stack_pointer[-2]; PyObject *res; - #line 543 "Python/bytecodes.c" + #line 591 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_2); res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); - #line 793 "Python/generated_cases.c.h" + #line 864 "Python/generated_cases.c.h" Py_DECREF(value2); Py_DECREF(value1); - #line 546 "Python/bytecodes.c" + #line 594 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 798 "Python/generated_cases.c.h" + #line 869 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -802,7 +873,7 @@ TARGET(RAISE_VARARGS) { PyObject **args = (stack_pointer - oparg); - #line 550 "Python/bytecodes.c" + #line 598 "Python/bytecodes.c" PyObject *cause = NULL, *exc = NULL; switch (oparg) { case 2: @@ -820,34 +891,31 @@ break; } if (true) { STACK_SHRINK(oparg); goto error; } - #line 824 "Python/generated_cases.c.h" + #line 895 "Python/generated_cases.c.h" } TARGET(INTERPRETER_EXIT) { PyObject *retval = stack_pointer[-1]; - #line 570 "Python/bytecodes.c" + #line 618 "Python/bytecodes.c" assert(frame == &entry_frame); assert(_PyFrame_IsIncomplete(frame)); STACK_SHRINK(1); // Since we're not going to DISPATCH() assert(EMPTY()); /* Restore previous cframe and return. */ tstate->cframe = cframe.previous; - tstate->cframe->use_tracing = cframe.use_tracing; assert(tstate->cframe->current_frame == frame->previous); assert(!_PyErr_Occurred(tstate)); _Py_LeaveRecursiveCallTstate(tstate); return retval; - #line 841 "Python/generated_cases.c.h" + #line 911 "Python/generated_cases.c.h" } TARGET(RETURN_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 584 "Python/bytecodes.c" + #line 631 "Python/bytecodes.c" STACK_SHRINK(1); assert(EMPTY()); _PyFrame_SetStackPointer(frame, stack_pointer); - TRACE_FUNCTION_EXIT(); - DTRACE_FUNCTION_EXIT(); _Py_LeaveRecursiveCallPy(tstate); assert(frame != &entry_frame); // GH-99729: We need to unlink the frame *before* clearing it: @@ -856,17 +924,36 @@ _PyEvalFrameClearAndPop(tstate, dying); _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 860 "Python/generated_cases.c.h" + #line 928 "Python/generated_cases.c.h" + } + + TARGET(INSTRUMENTED_RETURN_VALUE) { + PyObject *retval = stack_pointer[-1]; + #line 645 "Python/bytecodes.c" + int err = _Py_call_instrumentation_arg( + tstate, PY_MONITORING_EVENT_PY_RETURN, + frame, next_instr-1, retval); + if (err) goto error; + STACK_SHRINK(1); + assert(EMPTY()); + _PyFrame_SetStackPointer(frame, stack_pointer); + _Py_LeaveRecursiveCallPy(tstate); + assert(frame != &entry_frame); + // GH-99729: We need to unlink the frame *before* clearing it: + _PyInterpreterFrame *dying = frame; + frame = cframe.current_frame = dying->previous; + _PyEvalFrameClearAndPop(tstate, dying); + _PyFrame_StackPush(frame, retval); + goto resume_frame; + #line 949 "Python/generated_cases.c.h" } TARGET(RETURN_CONST) { - #line 600 "Python/bytecodes.c" + #line 663 "Python/bytecodes.c" PyObject *retval = GETITEM(frame->f_code->co_consts, oparg); Py_INCREF(retval); assert(EMPTY()); _PyFrame_SetStackPointer(frame, stack_pointer); - TRACE_FUNCTION_EXIT(); - DTRACE_FUNCTION_EXIT(); _Py_LeaveRecursiveCallPy(tstate); assert(frame != &entry_frame); // GH-99729: We need to unlink the frame *before* clearing it: @@ -875,13 +962,34 @@ _PyEvalFrameClearAndPop(tstate, dying); _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 879 "Python/generated_cases.c.h" + #line 966 "Python/generated_cases.c.h" + } + + TARGET(INSTRUMENTED_RETURN_CONST) { + #line 678 "Python/bytecodes.c" + PyObject *retval = GETITEM(frame->f_code->co_consts, oparg); + int err = _Py_call_instrumentation_arg( + tstate, PY_MONITORING_EVENT_PY_RETURN, + frame, next_instr-1, retval); + if (err) goto error; + Py_INCREF(retval); + assert(EMPTY()); + _PyFrame_SetStackPointer(frame, stack_pointer); + _Py_LeaveRecursiveCallPy(tstate); + assert(frame != &entry_frame); + // GH-99729: We need to unlink the frame *before* clearing it: + _PyInterpreterFrame *dying = frame; + frame = cframe.current_frame = dying->previous; + _PyEvalFrameClearAndPop(tstate, dying); + _PyFrame_StackPush(frame, retval); + goto resume_frame; + #line 987 "Python/generated_cases.c.h" } TARGET(GET_AITER) { PyObject *obj = stack_pointer[-1]; PyObject *iter; - #line 617 "Python/bytecodes.c" + #line 697 "Python/bytecodes.c" unaryfunc getter = NULL; PyTypeObject *type = Py_TYPE(obj); @@ -894,16 +1002,16 @@ "'async for' requires an object with " "__aiter__ method, got %.100s", type->tp_name); - #line 898 "Python/generated_cases.c.h" + #line 1006 "Python/generated_cases.c.h" Py_DECREF(obj); - #line 630 "Python/bytecodes.c" + #line 710 "Python/bytecodes.c" if (true) goto pop_1_error; } iter = (*getter)(obj); - #line 905 "Python/generated_cases.c.h" + #line 1013 "Python/generated_cases.c.h" Py_DECREF(obj); - #line 635 "Python/bytecodes.c" + #line 715 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; if (Py_TYPE(iter)->tp_as_async == NULL || @@ -916,7 +1024,7 @@ Py_DECREF(iter); if (true) goto pop_1_error; } - #line 920 "Python/generated_cases.c.h" + #line 1028 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -924,7 +1032,7 @@ TARGET(GET_ANEXT) { PyObject *aiter = stack_pointer[-1]; PyObject *awaitable; - #line 650 "Python/bytecodes.c" + #line 730 "Python/bytecodes.c" unaryfunc getter = NULL; PyObject *next_iter = NULL; PyTypeObject *type = Py_TYPE(aiter); @@ -968,7 +1076,7 @@ } } - #line 972 "Python/generated_cases.c.h" + #line 1080 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = awaitable; PREDICT(LOAD_CONST); @@ -979,16 +1087,16 @@ PREDICTED(GET_AWAITABLE); PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 697 "Python/bytecodes.c" + #line 777 "Python/bytecodes.c" iter = _PyCoro_GetAwaitableIter(iterable); if (iter == NULL) { format_awaitable_error(tstate, Py_TYPE(iterable), oparg); } - #line 990 "Python/generated_cases.c.h" + #line 1098 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 704 "Python/bytecodes.c" + #line 784 "Python/bytecodes.c" if (iter != NULL && PyCoro_CheckExact(iter)) { PyObject *yf = _PyGen_yf((PyGenObject*)iter); @@ -1006,7 +1114,7 @@ if (iter == NULL) goto pop_1_error; - #line 1010 "Python/generated_cases.c.h" + #line 1118 "Python/generated_cases.c.h" stack_pointer[-1] = iter; PREDICT(LOAD_CONST); DISPATCH(); @@ -1017,11 +1125,10 @@ PyObject *v = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; PyObject *retval; - #line 730 "Python/bytecodes.c" + #line 810 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PySendCache *cache = (_PySendCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_Send(receiver, next_instr); DISPATCH_SAME_OPARG(); @@ -1030,6 +1137,20 @@ DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ assert(frame != &entry_frame); + if ((Py_TYPE(receiver) == &PyGen_Type || + Py_TYPE(receiver) == &PyCoro_Type) && ((PyGenObject *)receiver)->gi_frame_state < FRAME_EXECUTING) + { + PyGenObject *gen = (PyGenObject *)receiver; + _PyInterpreterFrame *gen_frame = (_PyInterpreterFrame *)gen->gi_iframe; + frame->yield_offset = oparg; + STACK_SHRINK(1); + _PyFrame_StackPush(gen_frame, v); + gen->gi_frame_state = FRAME_EXECUTING; + gen->gi_exc_state.previous_item = tstate->exc_info; + tstate->exc_info = &gen->gi_exc_state; + JUMPBY(INLINE_CACHE_ENTRIES_SEND + oparg); + DISPATCH_INLINED(gen_frame); + } if (Py_IsNone(v) && PyIter_Check(receiver)) { retval = Py_TYPE(receiver)->tp_iternext(receiver); } @@ -1037,23 +1158,20 @@ retval = PyObject_CallMethodOneArg(receiver, &_Py_ID(send), v); } if (retval == NULL) { - if (tstate->c_tracefunc != NULL - && _PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) - call_exc_trace(tstate->c_tracefunc, tstate->c_traceobj, tstate, frame); + if (_PyErr_ExceptionMatches(tstate, PyExc_StopIteration) + ) { + monitor_raise(tstate, frame, next_instr-1); + } if (_PyGen_FetchStopIterationValue(&retval) == 0) { assert(retval != NULL); JUMPBY(oparg); } else { - assert(retval == NULL); goto error; } } - else { - assert(retval != NULL); - } Py_DECREF(v); - #line 1057 "Python/generated_cases.c.h" + #line 1175 "Python/generated_cases.c.h" stack_pointer[-1] = retval; next_instr += 1; DISPATCH(); @@ -1062,8 +1180,7 @@ TARGET(SEND_GEN) { PyObject *v = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; - #line 768 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 858 "Python/bytecodes.c" PyGenObject *gen = (PyGenObject *)receiver; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type && Py_TYPE(gen) != &PyCoro_Type, SEND); @@ -1078,12 +1195,35 @@ tstate->exc_info = &gen->gi_exc_state; JUMPBY(INLINE_CACHE_ENTRIES_SEND + oparg); DISPATCH_INLINED(gen_frame); - #line 1082 "Python/generated_cases.c.h" + #line 1199 "Python/generated_cases.c.h" + } + + TARGET(INSTRUMENTED_YIELD_VALUE) { + PyObject *retval = stack_pointer[-1]; + #line 875 "Python/bytecodes.c" + assert(frame != &entry_frame); + PyGenObject *gen = _PyFrame_GetGenerator(frame); + gen->gi_frame_state = FRAME_SUSPENDED; + _PyFrame_SetStackPointer(frame, stack_pointer - 1); + int err = _Py_call_instrumentation_arg( + tstate, PY_MONITORING_EVENT_PY_YIELD, + frame, next_instr-1, retval); + if (err) goto error; + tstate->exc_info = gen->gi_exc_state.previous_item; + gen->gi_exc_state.previous_item = NULL; + _Py_LeaveRecursiveCallPy(tstate); + _PyInterpreterFrame *gen_frame = frame; + frame = cframe.current_frame = frame->previous; + gen_frame->previous = NULL; + frame->prev_instr -= frame->yield_offset; + _PyFrame_StackPush(frame, retval); + goto resume_frame; + #line 1222 "Python/generated_cases.c.h" } TARGET(YIELD_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 786 "Python/bytecodes.c" + #line 895 "Python/bytecodes.c" // NOTE: It's important that YIELD_VALUE never raises an exception! // The compiler treats any exception raised here as a failed close() // or throw() call. @@ -1091,8 +1231,6 @@ PyGenObject *gen = _PyFrame_GetGenerator(frame); gen->gi_frame_state = FRAME_SUSPENDED; _PyFrame_SetStackPointer(frame, stack_pointer - 1); - TRACE_FUNCTION_EXIT(); - DTRACE_FUNCTION_EXIT(); tstate->exc_info = gen->gi_exc_state.previous_item; gen->gi_exc_state.previous_item = NULL; _Py_LeaveRecursiveCallPy(tstate); @@ -1102,15 +1240,15 @@ frame->prev_instr -= frame->yield_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1106 "Python/generated_cases.c.h" + #line 1244 "Python/generated_cases.c.h" } TARGET(POP_EXCEPT) { PyObject *exc_value = stack_pointer[-1]; - #line 807 "Python/bytecodes.c" + #line 914 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; Py_XSETREF(exc_info->exc_value, exc_value); - #line 1114 "Python/generated_cases.c.h" + #line 1252 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -1118,7 +1256,7 @@ TARGET(RERAISE) { PyObject *exc = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); - #line 812 "Python/bytecodes.c" + #line 919 "Python/bytecodes.c" assert(oparg >= 0 && oparg <= 2); if (oparg) { PyObject *lasti = values[0]; @@ -1136,26 +1274,26 @@ Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); goto exception_unwind; - #line 1140 "Python/generated_cases.c.h" + #line 1278 "Python/generated_cases.c.h" } TARGET(END_ASYNC_FOR) { PyObject *exc = stack_pointer[-1]; PyObject *awaitable = stack_pointer[-2]; - #line 832 "Python/bytecodes.c" + #line 939 "Python/bytecodes.c" assert(exc && PyExceptionInstance_Check(exc)); if (PyErr_GivenExceptionMatches(exc, PyExc_StopAsyncIteration)) { - #line 1149 "Python/generated_cases.c.h" + #line 1287 "Python/generated_cases.c.h" Py_DECREF(awaitable); Py_DECREF(exc); - #line 835 "Python/bytecodes.c" + #line 942 "Python/bytecodes.c" } else { Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); goto exception_unwind; } - #line 1159 "Python/generated_cases.c.h" + #line 1297 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } @@ -1166,23 +1304,23 @@ PyObject *sub_iter = stack_pointer[-3]; PyObject *none; PyObject *value; - #line 844 "Python/bytecodes.c" + #line 951 "Python/bytecodes.c" assert(throwflag); assert(exc_value && PyExceptionInstance_Check(exc_value)); if (PyErr_GivenExceptionMatches(exc_value, PyExc_StopIteration)) { value = Py_NewRef(((PyStopIterationObject *)exc_value)->value); - #line 1175 "Python/generated_cases.c.h" + #line 1313 "Python/generated_cases.c.h" Py_DECREF(sub_iter); Py_DECREF(last_sent_val); Py_DECREF(exc_value); - #line 849 "Python/bytecodes.c" + #line 956 "Python/bytecodes.c" none = Py_NewRef(Py_None); } else { _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); goto exception_unwind; } - #line 1186 "Python/generated_cases.c.h" + #line 1324 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = value; stack_pointer[-2] = none; @@ -1191,9 +1329,9 @@ TARGET(LOAD_ASSERTION_ERROR) { PyObject *value; - #line 858 "Python/bytecodes.c" + #line 965 "Python/bytecodes.c" value = Py_NewRef(PyExc_AssertionError); - #line 1197 "Python/generated_cases.c.h" + #line 1335 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -1201,7 +1339,7 @@ TARGET(LOAD_BUILD_CLASS) { PyObject *bc; - #line 862 "Python/bytecodes.c" + #line 969 "Python/bytecodes.c" if (PyDict_CheckExact(BUILTINS())) { bc = _PyDict_GetItemWithError(BUILTINS(), &_Py_ID(__build_class__)); @@ -1223,7 +1361,7 @@ if (true) goto error; } } - #line 1227 "Python/generated_cases.c.h" + #line 1365 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = bc; DISPATCH(); @@ -1231,33 +1369,33 @@ TARGET(STORE_NAME) { PyObject *v = stack_pointer[-1]; - #line 886 "Python/bytecodes.c" + #line 993 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); PyObject *ns = LOCALS(); int err; if (ns == NULL) { _PyErr_Format(tstate, PyExc_SystemError, "no locals found when storing %R", name); - #line 1242 "Python/generated_cases.c.h" + #line 1380 "Python/generated_cases.c.h" Py_DECREF(v); - #line 893 "Python/bytecodes.c" + #line 1000 "Python/bytecodes.c" if (true) goto pop_1_error; } if (PyDict_CheckExact(ns)) err = PyDict_SetItem(ns, name, v); else err = PyObject_SetItem(ns, name, v); - #line 1251 "Python/generated_cases.c.h" + #line 1389 "Python/generated_cases.c.h" Py_DECREF(v); - #line 900 "Python/bytecodes.c" + #line 1007 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1255 "Python/generated_cases.c.h" + #line 1393 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(DELETE_NAME) { - #line 904 "Python/bytecodes.c" + #line 1011 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); PyObject *ns = LOCALS(); int err; @@ -1274,7 +1412,7 @@ name); goto error; } - #line 1278 "Python/generated_cases.c.h" + #line 1416 "Python/generated_cases.c.h" DISPATCH(); } @@ -1282,11 +1420,10 @@ PREDICTED(UNPACK_SEQUENCE); static_assert(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE == 1, "incorrect cache size"); PyObject *seq = stack_pointer[-1]; - #line 930 "Python/bytecodes.c" + #line 1037 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_UnpackSequence(seq, next_instr, oparg); DISPATCH_SAME_OPARG(); @@ -1296,11 +1433,11 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; int res = unpack_iterable(tstate, seq, oparg, -1, top); - #line 1300 "Python/generated_cases.c.h" + #line 1437 "Python/generated_cases.c.h" Py_DECREF(seq); - #line 944 "Python/bytecodes.c" + #line 1050 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1304 "Python/generated_cases.c.h" + #line 1441 "Python/generated_cases.c.h" STACK_SHRINK(1); STACK_GROW(oparg); next_instr += 1; @@ -1310,14 +1447,14 @@ TARGET(UNPACK_SEQUENCE_TWO_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 948 "Python/bytecodes.c" + #line 1054 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE); assert(oparg == 2); STAT_INC(UNPACK_SEQUENCE, hit); values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1)); values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0)); - #line 1321 "Python/generated_cases.c.h" + #line 1458 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1328,7 +1465,7 @@ TARGET(UNPACK_SEQUENCE_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 958 "Python/bytecodes.c" + #line 1064 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -1336,7 +1473,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 1340 "Python/generated_cases.c.h" + #line 1477 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1347,7 +1484,7 @@ TARGET(UNPACK_SEQUENCE_LIST) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 969 "Python/bytecodes.c" + #line 1075 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -1355,7 +1492,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 1359 "Python/generated_cases.c.h" + #line 1496 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1365,15 +1502,15 @@ TARGET(UNPACK_EX) { PyObject *seq = stack_pointer[-1]; - #line 980 "Python/bytecodes.c" + #line 1086 "Python/bytecodes.c" int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); - #line 1373 "Python/generated_cases.c.h" + #line 1510 "Python/generated_cases.c.h" Py_DECREF(seq); - #line 984 "Python/bytecodes.c" + #line 1090 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1377 "Python/generated_cases.c.h" + #line 1514 "Python/generated_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8)); DISPATCH(); } @@ -1384,10 +1521,9 @@ PyObject *owner = stack_pointer[-1]; PyObject *v = stack_pointer[-2]; uint16_t counter = read_u16(&next_instr[0].cache); - #line 995 "Python/bytecodes.c" + #line 1101 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { - assert(cframe.use_tracing == 0); PyObject *name = GETITEM(frame->f_code->co_names, oparg); next_instr--; _Py_Specialize_StoreAttr(owner, next_instr, name); @@ -1401,12 +1537,12 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject *name = GETITEM(frame->f_code->co_names, oparg); int err = PyObject_SetAttr(owner, name, v); - #line 1405 "Python/generated_cases.c.h" + #line 1541 "Python/generated_cases.c.h" Py_DECREF(v); Py_DECREF(owner); - #line 1012 "Python/bytecodes.c" + #line 1117 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 1410 "Python/generated_cases.c.h" + #line 1546 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -1414,34 +1550,34 @@ TARGET(DELETE_ATTR) { PyObject *owner = stack_pointer[-1]; - #line 1016 "Python/bytecodes.c" + #line 1121 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); int err = PyObject_SetAttr(owner, name, (PyObject *)NULL); - #line 1421 "Python/generated_cases.c.h" + #line 1557 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1019 "Python/bytecodes.c" + #line 1124 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1425 "Python/generated_cases.c.h" + #line 1561 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(STORE_GLOBAL) { PyObject *v = stack_pointer[-1]; - #line 1023 "Python/bytecodes.c" + #line 1128 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); - #line 1435 "Python/generated_cases.c.h" + #line 1571 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1026 "Python/bytecodes.c" + #line 1131 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1439 "Python/generated_cases.c.h" + #line 1575 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(DELETE_GLOBAL) { - #line 1030 "Python/bytecodes.c" + #line 1135 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); int err; err = PyDict_DelItem(GLOBALS(), name); @@ -1453,13 +1589,13 @@ } goto error; } - #line 1457 "Python/generated_cases.c.h" + #line 1593 "Python/generated_cases.c.h" DISPATCH(); } TARGET(LOAD_NAME) { PyObject *v; - #line 1044 "Python/bytecodes.c" + #line 1149 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); PyObject *locals = LOCALS(); if (locals == NULL) { @@ -1518,7 +1654,7 @@ } } } - #line 1522 "Python/generated_cases.c.h" + #line 1658 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = v; DISPATCH(); @@ -1529,11 +1665,10 @@ static_assert(INLINE_CACHE_ENTRIES_LOAD_GLOBAL == 4, "incorrect cache size"); PyObject *null = NULL; PyObject *v; - #line 1111 "Python/bytecodes.c" + #line 1216 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); PyObject *name = GETITEM(frame->f_code->co_names, oparg>>1); next_instr--; _Py_Specialize_LoadGlobal(GLOBALS(), BUILTINS(), next_instr, name); @@ -1582,7 +1717,7 @@ } } null = NULL; - #line 1586 "Python/generated_cases.c.h" + #line 1721 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = v; @@ -1596,8 +1731,7 @@ PyObject *res; uint16_t index = read_u16(&next_instr[1].cache); uint16_t version = read_u16(&next_instr[2].cache); - #line 1166 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1270 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); @@ -1608,7 +1742,7 @@ Py_INCREF(res); STAT_INC(LOAD_GLOBAL, hit); null = NULL; - #line 1612 "Python/generated_cases.c.h" + #line 1746 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1623,12 +1757,12 @@ uint16_t index = read_u16(&next_instr[1].cache); uint16_t mod_version = read_u16(&next_instr[2].cache); uint16_t bltn_version = read_u16(&next_instr[3].cache); - #line 1180 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1283 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); DEOPT_IF(!PyDict_CheckExact(BUILTINS()), LOAD_GLOBAL); PyDictObject *mdict = (PyDictObject *)GLOBALS(); PyDictObject *bdict = (PyDictObject *)BUILTINS(); + assert(opcode == LOAD_GLOBAL_BUILTIN); DEOPT_IF(mdict->ma_keys->dk_version != mod_version, LOAD_GLOBAL); DEOPT_IF(bdict->ma_keys->dk_version != bltn_version, LOAD_GLOBAL); assert(DK_IS_UNICODE(bdict->ma_keys)); @@ -1638,7 +1772,7 @@ Py_INCREF(res); STAT_INC(LOAD_GLOBAL, hit); null = NULL; - #line 1642 "Python/generated_cases.c.h" + #line 1776 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1648,16 +1782,16 @@ } TARGET(DELETE_FAST) { - #line 1197 "Python/bytecodes.c" + #line 1300 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 1656 "Python/generated_cases.c.h" + #line 1790 "Python/generated_cases.c.h" DISPATCH(); } TARGET(MAKE_CELL) { - #line 1203 "Python/bytecodes.c" + #line 1306 "Python/bytecodes.c" // "initial" is probably NULL but not if it's an arg (or set // via PyFrame_LocalsToFast() before MAKE_CELL has run). PyObject *initial = GETLOCAL(oparg); @@ -1666,12 +1800,12 @@ goto resume_with_error; } SETLOCAL(oparg, cell); - #line 1670 "Python/generated_cases.c.h" + #line 1804 "Python/generated_cases.c.h" DISPATCH(); } TARGET(DELETE_DEREF) { - #line 1214 "Python/bytecodes.c" + #line 1317 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here. @@ -1682,13 +1816,13 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 1686 "Python/generated_cases.c.h" + #line 1820 "Python/generated_cases.c.h" DISPATCH(); } TARGET(LOAD_CLASSDEREF) { PyObject *value; - #line 1227 "Python/bytecodes.c" + #line 1330 "Python/bytecodes.c" PyObject *name, *locals = LOCALS(); assert(locals); assert(oparg >= 0 && oparg < frame->f_code->co_nlocalsplus); @@ -1720,7 +1854,7 @@ } Py_INCREF(value); } - #line 1724 "Python/generated_cases.c.h" + #line 1858 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -1728,7 +1862,7 @@ TARGET(LOAD_DEREF) { PyObject *value; - #line 1261 "Python/bytecodes.c" + #line 1364 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { @@ -1736,7 +1870,7 @@ if (true) goto error; } Py_INCREF(value); - #line 1740 "Python/generated_cases.c.h" + #line 1874 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -1744,18 +1878,18 @@ TARGET(STORE_DEREF) { PyObject *v = stack_pointer[-1]; - #line 1271 "Python/bytecodes.c" + #line 1374 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 1753 "Python/generated_cases.c.h" + #line 1887 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(COPY_FREE_VARS) { - #line 1278 "Python/bytecodes.c" + #line 1381 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = frame->f_code; assert(PyFunction_Check(frame->f_funcobj)); @@ -1766,22 +1900,22 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 1770 "Python/generated_cases.c.h" + #line 1904 "Python/generated_cases.c.h" DISPATCH(); } TARGET(BUILD_STRING) { PyObject **pieces = (stack_pointer - oparg); PyObject *str; - #line 1291 "Python/bytecodes.c" + #line 1394 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 1779 "Python/generated_cases.c.h" + #line 1913 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } - #line 1293 "Python/bytecodes.c" + #line 1396 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1785 "Python/generated_cases.c.h" + #line 1919 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -1791,10 +1925,10 @@ TARGET(BUILD_TUPLE) { PyObject **values = (stack_pointer - oparg); PyObject *tup; - #line 1297 "Python/bytecodes.c" + #line 1400 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1798 "Python/generated_cases.c.h" + #line 1932 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -1804,10 +1938,10 @@ TARGET(BUILD_LIST) { PyObject **values = (stack_pointer - oparg); PyObject *list; - #line 1302 "Python/bytecodes.c" + #line 1405 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1811 "Python/generated_cases.c.h" + #line 1945 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -1817,7 +1951,7 @@ TARGET(LIST_EXTEND) { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 1307 "Python/bytecodes.c" + #line 1410 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) && @@ -1828,13 +1962,13 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 1832 "Python/generated_cases.c.h" + #line 1966 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1318 "Python/bytecodes.c" + #line 1421 "Python/bytecodes.c" if (true) goto pop_1_error; } Py_DECREF(none_val); - #line 1838 "Python/generated_cases.c.h" + #line 1972 "Python/generated_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); DISPATCH(); @@ -1843,13 +1977,13 @@ TARGET(SET_UPDATE) { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 1325 "Python/bytecodes.c" + #line 1428 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 1849 "Python/generated_cases.c.h" + #line 1983 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1327 "Python/bytecodes.c" + #line 1430 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 1853 "Python/generated_cases.c.h" + #line 1987 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -1857,7 +1991,7 @@ TARGET(BUILD_SET) { PyObject **values = (stack_pointer - oparg); PyObject *set; - #line 1331 "Python/bytecodes.c" + #line 1434 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error; @@ -1872,7 +2006,7 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 1876 "Python/generated_cases.c.h" + #line 2010 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -1882,7 +2016,7 @@ TARGET(BUILD_MAP) { PyObject **values = (stack_pointer - oparg*2); PyObject *map; - #line 1348 "Python/bytecodes.c" + #line 1451 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2, @@ -1890,13 +2024,13 @@ if (map == NULL) goto error; - #line 1894 "Python/generated_cases.c.h" + #line 2028 "Python/generated_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } - #line 1356 "Python/bytecodes.c" + #line 1459 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 1900 "Python/generated_cases.c.h" + #line 2034 "Python/generated_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -1904,7 +2038,7 @@ } TARGET(SETUP_ANNOTATIONS) { - #line 1360 "Python/bytecodes.c" + #line 1463 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -1944,7 +2078,7 @@ Py_DECREF(ann_dict); } } - #line 1948 "Python/generated_cases.c.h" + #line 2082 "Python/generated_cases.c.h" DISPATCH(); } @@ -1952,7 +2086,7 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map; - #line 1402 "Python/bytecodes.c" + #line 1505 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1962,14 +2096,14 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 1966 "Python/generated_cases.c.h" + #line 2100 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); - #line 1412 "Python/bytecodes.c" + #line 1515 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 1973 "Python/generated_cases.c.h" + #line 2107 "Python/generated_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; DISPATCH(); @@ -1977,7 +2111,7 @@ TARGET(DICT_UPDATE) { PyObject *update = stack_pointer[-1]; - #line 1416 "Python/bytecodes.c" + #line 1519 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { @@ -1985,12 +2119,12 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 1989 "Python/generated_cases.c.h" + #line 2123 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1424 "Python/bytecodes.c" + #line 1527 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1994 "Python/generated_cases.c.h" + #line 2128 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); DISPATCH(); @@ -1998,17 +2132,17 @@ TARGET(DICT_MERGE) { PyObject *update = stack_pointer[-1]; - #line 1430 "Python/bytecodes.c" + #line 1533 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 2007 "Python/generated_cases.c.h" + #line 2141 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1435 "Python/bytecodes.c" + #line 1538 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2012 "Python/generated_cases.c.h" + #line 2146 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); PREDICT(CALL_FUNCTION_EX); @@ -2018,13 +2152,13 @@ TARGET(MAP_ADD) { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2]; - #line 1442 "Python/bytecodes.c" + #line 1545 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 2028 "Python/generated_cases.c.h" + #line 2162 "Python/generated_cases.c.h" STACK_SHRINK(2); PREDICT(JUMP_BACKWARD); DISPATCH(); @@ -2036,11 +2170,10 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1465 "Python/bytecodes.c" + #line 1568 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); PyObject *name = GETITEM(frame->f_code->co_names, oparg>>1); next_instr--; _Py_Specialize_LoadAttr(owner, next_instr, name); @@ -2071,9 +2204,9 @@ NULL | meth | arg1 | ... | argN */ - #line 2075 "Python/generated_cases.c.h" + #line 2208 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1500 "Python/bytecodes.c" + #line 1602 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -2082,12 +2215,12 @@ else { /* Classic, pushes one value. */ res = PyObject_GetAttr(owner, name); - #line 2086 "Python/generated_cases.c.h" + #line 2219 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1509 "Python/bytecodes.c" + #line 1611 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } - #line 2091 "Python/generated_cases.c.h" + #line 2224 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -2101,8 +2234,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1514 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1616 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2115,7 +2247,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2119 "Python/generated_cases.c.h" + #line 2251 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2130,8 +2262,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1531 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1632 "Python/bytecodes.c" DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; assert(dict != NULL); @@ -2144,7 +2275,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2148 "Python/generated_cases.c.h" + #line 2279 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2159,8 +2290,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1548 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1648 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2187,7 +2317,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2191 "Python/generated_cases.c.h" + #line 2321 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2202,8 +2332,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1579 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1678 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2213,7 +2342,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2217 "Python/generated_cases.c.h" + #line 2346 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2228,8 +2357,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 1593 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1691 "Python/bytecodes.c" DEOPT_IF(!PyType_Check(cls), LOAD_ATTR); DEOPT_IF(((PyTypeObject *)cls)->tp_version_tag != type_version, @@ -2241,7 +2369,7 @@ res = descr; assert(res != NULL); Py_INCREF(res); - #line 2245 "Python/generated_cases.c.h" + #line 2373 "Python/generated_cases.c.h" Py_DECREF(cls); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2255,8 +2383,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *fget = read_obj(&next_instr[5].cache); - #line 1609 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1706 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); @@ -2279,7 +2406,7 @@ new_frame->localsplus[0] = owner; JUMPBY(INLINE_CACHE_ENTRIES_LOAD_ATTR); DISPATCH_INLINED(new_frame); - #line 2283 "Python/generated_cases.c.h" + #line 2410 "Python/generated_cases.c.h" } TARGET(LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN) { @@ -2287,8 +2414,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *getattribute = read_obj(&next_instr[5].cache); - #line 1635 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1731 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); DEOPT_IF(cls->tp_version_tag != type_version, LOAD_ATTR); @@ -2313,7 +2439,7 @@ new_frame->localsplus[1] = Py_NewRef(name); JUMPBY(INLINE_CACHE_ENTRIES_LOAD_ATTR); DISPATCH_INLINED(new_frame); - #line 2317 "Python/generated_cases.c.h" + #line 2443 "Python/generated_cases.c.h" } TARGET(STORE_ATTR_INSTANCE_VALUE) { @@ -2321,8 +2447,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1663 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1758 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2340,7 +2465,7 @@ Py_DECREF(old_value); } Py_DECREF(owner); - #line 2344 "Python/generated_cases.c.h" + #line 2469 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2351,8 +2476,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t hint = read_u16(&next_instr[3].cache); - #line 1684 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1778 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2391,7 +2515,7 @@ /* PEP 509 */ dict->ma_version_tag = new_version; Py_DECREF(owner); - #line 2395 "Python/generated_cases.c.h" + #line 2519 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2402,8 +2526,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1726 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1819 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2413,7 +2536,7 @@ *(PyObject **)addr = value; Py_XDECREF(old_value); Py_DECREF(owner); - #line 2417 "Python/generated_cases.c.h" + #line 2540 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2425,11 +2548,10 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 1746 "Python/bytecodes.c" + #line 1838 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_CompareOp(left, right, next_instr, oparg); DISPATCH_SAME_OPARG(); @@ -2439,12 +2561,12 @@ #endif /* ENABLE_SPECIALIZATION */ assert((oparg >> 4) <= Py_GE); res = PyObject_RichCompare(left, right, oparg>>4); - #line 2443 "Python/generated_cases.c.h" + #line 2565 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 1760 "Python/bytecodes.c" + #line 1851 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 2448 "Python/generated_cases.c.h" + #line 2570 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2455,8 +2577,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 1764 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1855 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2468,7 +2589,7 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; Py_INCREF(res); - #line 2472 "Python/generated_cases.c.h" + #line 2593 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2479,8 +2600,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 1780 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1870 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -2496,7 +2616,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; Py_INCREF(res); - #line 2500 "Python/generated_cases.c.h" + #line 2620 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2507,8 +2627,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 1800 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 1889 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2521,7 +2640,7 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; Py_INCREF(res); - #line 2525 "Python/generated_cases.c.h" + #line 2644 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2532,14 +2651,14 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 1816 "Python/bytecodes.c" + #line 1904 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 2538 "Python/generated_cases.c.h" + #line 2657 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 1818 "Python/bytecodes.c" + #line 1906 "Python/bytecodes.c" b = Py_NewRef(res ? Py_True : Py_False); - #line 2543 "Python/generated_cases.c.h" + #line 2662 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -2549,15 +2668,15 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 1822 "Python/bytecodes.c" + #line 1910 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 2555 "Python/generated_cases.c.h" + #line 2674 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 1824 "Python/bytecodes.c" + #line 1912 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = Py_NewRef((res^oparg) ? Py_True : Py_False); - #line 2561 "Python/generated_cases.c.h" + #line 2680 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -2568,12 +2687,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 1829 "Python/bytecodes.c" + #line 1917 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 2574 "Python/generated_cases.c.h" + #line 2693 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 1831 "Python/bytecodes.c" + #line 1919 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -2581,10 +2700,10 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 2585 "Python/generated_cases.c.h" + #line 2704 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 1839 "Python/bytecodes.c" + #line 1927 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -2593,7 +2712,7 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 2597 "Python/generated_cases.c.h" + #line 2716 "Python/generated_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; DISPATCH(); @@ -2603,21 +2722,21 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 1850 "Python/bytecodes.c" + #line 1938 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 2610 "Python/generated_cases.c.h" + #line 2729 "Python/generated_cases.c.h" Py_DECREF(right); - #line 1853 "Python/bytecodes.c" + #line 1941 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 2617 "Python/generated_cases.c.h" + #line 2736 "Python/generated_cases.c.h" Py_DECREF(right); - #line 1858 "Python/bytecodes.c" + #line 1946 "Python/bytecodes.c" b = Py_NewRef(res ? Py_True : Py_False); - #line 2621 "Python/generated_cases.c.h" + #line 2740 "Python/generated_cases.c.h" stack_pointer[-1] = b; DISPATCH(); } @@ -2626,15 +2745,15 @@ PyObject *fromlist = stack_pointer[-1]; PyObject *level = stack_pointer[-2]; PyObject *res; - #line 1862 "Python/bytecodes.c" + #line 1950 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); res = import_name(tstate, frame, name, fromlist, level); - #line 2633 "Python/generated_cases.c.h" + #line 2752 "Python/generated_cases.c.h" Py_DECREF(level); Py_DECREF(fromlist); - #line 1865 "Python/bytecodes.c" + #line 1953 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 2638 "Python/generated_cases.c.h" + #line 2757 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -2643,29 +2762,29 @@ TARGET(IMPORT_FROM) { PyObject *from = stack_pointer[-1]; PyObject *res; - #line 1869 "Python/bytecodes.c" + #line 1957 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); res = import_from(tstate, from, name); if (res == NULL) goto error; - #line 2651 "Python/generated_cases.c.h" + #line 2770 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); } TARGET(JUMP_FORWARD) { - #line 1875 "Python/bytecodes.c" + #line 1963 "Python/bytecodes.c" JUMPBY(oparg); - #line 2660 "Python/generated_cases.c.h" + #line 2779 "Python/generated_cases.c.h" DISPATCH(); } TARGET(JUMP_BACKWARD) { PREDICTED(JUMP_BACKWARD); - #line 1879 "Python/bytecodes.c" + #line 1967 "Python/bytecodes.c" assert(oparg < INSTR_OFFSET()); JUMPBY(-oparg); - #line 2669 "Python/generated_cases.c.h" + #line 2788 "Python/generated_cases.c.h" CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -2673,7 +2792,7 @@ TARGET(POP_JUMP_IF_FALSE) { PREDICTED(POP_JUMP_IF_FALSE); PyObject *cond = stack_pointer[-1]; - #line 1885 "Python/bytecodes.c" + #line 1973 "Python/bytecodes.c" if (Py_IsTrue(cond)) { _Py_DECREF_NO_DEALLOC(cond); } @@ -2683,9 +2802,9 @@ } else { int err = PyObject_IsTrue(cond); - #line 2687 "Python/generated_cases.c.h" + #line 2806 "Python/generated_cases.c.h" Py_DECREF(cond); - #line 1895 "Python/bytecodes.c" + #line 1983 "Python/bytecodes.c" if (err == 0) { JUMPBY(oparg); } @@ -2693,14 +2812,14 @@ if (err < 0) goto pop_1_error; } } - #line 2697 "Python/generated_cases.c.h" + #line 2816 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_TRUE) { PyObject *cond = stack_pointer[-1]; - #line 1905 "Python/bytecodes.c" + #line 1993 "Python/bytecodes.c" if (Py_IsFalse(cond)) { _Py_DECREF_NO_DEALLOC(cond); } @@ -2710,9 +2829,9 @@ } else { int err = PyObject_IsTrue(cond); - #line 2714 "Python/generated_cases.c.h" + #line 2833 "Python/generated_cases.c.h" Py_DECREF(cond); - #line 1915 "Python/bytecodes.c" + #line 2003 "Python/bytecodes.c" if (err > 0) { JUMPBY(oparg); } @@ -2720,67 +2839,67 @@ if (err < 0) goto pop_1_error; } } - #line 2724 "Python/generated_cases.c.h" + #line 2843 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_NOT_NONE) { PyObject *value = stack_pointer[-1]; - #line 1925 "Python/bytecodes.c" + #line 2013 "Python/bytecodes.c" if (!Py_IsNone(value)) { - #line 2733 "Python/generated_cases.c.h" + #line 2852 "Python/generated_cases.c.h" Py_DECREF(value); - #line 1927 "Python/bytecodes.c" + #line 2015 "Python/bytecodes.c" JUMPBY(oparg); } else { _Py_DECREF_NO_DEALLOC(value); } - #line 2741 "Python/generated_cases.c.h" + #line 2860 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_NONE) { PyObject *value = stack_pointer[-1]; - #line 1935 "Python/bytecodes.c" + #line 2023 "Python/bytecodes.c" if (Py_IsNone(value)) { _Py_DECREF_NO_DEALLOC(value); JUMPBY(oparg); } else { - #line 2754 "Python/generated_cases.c.h" + #line 2873 "Python/generated_cases.c.h" Py_DECREF(value); - #line 1941 "Python/bytecodes.c" + #line 2029 "Python/bytecodes.c" } - #line 2758 "Python/generated_cases.c.h" + #line 2877 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(JUMP_BACKWARD_NO_INTERRUPT) { - #line 1945 "Python/bytecodes.c" + #line 2033 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here. * (see bpo-30039). */ JUMPBY(-oparg); - #line 2771 "Python/generated_cases.c.h" + #line 2890 "Python/generated_cases.c.h" DISPATCH(); } TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 1954 "Python/bytecodes.c" + #line 2042 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 2784 "Python/generated_cases.c.h" + #line 2903 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; DISPATCH(); @@ -2791,16 +2910,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 1962 "Python/bytecodes.c" + #line 2050 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 2800 "Python/generated_cases.c.h" + #line 2919 "Python/generated_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 1967 "Python/bytecodes.c" + #line 2055 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -2808,7 +2927,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_NewRef(Py_None); // Failure! } - #line 2812 "Python/generated_cases.c.h" + #line 2931 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; DISPATCH(); @@ -2817,10 +2936,10 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 1977 "Python/bytecodes.c" + #line 2065 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = Py_NewRef(match ? Py_True : Py_False); - #line 2824 "Python/generated_cases.c.h" + #line 2943 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; PREDICT(POP_JUMP_IF_FALSE); @@ -2830,10 +2949,10 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 1983 "Python/bytecodes.c" + #line 2071 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = Py_NewRef(match ? Py_True : Py_False); - #line 2837 "Python/generated_cases.c.h" + #line 2956 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; PREDICT(POP_JUMP_IF_FALSE); @@ -2844,11 +2963,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 1989 "Python/bytecodes.c" + #line 2077 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 2852 "Python/generated_cases.c.h" + #line 2971 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; DISPATCH(); @@ -2857,14 +2976,14 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 1995 "Python/bytecodes.c" + #line 2083 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 2864 "Python/generated_cases.c.h" + #line 2983 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1998 "Python/bytecodes.c" + #line 2086 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 2868 "Python/generated_cases.c.h" + #line 2987 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -2872,7 +2991,7 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2002 "Python/bytecodes.c" + #line 2090 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -2895,11 +3014,11 @@ if (iter == NULL) { goto error; } - #line 2899 "Python/generated_cases.c.h" + #line 3018 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2025 "Python/bytecodes.c" + #line 2113 "Python/bytecodes.c" } - #line 2903 "Python/generated_cases.c.h" + #line 3022 "Python/generated_cases.c.h" stack_pointer[-1] = iter; PREDICT(LOAD_CONST); DISPATCH(); @@ -2910,11 +3029,10 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2044 "Python/bytecodes.c" + #line 2132 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_ForIter(iter, next_instr, oparg); DISPATCH_SAME_OPARG(); @@ -2929,13 +3047,12 @@ if (!_PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) { goto error; } - else if (tstate->c_tracefunc != NULL) { - call_exc_trace(tstate->c_tracefunc, tstate->c_traceobj, tstate, frame); - } + monitor_raise(tstate, frame, next_instr-1); _PyErr_Clear(tstate); } /* iterator ended normally */ - assert(next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == END_FOR); + assert(next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == END_FOR || + next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == INSTRUMENTED_END_FOR); Py_DECREF(iter); STACK_SHRINK(1); /* Jump forward oparg, then skip following END_FOR instruction */ @@ -2943,18 +3060,48 @@ DISPATCH(); } // Common case: no jump, leave it to the code generator - #line 2947 "Python/generated_cases.c.h" + #line 3064 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; DISPATCH(); } + TARGET(INSTRUMENTED_FOR_ITER) { + #line 2165 "Python/bytecodes.c" + _Py_CODEUNIT *here = next_instr-1; + _Py_CODEUNIT *target; + PyObject *iter = TOP(); + PyObject *next = (*Py_TYPE(iter)->tp_iternext)(iter); + if (next != NULL) { + PUSH(next); + target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER; + } + else { + if (_PyErr_Occurred(tstate)) { + if (!_PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) { + goto error; + } + monitor_raise(tstate, frame, here); + _PyErr_Clear(tstate); + } + /* iterator ended normally */ + assert(next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == END_FOR || + next_instr[INLINE_CACHE_ENTRIES_FOR_ITER + oparg].op.code == INSTRUMENTED_END_FOR); + STACK_SHRINK(1); + Py_DECREF(iter); + /* Skip END_FOR */ + target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER + oparg + 1; + } + INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); + #line 3098 "Python/generated_cases.c.h" + DISPATCH(); + } + TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2079 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2193 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -2974,7 +3121,7 @@ DISPATCH(); end_for_iter_list: // Common case: no jump, leave it to the code generator - #line 2978 "Python/generated_cases.c.h" + #line 3125 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -2984,8 +3131,7 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2102 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2215 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3005,7 +3151,7 @@ DISPATCH(); end_for_iter_tuple: // Common case: no jump, leave it to the code generator - #line 3009 "Python/generated_cases.c.h" + #line 3155 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3015,8 +3161,7 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2125 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2237 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3034,7 +3179,7 @@ if (next == NULL) { goto error; } - #line 3038 "Python/generated_cases.c.h" + #line 3183 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3043,8 +3188,7 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1]; - #line 2146 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2257 "Python/bytecodes.c" PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); DEOPT_IF(gen->gi_frame_state >= FRAME_EXECUTING, FOR_ITER); @@ -3056,16 +3200,17 @@ gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; JUMPBY(INLINE_CACHE_ENTRIES_FOR_ITER + oparg); - assert(next_instr->op.code == END_FOR); + assert(next_instr->op.code == END_FOR || + next_instr->op.code == INSTRUMENTED_END_FOR); DISPATCH_INLINED(gen_frame); - #line 3062 "Python/generated_cases.c.h" + #line 3207 "Python/generated_cases.c.h" } TARGET(BEFORE_ASYNC_WITH) { PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2163 "Python/bytecodes.c" + #line 2274 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { @@ -3088,16 +3233,16 @@ Py_DECREF(enter); goto error; } - #line 3092 "Python/generated_cases.c.h" + #line 3237 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2186 "Python/bytecodes.c" + #line 2297 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3101 "Python/generated_cases.c.h" + #line 3246 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3109,7 +3254,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2196 "Python/bytecodes.c" + #line 2307 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */ @@ -3135,16 +3280,16 @@ Py_DECREF(enter); goto error; } - #line 3139 "Python/generated_cases.c.h" + #line 3284 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2222 "Python/bytecodes.c" + #line 2333 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3148 "Python/generated_cases.c.h" + #line 3293 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3156,7 +3301,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2231 "Python/bytecodes.c" + #line 2342 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -3177,7 +3322,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 3181 "Python/generated_cases.c.h" + #line 3326 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3186,7 +3331,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2254 "Python/bytecodes.c" + #line 2365 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3196,7 +3341,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 3200 "Python/generated_cases.c.h" + #line 3345 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -3210,9 +3355,8 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2266 "Python/bytecodes.c" + #line 2377 "Python/bytecodes.c" /* Cached method object */ - assert(cframe.use_tracing == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3228,7 +3372,7 @@ assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; assert(oparg & 1); - #line 3232 "Python/generated_cases.c.h" + #line 3376 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3242,8 +3386,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2286 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2396 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); assert(self_cls->tp_dictoffset == 0); @@ -3253,7 +3396,7 @@ res2 = Py_NewRef(descr); res = self; assert(oparg & 1); - #line 3257 "Python/generated_cases.c.h" + #line 3400 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3267,8 +3410,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2299 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2408 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); Py_ssize_t dictoffset = self_cls->tp_dictoffset; @@ -3282,7 +3424,7 @@ res2 = Py_NewRef(descr); res = self; assert(oparg & 1); - #line 3286 "Python/generated_cases.c.h" + #line 3428 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3291,14 +3433,31 @@ } TARGET(KW_NAMES) { - #line 2316 "Python/bytecodes.c" + #line 2424 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(frame->f_code->co_consts)); kwnames = GETITEM(frame->f_code->co_consts, oparg); - #line 3299 "Python/generated_cases.c.h" + #line 3441 "Python/generated_cases.c.h" DISPATCH(); } + TARGET(INSTRUMENTED_CALL) { + #line 2430 "Python/bytecodes.c" + int is_meth = PEEK(oparg+2) != NULL; + int total_args = oparg + is_meth; + PyObject *function = PEEK(total_args + 1); + PyObject *arg = total_args == 0 ? + &_PyInstrumentation_MISSING : PEEK(total_args); + int err = _Py_call_instrumentation_2args( + tstate, PY_MONITORING_EVENT_CALL, + frame, next_instr-1, function, arg); + if (err) goto error; + _PyCallCache *cache = (_PyCallCache *)next_instr; + INCREMENT_ADAPTIVE_COUNTER(cache->counter); + GO_TO_INSTRUCTION(CALL); + #line 3459 "Python/generated_cases.c.h" + } + TARGET(CALL) { PREDICTED(CALL); static_assert(INLINE_CACHE_ENTRIES_CALL == 3, "incorrect cache size"); @@ -3306,7 +3465,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2352 "Python/bytecodes.c" + #line 2475 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3317,7 +3476,6 @@ #if ENABLE_SPECIALIZATION _PyCallCache *cache = (_PyCallCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_Call(callable, next_instr, total_args, kwnames); DISPATCH_SAME_OPARG(); @@ -3360,16 +3518,26 @@ DISPATCH_INLINED(new_frame); } /* Callable is not a normal Python function */ - if (cframe.use_tracing) { - res = trace_call_function( - tstate, callable, args, - positional_args, kwnames); - } - else { - res = PyObject_Vectorcall( - callable, args, - positional_args | PY_VECTORCALL_ARGUMENTS_OFFSET, - kwnames); + res = PyObject_Vectorcall( + callable, args, + positional_args | PY_VECTORCALL_ARGUMENTS_OFFSET, + kwnames); + if (opcode == INSTRUMENTED_CALL) { + PyObject *arg = total_args == 0 ? + &_PyInstrumentation_MISSING : PEEK(total_args); + if (res == NULL) { + _Py_call_instrumentation_exc2( + tstate, PY_MONITORING_EVENT_C_RAISE, + frame, next_instr-1, callable, arg); + } + else { + int err = _Py_call_instrumentation_2args( + tstate, PY_MONITORING_EVENT_C_RETURN, + frame, next_instr-1, callable, arg); + if (err < 0) { + Py_CLEAR(res); + } + } } kwnames = NULL; assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); @@ -3378,7 +3546,7 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3382 "Python/generated_cases.c.h" + #line 3550 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3390,7 +3558,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2430 "Python/bytecodes.c" + #line 2562 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -3400,7 +3568,7 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 3404 "Python/generated_cases.c.h" + #line 3572 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -3409,7 +3577,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2442 "Python/bytecodes.c" + #line 2574 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -3434,7 +3602,7 @@ STACK_SHRINK(oparg + 2); JUMPBY(INLINE_CACHE_ENTRIES_CALL); DISPATCH_INLINED(new_frame); - #line 3438 "Python/generated_cases.c.h" + #line 3606 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -3442,7 +3610,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2469 "Python/bytecodes.c" + #line 2601 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -3477,7 +3645,7 @@ STACK_SHRINK(oparg + 2); JUMPBY(INLINE_CACHE_ENTRIES_CALL); DISPATCH_INLINED(new_frame); - #line 3481 "Python/generated_cases.c.h" + #line 3649 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -3485,9 +3653,8 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2506 "Python/bytecodes.c" + #line 2638 "Python/bytecodes.c" assert(kwnames == NULL); - assert(cframe.use_tracing == 0); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); PyObject *obj = args[0]; @@ -3496,7 +3663,7 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 3500 "Python/generated_cases.c.h" + #line 3667 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3509,9 +3676,8 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2519 "Python/bytecodes.c" + #line 2650 "Python/bytecodes.c" assert(kwnames == NULL); - assert(cframe.use_tracing == 0); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); DEOPT_IF(callable != (PyObject *)&PyUnicode_Type, CALL); @@ -3521,7 +3687,7 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3525 "Python/generated_cases.c.h" + #line 3691 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3535,7 +3701,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2534 "Python/bytecodes.c" + #line 2664 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -3546,7 +3712,7 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3550 "Python/generated_cases.c.h" + #line 3716 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3560,7 +3726,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2548 "Python/bytecodes.c" + #line 2678 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3582,7 +3748,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3586 "Python/generated_cases.c.h" + #line 3752 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3596,8 +3762,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2573 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2703 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -3625,7 +3790,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3629 "Python/generated_cases.c.h" + #line 3794 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3639,8 +3804,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2605 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2734 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -3672,7 +3836,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 3676 "Python/generated_cases.c.h" + #line 3840 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3686,8 +3850,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2641 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2769 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -3719,7 +3882,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3723 "Python/generated_cases.c.h" + #line 3886 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3733,8 +3896,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2677 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2804 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -3759,7 +3921,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3763 "Python/generated_cases.c.h" + #line 3925 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3772,8 +3934,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2705 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2831 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -3800,7 +3961,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3804 "Python/generated_cases.c.h" + #line 3965 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3812,8 +3973,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2736 "Python/bytecodes.c" - assert(cframe.use_tracing == 0); + #line 2861 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -3831,14 +3991,14 @@ JUMPBY(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 3835 "Python/generated_cases.c.h" + #line 3995 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2757 "Python/bytecodes.c" + #line 2881 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -3869,7 +4029,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3873 "Python/generated_cases.c.h" + #line 4033 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3882,7 +4042,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2791 "Python/bytecodes.c" + #line 2915 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3911,7 +4071,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3915 "Python/generated_cases.c.h" + #line 4075 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3924,7 +4084,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2823 "Python/bytecodes.c" + #line 2947 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -3953,7 +4113,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3957 "Python/generated_cases.c.h" + #line 4117 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3966,7 +4126,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2855 "Python/bytecodes.c" + #line 2979 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -3994,7 +4154,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3998 "Python/generated_cases.c.h" + #line 4158 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4003,18 +4163,22 @@ DISPATCH(); } + TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { + #line 3010 "Python/bytecodes.c" + GO_TO_INSTRUCTION(CALL_FUNCTION_EX); + #line 4170 "Python/generated_cases.c.h" + } + TARGET(CALL_FUNCTION_EX) { PREDICTED(CALL_FUNCTION_EX); PyObject *kwargs = (oparg & 1) ? stack_pointer[-(((oparg & 1) ? 1 : 0))] : NULL; PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 2886 "Python/bytecodes.c" - if (oparg & 1) { - // DICT_MERGE is called before this opcode if there are kwargs. - // It converts all dict subtypes in kwargs into regular dicts. - assert(PyDict_CheckExact(kwargs)); - } + #line 3014 "Python/bytecodes.c" + // DICT_MERGE is called before this opcode if there are kwargs. + // It converts all dict subtypes in kwargs into regular dicts. + assert(kwargs == NULL || PyDict_CheckExact(kwargs)); if (!PyTuple_CheckExact(callargs)) { if (check_args_iterable(tstate, func, callargs) < 0) { goto error; @@ -4026,17 +4190,42 @@ Py_SETREF(callargs, tuple); } assert(PyTuple_CheckExact(callargs)); - - result = do_call_core(tstate, func, callargs, kwargs, cframe.use_tracing); - #line 4032 "Python/generated_cases.c.h" + EVAL_CALL_STAT_INC_IF_FUNCTION(EVAL_CALL_FUNCTION_EX, func); + if (opcode == INSTRUMENTED_CALL_FUNCTION_EX && + !PyFunction_Check(func) && !PyMethod_Check(func) + ) { + PyObject *arg = PyTuple_GET_SIZE(callargs) > 0 ? + PyTuple_GET_ITEM(callargs, 0) : Py_None; + int err = _Py_call_instrumentation_2args( + tstate, PY_MONITORING_EVENT_CALL, + frame, next_instr-1, func, arg); + if (err) goto error; + result = PyObject_Call(func, callargs, kwargs); + if (result == NULL) { + _Py_call_instrumentation_exc2( + tstate, PY_MONITORING_EVENT_C_RAISE, + frame, next_instr-1, func, arg); + } + else { + int err = _Py_call_instrumentation_2args( + tstate, PY_MONITORING_EVENT_C_RETURN, + frame, next_instr-1, func, arg); + if (err < 0) { + Py_CLEAR(result); + } + } + } + else { + result = PyObject_Call(func, callargs, kwargs); + } + #line 4222 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 2905 "Python/bytecodes.c" - + #line 3057 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4040 "Python/generated_cases.c.h" + #line 4229 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4051,7 +4240,7 @@ PyObject *kwdefaults = (oparg & 0x02) ? stack_pointer[-(1 + ((oparg & 0x08) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0))] : NULL; PyObject *defaults = (oparg & 0x01) ? stack_pointer[-(1 + ((oparg & 0x08) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0) + ((oparg & 0x01) ? 1 : 0))] : NULL; PyObject *func; - #line 2916 "Python/bytecodes.c" + #line 3067 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4080,14 +4269,14 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4084 "Python/generated_cases.c.h" + #line 4273 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 0x01) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x08) ? 1 : 0)); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 2947 "Python/bytecodes.c" + #line 3098 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4108,7 +4297,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4112 "Python/generated_cases.c.h" + #line 4301 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4116,15 +4305,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 2970 "Python/bytecodes.c" + #line 3121 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4122 "Python/generated_cases.c.h" + #line 4311 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 2972 "Python/bytecodes.c" + #line 3123 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4128 "Python/generated_cases.c.h" + #line 4317 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4135,7 +4324,7 @@ PyObject *fmt_spec = ((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? stack_pointer[-((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0))] : NULL; PyObject *value = stack_pointer[-(1 + (((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0))]; PyObject *result; - #line 2976 "Python/bytecodes.c" + #line 3127 "Python/bytecodes.c" /* Handles f-string value formatting. */ PyObject *(*conv_fn)(PyObject *); int which_conversion = oparg & FVC_MASK; @@ -4170,7 +4359,7 @@ Py_DECREF(value); Py_XDECREF(fmt_spec); if (result == NULL) { STACK_SHRINK((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0)); goto pop_1_error; } - #line 4174 "Python/generated_cases.c.h" + #line 4363 "Python/generated_cases.c.h" STACK_SHRINK((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0)); stack_pointer[-1] = result; DISPATCH(); @@ -4179,10 +4368,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3013 "Python/bytecodes.c" + #line 3164 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4186 "Python/generated_cases.c.h" + #line 4375 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4194,11 +4383,10 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3018 "Python/bytecodes.c" + #line 3169 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { - assert(cframe.use_tracing == 0); next_instr--; _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, &GETLOCAL(0)); DISPATCH_SAME_OPARG(); @@ -4210,12 +4398,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4214 "Python/generated_cases.c.h" + #line 4402 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3034 "Python/bytecodes.c" + #line 3184 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4219 "Python/generated_cases.c.h" + #line 4407 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4225,27 +4413,153 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3039 "Python/bytecodes.c" + #line 3189 "Python/bytecodes.c" assert(oparg >= 2); - #line 4231 "Python/generated_cases.c.h" + #line 4419 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } + TARGET(INSTRUMENTED_LINE) { + #line 3193 "Python/bytecodes.c" + _Py_CODEUNIT *here = next_instr-1; + _PyFrame_SetStackPointer(frame, stack_pointer); + int original_opcode = _Py_call_instrumentation_line( + tstate, frame, here); + stack_pointer = _PyFrame_GetStackPointer(frame); + if (original_opcode < 0) { + next_instr = here+1; + goto error; + } + next_instr = frame->prev_instr; + if (next_instr != here) { + DISPATCH(); + } + if (_PyOpcode_Caches[original_opcode]) { + _PyBinaryOpCache *cache = (_PyBinaryOpCache *)(next_instr+1); + INCREMENT_ADAPTIVE_COUNTER(cache->counter); + } + opcode = original_opcode; + DISPATCH_GOTO(); + #line 4446 "Python/generated_cases.c.h" + } + + TARGET(INSTRUMENTED_INSTRUCTION) { + #line 3215 "Python/bytecodes.c" + int next_opcode = _Py_call_instrumentation_instruction( + tstate, frame, next_instr-1); + if (next_opcode < 0) goto error; + next_instr--; + if (_PyOpcode_Caches[next_opcode]) { + _PyBinaryOpCache *cache = (_PyBinaryOpCache *)(next_instr+1); + INCREMENT_ADAPTIVE_COUNTER(cache->counter); + } + assert(next_opcode > 0 && next_opcode < 256); + opcode = next_opcode; + DISPATCH_GOTO(); + #line 4462 "Python/generated_cases.c.h" + } + + TARGET(INSTRUMENTED_JUMP_FORWARD) { + #line 3229 "Python/bytecodes.c" + INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); + #line 4468 "Python/generated_cases.c.h" + DISPATCH(); + } + + TARGET(INSTRUMENTED_JUMP_BACKWARD) { + #line 3233 "Python/bytecodes.c" + INSTRUMENTED_JUMP(next_instr-1, next_instr-oparg, PY_MONITORING_EVENT_JUMP); + #line 4475 "Python/generated_cases.c.h" + CHECK_EVAL_BREAKER(); + DISPATCH(); + } + + TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { + #line 3238 "Python/bytecodes.c" + PyObject *cond = POP(); + int err = PyObject_IsTrue(cond); + Py_DECREF(cond); + if (err < 0) goto error; + _Py_CODEUNIT *here = next_instr-1; + assert(err == 0 || err == 1); + int offset = err*oparg; + INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); + #line 4490 "Python/generated_cases.c.h" + DISPATCH(); + } + + TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { + #line 3249 "Python/bytecodes.c" + PyObject *cond = POP(); + int err = PyObject_IsTrue(cond); + Py_DECREF(cond); + if (err < 0) goto error; + _Py_CODEUNIT *here = next_instr-1; + assert(err == 0 || err == 1); + int offset = (1-err)*oparg; + INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); + #line 4504 "Python/generated_cases.c.h" + DISPATCH(); + } + + TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { + #line 3260 "Python/bytecodes.c" + PyObject *value = POP(); + _Py_CODEUNIT *here = next_instr-1; + int offset; + if (Py_IsNone(value)) { + _Py_DECREF_NO_DEALLOC(value); + offset = oparg; + } + else { + Py_DECREF(value); + offset = 0; + } + INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); + #line 4522 "Python/generated_cases.c.h" + DISPATCH(); + } + + TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { + #line 3275 "Python/bytecodes.c" + PyObject *value = POP(); + _Py_CODEUNIT *here = next_instr-1; + int offset; + if (Py_IsNone(value)) { + _Py_DECREF_NO_DEALLOC(value); + offset = 0; + } + else { + Py_DECREF(value); + offset = oparg; + } + INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); + #line 4540 "Python/generated_cases.c.h" + DISPATCH(); + } + TARGET(EXTENDED_ARG) { - #line 3043 "Python/bytecodes.c" + #line 3290 "Python/bytecodes.c" assert(oparg); - assert(cframe.use_tracing == 0); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 4245 "Python/generated_cases.c.h" + #line 4551 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3052 "Python/bytecodes.c" + #line 3298 "Python/bytecodes.c" + assert(0 && "Executing a cache."); + Py_UNREACHABLE(); + #line 4558 "Python/generated_cases.c.h" + } + + TARGET(RESERVED) { + #line 3303 "Python/bytecodes.c" + assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 4251 "Python/generated_cases.c.h" + #line 4565 "Python/generated_cases.c.h" } diff --git a/Python/importlib.h b/Python/importlib.h deleted file mode 100644 index 586f3b21f46246..00000000000000 --- a/Python/importlib.h +++ /dev/null @@ -1,1783 +0,0 @@ -/* Auto-generated by Programs/_freeze_importlib.c */ -const unsigned char _Py_M__importlib_bootstrap[] = { - 99,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, - 0,4,0,0,0,64,0,0,0,115,194,1,0,0,100,0, - 90,0,100,1,97,1,100,2,100,3,132,0,90,2,100,4, - 100,5,132,0,90,3,105,0,90,4,105,0,90,5,71,0, - 100,6,100,7,132,0,100,7,101,6,131,3,90,7,71,0, - 100,8,100,9,132,0,100,9,131,2,90,8,71,0,100,10, - 100,11,132,0,100,11,131,2,90,9,71,0,100,12,100,13, - 132,0,100,13,131,2,90,10,100,14,100,15,132,0,90,11, - 100,16,100,17,132,0,90,12,100,18,100,19,132,0,90,13, - 100,20,100,21,156,1,100,22,100,23,132,2,90,14,100,24, - 100,25,132,0,90,15,100,26,100,27,132,0,90,16,100,28, - 100,29,132,0,90,17,100,30,100,31,132,0,90,18,71,0, - 100,32,100,33,132,0,100,33,131,2,90,19,100,1,100,1, - 100,34,156,2,100,35,100,36,132,2,90,20,100,94,100,37, - 100,38,132,1,90,21,100,39,100,40,156,1,100,41,100,42, - 132,2,90,22,100,43,100,44,132,0,90,23,100,45,100,46, - 132,0,90,24,100,47,100,48,132,0,90,25,100,49,100,50, - 132,0,90,26,100,51,100,52,132,0,90,27,100,53,100,54, - 132,0,90,28,71,0,100,55,100,56,132,0,100,56,131,2, - 90,29,71,0,100,57,100,58,132,0,100,58,131,2,90,30, - 71,0,100,59,100,60,132,0,100,60,131,2,90,31,100,61, - 100,62,132,0,90,32,100,63,100,64,132,0,90,33,100,95, - 100,65,100,66,132,1,90,34,100,67,100,68,132,0,90,35, - 100,69,90,36,101,36,100,70,23,0,90,37,100,71,100,72, - 132,0,90,38,101,39,131,0,90,40,100,73,100,74,132,0, - 90,41,100,96,100,76,100,77,132,1,90,42,100,39,100,78, - 156,1,100,79,100,80,132,2,90,43,100,81,100,82,132,0, - 90,44,100,97,100,84,100,85,132,1,90,45,100,86,100,87, - 132,0,90,46,100,88,100,89,132,0,90,47,100,90,100,91, - 132,0,90,48,100,92,100,93,132,0,90,49,100,1,83,0, - 41,98,97,83,1,0,0,67,111,114,101,32,105,109,112,108, - 101,109,101,110,116,97,116,105,111,110,32,111,102,32,105,109, - 112,111,114,116,46,10,10,84,104,105,115,32,109,111,100,117, - 108,101,32,105,115,32,78,79,84,32,109,101,97,110,116,32, - 116,111,32,98,101,32,100,105,114,101,99,116,108,121,32,105, - 109,112,111,114,116,101,100,33,32,73,116,32,104,97,115,32, - 98,101,101,110,32,100,101,115,105,103,110,101,100,32,115,117, - 99,104,10,116,104,97,116,32,105,116,32,99,97,110,32,98, - 101,32,98,111,111,116,115,116,114,97,112,112,101,100,32,105, - 110,116,111,32,80,121,116,104,111,110,32,97,115,32,116,104, - 101,32,105,109,112,108,101,109,101,110,116,97,116,105,111,110, - 32,111,102,32,105,109,112,111,114,116,46,32,65,115,10,115, - 117,99,104,32,105,116,32,114,101,113,117,105,114,101,115,32, - 116,104,101,32,105,110,106,101,99,116,105,111,110,32,111,102, - 32,115,112,101,99,105,102,105,99,32,109,111,100,117,108,101, - 115,32,97,110,100,32,97,116,116,114,105,98,117,116,101,115, - 32,105,110,32,111,114,100,101,114,32,116,111,10,119,111,114, - 107,46,32,79,110,101,32,115,104,111,117,108,100,32,117,115, - 101,32,105,109,112,111,114,116,108,105,98,32,97,115,32,116, - 104,101,32,112,117,98,108,105,99,45,102,97,99,105,110,103, - 32,118,101,114,115,105,111,110,32,111,102,32,116,104,105,115, - 32,109,111,100,117,108,101,46,10,10,78,99,2,0,0,0, - 0,0,0,0,0,0,0,0,3,0,0,0,7,0,0,0, - 67,0,0,0,115,56,0,0,0,100,1,68,0,93,32,125, - 2,116,0,124,1,124,2,131,2,114,4,116,1,124,0,124, - 2,116,2,124,1,124,2,131,2,131,3,1,0,113,4,124, - 0,106,3,160,4,124,1,106,3,161,1,1,0,100,2,83, - 0,41,3,122,47,83,105,109,112,108,101,32,115,117,98,115, - 116,105,116,117,116,101,32,102,111,114,32,102,117,110,99,116, - 111,111,108,115,46,117,112,100,97,116,101,95,119,114,97,112, - 112,101,114,46,41,4,218,10,95,95,109,111,100,117,108,101, - 95,95,218,8,95,95,110,97,109,101,95,95,218,12,95,95, - 113,117,97,108,110,97,109,101,95,95,218,7,95,95,100,111, - 99,95,95,78,41,5,218,7,104,97,115,97,116,116,114,218, - 7,115,101,116,97,116,116,114,218,7,103,101,116,97,116,116, - 114,218,8,95,95,100,105,99,116,95,95,218,6,117,112,100, - 97,116,101,41,3,90,3,110,101,119,90,3,111,108,100,218, - 7,114,101,112,108,97,99,101,169,0,114,10,0,0,0,250, - 29,60,102,114,111,122,101,110,32,105,109,112,111,114,116,108, - 105,98,46,95,98,111,111,116,115,116,114,97,112,62,218,5, - 95,119,114,97,112,27,0,0,0,115,8,0,0,0,0,2, - 8,1,10,1,20,1,114,12,0,0,0,99,1,0,0,0, - 0,0,0,0,0,0,0,0,1,0,0,0,2,0,0,0, - 67,0,0,0,115,12,0,0,0,116,0,116,1,131,1,124, - 0,131,1,83,0,169,1,78,41,2,218,4,116,121,112,101, - 218,3,115,121,115,169,1,218,4,110,97,109,101,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,218,11,95,110, - 101,119,95,109,111,100,117,108,101,35,0,0,0,115,2,0, - 0,0,0,1,114,18,0,0,0,99,0,0,0,0,0,0, - 0,0,0,0,0,0,0,0,0,0,1,0,0,0,64,0, - 0,0,115,12,0,0,0,101,0,90,1,100,0,90,2,100, - 1,83,0,41,2,218,14,95,68,101,97,100,108,111,99,107, - 69,114,114,111,114,78,41,3,114,1,0,0,0,114,0,0, - 0,0,114,2,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,19,0,0,0, - 48,0,0,0,115,2,0,0,0,8,1,114,19,0,0,0, - 99,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, - 0,2,0,0,0,64,0,0,0,115,56,0,0,0,101,0, - 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, - 90,4,100,4,100,5,132,0,90,5,100,6,100,7,132,0, - 90,6,100,8,100,9,132,0,90,7,100,10,100,11,132,0, - 90,8,100,12,83,0,41,13,218,11,95,77,111,100,117,108, - 101,76,111,99,107,122,169,65,32,114,101,99,117,114,115,105, - 118,101,32,108,111,99,107,32,105,109,112,108,101,109,101,110, - 116,97,116,105,111,110,32,119,104,105,99,104,32,105,115,32, - 97,98,108,101,32,116,111,32,100,101,116,101,99,116,32,100, - 101,97,100,108,111,99,107,115,10,32,32,32,32,40,101,46, - 103,46,32,116,104,114,101,97,100,32,49,32,116,114,121,105, - 110,103,32,116,111,32,116,97,107,101,32,108,111,99,107,115, - 32,65,32,116,104,101,110,32,66,44,32,97,110,100,32,116, - 104,114,101,97,100,32,50,32,116,114,121,105,110,103,32,116, - 111,10,32,32,32,32,116,97,107,101,32,108,111,99,107,115, - 32,66,32,116,104,101,110,32,65,41,46,10,32,32,32,32, - 99,2,0,0,0,0,0,0,0,0,0,0,0,2,0,0, - 0,2,0,0,0,67,0,0,0,115,48,0,0,0,116,0, - 160,1,161,0,124,0,95,2,116,0,160,1,161,0,124,0, - 95,3,124,1,124,0,95,4,100,0,124,0,95,5,100,1, - 124,0,95,6,100,1,124,0,95,7,100,0,83,0,169,2, - 78,233,0,0,0,0,41,8,218,7,95,116,104,114,101,97, - 100,90,13,97,108,108,111,99,97,116,101,95,108,111,99,107, - 218,4,108,111,99,107,218,6,119,97,107,101,117,112,114,17, - 0,0,0,218,5,111,119,110,101,114,218,5,99,111,117,110, - 116,218,7,119,97,105,116,101,114,115,169,2,218,4,115,101, - 108,102,114,17,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,218,8,95,95,105,110,105,116,95,95, - 58,0,0,0,115,12,0,0,0,0,1,10,1,10,1,6, - 1,6,1,6,1,122,20,95,77,111,100,117,108,101,76,111, - 99,107,46,95,95,105,110,105,116,95,95,99,1,0,0,0, - 0,0,0,0,0,0,0,0,4,0,0,0,3,0,0,0, - 67,0,0,0,115,60,0,0,0,116,0,160,1,161,0,125, - 1,124,0,106,2,125,2,116,3,160,4,124,2,161,1,125, - 3,124,3,100,0,107,8,114,36,100,1,83,0,124,3,106, - 2,125,2,124,2,124,1,107,2,114,14,100,2,83,0,113, - 14,100,0,83,0,41,3,78,70,84,41,5,114,23,0,0, - 0,218,9,103,101,116,95,105,100,101,110,116,114,26,0,0, - 0,218,12,95,98,108,111,99,107,105,110,103,95,111,110,218, - 3,103,101,116,41,4,114,30,0,0,0,90,2,109,101,218, - 3,116,105,100,114,24,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,218,12,104,97,115,95,100,101, - 97,100,108,111,99,107,66,0,0,0,115,16,0,0,0,0, - 2,8,1,6,2,10,1,8,1,4,1,6,1,8,1,122, - 24,95,77,111,100,117,108,101,76,111,99,107,46,104,97,115, - 95,100,101,97,100,108,111,99,107,99,1,0,0,0,0,0, - 0,0,0,0,0,0,2,0,0,0,9,0,0,0,67,0, - 0,0,115,178,0,0,0,116,0,160,1,161,0,125,1,124, - 0,116,2,124,1,60,0,122,148,124,0,106,3,143,110,1, - 0,124,0,106,4,100,1,107,2,115,46,124,0,106,5,124, - 1,107,2,114,84,124,1,124,0,95,5,124,0,4,0,106, - 4,100,2,55,0,2,0,95,4,87,0,53,0,81,0,82, - 0,163,0,87,0,162,86,100,3,83,0,124,0,160,6,161, - 0,114,104,116,7,100,4,124,0,22,0,131,1,130,1,124, - 0,106,8,160,9,100,5,161,1,114,130,124,0,4,0,106, - 10,100,2,55,0,2,0,95,10,87,0,53,0,81,0,82, - 0,88,0,124,0,106,8,160,9,161,0,1,0,124,0,106, - 8,160,11,161,0,1,0,113,18,87,0,53,0,116,2,124, - 1,61,0,88,0,100,6,83,0,41,7,122,185,10,32,32, - 32,32,32,32,32,32,65,99,113,117,105,114,101,32,116,104, - 101,32,109,111,100,117,108,101,32,108,111,99,107,46,32,32, - 73,102,32,97,32,112,111,116,101,110,116,105,97,108,32,100, - 101,97,100,108,111,99,107,32,105,115,32,100,101,116,101,99, - 116,101,100,44,10,32,32,32,32,32,32,32,32,97,32,95, - 68,101,97,100,108,111,99,107,69,114,114,111,114,32,105,115, - 32,114,97,105,115,101,100,46,10,32,32,32,32,32,32,32, - 32,79,116,104,101,114,119,105,115,101,44,32,116,104,101,32, - 108,111,99,107,32,105,115,32,97,108,119,97,121,115,32,97, - 99,113,117,105,114,101,100,32,97,110,100,32,84,114,117,101, - 32,105,115,32,114,101,116,117,114,110,101,100,46,10,32,32, - 32,32,32,32,32,32,114,22,0,0,0,233,1,0,0,0, - 84,122,23,100,101,97,100,108,111,99,107,32,100,101,116,101, - 99,116,101,100,32,98,121,32,37,114,70,78,41,12,114,23, - 0,0,0,114,32,0,0,0,114,33,0,0,0,114,24,0, - 0,0,114,27,0,0,0,114,26,0,0,0,114,36,0,0, - 0,114,19,0,0,0,114,25,0,0,0,218,7,97,99,113, - 117,105,114,101,114,28,0,0,0,218,7,114,101,108,101,97, - 115,101,169,2,114,30,0,0,0,114,35,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,38,0, - 0,0,78,0,0,0,115,30,0,0,0,0,6,8,1,8, - 1,2,2,8,1,20,1,6,1,14,1,18,1,8,1,12, - 1,12,1,24,2,10,1,16,2,122,19,95,77,111,100,117, - 108,101,76,111,99,107,46,97,99,113,117,105,114,101,99,1, - 0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,9, - 0,0,0,67,0,0,0,115,122,0,0,0,116,0,160,1, - 161,0,125,1,124,0,106,2,143,98,1,0,124,0,106,3, - 124,1,107,3,114,34,116,4,100,1,131,1,130,1,124,0, - 106,5,100,2,107,4,115,48,74,0,130,1,124,0,4,0, - 106,5,100,3,56,0,2,0,95,5,124,0,106,5,100,2, - 107,2,114,108,100,0,124,0,95,3,124,0,106,6,114,108, - 124,0,4,0,106,6,100,3,56,0,2,0,95,6,124,0, - 106,7,160,8,161,0,1,0,87,0,53,0,81,0,82,0, - 88,0,100,0,83,0,41,4,78,250,31,99,97,110,110,111, - 116,32,114,101,108,101,97,115,101,32,117,110,45,97,99,113, - 117,105,114,101,100,32,108,111,99,107,114,22,0,0,0,114, - 37,0,0,0,41,9,114,23,0,0,0,114,32,0,0,0, - 114,24,0,0,0,114,26,0,0,0,218,12,82,117,110,116, - 105,109,101,69,114,114,111,114,114,27,0,0,0,114,28,0, - 0,0,114,25,0,0,0,114,39,0,0,0,114,40,0,0, - 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 114,39,0,0,0,103,0,0,0,115,22,0,0,0,0,1, - 8,1,8,1,10,1,8,1,14,1,14,1,10,1,6,1, - 6,1,14,1,122,19,95,77,111,100,117,108,101,76,111,99, - 107,46,114,101,108,101,97,115,101,99,1,0,0,0,0,0, - 0,0,0,0,0,0,1,0,0,0,5,0,0,0,67,0, - 0,0,115,22,0,0,0,100,1,124,0,106,0,155,2,100, - 2,116,1,124,0,131,1,155,0,157,4,83,0,41,3,78, - 122,12,95,77,111,100,117,108,101,76,111,99,107,40,250,5, - 41,32,97,116,32,169,2,114,17,0,0,0,218,2,105,100, - 169,1,114,30,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,218,8,95,95,114,101,112,114,95,95, - 116,0,0,0,115,2,0,0,0,0,1,122,20,95,77,111, - 100,117,108,101,76,111,99,107,46,95,95,114,101,112,114,95, - 95,78,41,9,114,1,0,0,0,114,0,0,0,0,114,2, - 0,0,0,114,3,0,0,0,114,31,0,0,0,114,36,0, - 0,0,114,38,0,0,0,114,39,0,0,0,114,47,0,0, - 0,114,10,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,114,20,0,0,0,52,0,0,0,115,12, - 0,0,0,8,1,4,5,8,8,8,12,8,25,8,13,114, - 20,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, - 0,0,0,0,0,2,0,0,0,64,0,0,0,115,48,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, - 100,7,132,0,90,6,100,8,100,9,132,0,90,7,100,10, - 83,0,41,11,218,16,95,68,117,109,109,121,77,111,100,117, - 108,101,76,111,99,107,122,86,65,32,115,105,109,112,108,101, - 32,95,77,111,100,117,108,101,76,111,99,107,32,101,113,117, - 105,118,97,108,101,110,116,32,102,111,114,32,80,121,116,104, - 111,110,32,98,117,105,108,100,115,32,119,105,116,104,111,117, - 116,10,32,32,32,32,109,117,108,116,105,45,116,104,114,101, - 97,100,105,110,103,32,115,117,112,112,111,114,116,46,99,2, - 0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,2, - 0,0,0,67,0,0,0,115,16,0,0,0,124,1,124,0, - 95,0,100,1,124,0,95,1,100,0,83,0,114,21,0,0, - 0,41,2,114,17,0,0,0,114,27,0,0,0,114,29,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,31,0,0,0,124,0,0,0,115,4,0,0,0,0, - 1,6,1,122,25,95,68,117,109,109,121,77,111,100,117,108, - 101,76,111,99,107,46,95,95,105,110,105,116,95,95,99,1, - 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,3, - 0,0,0,67,0,0,0,115,18,0,0,0,124,0,4,0, - 106,0,100,1,55,0,2,0,95,0,100,2,83,0,41,3, - 78,114,37,0,0,0,84,41,1,114,27,0,0,0,114,46, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,38,0,0,0,128,0,0,0,115,4,0,0,0, - 0,1,14,1,122,24,95,68,117,109,109,121,77,111,100,117, - 108,101,76,111,99,107,46,97,99,113,117,105,114,101,99,1, - 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,3, - 0,0,0,67,0,0,0,115,36,0,0,0,124,0,106,0, - 100,1,107,2,114,18,116,1,100,2,131,1,130,1,124,0, - 4,0,106,0,100,3,56,0,2,0,95,0,100,0,83,0, - 41,4,78,114,22,0,0,0,114,41,0,0,0,114,37,0, - 0,0,41,2,114,27,0,0,0,114,42,0,0,0,114,46, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,39,0,0,0,132,0,0,0,115,6,0,0,0, - 0,1,10,1,8,1,122,24,95,68,117,109,109,121,77,111, - 100,117,108,101,76,111,99,107,46,114,101,108,101,97,115,101, - 99,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0, - 0,5,0,0,0,67,0,0,0,115,22,0,0,0,100,1, - 124,0,106,0,155,2,100,2,116,1,124,0,131,1,155,0, - 157,4,83,0,41,3,78,122,17,95,68,117,109,109,121,77, - 111,100,117,108,101,76,111,99,107,40,114,43,0,0,0,114, - 44,0,0,0,114,46,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,47,0,0,0,137,0,0, - 0,115,2,0,0,0,0,1,122,25,95,68,117,109,109,121, - 77,111,100,117,108,101,76,111,99,107,46,95,95,114,101,112, - 114,95,95,78,41,8,114,1,0,0,0,114,0,0,0,0, - 114,2,0,0,0,114,3,0,0,0,114,31,0,0,0,114, - 38,0,0,0,114,39,0,0,0,114,47,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,48,0,0,0,120,0,0,0,115,10,0,0,0, - 8,1,4,3,8,4,8,4,8,5,114,48,0,0,0,99, - 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, - 2,0,0,0,64,0,0,0,115,36,0,0,0,101,0,90, - 1,100,0,90,2,100,1,100,2,132,0,90,3,100,3,100, - 4,132,0,90,4,100,5,100,6,132,0,90,5,100,7,83, - 0,41,8,218,18,95,77,111,100,117,108,101,76,111,99,107, - 77,97,110,97,103,101,114,99,2,0,0,0,0,0,0,0, - 0,0,0,0,2,0,0,0,2,0,0,0,67,0,0,0, - 115,16,0,0,0,124,1,124,0,95,0,100,0,124,0,95, - 1,100,0,83,0,114,13,0,0,0,41,2,218,5,95,110, - 97,109,101,218,5,95,108,111,99,107,114,29,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,31, - 0,0,0,143,0,0,0,115,4,0,0,0,0,1,6,1, - 122,27,95,77,111,100,117,108,101,76,111,99,107,77,97,110, - 97,103,101,114,46,95,95,105,110,105,116,95,95,99,1,0, - 0,0,0,0,0,0,0,0,0,0,1,0,0,0,2,0, - 0,0,67,0,0,0,115,26,0,0,0,116,0,124,0,106, - 1,131,1,124,0,95,2,124,0,106,2,160,3,161,0,1, - 0,100,0,83,0,114,13,0,0,0,41,4,218,16,95,103, - 101,116,95,109,111,100,117,108,101,95,108,111,99,107,114,50, - 0,0,0,114,51,0,0,0,114,38,0,0,0,114,46,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,218,9,95,95,101,110,116,101,114,95,95,147,0,0,0, - 115,4,0,0,0,0,1,12,1,122,28,95,77,111,100,117, - 108,101,76,111,99,107,77,97,110,97,103,101,114,46,95,95, - 101,110,116,101,114,95,95,99,1,0,0,0,0,0,0,0, - 0,0,0,0,3,0,0,0,2,0,0,0,79,0,0,0, - 115,14,0,0,0,124,0,106,0,160,1,161,0,1,0,100, - 0,83,0,114,13,0,0,0,41,2,114,51,0,0,0,114, - 39,0,0,0,41,3,114,30,0,0,0,218,4,97,114,103, - 115,90,6,107,119,97,114,103,115,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,218,8,95,95,101,120,105,116, - 95,95,151,0,0,0,115,2,0,0,0,0,1,122,27,95, - 77,111,100,117,108,101,76,111,99,107,77,97,110,97,103,101, - 114,46,95,95,101,120,105,116,95,95,78,41,6,114,1,0, - 0,0,114,0,0,0,0,114,2,0,0,0,114,31,0,0, - 0,114,53,0,0,0,114,55,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 49,0,0,0,141,0,0,0,115,6,0,0,0,8,2,8, - 4,8,4,114,49,0,0,0,99,1,0,0,0,0,0,0, - 0,0,0,0,0,3,0,0,0,8,0,0,0,67,0,0, - 0,115,130,0,0,0,116,0,160,1,161,0,1,0,122,106, - 122,14,116,3,124,0,25,0,131,0,125,1,87,0,110,24, - 4,0,116,4,107,10,114,48,1,0,1,0,1,0,100,1, - 125,1,89,0,110,2,88,0,124,1,100,1,107,8,114,112, - 116,5,100,1,107,8,114,76,116,6,124,0,131,1,125,1, - 110,8,116,7,124,0,131,1,125,1,124,0,102,1,100,2, - 100,3,132,1,125,2,116,8,160,9,124,1,124,2,161,2, - 116,3,124,0,60,0,87,0,53,0,116,0,160,2,161,0, - 1,0,88,0,124,1,83,0,41,4,122,139,71,101,116,32, - 111,114,32,99,114,101,97,116,101,32,116,104,101,32,109,111, - 100,117,108,101,32,108,111,99,107,32,102,111,114,32,97,32, - 103,105,118,101,110,32,109,111,100,117,108,101,32,110,97,109, - 101,46,10,10,32,32,32,32,65,99,113,117,105,114,101,47, - 114,101,108,101,97,115,101,32,105,110,116,101,114,110,97,108, - 108,121,32,116,104,101,32,103,108,111,98,97,108,32,105,109, - 112,111,114,116,32,108,111,99,107,32,116,111,32,112,114,111, - 116,101,99,116,10,32,32,32,32,95,109,111,100,117,108,101, - 95,108,111,99,107,115,46,78,99,2,0,0,0,0,0,0, - 0,0,0,0,0,2,0,0,0,8,0,0,0,83,0,0, - 0,115,48,0,0,0,116,0,160,1,161,0,1,0,122,24, - 116,3,160,4,124,1,161,1,124,0,107,8,114,30,116,3, - 124,1,61,0,87,0,53,0,116,0,160,2,161,0,1,0, - 88,0,100,0,83,0,114,13,0,0,0,41,5,218,4,95, - 105,109,112,218,12,97,99,113,117,105,114,101,95,108,111,99, - 107,218,12,114,101,108,101,97,115,101,95,108,111,99,107,218, - 13,95,109,111,100,117,108,101,95,108,111,99,107,115,114,34, - 0,0,0,41,2,218,3,114,101,102,114,17,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,2, - 99,98,176,0,0,0,115,10,0,0,0,0,1,8,1,2, - 4,14,1,10,2,122,28,95,103,101,116,95,109,111,100,117, - 108,101,95,108,111,99,107,46,60,108,111,99,97,108,115,62, - 46,99,98,41,10,114,56,0,0,0,114,57,0,0,0,114, - 58,0,0,0,114,59,0,0,0,218,8,75,101,121,69,114, - 114,111,114,114,23,0,0,0,114,48,0,0,0,114,20,0, - 0,0,218,8,95,119,101,97,107,114,101,102,114,60,0,0, - 0,41,3,114,17,0,0,0,114,24,0,0,0,114,61,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,52,0,0,0,157,0,0,0,115,28,0,0,0,0, - 6,8,1,2,1,2,1,14,1,14,1,10,2,8,1,8, - 1,10,2,8,2,12,11,20,2,10,2,114,52,0,0,0, - 99,1,0,0,0,0,0,0,0,0,0,0,0,2,0,0, - 0,8,0,0,0,67,0,0,0,115,54,0,0,0,116,0, - 124,0,131,1,125,1,122,12,124,1,160,1,161,0,1,0, - 87,0,110,20,4,0,116,2,107,10,114,40,1,0,1,0, - 1,0,89,0,110,10,88,0,124,1,160,3,161,0,1,0, - 100,1,83,0,41,2,122,189,65,99,113,117,105,114,101,115, - 32,116,104,101,110,32,114,101,108,101,97,115,101,115,32,116, - 104,101,32,109,111,100,117,108,101,32,108,111,99,107,32,102, - 111,114,32,97,32,103,105,118,101,110,32,109,111,100,117,108, - 101,32,110,97,109,101,46,10,10,32,32,32,32,84,104,105, - 115,32,105,115,32,117,115,101,100,32,116,111,32,101,110,115, - 117,114,101,32,97,32,109,111,100,117,108,101,32,105,115,32, - 99,111,109,112,108,101,116,101,108,121,32,105,110,105,116,105, - 97,108,105,122,101,100,44,32,105,110,32,116,104,101,10,32, - 32,32,32,101,118,101,110,116,32,105,116,32,105,115,32,98, - 101,105,110,103,32,105,109,112,111,114,116,101,100,32,98,121, - 32,97,110,111,116,104,101,114,32,116,104,114,101,97,100,46, - 10,32,32,32,32,78,41,4,114,52,0,0,0,114,38,0, - 0,0,114,19,0,0,0,114,39,0,0,0,41,2,114,17, - 0,0,0,114,24,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,19,95,108,111,99,107,95,117, - 110,108,111,99,107,95,109,111,100,117,108,101,194,0,0,0, - 115,12,0,0,0,0,6,8,1,2,1,12,1,14,3,6, - 2,114,64,0,0,0,99,1,0,0,0,0,0,0,0,0, - 0,0,0,3,0,0,0,3,0,0,0,79,0,0,0,115, - 10,0,0,0,124,0,124,1,124,2,142,1,83,0,41,1, - 97,46,1,0,0,114,101,109,111,118,101,95,105,109,112,111, - 114,116,108,105,98,95,102,114,97,109,101,115,32,105,110,32, - 105,109,112,111,114,116,46,99,32,119,105,108,108,32,97,108, - 119,97,121,115,32,114,101,109,111,118,101,32,115,101,113,117, - 101,110,99,101,115,10,32,32,32,32,111,102,32,105,109,112, - 111,114,116,108,105,98,32,102,114,97,109,101,115,32,116,104, - 97,116,32,101,110,100,32,119,105,116,104,32,97,32,99,97, - 108,108,32,116,111,32,116,104,105,115,32,102,117,110,99,116, - 105,111,110,10,10,32,32,32,32,85,115,101,32,105,116,32, - 105,110,115,116,101,97,100,32,111,102,32,97,32,110,111,114, - 109,97,108,32,99,97,108,108,32,105,110,32,112,108,97,99, - 101,115,32,119,104,101,114,101,32,105,110,99,108,117,100,105, - 110,103,32,116,104,101,32,105,109,112,111,114,116,108,105,98, - 10,32,32,32,32,102,114,97,109,101,115,32,105,110,116,114, - 111,100,117,99,101,115,32,117,110,119,97,110,116,101,100,32, - 110,111,105,115,101,32,105,110,116,111,32,116,104,101,32,116, - 114,97,99,101,98,97,99,107,32,40,101,46,103,46,32,119, - 104,101,110,32,101,120,101,99,117,116,105,110,103,10,32,32, - 32,32,109,111,100,117,108,101,32,99,111,100,101,41,10,32, - 32,32,32,114,10,0,0,0,41,3,218,1,102,114,54,0, - 0,0,90,4,107,119,100,115,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,25,95,99,97,108,108,95,119, - 105,116,104,95,102,114,97,109,101,115,95,114,101,109,111,118, - 101,100,211,0,0,0,115,2,0,0,0,0,8,114,66,0, - 0,0,114,37,0,0,0,41,1,218,9,118,101,114,98,111, - 115,105,116,121,99,1,0,0,0,0,0,0,0,1,0,0, - 0,3,0,0,0,4,0,0,0,71,0,0,0,115,54,0, - 0,0,116,0,106,1,106,2,124,1,107,5,114,50,124,0, - 160,3,100,1,161,1,115,30,100,2,124,0,23,0,125,0, - 116,4,124,0,106,5,124,2,142,0,116,0,106,6,100,3, - 141,2,1,0,100,4,83,0,41,5,122,61,80,114,105,110, - 116,32,116,104,101,32,109,101,115,115,97,103,101,32,116,111, - 32,115,116,100,101,114,114,32,105,102,32,45,118,47,80,89, - 84,72,79,78,86,69,82,66,79,83,69,32,105,115,32,116, - 117,114,110,101,100,32,111,110,46,41,2,250,1,35,122,7, - 105,109,112,111,114,116,32,122,2,35,32,41,1,90,4,102, - 105,108,101,78,41,7,114,15,0,0,0,218,5,102,108,97, - 103,115,218,7,118,101,114,98,111,115,101,218,10,115,116,97, - 114,116,115,119,105,116,104,218,5,112,114,105,110,116,218,6, - 102,111,114,109,97,116,218,6,115,116,100,101,114,114,41,3, - 218,7,109,101,115,115,97,103,101,114,67,0,0,0,114,54, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,218,16,95,118,101,114,98,111,115,101,95,109,101,115, - 115,97,103,101,222,0,0,0,115,8,0,0,0,0,2,12, - 1,10,1,8,1,114,76,0,0,0,99,1,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,3, - 0,0,0,115,26,0,0,0,135,0,102,1,100,1,100,2, - 132,8,125,1,116,0,124,1,136,0,131,2,1,0,124,1, - 83,0,41,3,122,49,68,101,99,111,114,97,116,111,114,32, - 116,111,32,118,101,114,105,102,121,32,116,104,101,32,110,97, - 109,101,100,32,109,111,100,117,108,101,32,105,115,32,98,117, - 105,108,116,45,105,110,46,99,2,0,0,0,0,0,0,0, - 0,0,0,0,2,0,0,0,4,0,0,0,19,0,0,0, - 115,38,0,0,0,124,1,116,0,106,1,107,7,114,28,116, - 2,124,1,155,2,100,1,157,2,124,1,100,2,141,2,130, - 1,136,0,124,0,124,1,131,2,83,0,41,3,78,250,25, - 32,105,115,32,110,111,116,32,97,32,98,117,105,108,116,45, - 105,110,32,109,111,100,117,108,101,114,16,0,0,0,41,3, - 114,15,0,0,0,218,20,98,117,105,108,116,105,110,95,109, - 111,100,117,108,101,95,110,97,109,101,115,218,11,73,109,112, - 111,114,116,69,114,114,111,114,169,2,114,30,0,0,0,218, - 8,102,117,108,108,110,97,109,101,169,1,218,3,102,120,110, - 114,10,0,0,0,114,11,0,0,0,218,25,95,114,101,113, - 117,105,114,101,115,95,98,117,105,108,116,105,110,95,119,114, - 97,112,112,101,114,232,0,0,0,115,10,0,0,0,0,1, - 10,1,10,1,2,255,6,2,122,52,95,114,101,113,117,105, - 114,101,115,95,98,117,105,108,116,105,110,46,60,108,111,99, - 97,108,115,62,46,95,114,101,113,117,105,114,101,115,95,98, - 117,105,108,116,105,110,95,119,114,97,112,112,101,114,169,1, - 114,12,0,0,0,41,2,114,83,0,0,0,114,84,0,0, - 0,114,10,0,0,0,114,82,0,0,0,114,11,0,0,0, - 218,17,95,114,101,113,117,105,114,101,115,95,98,117,105,108, - 116,105,110,230,0,0,0,115,6,0,0,0,0,2,12,5, - 10,1,114,86,0,0,0,99,1,0,0,0,0,0,0,0, - 0,0,0,0,2,0,0,0,3,0,0,0,3,0,0,0, - 115,26,0,0,0,135,0,102,1,100,1,100,2,132,8,125, - 1,116,0,124,1,136,0,131,2,1,0,124,1,83,0,41, - 3,122,47,68,101,99,111,114,97,116,111,114,32,116,111,32, - 118,101,114,105,102,121,32,116,104,101,32,110,97,109,101,100, - 32,109,111,100,117,108,101,32,105,115,32,102,114,111,122,101, - 110,46,99,2,0,0,0,0,0,0,0,0,0,0,0,2, - 0,0,0,4,0,0,0,19,0,0,0,115,38,0,0,0, - 116,0,160,1,124,1,161,1,115,28,116,2,124,1,155,2, - 100,1,157,2,124,1,100,2,141,2,130,1,136,0,124,0, - 124,1,131,2,83,0,169,3,78,122,23,32,105,115,32,110, - 111,116,32,97,32,102,114,111,122,101,110,32,109,111,100,117, - 108,101,114,16,0,0,0,41,3,114,56,0,0,0,218,9, - 105,115,95,102,114,111,122,101,110,114,79,0,0,0,114,80, - 0,0,0,114,82,0,0,0,114,10,0,0,0,114,11,0, - 0,0,218,24,95,114,101,113,117,105,114,101,115,95,102,114, - 111,122,101,110,95,119,114,97,112,112,101,114,243,0,0,0, - 115,10,0,0,0,0,1,10,1,10,1,2,255,6,2,122, - 50,95,114,101,113,117,105,114,101,115,95,102,114,111,122,101, - 110,46,60,108,111,99,97,108,115,62,46,95,114,101,113,117, - 105,114,101,115,95,102,114,111,122,101,110,95,119,114,97,112, - 112,101,114,114,85,0,0,0,41,2,114,83,0,0,0,114, - 89,0,0,0,114,10,0,0,0,114,82,0,0,0,114,11, - 0,0,0,218,16,95,114,101,113,117,105,114,101,115,95,102, - 114,111,122,101,110,241,0,0,0,115,6,0,0,0,0,2, - 12,5,10,1,114,90,0,0,0,99,2,0,0,0,0,0, - 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, - 0,0,115,62,0,0,0,116,0,124,1,124,0,131,2,125, - 2,124,1,116,1,106,2,107,6,114,50,116,1,106,2,124, - 1,25,0,125,3,116,3,124,2,124,3,131,2,1,0,116, - 1,106,2,124,1,25,0,83,0,116,4,124,2,131,1,83, - 0,100,1,83,0,41,2,122,128,76,111,97,100,32,116,104, - 101,32,115,112,101,99,105,102,105,101,100,32,109,111,100,117, - 108,101,32,105,110,116,111,32,115,121,115,46,109,111,100,117, - 108,101,115,32,97,110,100,32,114,101,116,117,114,110,32,105, - 116,46,10,10,32,32,32,32,84,104,105,115,32,109,101,116, - 104,111,100,32,105,115,32,100,101,112,114,101,99,97,116,101, - 100,46,32,32,85,115,101,32,108,111,97,100,101,114,46,101, - 120,101,99,95,109,111,100,117,108,101,32,105,110,115,116,101, - 97,100,46,10,10,32,32,32,32,78,41,5,218,16,115,112, - 101,99,95,102,114,111,109,95,108,111,97,100,101,114,114,15, - 0,0,0,218,7,109,111,100,117,108,101,115,218,5,95,101, - 120,101,99,218,5,95,108,111,97,100,41,4,114,30,0,0, - 0,114,81,0,0,0,218,4,115,112,101,99,218,6,109,111, - 100,117,108,101,114,10,0,0,0,114,10,0,0,0,114,11, - 0,0,0,218,17,95,108,111,97,100,95,109,111,100,117,108, - 101,95,115,104,105,109,253,0,0,0,115,12,0,0,0,0, - 6,10,1,10,1,10,1,10,1,10,2,114,97,0,0,0, - 99,1,0,0,0,0,0,0,0,0,0,0,0,5,0,0, - 0,8,0,0,0,67,0,0,0,115,240,0,0,0,116,0, - 124,0,100,1,100,0,131,3,125,1,116,1,124,1,100,2, - 131,2,114,56,122,12,124,1,160,2,124,0,161,1,87,0, - 83,0,4,0,116,3,107,10,114,54,1,0,1,0,1,0, - 89,0,110,2,88,0,122,10,124,0,106,4,125,2,87,0, - 110,20,4,0,116,5,107,10,114,86,1,0,1,0,1,0, - 89,0,110,18,88,0,124,2,100,0,107,9,114,104,116,6, - 124,2,131,1,83,0,122,10,124,0,106,7,125,3,87,0, - 110,24,4,0,116,5,107,10,114,138,1,0,1,0,1,0, - 100,3,125,3,89,0,110,2,88,0,122,10,124,0,106,8, - 125,4,87,0,110,66,4,0,116,5,107,10,114,216,1,0, - 1,0,1,0,124,1,100,0,107,8,114,190,100,4,124,3, - 155,2,100,5,157,3,6,0,89,0,83,0,100,4,124,3, - 155,2,100,6,124,1,155,2,100,7,157,5,6,0,89,0, - 83,0,89,0,110,20,88,0,100,4,124,3,155,2,100,8, - 124,4,155,2,100,5,157,5,83,0,100,0,83,0,41,9, - 78,218,10,95,95,108,111,97,100,101,114,95,95,218,11,109, - 111,100,117,108,101,95,114,101,112,114,250,1,63,250,8,60, - 109,111,100,117,108,101,32,250,1,62,250,2,32,40,250,2, - 41,62,250,6,32,102,114,111,109,32,41,9,114,6,0,0, - 0,114,4,0,0,0,114,99,0,0,0,218,9,69,120,99, - 101,112,116,105,111,110,218,8,95,95,115,112,101,99,95,95, - 218,14,65,116,116,114,105,98,117,116,101,69,114,114,111,114, - 218,22,95,109,111,100,117,108,101,95,114,101,112,114,95,102, - 114,111,109,95,115,112,101,99,114,1,0,0,0,218,8,95, - 95,102,105,108,101,95,95,41,5,114,96,0,0,0,218,6, - 108,111,97,100,101,114,114,95,0,0,0,114,17,0,0,0, - 218,8,102,105,108,101,110,97,109,101,114,10,0,0,0,114, - 10,0,0,0,114,11,0,0,0,218,12,95,109,111,100,117, - 108,101,95,114,101,112,114,13,1,0,0,115,46,0,0,0, - 0,2,12,1,10,4,2,1,12,1,14,1,6,1,2,1, - 10,1,14,1,6,2,8,1,8,4,2,1,10,1,14,1, - 10,1,2,1,10,1,14,1,8,1,16,2,28,2,114,113, - 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, - 0,0,0,0,4,0,0,0,64,0,0,0,115,114,0,0, - 0,101,0,90,1,100,0,90,2,100,1,90,3,100,2,100, - 2,100,2,100,3,156,3,100,4,100,5,132,2,90,4,100, - 6,100,7,132,0,90,5,100,8,100,9,132,0,90,6,101, - 7,100,10,100,11,132,0,131,1,90,8,101,8,106,9,100, - 12,100,11,132,0,131,1,90,8,101,7,100,13,100,14,132, - 0,131,1,90,10,101,7,100,15,100,16,132,0,131,1,90, - 11,101,11,106,9,100,17,100,16,132,0,131,1,90,11,100, - 2,83,0,41,18,218,10,77,111,100,117,108,101,83,112,101, - 99,97,208,5,0,0,84,104,101,32,115,112,101,99,105,102, - 105,99,97,116,105,111,110,32,102,111,114,32,97,32,109,111, - 100,117,108,101,44,32,117,115,101,100,32,102,111,114,32,108, - 111,97,100,105,110,103,46,10,10,32,32,32,32,65,32,109, - 111,100,117,108,101,39,115,32,115,112,101,99,32,105,115,32, - 116,104,101,32,115,111,117,114,99,101,32,102,111,114,32,105, - 110,102,111,114,109,97,116,105,111,110,32,97,98,111,117,116, - 32,116,104,101,32,109,111,100,117,108,101,46,32,32,70,111, - 114,10,32,32,32,32,100,97,116,97,32,97,115,115,111,99, - 105,97,116,101,100,32,119,105,116,104,32,116,104,101,32,109, - 111,100,117,108,101,44,32,105,110,99,108,117,100,105,110,103, - 32,115,111,117,114,99,101,44,32,117,115,101,32,116,104,101, - 32,115,112,101,99,39,115,10,32,32,32,32,108,111,97,100, - 101,114,46,10,10,32,32,32,32,96,110,97,109,101,96,32, - 105,115,32,116,104,101,32,97,98,115,111,108,117,116,101,32, - 110,97,109,101,32,111,102,32,116,104,101,32,109,111,100,117, - 108,101,46,32,32,96,108,111,97,100,101,114,96,32,105,115, - 32,116,104,101,32,108,111,97,100,101,114,10,32,32,32,32, - 116,111,32,117,115,101,32,119,104,101,110,32,108,111,97,100, - 105,110,103,32,116,104,101,32,109,111,100,117,108,101,46,32, - 32,96,112,97,114,101,110,116,96,32,105,115,32,116,104,101, - 32,110,97,109,101,32,111,102,32,116,104,101,10,32,32,32, - 32,112,97,99,107,97,103,101,32,116,104,101,32,109,111,100, - 117,108,101,32,105,115,32,105,110,46,32,32,84,104,101,32, - 112,97,114,101,110,116,32,105,115,32,100,101,114,105,118,101, - 100,32,102,114,111,109,32,116,104,101,32,110,97,109,101,46, - 10,10,32,32,32,32,96,105,115,95,112,97,99,107,97,103, - 101,96,32,100,101,116,101,114,109,105,110,101,115,32,105,102, - 32,116,104,101,32,109,111,100,117,108,101,32,105,115,32,99, - 111,110,115,105,100,101,114,101,100,32,97,32,112,97,99,107, - 97,103,101,32,111,114,10,32,32,32,32,110,111,116,46,32, - 32,79,110,32,109,111,100,117,108,101,115,32,116,104,105,115, - 32,105,115,32,114,101,102,108,101,99,116,101,100,32,98,121, - 32,116,104,101,32,96,95,95,112,97,116,104,95,95,96,32, - 97,116,116,114,105,98,117,116,101,46,10,10,32,32,32,32, - 96,111,114,105,103,105,110,96,32,105,115,32,116,104,101,32, - 115,112,101,99,105,102,105,99,32,108,111,99,97,116,105,111, - 110,32,117,115,101,100,32,98,121,32,116,104,101,32,108,111, - 97,100,101,114,32,102,114,111,109,32,119,104,105,99,104,32, - 116,111,10,32,32,32,32,108,111,97,100,32,116,104,101,32, - 109,111,100,117,108,101,44,32,105,102,32,116,104,97,116,32, - 105,110,102,111,114,109,97,116,105,111,110,32,105,115,32,97, - 118,97,105,108,97,98,108,101,46,32,32,87,104,101,110,32, - 102,105,108,101,110,97,109,101,32,105,115,10,32,32,32,32, - 115,101,116,44,32,111,114,105,103,105,110,32,119,105,108,108, - 32,109,97,116,99,104,46,10,10,32,32,32,32,96,104,97, - 115,95,108,111,99,97,116,105,111,110,96,32,105,110,100,105, - 99,97,116,101,115,32,116,104,97,116,32,97,32,115,112,101, - 99,39,115,32,34,111,114,105,103,105,110,34,32,114,101,102, - 108,101,99,116,115,32,97,32,108,111,99,97,116,105,111,110, - 46,10,32,32,32,32,87,104,101,110,32,116,104,105,115,32, - 105,115,32,84,114,117,101,44,32,96,95,95,102,105,108,101, - 95,95,96,32,97,116,116,114,105,98,117,116,101,32,111,102, - 32,116,104,101,32,109,111,100,117,108,101,32,105,115,32,115, - 101,116,46,10,10,32,32,32,32,96,99,97,99,104,101,100, - 96,32,105,115,32,116,104,101,32,108,111,99,97,116,105,111, - 110,32,111,102,32,116,104,101,32,99,97,99,104,101,100,32, - 98,121,116,101,99,111,100,101,32,102,105,108,101,44,32,105, - 102,32,97,110,121,46,32,32,73,116,10,32,32,32,32,99, - 111,114,114,101,115,112,111,110,100,115,32,116,111,32,116,104, - 101,32,96,95,95,99,97,99,104,101,100,95,95,96,32,97, - 116,116,114,105,98,117,116,101,46,10,10,32,32,32,32,96, - 115,117,98,109,111,100,117,108,101,95,115,101,97,114,99,104, - 95,108,111,99,97,116,105,111,110,115,96,32,105,115,32,116, - 104,101,32,115,101,113,117,101,110,99,101,32,111,102,32,112, - 97,116,104,32,101,110,116,114,105,101,115,32,116,111,10,32, - 32,32,32,115,101,97,114,99,104,32,119,104,101,110,32,105, - 109,112,111,114,116,105,110,103,32,115,117,98,109,111,100,117, - 108,101,115,46,32,32,73,102,32,115,101,116,44,32,105,115, - 95,112,97,99,107,97,103,101,32,115,104,111,117,108,100,32, - 98,101,10,32,32,32,32,84,114,117,101,45,45,97,110,100, - 32,70,97,108,115,101,32,111,116,104,101,114,119,105,115,101, - 46,10,10,32,32,32,32,80,97,99,107,97,103,101,115,32, - 97,114,101,32,115,105,109,112,108,121,32,109,111,100,117,108, - 101,115,32,116,104,97,116,32,40,109,97,121,41,32,104,97, - 118,101,32,115,117,98,109,111,100,117,108,101,115,46,32,32, - 73,102,32,97,32,115,112,101,99,10,32,32,32,32,104,97, - 115,32,97,32,110,111,110,45,78,111,110,101,32,118,97,108, - 117,101,32,105,110,32,96,115,117,98,109,111,100,117,108,101, - 95,115,101,97,114,99,104,95,108,111,99,97,116,105,111,110, - 115,96,44,32,116,104,101,32,105,109,112,111,114,116,10,32, - 32,32,32,115,121,115,116,101,109,32,119,105,108,108,32,99, - 111,110,115,105,100,101,114,32,109,111,100,117,108,101,115,32, - 108,111,97,100,101,100,32,102,114,111,109,32,116,104,101,32, - 115,112,101,99,32,97,115,32,112,97,99,107,97,103,101,115, - 46,10,10,32,32,32,32,79,110,108,121,32,102,105,110,100, - 101,114,115,32,40,115,101,101,32,105,109,112,111,114,116,108, - 105,98,46,97,98,99,46,77,101,116,97,80,97,116,104,70, - 105,110,100,101,114,32,97,110,100,10,32,32,32,32,105,109, - 112,111,114,116,108,105,98,46,97,98,99,46,80,97,116,104, - 69,110,116,114,121,70,105,110,100,101,114,41,32,115,104,111, - 117,108,100,32,109,111,100,105,102,121,32,77,111,100,117,108, - 101,83,112,101,99,32,105,110,115,116,97,110,99,101,115,46, - 10,10,32,32,32,32,78,41,3,218,6,111,114,105,103,105, - 110,218,12,108,111,97,100,101,114,95,115,116,97,116,101,218, - 10,105,115,95,112,97,99,107,97,103,101,99,3,0,0,0, - 0,0,0,0,3,0,0,0,6,0,0,0,2,0,0,0, - 67,0,0,0,115,54,0,0,0,124,1,124,0,95,0,124, - 2,124,0,95,1,124,3,124,0,95,2,124,4,124,0,95, - 3,124,5,114,32,103,0,110,2,100,0,124,0,95,4,100, - 1,124,0,95,5,100,0,124,0,95,6,100,0,83,0,41, - 2,78,70,41,7,114,17,0,0,0,114,111,0,0,0,114, - 115,0,0,0,114,116,0,0,0,218,26,115,117,98,109,111, - 100,117,108,101,95,115,101,97,114,99,104,95,108,111,99,97, - 116,105,111,110,115,218,13,95,115,101,116,95,102,105,108,101, - 97,116,116,114,218,7,95,99,97,99,104,101,100,41,6,114, - 30,0,0,0,114,17,0,0,0,114,111,0,0,0,114,115, - 0,0,0,114,116,0,0,0,114,117,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,114,31,0,0, - 0,86,1,0,0,115,14,0,0,0,0,2,6,1,6,1, - 6,1,6,1,14,3,6,1,122,19,77,111,100,117,108,101, - 83,112,101,99,46,95,95,105,110,105,116,95,95,99,1,0, - 0,0,0,0,0,0,0,0,0,0,2,0,0,0,5,0, - 0,0,67,0,0,0,115,106,0,0,0,100,1,124,0,106, - 0,155,2,157,2,100,2,124,0,106,1,155,2,157,2,103, - 2,125,1,124,0,106,2,100,0,107,9,114,52,124,1,160, - 3,100,3,124,0,106,2,155,2,157,2,161,1,1,0,124, - 0,106,4,100,0,107,9,114,80,124,1,160,3,100,4,124, - 0,106,4,155,0,157,2,161,1,1,0,124,0,106,5,106, - 6,155,0,100,5,100,6,160,7,124,1,161,1,155,0,100, - 7,157,4,83,0,41,8,78,122,5,110,97,109,101,61,122, - 7,108,111,97,100,101,114,61,122,7,111,114,105,103,105,110, - 61,122,27,115,117,98,109,111,100,117,108,101,95,115,101,97, - 114,99,104,95,108,111,99,97,116,105,111,110,115,61,250,1, - 40,122,2,44,32,250,1,41,41,8,114,17,0,0,0,114, - 111,0,0,0,114,115,0,0,0,218,6,97,112,112,101,110, - 100,114,118,0,0,0,218,9,95,95,99,108,97,115,115,95, - 95,114,1,0,0,0,218,4,106,111,105,110,41,2,114,30, - 0,0,0,114,54,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,114,47,0,0,0,98,1,0,0, - 115,16,0,0,0,0,1,10,1,10,255,4,2,10,1,18, - 1,10,1,18,1,122,19,77,111,100,117,108,101,83,112,101, - 99,46,95,95,114,101,112,114,95,95,99,2,0,0,0,0, - 0,0,0,0,0,0,0,3,0,0,0,8,0,0,0,67, - 0,0,0,115,108,0,0,0,124,0,106,0,125,2,122,72, - 124,0,106,1,124,1,106,1,107,2,111,76,124,0,106,2, - 124,1,106,2,107,2,111,76,124,0,106,3,124,1,106,3, - 107,2,111,76,124,2,124,1,106,0,107,2,111,76,124,0, - 106,4,124,1,106,4,107,2,111,76,124,0,106,5,124,1, - 106,5,107,2,87,0,83,0,4,0,116,6,107,10,114,102, - 1,0,1,0,1,0,116,7,6,0,89,0,83,0,88,0, - 100,0,83,0,114,13,0,0,0,41,8,114,118,0,0,0, - 114,17,0,0,0,114,111,0,0,0,114,115,0,0,0,218, - 6,99,97,99,104,101,100,218,12,104,97,115,95,108,111,99, - 97,116,105,111,110,114,108,0,0,0,218,14,78,111,116,73, - 109,112,108,101,109,101,110,116,101,100,41,3,114,30,0,0, - 0,90,5,111,116,104,101,114,90,4,115,109,115,108,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,218,6,95, - 95,101,113,95,95,107,1,0,0,115,30,0,0,0,0,1, - 6,1,2,1,12,1,10,255,2,2,10,254,2,3,8,253, - 2,4,10,252,2,5,10,251,4,6,14,1,122,17,77,111, - 100,117,108,101,83,112,101,99,46,95,95,101,113,95,95,99, - 1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0, - 3,0,0,0,67,0,0,0,115,58,0,0,0,124,0,106, - 0,100,0,107,8,114,52,124,0,106,1,100,0,107,9,114, - 52,124,0,106,2,114,52,116,3,100,0,107,8,114,38,116, - 4,130,1,116,3,160,5,124,0,106,1,161,1,124,0,95, - 0,124,0,106,0,83,0,114,13,0,0,0,41,6,114,120, - 0,0,0,114,115,0,0,0,114,119,0,0,0,218,19,95, - 98,111,111,116,115,116,114,97,112,95,101,120,116,101,114,110, - 97,108,218,19,78,111,116,73,109,112,108,101,109,101,110,116, - 101,100,69,114,114,111,114,90,11,95,103,101,116,95,99,97, - 99,104,101,100,114,46,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,126,0,0,0,119,1,0, - 0,115,12,0,0,0,0,2,10,1,16,1,8,1,4,1, - 14,1,122,17,77,111,100,117,108,101,83,112,101,99,46,99, - 97,99,104,101,100,99,2,0,0,0,0,0,0,0,0,0, - 0,0,2,0,0,0,2,0,0,0,67,0,0,0,115,10, - 0,0,0,124,1,124,0,95,0,100,0,83,0,114,13,0, - 0,0,41,1,114,120,0,0,0,41,2,114,30,0,0,0, - 114,126,0,0,0,114,10,0,0,0,114,10,0,0,0,114, - 11,0,0,0,114,126,0,0,0,128,1,0,0,115,2,0, - 0,0,0,2,99,1,0,0,0,0,0,0,0,0,0,0, - 0,1,0,0,0,3,0,0,0,67,0,0,0,115,36,0, - 0,0,124,0,106,0,100,1,107,8,114,26,124,0,106,1, - 160,2,100,2,161,1,100,3,25,0,83,0,124,0,106,1, - 83,0,100,1,83,0,41,4,122,32,84,104,101,32,110,97, - 109,101,32,111,102,32,116,104,101,32,109,111,100,117,108,101, - 39,115,32,112,97,114,101,110,116,46,78,218,1,46,114,22, - 0,0,0,41,3,114,118,0,0,0,114,17,0,0,0,218, - 10,114,112,97,114,116,105,116,105,111,110,114,46,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 6,112,97,114,101,110,116,132,1,0,0,115,6,0,0,0, - 0,3,10,1,16,2,122,17,77,111,100,117,108,101,83,112, - 101,99,46,112,97,114,101,110,116,99,1,0,0,0,0,0, - 0,0,0,0,0,0,1,0,0,0,1,0,0,0,67,0, - 0,0,115,6,0,0,0,124,0,106,0,83,0,114,13,0, - 0,0,41,1,114,119,0,0,0,114,46,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,127,0, - 0,0,140,1,0,0,115,2,0,0,0,0,2,122,23,77, - 111,100,117,108,101,83,112,101,99,46,104,97,115,95,108,111, - 99,97,116,105,111,110,99,2,0,0,0,0,0,0,0,0, - 0,0,0,2,0,0,0,2,0,0,0,67,0,0,0,115, - 14,0,0,0,116,0,124,1,131,1,124,0,95,1,100,0, - 83,0,114,13,0,0,0,41,2,218,4,98,111,111,108,114, - 119,0,0,0,41,2,114,30,0,0,0,218,5,118,97,108, - 117,101,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,127,0,0,0,144,1,0,0,115,2,0,0,0,0, - 2,41,12,114,1,0,0,0,114,0,0,0,0,114,2,0, - 0,0,114,3,0,0,0,114,31,0,0,0,114,47,0,0, - 0,114,129,0,0,0,218,8,112,114,111,112,101,114,116,121, - 114,126,0,0,0,218,6,115,101,116,116,101,114,114,134,0, - 0,0,114,127,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,114,0,0,0, - 49,1,0,0,115,32,0,0,0,8,1,4,36,4,1,2, - 255,12,12,8,9,8,12,2,1,10,8,4,1,10,3,2, - 1,10,7,2,1,10,3,4,1,114,114,0,0,0,169,2, - 114,115,0,0,0,114,117,0,0,0,99,2,0,0,0,0, - 0,0,0,2,0,0,0,6,0,0,0,8,0,0,0,67, - 0,0,0,115,154,0,0,0,116,0,124,1,100,1,131,2, - 114,74,116,1,100,2,107,8,114,22,116,2,130,1,116,1, - 106,3,125,4,124,3,100,2,107,8,114,48,124,4,124,0, - 124,1,100,3,141,2,83,0,124,3,114,56,103,0,110,2, - 100,2,125,5,124,4,124,0,124,1,124,5,100,4,141,3, - 83,0,124,3,100,2,107,8,114,138,116,0,124,1,100,5, - 131,2,114,134,122,14,124,1,160,4,124,0,161,1,125,3, - 87,0,113,138,4,0,116,5,107,10,114,130,1,0,1,0, - 1,0,100,2,125,3,89,0,113,138,88,0,110,4,100,6, - 125,3,116,6,124,0,124,1,124,2,124,3,100,7,141,4, - 83,0,41,8,122,53,82,101,116,117,114,110,32,97,32,109, - 111,100,117,108,101,32,115,112,101,99,32,98,97,115,101,100, - 32,111,110,32,118,97,114,105,111,117,115,32,108,111,97,100, - 101,114,32,109,101,116,104,111,100,115,46,90,12,103,101,116, - 95,102,105,108,101,110,97,109,101,78,41,1,114,111,0,0, - 0,41,2,114,111,0,0,0,114,118,0,0,0,114,117,0, - 0,0,70,114,139,0,0,0,41,7,114,4,0,0,0,114, - 130,0,0,0,114,131,0,0,0,218,23,115,112,101,99,95, - 102,114,111,109,95,102,105,108,101,95,108,111,99,97,116,105, - 111,110,114,117,0,0,0,114,79,0,0,0,114,114,0,0, - 0,41,6,114,17,0,0,0,114,111,0,0,0,114,115,0, - 0,0,114,117,0,0,0,114,140,0,0,0,90,6,115,101, - 97,114,99,104,114,10,0,0,0,114,10,0,0,0,114,11, - 0,0,0,114,91,0,0,0,149,1,0,0,115,36,0,0, - 0,0,2,10,1,8,1,4,1,6,2,8,1,12,1,12, - 1,6,1,2,255,6,3,8,1,10,1,2,1,14,1,14, - 1,12,3,4,2,114,91,0,0,0,99,3,0,0,0,0, - 0,0,0,0,0,0,0,8,0,0,0,8,0,0,0,67, - 0,0,0,115,56,1,0,0,122,10,124,0,106,0,125,3, - 87,0,110,20,4,0,116,1,107,10,114,30,1,0,1,0, - 1,0,89,0,110,14,88,0,124,3,100,0,107,9,114,44, - 124,3,83,0,124,0,106,2,125,4,124,1,100,0,107,8, - 114,90,122,10,124,0,106,3,125,1,87,0,110,20,4,0, - 116,1,107,10,114,88,1,0,1,0,1,0,89,0,110,2, - 88,0,122,10,124,0,106,4,125,5,87,0,110,24,4,0, - 116,1,107,10,114,124,1,0,1,0,1,0,100,0,125,5, - 89,0,110,2,88,0,124,2,100,0,107,8,114,184,124,5, - 100,0,107,8,114,180,122,10,124,1,106,5,125,2,87,0, - 113,184,4,0,116,1,107,10,114,176,1,0,1,0,1,0, - 100,0,125,2,89,0,113,184,88,0,110,4,124,5,125,2, - 122,10,124,0,106,6,125,6,87,0,110,24,4,0,116,1, - 107,10,114,218,1,0,1,0,1,0,100,0,125,6,89,0, - 110,2,88,0,122,14,116,7,124,0,106,8,131,1,125,7, - 87,0,110,26,4,0,116,1,107,10,144,1,114,4,1,0, - 1,0,1,0,100,0,125,7,89,0,110,2,88,0,116,9, - 124,4,124,1,124,2,100,1,141,3,125,3,124,5,100,0, - 107,8,144,1,114,34,100,2,110,2,100,3,124,3,95,10, - 124,6,124,3,95,11,124,7,124,3,95,12,124,3,83,0, - 41,4,78,169,1,114,115,0,0,0,70,84,41,13,114,107, - 0,0,0,114,108,0,0,0,114,1,0,0,0,114,98,0, - 0,0,114,110,0,0,0,218,7,95,79,82,73,71,73,78, - 218,10,95,95,99,97,99,104,101,100,95,95,218,4,108,105, - 115,116,218,8,95,95,112,97,116,104,95,95,114,114,0,0, - 0,114,119,0,0,0,114,126,0,0,0,114,118,0,0,0, - 41,8,114,96,0,0,0,114,111,0,0,0,114,115,0,0, - 0,114,95,0,0,0,114,17,0,0,0,90,8,108,111,99, - 97,116,105,111,110,114,126,0,0,0,114,118,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,17, - 95,115,112,101,99,95,102,114,111,109,95,109,111,100,117,108, - 101,175,1,0,0,115,72,0,0,0,0,2,2,1,10,1, - 14,1,6,2,8,1,4,2,6,1,8,1,2,1,10,1, - 14,2,6,1,2,1,10,1,14,1,10,1,8,1,8,1, - 2,1,10,1,14,1,12,2,4,1,2,1,10,1,14,1, - 10,1,2,1,14,1,16,1,10,2,14,1,20,1,6,1, - 6,1,114,146,0,0,0,70,169,1,218,8,111,118,101,114, - 114,105,100,101,99,2,0,0,0,0,0,0,0,1,0,0, - 0,5,0,0,0,8,0,0,0,67,0,0,0,115,226,1, - 0,0,124,2,115,20,116,0,124,1,100,1,100,0,131,3, - 100,0,107,8,114,54,122,12,124,0,106,1,124,1,95,2, - 87,0,110,20,4,0,116,3,107,10,114,52,1,0,1,0, - 1,0,89,0,110,2,88,0,124,2,115,74,116,0,124,1, - 100,2,100,0,131,3,100,0,107,8,114,178,124,0,106,4, - 125,3,124,3,100,0,107,8,114,146,124,0,106,5,100,0, - 107,9,114,146,116,6,100,0,107,8,114,110,116,7,130,1, - 116,6,106,8,125,4,124,4,160,9,124,4,161,1,125,3, - 124,0,106,5,124,3,95,10,124,3,124,0,95,4,100,0, - 124,1,95,11,122,10,124,3,124,1,95,12,87,0,110,20, - 4,0,116,3,107,10,114,176,1,0,1,0,1,0,89,0, - 110,2,88,0,124,2,115,198,116,0,124,1,100,3,100,0, - 131,3,100,0,107,8,114,232,122,12,124,0,106,13,124,1, - 95,14,87,0,110,20,4,0,116,3,107,10,114,230,1,0, - 1,0,1,0,89,0,110,2,88,0,122,10,124,0,124,1, - 95,15,87,0,110,22,4,0,116,3,107,10,144,1,114,8, - 1,0,1,0,1,0,89,0,110,2,88,0,124,2,144,1, - 115,34,116,0,124,1,100,4,100,0,131,3,100,0,107,8, - 144,1,114,82,124,0,106,5,100,0,107,9,144,1,114,82, - 122,12,124,0,106,5,124,1,95,16,87,0,110,22,4,0, - 116,3,107,10,144,1,114,80,1,0,1,0,1,0,89,0, - 110,2,88,0,124,0,106,17,144,1,114,222,124,2,144,1, - 115,114,116,0,124,1,100,5,100,0,131,3,100,0,107,8, - 144,1,114,150,122,12,124,0,106,18,124,1,95,11,87,0, - 110,22,4,0,116,3,107,10,144,1,114,148,1,0,1,0, - 1,0,89,0,110,2,88,0,124,2,144,1,115,174,116,0, - 124,1,100,6,100,0,131,3,100,0,107,8,144,1,114,222, - 124,0,106,19,100,0,107,9,144,1,114,222,122,12,124,0, - 106,19,124,1,95,20,87,0,110,22,4,0,116,3,107,10, - 144,1,114,220,1,0,1,0,1,0,89,0,110,2,88,0, - 124,1,83,0,41,7,78,114,1,0,0,0,114,98,0,0, - 0,218,11,95,95,112,97,99,107,97,103,101,95,95,114,145, - 0,0,0,114,110,0,0,0,114,143,0,0,0,41,21,114, - 6,0,0,0,114,17,0,0,0,114,1,0,0,0,114,108, - 0,0,0,114,111,0,0,0,114,118,0,0,0,114,130,0, - 0,0,114,131,0,0,0,218,16,95,78,97,109,101,115,112, - 97,99,101,76,111,97,100,101,114,218,7,95,95,110,101,119, - 95,95,90,5,95,112,97,116,104,114,110,0,0,0,114,98, - 0,0,0,114,134,0,0,0,114,149,0,0,0,114,107,0, - 0,0,114,145,0,0,0,114,127,0,0,0,114,115,0,0, - 0,114,126,0,0,0,114,143,0,0,0,41,5,114,95,0, - 0,0,114,96,0,0,0,114,148,0,0,0,114,111,0,0, - 0,114,150,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,18,95,105,110,105,116,95,109,111,100, - 117,108,101,95,97,116,116,114,115,220,1,0,0,115,96,0, - 0,0,0,4,20,1,2,1,12,1,14,1,6,2,20,1, - 6,1,8,2,10,1,8,1,4,1,6,2,10,1,8,1, - 6,11,6,1,2,1,10,1,14,1,6,2,20,1,2,1, - 12,1,14,1,6,2,2,1,10,1,16,1,6,2,24,1, - 12,1,2,1,12,1,16,1,6,2,8,1,24,1,2,1, - 12,1,16,1,6,2,24,1,12,1,2,1,12,1,16,1, - 6,1,114,152,0,0,0,99,1,0,0,0,0,0,0,0, - 0,0,0,0,2,0,0,0,3,0,0,0,67,0,0,0, - 115,82,0,0,0,100,1,125,1,116,0,124,0,106,1,100, - 2,131,2,114,30,124,0,106,1,160,2,124,0,161,1,125, - 1,110,20,116,0,124,0,106,1,100,3,131,2,114,50,116, - 3,100,4,131,1,130,1,124,1,100,1,107,8,114,68,116, - 4,124,0,106,5,131,1,125,1,116,6,124,0,124,1,131, - 2,1,0,124,1,83,0,41,5,122,43,67,114,101,97,116, - 101,32,97,32,109,111,100,117,108,101,32,98,97,115,101,100, - 32,111,110,32,116,104,101,32,112,114,111,118,105,100,101,100, - 32,115,112,101,99,46,78,218,13,99,114,101,97,116,101,95, - 109,111,100,117,108,101,218,11,101,120,101,99,95,109,111,100, - 117,108,101,122,66,108,111,97,100,101,114,115,32,116,104,97, - 116,32,100,101,102,105,110,101,32,101,120,101,99,95,109,111, - 100,117,108,101,40,41,32,109,117,115,116,32,97,108,115,111, - 32,100,101,102,105,110,101,32,99,114,101,97,116,101,95,109, - 111,100,117,108,101,40,41,41,7,114,4,0,0,0,114,111, - 0,0,0,114,153,0,0,0,114,79,0,0,0,114,18,0, - 0,0,114,17,0,0,0,114,152,0,0,0,169,2,114,95, - 0,0,0,114,96,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,16,109,111,100,117,108,101,95, - 102,114,111,109,95,115,112,101,99,36,2,0,0,115,18,0, - 0,0,0,3,4,1,12,3,14,1,12,1,8,2,8,1, - 10,1,10,1,114,156,0,0,0,99,1,0,0,0,0,0, - 0,0,0,0,0,0,2,0,0,0,5,0,0,0,67,0, - 0,0,115,126,0,0,0,124,0,106,0,100,1,107,8,114, - 14,100,2,110,4,124,0,106,0,125,1,124,0,106,1,100, - 1,107,8,114,74,124,0,106,2,100,1,107,8,114,52,100, - 3,124,1,155,2,100,4,157,3,83,0,100,3,124,1,155, - 2,100,5,124,0,106,2,155,2,100,6,157,5,83,0,110, - 48,124,0,106,3,114,100,100,3,124,1,155,2,100,7,124, - 0,106,1,155,2,100,4,157,5,83,0,100,3,124,0,106, - 0,155,2,100,5,124,0,106,1,155,0,100,6,157,5,83, - 0,100,1,83,0,41,8,122,38,82,101,116,117,114,110,32, - 116,104,101,32,114,101,112,114,32,116,111,32,117,115,101,32, - 102,111,114,32,116,104,101,32,109,111,100,117,108,101,46,78, - 114,100,0,0,0,114,101,0,0,0,114,102,0,0,0,114, - 103,0,0,0,114,104,0,0,0,114,105,0,0,0,41,4, - 114,17,0,0,0,114,115,0,0,0,114,111,0,0,0,114, - 127,0,0,0,41,2,114,95,0,0,0,114,17,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 109,0,0,0,53,2,0,0,115,16,0,0,0,0,3,20, - 1,10,1,10,1,12,2,22,2,6,1,20,2,114,109,0, - 0,0,99,2,0,0,0,0,0,0,0,0,0,0,0,4, - 0,0,0,10,0,0,0,67,0,0,0,115,206,0,0,0, - 124,0,106,0,125,2,116,1,124,2,131,1,143,182,1,0, - 116,2,106,3,160,4,124,2,161,1,124,1,107,9,114,56, - 100,1,124,2,155,2,100,2,157,3,125,3,116,5,124,3, - 124,2,100,3,141,2,130,1,122,106,124,0,106,7,100,4, - 107,8,114,108,124,0,106,8,100,4,107,8,114,92,116,5, - 100,5,124,0,106,0,100,3,141,2,130,1,116,9,124,0, - 124,1,100,6,100,7,141,3,1,0,110,52,116,9,124,0, - 124,1,100,6,100,7,141,3,1,0,116,10,124,0,106,7, - 100,8,131,2,115,148,124,0,106,7,160,11,124,2,161,1, - 1,0,110,12,124,0,106,7,160,12,124,1,161,1,1,0, - 87,0,53,0,116,2,106,3,160,6,124,0,106,0,161,1, - 125,1,124,1,116,2,106,3,124,0,106,0,60,0,88,0, - 87,0,53,0,81,0,82,0,88,0,124,1,83,0,41,9, - 122,70,69,120,101,99,117,116,101,32,116,104,101,32,115,112, - 101,99,39,115,32,115,112,101,99,105,102,105,101,100,32,109, - 111,100,117,108,101,32,105,110,32,97,110,32,101,120,105,115, - 116,105,110,103,32,109,111,100,117,108,101,39,115,32,110,97, - 109,101,115,112,97,99,101,46,122,7,109,111,100,117,108,101, - 32,122,19,32,110,111,116,32,105,110,32,115,121,115,46,109, - 111,100,117,108,101,115,114,16,0,0,0,78,250,14,109,105, - 115,115,105,110,103,32,108,111,97,100,101,114,84,114,147,0, - 0,0,114,154,0,0,0,41,13,114,17,0,0,0,114,49, - 0,0,0,114,15,0,0,0,114,92,0,0,0,114,34,0, - 0,0,114,79,0,0,0,218,3,112,111,112,114,111,0,0, - 0,114,118,0,0,0,114,152,0,0,0,114,4,0,0,0, - 218,11,108,111,97,100,95,109,111,100,117,108,101,114,154,0, - 0,0,41,4,114,95,0,0,0,114,96,0,0,0,114,17, - 0,0,0,218,3,109,115,103,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,114,93,0,0,0,70,2,0,0, - 115,34,0,0,0,0,2,6,1,10,1,16,1,12,1,12, - 1,2,1,10,1,10,1,14,2,16,2,14,1,12,4,14, - 2,16,4,14,1,24,1,114,93,0,0,0,99,1,0,0, - 0,0,0,0,0,0,0,0,0,2,0,0,0,8,0,0, - 0,67,0,0,0,115,26,1,0,0,122,18,124,0,106,0, - 160,1,124,0,106,2,161,1,1,0,87,0,110,52,1,0, - 1,0,1,0,124,0,106,2,116,3,106,4,107,6,114,64, - 116,3,106,4,160,5,124,0,106,2,161,1,125,1,124,1, - 116,3,106,4,124,0,106,2,60,0,130,0,89,0,110,2, - 88,0,116,3,106,4,160,5,124,0,106,2,161,1,125,1, - 124,1,116,3,106,4,124,0,106,2,60,0,116,6,124,1, - 100,1,100,0,131,3,100,0,107,8,114,148,122,12,124,0, - 106,0,124,1,95,7,87,0,110,20,4,0,116,8,107,10, - 114,146,1,0,1,0,1,0,89,0,110,2,88,0,116,6, - 124,1,100,2,100,0,131,3,100,0,107,8,114,226,122,40, - 124,1,106,9,124,1,95,10,116,11,124,1,100,3,131,2, - 115,202,124,0,106,2,160,12,100,4,161,1,100,5,25,0, - 124,1,95,10,87,0,110,20,4,0,116,8,107,10,114,224, - 1,0,1,0,1,0,89,0,110,2,88,0,116,6,124,1, - 100,6,100,0,131,3,100,0,107,8,144,1,114,22,122,10, - 124,0,124,1,95,13,87,0,110,22,4,0,116,8,107,10, - 144,1,114,20,1,0,1,0,1,0,89,0,110,2,88,0, - 124,1,83,0,41,7,78,114,98,0,0,0,114,149,0,0, - 0,114,145,0,0,0,114,132,0,0,0,114,22,0,0,0, - 114,107,0,0,0,41,14,114,111,0,0,0,114,159,0,0, - 0,114,17,0,0,0,114,15,0,0,0,114,92,0,0,0, - 114,158,0,0,0,114,6,0,0,0,114,98,0,0,0,114, - 108,0,0,0,114,1,0,0,0,114,149,0,0,0,114,4, - 0,0,0,114,133,0,0,0,114,107,0,0,0,114,155,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,218,25,95,108,111,97,100,95,98,97,99,107,119,97,114, - 100,95,99,111,109,112,97,116,105,98,108,101,100,2,0,0, - 115,54,0,0,0,0,4,2,1,18,1,6,1,12,1,14, - 1,12,1,8,3,14,1,12,1,16,1,2,1,12,1,14, - 1,6,1,16,1,2,4,8,1,10,1,22,1,14,1,6, - 1,18,1,2,1,10,1,16,1,6,1,114,161,0,0,0, - 99,1,0,0,0,0,0,0,0,0,0,0,0,2,0,0, - 0,11,0,0,0,67,0,0,0,115,220,0,0,0,124,0, - 106,0,100,0,107,9,114,30,116,1,124,0,106,0,100,1, - 131,2,115,30,116,2,124,0,131,1,83,0,116,3,124,0, - 131,1,125,1,100,2,124,0,95,4,122,162,124,1,116,5, - 106,6,124,0,106,7,60,0,122,52,124,0,106,0,100,0, - 107,8,114,96,124,0,106,8,100,0,107,8,114,108,116,9, - 100,4,124,0,106,7,100,5,141,2,130,1,110,12,124,0, - 106,0,160,10,124,1,161,1,1,0,87,0,110,50,1,0, - 1,0,1,0,122,14,116,5,106,6,124,0,106,7,61,0, - 87,0,110,20,4,0,116,11,107,10,114,152,1,0,1,0, - 1,0,89,0,110,2,88,0,130,0,89,0,110,2,88,0, - 116,5,106,6,160,12,124,0,106,7,161,1,125,1,124,1, - 116,5,106,6,124,0,106,7,60,0,116,13,100,6,124,0, - 106,7,124,0,106,0,131,3,1,0,87,0,53,0,100,3, - 124,0,95,4,88,0,124,1,83,0,41,7,78,114,154,0, - 0,0,84,70,114,157,0,0,0,114,16,0,0,0,122,18, - 105,109,112,111,114,116,32,123,33,114,125,32,35,32,123,33, - 114,125,41,14,114,111,0,0,0,114,4,0,0,0,114,161, - 0,0,0,114,156,0,0,0,90,13,95,105,110,105,116,105, - 97,108,105,122,105,110,103,114,15,0,0,0,114,92,0,0, - 0,114,17,0,0,0,114,118,0,0,0,114,79,0,0,0, - 114,154,0,0,0,114,62,0,0,0,114,158,0,0,0,114, - 76,0,0,0,114,155,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,218,14,95,108,111,97,100,95, - 117,110,108,111,99,107,101,100,137,2,0,0,115,46,0,0, - 0,0,2,10,2,12,1,8,2,8,5,6,1,2,1,12, - 1,2,1,10,1,10,1,16,3,16,1,6,1,2,1,14, - 1,14,1,6,1,8,5,14,1,12,1,20,2,8,2,114, - 162,0,0,0,99,1,0,0,0,0,0,0,0,0,0,0, - 0,1,0,0,0,10,0,0,0,67,0,0,0,115,42,0, - 0,0,116,0,124,0,106,1,131,1,143,22,1,0,116,2, - 124,0,131,1,87,0,2,0,53,0,81,0,82,0,163,0, - 83,0,81,0,82,0,88,0,100,1,83,0,41,2,122,191, - 82,101,116,117,114,110,32,97,32,110,101,119,32,109,111,100, - 117,108,101,32,111,98,106,101,99,116,44,32,108,111,97,100, - 101,100,32,98,121,32,116,104,101,32,115,112,101,99,39,115, - 32,108,111,97,100,101,114,46,10,10,32,32,32,32,84,104, - 101,32,109,111,100,117,108,101,32,105,115,32,110,111,116,32, - 97,100,100,101,100,32,116,111,32,105,116,115,32,112,97,114, - 101,110,116,46,10,10,32,32,32,32,73,102,32,97,32,109, - 111,100,117,108,101,32,105,115,32,97,108,114,101,97,100,121, - 32,105,110,32,115,121,115,46,109,111,100,117,108,101,115,44, - 32,116,104,97,116,32,101,120,105,115,116,105,110,103,32,109, - 111,100,117,108,101,32,103,101,116,115,10,32,32,32,32,99, - 108,111,98,98,101,114,101,100,46,10,10,32,32,32,32,78, - 41,3,114,49,0,0,0,114,17,0,0,0,114,162,0,0, - 0,41,1,114,95,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,114,94,0,0,0,179,2,0,0, - 115,4,0,0,0,0,9,12,1,114,94,0,0,0,99,0, - 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4, - 0,0,0,64,0,0,0,115,140,0,0,0,101,0,90,1, - 100,0,90,2,100,1,90,3,100,2,90,4,101,5,100,3, - 100,4,132,0,131,1,90,6,101,7,100,20,100,6,100,7, - 132,1,131,1,90,8,101,7,100,21,100,8,100,9,132,1, - 131,1,90,9,101,7,100,10,100,11,132,0,131,1,90,10, - 101,7,100,12,100,13,132,0,131,1,90,11,101,7,101,12, - 100,14,100,15,132,0,131,1,131,1,90,13,101,7,101,12, - 100,16,100,17,132,0,131,1,131,1,90,14,101,7,101,12, - 100,18,100,19,132,0,131,1,131,1,90,15,101,7,101,16, - 131,1,90,17,100,5,83,0,41,22,218,15,66,117,105,108, - 116,105,110,73,109,112,111,114,116,101,114,122,144,77,101,116, - 97,32,112,97,116,104,32,105,109,112,111,114,116,32,102,111, - 114,32,98,117,105,108,116,45,105,110,32,109,111,100,117,108, - 101,115,46,10,10,32,32,32,32,65,108,108,32,109,101,116, - 104,111,100,115,32,97,114,101,32,101,105,116,104,101,114,32, - 99,108,97,115,115,32,111,114,32,115,116,97,116,105,99,32, - 109,101,116,104,111,100,115,32,116,111,32,97,118,111,105,100, - 32,116,104,101,32,110,101,101,100,32,116,111,10,32,32,32, - 32,105,110,115,116,97,110,116,105,97,116,101,32,116,104,101, - 32,99,108,97,115,115,46,10,10,32,32,32,32,122,8,98, - 117,105,108,116,45,105,110,99,1,0,0,0,0,0,0,0, - 0,0,0,0,1,0,0,0,5,0,0,0,67,0,0,0, - 115,22,0,0,0,100,1,124,0,106,0,155,2,100,2,116, - 1,106,2,155,0,100,3,157,5,83,0,169,4,122,115,82, - 101,116,117,114,110,32,114,101,112,114,32,102,111,114,32,116, - 104,101,32,109,111,100,117,108,101,46,10,10,32,32,32,32, - 32,32,32,32,84,104,101,32,109,101,116,104,111,100,32,105, - 115,32,100,101,112,114,101,99,97,116,101,100,46,32,32,84, - 104,101,32,105,109,112,111,114,116,32,109,97,99,104,105,110, - 101,114,121,32,100,111,101,115,32,116,104,101,32,106,111,98, - 32,105,116,115,101,108,102,46,10,10,32,32,32,32,32,32, - 32,32,114,101,0,0,0,114,103,0,0,0,114,104,0,0, - 0,41,3,114,1,0,0,0,114,163,0,0,0,114,142,0, - 0,0,41,1,114,96,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,99,0,0,0,205,2,0, - 0,115,2,0,0,0,0,7,122,27,66,117,105,108,116,105, - 110,73,109,112,111,114,116,101,114,46,109,111,100,117,108,101, - 95,114,101,112,114,78,99,4,0,0,0,0,0,0,0,0, - 0,0,0,4,0,0,0,5,0,0,0,67,0,0,0,115, - 46,0,0,0,124,2,100,0,107,9,114,12,100,0,83,0, - 116,0,160,1,124,1,161,1,114,38,116,2,124,1,124,0, - 124,0,106,3,100,1,141,3,83,0,100,0,83,0,100,0, - 83,0,169,2,78,114,141,0,0,0,41,4,114,56,0,0, - 0,90,10,105,115,95,98,117,105,108,116,105,110,114,91,0, - 0,0,114,142,0,0,0,169,4,218,3,99,108,115,114,81, - 0,0,0,218,4,112,97,116,104,218,6,116,97,114,103,101, - 116,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 218,9,102,105,110,100,95,115,112,101,99,214,2,0,0,115, - 10,0,0,0,0,2,8,1,4,1,10,1,16,2,122,25, - 66,117,105,108,116,105,110,73,109,112,111,114,116,101,114,46, - 102,105,110,100,95,115,112,101,99,99,3,0,0,0,0,0, - 0,0,0,0,0,0,4,0,0,0,4,0,0,0,67,0, - 0,0,115,30,0,0,0,124,0,160,0,124,1,124,2,161, - 2,125,3,124,3,100,1,107,9,114,26,124,3,106,1,83, - 0,100,1,83,0,41,2,122,175,70,105,110,100,32,116,104, - 101,32,98,117,105,108,116,45,105,110,32,109,111,100,117,108, - 101,46,10,10,32,32,32,32,32,32,32,32,73,102,32,39, - 112,97,116,104,39,32,105,115,32,101,118,101,114,32,115,112, - 101,99,105,102,105,101,100,32,116,104,101,110,32,116,104,101, - 32,115,101,97,114,99,104,32,105,115,32,99,111,110,115,105, - 100,101,114,101,100,32,97,32,102,97,105,108,117,114,101,46, - 10,10,32,32,32,32,32,32,32,32,84,104,105,115,32,109, - 101,116,104,111,100,32,105,115,32,100,101,112,114,101,99,97, - 116,101,100,46,32,32,85,115,101,32,102,105,110,100,95,115, - 112,101,99,40,41,32,105,110,115,116,101,97,100,46,10,10, - 32,32,32,32,32,32,32,32,78,41,2,114,170,0,0,0, - 114,111,0,0,0,41,4,114,167,0,0,0,114,81,0,0, - 0,114,168,0,0,0,114,95,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,218,11,102,105,110,100, - 95,109,111,100,117,108,101,223,2,0,0,115,4,0,0,0, - 0,9,12,1,122,27,66,117,105,108,116,105,110,73,109,112, - 111,114,116,101,114,46,102,105,110,100,95,109,111,100,117,108, - 101,99,2,0,0,0,0,0,0,0,0,0,0,0,2,0, - 0,0,4,0,0,0,67,0,0,0,115,46,0,0,0,124, - 1,106,0,116,1,106,2,107,7,114,34,116,3,124,1,106, - 0,155,2,100,1,157,2,124,1,106,0,100,2,141,2,130, - 1,116,4,116,5,106,6,124,1,131,2,83,0,41,3,122, - 24,67,114,101,97,116,101,32,97,32,98,117,105,108,116,45, - 105,110,32,109,111,100,117,108,101,114,77,0,0,0,114,16, - 0,0,0,41,7,114,17,0,0,0,114,15,0,0,0,114, - 78,0,0,0,114,79,0,0,0,114,66,0,0,0,114,56, - 0,0,0,90,14,99,114,101,97,116,101,95,98,117,105,108, - 116,105,110,41,2,114,30,0,0,0,114,95,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,153, - 0,0,0,235,2,0,0,115,10,0,0,0,0,3,12,1, - 12,1,4,255,6,2,122,29,66,117,105,108,116,105,110,73, - 109,112,111,114,116,101,114,46,99,114,101,97,116,101,95,109, - 111,100,117,108,101,99,2,0,0,0,0,0,0,0,0,0, - 0,0,2,0,0,0,3,0,0,0,67,0,0,0,115,16, - 0,0,0,116,0,116,1,106,2,124,1,131,2,1,0,100, - 1,83,0,41,2,122,22,69,120,101,99,32,97,32,98,117, - 105,108,116,45,105,110,32,109,111,100,117,108,101,78,41,3, - 114,66,0,0,0,114,56,0,0,0,90,12,101,120,101,99, - 95,98,117,105,108,116,105,110,41,2,114,30,0,0,0,114, - 96,0,0,0,114,10,0,0,0,114,10,0,0,0,114,11, - 0,0,0,114,154,0,0,0,243,2,0,0,115,2,0,0, - 0,0,3,122,27,66,117,105,108,116,105,110,73,109,112,111, - 114,116,101,114,46,101,120,101,99,95,109,111,100,117,108,101, - 99,2,0,0,0,0,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, - 83,0,41,2,122,57,82,101,116,117,114,110,32,78,111,110, - 101,32,97,115,32,98,117,105,108,116,45,105,110,32,109,111, - 100,117,108,101,115,32,100,111,32,110,111,116,32,104,97,118, - 101,32,99,111,100,101,32,111,98,106,101,99,116,115,46,78, - 114,10,0,0,0,169,2,114,167,0,0,0,114,81,0,0, - 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 218,8,103,101,116,95,99,111,100,101,248,2,0,0,115,2, - 0,0,0,0,4,122,24,66,117,105,108,116,105,110,73,109, - 112,111,114,116,101,114,46,103,101,116,95,99,111,100,101,99, - 2,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0, - 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,122,56,82,101,116,117,114,110,32,78,111,110,101, - 32,97,115,32,98,117,105,108,116,45,105,110,32,109,111,100, - 117,108,101,115,32,100,111,32,110,111,116,32,104,97,118,101, - 32,115,111,117,114,99,101,32,99,111,100,101,46,78,114,10, - 0,0,0,114,172,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,10,103,101,116,95,115,111,117, - 114,99,101,254,2,0,0,115,2,0,0,0,0,4,122,26, - 66,117,105,108,116,105,110,73,109,112,111,114,116,101,114,46, - 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,1,0,0,0,67, - 0,0,0,115,4,0,0,0,100,1,83,0,41,2,122,52, - 82,101,116,117,114,110,32,70,97,108,115,101,32,97,115,32, - 98,117,105,108,116,45,105,110,32,109,111,100,117,108,101,115, - 32,97,114,101,32,110,101,118,101,114,32,112,97,99,107,97, - 103,101,115,46,70,114,10,0,0,0,114,172,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,117, - 0,0,0,4,3,0,0,115,2,0,0,0,0,4,122,26, - 66,117,105,108,116,105,110,73,109,112,111,114,116,101,114,46, - 105,115,95,112,97,99,107,97,103,101,41,2,78,78,41,1, - 78,41,18,114,1,0,0,0,114,0,0,0,0,114,2,0, - 0,0,114,3,0,0,0,114,142,0,0,0,218,12,115,116, - 97,116,105,99,109,101,116,104,111,100,114,99,0,0,0,218, - 11,99,108,97,115,115,109,101,116,104,111,100,114,170,0,0, - 0,114,171,0,0,0,114,153,0,0,0,114,154,0,0,0, - 114,86,0,0,0,114,173,0,0,0,114,174,0,0,0,114, - 117,0,0,0,114,97,0,0,0,114,159,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,163,0,0,0,194,2,0,0,115,44,0,0,0, - 8,2,4,7,4,2,2,1,10,8,2,1,12,8,2,1, - 12,11,2,1,10,7,2,1,10,4,2,1,2,1,12,4, - 2,1,2,1,12,4,2,1,2,1,12,4,114,163,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,0,0, - 0,0,4,0,0,0,64,0,0,0,115,144,0,0,0,101, - 0,90,1,100,0,90,2,100,1,90,3,100,2,90,4,101, - 5,100,3,100,4,132,0,131,1,90,6,101,7,100,22,100, - 6,100,7,132,1,131,1,90,8,101,7,100,23,100,8,100, - 9,132,1,131,1,90,9,101,7,100,10,100,11,132,0,131, - 1,90,10,101,5,100,12,100,13,132,0,131,1,90,11,101, - 7,100,14,100,15,132,0,131,1,90,12,101,7,101,13,100, - 16,100,17,132,0,131,1,131,1,90,14,101,7,101,13,100, - 18,100,19,132,0,131,1,131,1,90,15,101,7,101,13,100, - 20,100,21,132,0,131,1,131,1,90,16,100,5,83,0,41, - 24,218,14,70,114,111,122,101,110,73,109,112,111,114,116,101, - 114,122,142,77,101,116,97,32,112,97,116,104,32,105,109,112, - 111,114,116,32,102,111,114,32,102,114,111,122,101,110,32,109, - 111,100,117,108,101,115,46,10,10,32,32,32,32,65,108,108, - 32,109,101,116,104,111,100,115,32,97,114,101,32,101,105,116, - 104,101,114,32,99,108,97,115,115,32,111,114,32,115,116,97, - 116,105,99,32,109,101,116,104,111,100,115,32,116,111,32,97, - 118,111,105,100,32,116,104,101,32,110,101,101,100,32,116,111, - 10,32,32,32,32,105,110,115,116,97,110,116,105,97,116,101, - 32,116,104,101,32,99,108,97,115,115,46,10,10,32,32,32, - 32,90,6,102,114,111,122,101,110,99,1,0,0,0,0,0, - 0,0,0,0,0,0,1,0,0,0,5,0,0,0,67,0, - 0,0,115,22,0,0,0,100,1,124,0,106,0,155,2,100, - 2,116,1,106,2,155,0,100,3,157,5,83,0,114,164,0, - 0,0,41,3,114,1,0,0,0,114,177,0,0,0,114,142, - 0,0,0,41,1,218,1,109,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,114,99,0,0,0,24,3,0,0, - 115,2,0,0,0,0,7,122,26,70,114,111,122,101,110,73, - 109,112,111,114,116,101,114,46,109,111,100,117,108,101,95,114, - 101,112,114,78,99,4,0,0,0,0,0,0,0,0,0,0, - 0,4,0,0,0,5,0,0,0,67,0,0,0,115,34,0, - 0,0,116,0,160,1,124,1,161,1,114,26,116,2,124,1, - 124,0,124,0,106,3,100,1,141,3,83,0,100,0,83,0, - 100,0,83,0,114,165,0,0,0,41,4,114,56,0,0,0, - 114,88,0,0,0,114,91,0,0,0,114,142,0,0,0,114, - 166,0,0,0,114,10,0,0,0,114,10,0,0,0,114,11, - 0,0,0,114,170,0,0,0,33,3,0,0,115,6,0,0, - 0,0,2,10,1,16,2,122,24,70,114,111,122,101,110,73, - 109,112,111,114,116,101,114,46,102,105,110,100,95,115,112,101, - 99,99,3,0,0,0,0,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,67,0,0,0,115,18,0,0,0,116, - 0,160,1,124,1,161,1,114,14,124,0,83,0,100,1,83, - 0,41,2,122,93,70,105,110,100,32,97,32,102,114,111,122, - 101,110,32,109,111,100,117,108,101,46,10,10,32,32,32,32, - 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, - 85,115,101,32,102,105,110,100,95,115,112,101,99,40,41,32, - 105,110,115,116,101,97,100,46,10,10,32,32,32,32,32,32, - 32,32,78,41,2,114,56,0,0,0,114,88,0,0,0,41, - 3,114,167,0,0,0,114,81,0,0,0,114,168,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 171,0,0,0,40,3,0,0,115,2,0,0,0,0,7,122, - 26,70,114,111,122,101,110,73,109,112,111,114,116,101,114,46, - 102,105,110,100,95,109,111,100,117,108,101,99,2,0,0,0, - 0,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, - 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, - 42,85,115,101,32,100,101,102,97,117,108,116,32,115,101,109, - 97,110,116,105,99,115,32,102,111,114,32,109,111,100,117,108, - 101,32,99,114,101,97,116,105,111,110,46,78,114,10,0,0, - 0,41,2,114,167,0,0,0,114,95,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,114,153,0,0, - 0,49,3,0,0,115,2,0,0,0,0,2,122,28,70,114, - 111,122,101,110,73,109,112,111,114,116,101,114,46,99,114,101, - 97,116,101,95,109,111,100,117,108,101,99,1,0,0,0,0, - 0,0,0,0,0,0,0,3,0,0,0,4,0,0,0,67, - 0,0,0,115,64,0,0,0,124,0,106,0,106,1,125,1, - 116,2,160,3,124,1,161,1,115,36,116,4,124,1,155,2, - 100,1,157,2,124,1,100,2,141,2,130,1,116,5,116,2, - 106,6,124,1,131,2,125,2,116,7,124,2,124,0,106,8, - 131,2,1,0,100,0,83,0,114,87,0,0,0,41,9,114, - 107,0,0,0,114,17,0,0,0,114,56,0,0,0,114,88, - 0,0,0,114,79,0,0,0,114,66,0,0,0,218,17,103, - 101,116,95,102,114,111,122,101,110,95,111,98,106,101,99,116, - 218,4,101,120,101,99,114,7,0,0,0,41,3,114,96,0, - 0,0,114,17,0,0,0,218,4,99,111,100,101,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,114,154,0,0, - 0,53,3,0,0,115,10,0,0,0,0,2,8,1,10,1, - 18,1,12,1,122,26,70,114,111,122,101,110,73,109,112,111, - 114,116,101,114,46,101,120,101,99,95,109,111,100,117,108,101, - 99,2,0,0,0,0,0,0,0,0,0,0,0,2,0,0, - 0,3,0,0,0,67,0,0,0,115,10,0,0,0,116,0, - 124,0,124,1,131,2,83,0,41,1,122,95,76,111,97,100, - 32,97,32,102,114,111,122,101,110,32,109,111,100,117,108,101, - 46,10,10,32,32,32,32,32,32,32,32,84,104,105,115,32, - 109,101,116,104,111,100,32,105,115,32,100,101,112,114,101,99, - 97,116,101,100,46,32,32,85,115,101,32,101,120,101,99,95, - 109,111,100,117,108,101,40,41,32,105,110,115,116,101,97,100, - 46,10,10,32,32,32,32,32,32,32,32,41,1,114,97,0, - 0,0,114,172,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,159,0,0,0,61,3,0,0,115, - 2,0,0,0,0,7,122,26,70,114,111,122,101,110,73,109, - 112,111,114,116,101,114,46,108,111,97,100,95,109,111,100,117, - 108,101,99,2,0,0,0,0,0,0,0,0,0,0,0,2, - 0,0,0,3,0,0,0,67,0,0,0,115,10,0,0,0, - 116,0,160,1,124,1,161,1,83,0,41,1,122,45,82,101, - 116,117,114,110,32,116,104,101,32,99,111,100,101,32,111,98, - 106,101,99,116,32,102,111,114,32,116,104,101,32,102,114,111, - 122,101,110,32,109,111,100,117,108,101,46,41,2,114,56,0, - 0,0,114,179,0,0,0,114,172,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,173,0,0,0, - 70,3,0,0,115,2,0,0,0,0,4,122,23,70,114,111, - 122,101,110,73,109,112,111,114,116,101,114,46,103,101,116,95, - 99,111,100,101,99,2,0,0,0,0,0,0,0,0,0,0, - 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, - 0,0,100,1,83,0,41,2,122,54,82,101,116,117,114,110, - 32,78,111,110,101,32,97,115,32,102,114,111,122,101,110,32, - 109,111,100,117,108,101,115,32,100,111,32,110,111,116,32,104, - 97,118,101,32,115,111,117,114,99,101,32,99,111,100,101,46, - 78,114,10,0,0,0,114,172,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,114,174,0,0,0,76, - 3,0,0,115,2,0,0,0,0,4,122,25,70,114,111,122, - 101,110,73,109,112,111,114,116,101,114,46,103,101,116,95,115, - 111,117,114,99,101,99,2,0,0,0,0,0,0,0,0,0, - 0,0,2,0,0,0,3,0,0,0,67,0,0,0,115,10, - 0,0,0,116,0,160,1,124,1,161,1,83,0,41,1,122, - 46,82,101,116,117,114,110,32,84,114,117,101,32,105,102,32, - 116,104,101,32,102,114,111,122,101,110,32,109,111,100,117,108, - 101,32,105,115,32,97,32,112,97,99,107,97,103,101,46,41, - 2,114,56,0,0,0,90,17,105,115,95,102,114,111,122,101, - 110,95,112,97,99,107,97,103,101,114,172,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,117,0, - 0,0,82,3,0,0,115,2,0,0,0,0,4,122,25,70, - 114,111,122,101,110,73,109,112,111,114,116,101,114,46,105,115, - 95,112,97,99,107,97,103,101,41,2,78,78,41,1,78,41, - 17,114,1,0,0,0,114,0,0,0,0,114,2,0,0,0, - 114,3,0,0,0,114,142,0,0,0,114,175,0,0,0,114, - 99,0,0,0,114,176,0,0,0,114,170,0,0,0,114,171, - 0,0,0,114,153,0,0,0,114,154,0,0,0,114,159,0, - 0,0,114,90,0,0,0,114,173,0,0,0,114,174,0,0, - 0,114,117,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,114,177,0,0,0,13, - 3,0,0,115,46,0,0,0,8,2,4,7,4,2,2,1, - 10,8,2,1,12,6,2,1,12,8,2,1,10,3,2,1, - 10,7,2,1,10,8,2,1,2,1,12,4,2,1,2,1, - 12,4,2,1,2,1,114,177,0,0,0,99,0,0,0,0, - 0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0, - 64,0,0,0,115,32,0,0,0,101,0,90,1,100,0,90, - 2,100,1,90,3,100,2,100,3,132,0,90,4,100,4,100, - 5,132,0,90,5,100,6,83,0,41,7,218,18,95,73,109, - 112,111,114,116,76,111,99,107,67,111,110,116,101,120,116,122, - 36,67,111,110,116,101,120,116,32,109,97,110,97,103,101,114, - 32,102,111,114,32,116,104,101,32,105,109,112,111,114,116,32, - 108,111,99,107,46,99,1,0,0,0,0,0,0,0,0,0, - 0,0,1,0,0,0,2,0,0,0,67,0,0,0,115,12, - 0,0,0,116,0,160,1,161,0,1,0,100,1,83,0,41, - 2,122,24,65,99,113,117,105,114,101,32,116,104,101,32,105, - 109,112,111,114,116,32,108,111,99,107,46,78,41,2,114,56, - 0,0,0,114,57,0,0,0,114,46,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,114,53,0,0, - 0,95,3,0,0,115,2,0,0,0,0,2,122,28,95,73, - 109,112,111,114,116,76,111,99,107,67,111,110,116,101,120,116, - 46,95,95,101,110,116,101,114,95,95,99,4,0,0,0,0, - 0,0,0,0,0,0,0,4,0,0,0,2,0,0,0,67, - 0,0,0,115,12,0,0,0,116,0,160,1,161,0,1,0, - 100,1,83,0,41,2,122,60,82,101,108,101,97,115,101,32, - 116,104,101,32,105,109,112,111,114,116,32,108,111,99,107,32, - 114,101,103,97,114,100,108,101,115,115,32,111,102,32,97,110, - 121,32,114,97,105,115,101,100,32,101,120,99,101,112,116,105, - 111,110,115,46,78,41,2,114,56,0,0,0,114,58,0,0, - 0,41,4,114,30,0,0,0,218,8,101,120,99,95,116,121, - 112,101,218,9,101,120,99,95,118,97,108,117,101,218,13,101, - 120,99,95,116,114,97,99,101,98,97,99,107,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,55,0,0,0, - 99,3,0,0,115,2,0,0,0,0,2,122,27,95,73,109, - 112,111,114,116,76,111,99,107,67,111,110,116,101,120,116,46, - 95,95,101,120,105,116,95,95,78,41,6,114,1,0,0,0, - 114,0,0,0,0,114,2,0,0,0,114,3,0,0,0,114, - 53,0,0,0,114,55,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,182,0, - 0,0,91,3,0,0,115,6,0,0,0,8,2,4,2,8, - 4,114,182,0,0,0,99,3,0,0,0,0,0,0,0,0, - 0,0,0,5,0,0,0,5,0,0,0,67,0,0,0,115, - 66,0,0,0,124,1,160,0,100,1,124,2,100,2,24,0, - 161,2,125,3,116,1,124,3,131,1,124,2,107,0,114,36, - 116,2,100,3,131,1,130,1,124,3,100,4,25,0,125,4, - 124,0,114,62,124,4,155,0,100,1,124,0,155,0,157,3, - 83,0,124,4,83,0,41,5,122,50,82,101,115,111,108,118, - 101,32,97,32,114,101,108,97,116,105,118,101,32,109,111,100, - 117,108,101,32,110,97,109,101,32,116,111,32,97,110,32,97, - 98,115,111,108,117,116,101,32,111,110,101,46,114,132,0,0, - 0,114,37,0,0,0,122,50,97,116,116,101,109,112,116,101, - 100,32,114,101,108,97,116,105,118,101,32,105,109,112,111,114, - 116,32,98,101,121,111,110,100,32,116,111,112,45,108,101,118, - 101,108,32,112,97,99,107,97,103,101,114,22,0,0,0,41, - 3,218,6,114,115,112,108,105,116,218,3,108,101,110,114,79, - 0,0,0,41,5,114,17,0,0,0,218,7,112,97,99,107, - 97,103,101,218,5,108,101,118,101,108,90,4,98,105,116,115, - 90,4,98,97,115,101,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,13,95,114,101,115,111,108,118,101,95, - 110,97,109,101,104,3,0,0,115,10,0,0,0,0,2,16, - 1,12,1,8,1,8,1,114,190,0,0,0,99,3,0,0, - 0,0,0,0,0,0,0,0,0,4,0,0,0,4,0,0, - 0,67,0,0,0,115,34,0,0,0,124,0,160,0,124,1, - 124,2,161,2,125,3,124,3,100,0,107,8,114,24,100,0, - 83,0,116,1,124,1,124,3,131,2,83,0,114,13,0,0, - 0,41,2,114,171,0,0,0,114,91,0,0,0,41,4,218, - 6,102,105,110,100,101,114,114,17,0,0,0,114,168,0,0, - 0,114,111,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,17,95,102,105,110,100,95,115,112,101, - 99,95,108,101,103,97,99,121,113,3,0,0,115,8,0,0, - 0,0,3,12,1,8,1,4,1,114,192,0,0,0,99,3, - 0,0,0,0,0,0,0,0,0,0,0,10,0,0,0,10, - 0,0,0,67,0,0,0,115,12,1,0,0,116,0,106,1, - 125,3,124,3,100,1,107,8,114,22,116,2,100,2,131,1, - 130,1,124,3,115,38,116,3,160,4,100,3,116,5,161,2, - 1,0,124,0,116,0,106,6,107,6,125,4,124,3,68,0, - 93,210,125,5,116,7,131,0,143,84,1,0,122,10,124,5, - 106,8,125,6,87,0,110,54,4,0,116,9,107,10,114,128, - 1,0,1,0,1,0,116,10,124,5,124,0,124,1,131,3, - 125,7,124,7,100,1,107,8,114,124,89,0,87,0,53,0, - 81,0,82,0,163,0,113,52,89,0,110,14,88,0,124,6, - 124,0,124,1,124,2,131,3,125,7,87,0,53,0,81,0, - 82,0,88,0,124,7,100,1,107,9,114,52,124,4,144,0, - 115,254,124,0,116,0,106,6,107,6,144,0,114,254,116,0, - 106,6,124,0,25,0,125,8,122,10,124,8,106,11,125,9, - 87,0,110,28,4,0,116,9,107,10,114,226,1,0,1,0, - 1,0,124,7,6,0,89,0,2,0,1,0,83,0,88,0, - 124,9,100,1,107,8,114,244,124,7,2,0,1,0,83,0, - 124,9,2,0,1,0,83,0,113,52,124,7,2,0,1,0, - 83,0,113,52,100,1,83,0,41,4,122,21,70,105,110,100, - 32,97,32,109,111,100,117,108,101,39,115,32,115,112,101,99, - 46,78,122,53,115,121,115,46,109,101,116,97,95,112,97,116, - 104,32,105,115,32,78,111,110,101,44,32,80,121,116,104,111, - 110,32,105,115,32,108,105,107,101,108,121,32,115,104,117,116, - 116,105,110,103,32,100,111,119,110,122,22,115,121,115,46,109, - 101,116,97,95,112,97,116,104,32,105,115,32,101,109,112,116, - 121,41,12,114,15,0,0,0,218,9,109,101,116,97,95,112, - 97,116,104,114,79,0,0,0,218,9,95,119,97,114,110,105, - 110,103,115,218,4,119,97,114,110,218,13,73,109,112,111,114, - 116,87,97,114,110,105,110,103,114,92,0,0,0,114,182,0, - 0,0,114,170,0,0,0,114,108,0,0,0,114,192,0,0, - 0,114,107,0,0,0,41,10,114,17,0,0,0,114,168,0, - 0,0,114,169,0,0,0,114,193,0,0,0,90,9,105,115, - 95,114,101,108,111,97,100,114,191,0,0,0,114,170,0,0, - 0,114,95,0,0,0,114,96,0,0,0,114,107,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 10,95,102,105,110,100,95,115,112,101,99,122,3,0,0,115, - 54,0,0,0,0,2,6,1,8,2,8,3,4,1,12,5, - 10,1,8,1,8,1,2,1,10,1,14,1,12,1,8,1, - 20,2,22,1,8,2,18,1,10,1,2,1,10,1,14,4, - 14,2,8,1,8,2,10,2,10,2,114,197,0,0,0,99, - 3,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0, - 4,0,0,0,67,0,0,0,115,108,0,0,0,116,0,124, - 0,116,1,131,2,115,28,116,2,100,1,116,3,124,0,131, - 1,155,0,157,2,131,1,130,1,124,2,100,2,107,0,114, - 44,116,4,100,3,131,1,130,1,124,2,100,2,107,4,114, - 84,116,0,124,1,116,1,131,2,115,72,116,2,100,4,131, - 1,130,1,110,12,124,1,115,84,116,5,100,5,131,1,130, - 1,124,0,115,104,124,2,100,2,107,2,114,104,116,4,100, - 6,131,1,130,1,100,7,83,0,41,8,122,28,86,101,114, - 105,102,121,32,97,114,103,117,109,101,110,116,115,32,97,114, - 101,32,34,115,97,110,101,34,46,122,29,109,111,100,117,108, - 101,32,110,97,109,101,32,109,117,115,116,32,98,101,32,115, - 116,114,44,32,110,111,116,32,114,22,0,0,0,122,18,108, - 101,118,101,108,32,109,117,115,116,32,98,101,32,62,61,32, - 48,122,31,95,95,112,97,99,107,97,103,101,95,95,32,110, - 111,116,32,115,101,116,32,116,111,32,97,32,115,116,114,105, - 110,103,122,54,97,116,116,101,109,112,116,101,100,32,114,101, - 108,97,116,105,118,101,32,105,109,112,111,114,116,32,119,105, - 116,104,32,110,111,32,107,110,111,119,110,32,112,97,114,101, - 110,116,32,112,97,99,107,97,103,101,122,17,69,109,112,116, - 121,32,109,111,100,117,108,101,32,110,97,109,101,78,41,6, - 218,10,105,115,105,110,115,116,97,110,99,101,218,3,115,116, - 114,218,9,84,121,112,101,69,114,114,111,114,114,14,0,0, - 0,218,10,86,97,108,117,101,69,114,114,111,114,114,79,0, - 0,0,169,3,114,17,0,0,0,114,188,0,0,0,114,189, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,218,13,95,115,97,110,105,116,121,95,99,104,101,99, - 107,169,3,0,0,115,22,0,0,0,0,2,10,1,18,1, - 8,1,8,1,8,1,10,1,10,1,4,1,8,2,12,1, - 114,203,0,0,0,250,16,78,111,32,109,111,100,117,108,101, - 32,110,97,109,101,100,32,122,4,123,33,114,125,99,2,0, - 0,0,0,0,0,0,0,0,0,0,8,0,0,0,8,0, - 0,0,67,0,0,0,115,222,0,0,0,100,0,125,2,124, - 0,160,0,100,1,161,1,100,2,25,0,125,3,124,3,114, - 136,124,3,116,1,106,2,107,7,114,42,116,3,124,1,124, - 3,131,2,1,0,124,0,116,1,106,2,107,6,114,62,116, - 1,106,2,124,0,25,0,83,0,116,1,106,2,124,3,25, - 0,125,4,122,10,124,4,106,4,125,2,87,0,110,52,4, - 0,116,5,107,10,114,134,1,0,1,0,1,0,100,3,124, - 0,155,2,100,4,124,3,155,2,100,5,157,5,125,5,116, - 6,124,5,124,0,100,6,141,2,100,0,130,2,89,0,110, - 2,88,0,116,7,124,0,124,2,131,2,125,6,124,6,100, - 0,107,8,114,174,116,6,100,3,124,0,155,2,157,2,124, - 0,100,6,141,2,130,1,110,8,116,8,124,6,131,1,125, - 7,124,3,114,218,116,1,106,2,124,3,25,0,125,4,116, - 9,124,4,124,0,160,0,100,1,161,1,100,7,25,0,124, - 7,131,3,1,0,124,7,83,0,41,8,78,114,132,0,0, - 0,114,22,0,0,0,114,204,0,0,0,122,2,59,32,122, - 17,32,105,115,32,110,111,116,32,97,32,112,97,99,107,97, - 103,101,114,16,0,0,0,233,2,0,0,0,41,10,114,133, - 0,0,0,114,15,0,0,0,114,92,0,0,0,114,66,0, - 0,0,114,145,0,0,0,114,108,0,0,0,218,19,77,111, - 100,117,108,101,78,111,116,70,111,117,110,100,69,114,114,111, - 114,114,197,0,0,0,114,162,0,0,0,114,5,0,0,0, - 41,8,114,17,0,0,0,218,7,105,109,112,111,114,116,95, - 114,168,0,0,0,114,134,0,0,0,90,13,112,97,114,101, - 110,116,95,109,111,100,117,108,101,114,160,0,0,0,114,95, - 0,0,0,114,96,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,23,95,102,105,110,100,95,97, - 110,100,95,108,111,97,100,95,117,110,108,111,99,107,101,100, - 188,3,0,0,115,42,0,0,0,0,1,4,1,14,1,4, - 1,10,1,10,2,10,1,10,1,10,1,2,1,10,1,14, - 1,18,1,20,1,10,1,8,1,20,2,8,1,4,2,10, - 1,22,1,114,208,0,0,0,99,2,0,0,0,0,0,0, - 0,0,0,0,0,4,0,0,0,10,0,0,0,67,0,0, - 0,115,108,0,0,0,116,0,124,0,131,1,143,50,1,0, - 116,1,106,2,160,3,124,0,116,4,161,2,125,2,124,2, - 116,4,107,8,114,54,116,5,124,0,124,1,131,2,87,0, - 2,0,53,0,81,0,82,0,163,0,83,0,87,0,53,0, - 81,0,82,0,88,0,124,2,100,1,107,8,114,96,100,2, - 124,0,155,0,100,3,157,3,125,3,116,6,124,3,124,0, - 100,4,141,2,130,1,116,7,124,0,131,1,1,0,124,2, - 83,0,41,5,122,25,70,105,110,100,32,97,110,100,32,108, - 111,97,100,32,116,104,101,32,109,111,100,117,108,101,46,78, - 122,10,105,109,112,111,114,116,32,111,102,32,122,28,32,104, - 97,108,116,101,100,59,32,78,111,110,101,32,105,110,32,115, - 121,115,46,109,111,100,117,108,101,115,114,16,0,0,0,41, - 8,114,49,0,0,0,114,15,0,0,0,114,92,0,0,0, - 114,34,0,0,0,218,14,95,78,69,69,68,83,95,76,79, - 65,68,73,78,71,114,208,0,0,0,114,206,0,0,0,114, - 64,0,0,0,41,4,114,17,0,0,0,114,207,0,0,0, - 114,96,0,0,0,114,75,0,0,0,114,10,0,0,0,114, - 10,0,0,0,114,11,0,0,0,218,14,95,102,105,110,100, - 95,97,110,100,95,108,111,97,100,218,3,0,0,115,18,0, - 0,0,0,2,10,1,14,1,8,1,32,2,8,1,12,2, - 12,2,8,1,114,210,0,0,0,114,22,0,0,0,99,3, - 0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,4, - 0,0,0,67,0,0,0,115,42,0,0,0,116,0,124,0, - 124,1,124,2,131,3,1,0,124,2,100,1,107,4,114,32, - 116,1,124,0,124,1,124,2,131,3,125,0,116,2,124,0, - 116,3,131,2,83,0,41,2,97,50,1,0,0,73,109,112, - 111,114,116,32,97,110,100,32,114,101,116,117,114,110,32,116, - 104,101,32,109,111,100,117,108,101,32,98,97,115,101,100,32, - 111,110,32,105,116,115,32,110,97,109,101,44,32,116,104,101, - 32,112,97,99,107,97,103,101,32,116,104,101,32,99,97,108, - 108,32,105,115,10,32,32,32,32,98,101,105,110,103,32,109, - 97,100,101,32,102,114,111,109,44,32,97,110,100,32,116,104, - 101,32,108,101,118,101,108,32,97,100,106,117,115,116,109,101, - 110,116,46,10,10,32,32,32,32,84,104,105,115,32,102,117, - 110,99,116,105,111,110,32,114,101,112,114,101,115,101,110,116, - 115,32,116,104,101,32,103,114,101,97,116,101,115,116,32,99, - 111,109,109,111,110,32,100,101,110,111,109,105,110,97,116,111, - 114,32,111,102,32,102,117,110,99,116,105,111,110,97,108,105, - 116,121,10,32,32,32,32,98,101,116,119,101,101,110,32,105, - 109,112,111,114,116,95,109,111,100,117,108,101,32,97,110,100, - 32,95,95,105,109,112,111,114,116,95,95,46,32,84,104,105, - 115,32,105,110,99,108,117,100,101,115,32,115,101,116,116,105, - 110,103,32,95,95,112,97,99,107,97,103,101,95,95,32,105, - 102,10,32,32,32,32,116,104,101,32,108,111,97,100,101,114, - 32,100,105,100,32,110,111,116,46,10,10,32,32,32,32,114, - 22,0,0,0,41,4,114,203,0,0,0,114,190,0,0,0, - 114,210,0,0,0,218,11,95,103,99,100,95,105,109,112,111, - 114,116,114,202,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,211,0,0,0,234,3,0,0,115, - 8,0,0,0,0,9,12,1,8,1,12,1,114,211,0,0, - 0,169,1,218,9,114,101,99,117,114,115,105,118,101,99,3, - 0,0,0,0,0,0,0,1,0,0,0,8,0,0,0,11, - 0,0,0,67,0,0,0,115,228,0,0,0,124,1,68,0, - 93,218,125,4,116,0,124,4,116,1,131,2,115,66,124,3, - 114,34,124,0,106,2,100,1,23,0,125,5,110,4,100,2, - 125,5,116,3,100,3,124,5,155,0,100,4,116,4,124,4, - 131,1,106,2,155,0,157,4,131,1,130,1,113,4,124,4, - 100,5,107,2,114,108,124,3,115,222,116,5,124,0,100,6, - 131,2,114,222,116,6,124,0,124,0,106,7,124,2,100,7, - 100,8,141,4,1,0,113,4,116,5,124,0,124,4,131,2, - 115,4,124,0,106,2,155,0,100,9,124,4,155,0,157,3, - 125,6,122,14,116,8,124,2,124,6,131,2,1,0,87,0, - 113,4,4,0,116,9,107,10,114,220,1,0,125,7,1,0, - 122,42,124,7,106,10,124,6,107,2,114,202,116,11,106,12, - 160,13,124,6,116,14,161,2,100,10,107,9,114,202,87,0, - 89,0,162,8,113,4,130,0,87,0,53,0,100,10,125,7, - 126,7,88,0,89,0,113,4,88,0,113,4,124,0,83,0, - 41,11,122,238,70,105,103,117,114,101,32,111,117,116,32,119, - 104,97,116,32,95,95,105,109,112,111,114,116,95,95,32,115, - 104,111,117,108,100,32,114,101,116,117,114,110,46,10,10,32, - 32,32,32,84,104,101,32,105,109,112,111,114,116,95,32,112, - 97,114,97,109,101,116,101,114,32,105,115,32,97,32,99,97, - 108,108,97,98,108,101,32,119,104,105,99,104,32,116,97,107, - 101,115,32,116,104,101,32,110,97,109,101,32,111,102,32,109, - 111,100,117,108,101,32,116,111,10,32,32,32,32,105,109,112, - 111,114,116,46,32,73,116,32,105,115,32,114,101,113,117,105, - 114,101,100,32,116,111,32,100,101,99,111,117,112,108,101,32, - 116,104,101,32,102,117,110,99,116,105,111,110,32,102,114,111, - 109,32,97,115,115,117,109,105,110,103,32,105,109,112,111,114, - 116,108,105,98,39,115,10,32,32,32,32,105,109,112,111,114, - 116,32,105,109,112,108,101,109,101,110,116,97,116,105,111,110, - 32,105,115,32,100,101,115,105,114,101,100,46,10,10,32,32, - 32,32,122,8,46,95,95,97,108,108,95,95,122,13,96,96, - 102,114,111,109,32,108,105,115,116,39,39,122,8,73,116,101, - 109,32,105,110,32,122,18,32,109,117,115,116,32,98,101,32, - 115,116,114,44,32,110,111,116,32,250,1,42,218,7,95,95, - 97,108,108,95,95,84,114,212,0,0,0,114,132,0,0,0, - 78,41,15,114,198,0,0,0,114,199,0,0,0,114,1,0, - 0,0,114,200,0,0,0,114,14,0,0,0,114,4,0,0, - 0,218,16,95,104,97,110,100,108,101,95,102,114,111,109,108, - 105,115,116,114,215,0,0,0,114,66,0,0,0,114,206,0, - 0,0,114,17,0,0,0,114,15,0,0,0,114,92,0,0, - 0,114,34,0,0,0,114,209,0,0,0,41,8,114,96,0, - 0,0,218,8,102,114,111,109,108,105,115,116,114,207,0,0, - 0,114,213,0,0,0,218,1,120,90,5,119,104,101,114,101, - 90,9,102,114,111,109,95,110,97,109,101,90,3,101,120,99, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 216,0,0,0,249,3,0,0,115,44,0,0,0,0,10,8, - 1,10,1,4,1,12,2,4,1,28,2,8,1,14,1,10, - 1,2,255,8,2,10,1,16,1,2,1,14,1,16,4,10, - 1,16,255,2,2,8,1,22,1,114,216,0,0,0,99,1, - 0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,6, - 0,0,0,67,0,0,0,115,146,0,0,0,124,0,160,0, - 100,1,161,1,125,1,124,0,160,0,100,2,161,1,125,2, - 124,1,100,3,107,9,114,82,124,2,100,3,107,9,114,78, - 124,1,124,2,106,1,107,3,114,78,116,2,106,3,100,4, - 124,1,155,2,100,5,124,2,106,1,155,2,100,6,157,5, - 116,4,100,7,100,8,141,3,1,0,124,1,83,0,124,2, - 100,3,107,9,114,96,124,2,106,1,83,0,116,2,106,3, - 100,9,116,4,100,7,100,8,141,3,1,0,124,0,100,10, - 25,0,125,1,100,11,124,0,107,7,114,142,124,1,160,5, - 100,12,161,1,100,13,25,0,125,1,124,1,83,0,41,14, - 122,167,67,97,108,99,117,108,97,116,101,32,119,104,97,116, - 32,95,95,112,97,99,107,97,103,101,95,95,32,115,104,111, - 117,108,100,32,98,101,46,10,10,32,32,32,32,95,95,112, - 97,99,107,97,103,101,95,95,32,105,115,32,110,111,116,32, - 103,117,97,114,97,110,116,101,101,100,32,116,111,32,98,101, - 32,100,101,102,105,110,101,100,32,111,114,32,99,111,117,108, - 100,32,98,101,32,115,101,116,32,116,111,32,78,111,110,101, - 10,32,32,32,32,116,111,32,114,101,112,114,101,115,101,110, - 116,32,116,104,97,116,32,105,116,115,32,112,114,111,112,101, - 114,32,118,97,108,117,101,32,105,115,32,117,110,107,110,111, - 119,110,46,10,10,32,32,32,32,114,149,0,0,0,114,107, - 0,0,0,78,122,32,95,95,112,97,99,107,97,103,101,95, - 95,32,33,61,32,95,95,115,112,101,99,95,95,46,112,97, - 114,101,110,116,32,40,122,4,32,33,61,32,114,122,0,0, - 0,233,3,0,0,0,41,1,90,10,115,116,97,99,107,108, - 101,118,101,108,122,89,99,97,110,39,116,32,114,101,115,111, - 108,118,101,32,112,97,99,107,97,103,101,32,102,114,111,109, - 32,95,95,115,112,101,99,95,95,32,111,114,32,95,95,112, - 97,99,107,97,103,101,95,95,44,32,102,97,108,108,105,110, - 103,32,98,97,99,107,32,111,110,32,95,95,110,97,109,101, - 95,95,32,97,110,100,32,95,95,112,97,116,104,95,95,114, - 1,0,0,0,114,145,0,0,0,114,132,0,0,0,114,22, - 0,0,0,41,6,114,34,0,0,0,114,134,0,0,0,114, - 194,0,0,0,114,195,0,0,0,114,196,0,0,0,114,133, - 0,0,0,41,3,218,7,103,108,111,98,97,108,115,114,188, - 0,0,0,114,95,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,17,95,99,97,108,99,95,95, - 95,112,97,99,107,97,103,101,95,95,30,4,0,0,115,38, - 0,0,0,0,7,10,1,10,1,8,1,18,1,22,2,2, - 0,2,254,6,3,4,1,8,1,6,2,6,2,2,0,2, - 254,6,3,8,1,8,1,14,1,114,221,0,0,0,114,10, - 0,0,0,99,5,0,0,0,0,0,0,0,0,0,0,0, - 9,0,0,0,5,0,0,0,67,0,0,0,115,180,0,0, - 0,124,4,100,1,107,2,114,18,116,0,124,0,131,1,125, - 5,110,36,124,1,100,2,107,9,114,30,124,1,110,2,105, - 0,125,6,116,1,124,6,131,1,125,7,116,0,124,0,124, - 7,124,4,131,3,125,5,124,3,115,150,124,4,100,1,107, - 2,114,84,116,0,124,0,160,2,100,3,161,1,100,1,25, - 0,131,1,83,0,124,0,115,92,124,5,83,0,116,3,124, - 0,131,1,116,3,124,0,160,2,100,3,161,1,100,1,25, - 0,131,1,24,0,125,8,116,4,106,5,124,5,106,6,100, - 2,116,3,124,5,106,6,131,1,124,8,24,0,133,2,25, - 0,25,0,83,0,110,26,116,7,124,5,100,4,131,2,114, - 172,116,8,124,5,124,3,116,0,131,3,83,0,124,5,83, - 0,100,2,83,0,41,5,97,215,1,0,0,73,109,112,111, - 114,116,32,97,32,109,111,100,117,108,101,46,10,10,32,32, - 32,32,84,104,101,32,39,103,108,111,98,97,108,115,39,32, - 97,114,103,117,109,101,110,116,32,105,115,32,117,115,101,100, - 32,116,111,32,105,110,102,101,114,32,119,104,101,114,101,32, - 116,104,101,32,105,109,112,111,114,116,32,105,115,32,111,99, - 99,117,114,114,105,110,103,32,102,114,111,109,10,32,32,32, - 32,116,111,32,104,97,110,100,108,101,32,114,101,108,97,116, - 105,118,101,32,105,109,112,111,114,116,115,46,32,84,104,101, - 32,39,108,111,99,97,108,115,39,32,97,114,103,117,109,101, - 110,116,32,105,115,32,105,103,110,111,114,101,100,46,32,84, - 104,101,10,32,32,32,32,39,102,114,111,109,108,105,115,116, - 39,32,97,114,103,117,109,101,110,116,32,115,112,101,99,105, - 102,105,101,115,32,119,104,97,116,32,115,104,111,117,108,100, - 32,101,120,105,115,116,32,97,115,32,97,116,116,114,105,98, - 117,116,101,115,32,111,110,32,116,104,101,32,109,111,100,117, - 108,101,10,32,32,32,32,98,101,105,110,103,32,105,109,112, - 111,114,116,101,100,32,40,101,46,103,46,32,96,96,102,114, - 111,109,32,109,111,100,117,108,101,32,105,109,112,111,114,116, - 32,60,102,114,111,109,108,105,115,116,62,96,96,41,46,32, - 32,84,104,101,32,39,108,101,118,101,108,39,10,32,32,32, - 32,97,114,103,117,109,101,110,116,32,114,101,112,114,101,115, - 101,110,116,115,32,116,104,101,32,112,97,99,107,97,103,101, - 32,108,111,99,97,116,105,111,110,32,116,111,32,105,109,112, - 111,114,116,32,102,114,111,109,32,105,110,32,97,32,114,101, - 108,97,116,105,118,101,10,32,32,32,32,105,109,112,111,114, - 116,32,40,101,46,103,46,32,96,96,102,114,111,109,32,46, - 46,112,107,103,32,105,109,112,111,114,116,32,109,111,100,96, - 96,32,119,111,117,108,100,32,104,97,118,101,32,97,32,39, - 108,101,118,101,108,39,32,111,102,32,50,41,46,10,10,32, - 32,32,32,114,22,0,0,0,78,114,132,0,0,0,114,145, - 0,0,0,41,9,114,211,0,0,0,114,221,0,0,0,218, - 9,112,97,114,116,105,116,105,111,110,114,187,0,0,0,114, - 15,0,0,0,114,92,0,0,0,114,1,0,0,0,114,4, - 0,0,0,114,216,0,0,0,41,9,114,17,0,0,0,114, - 220,0,0,0,218,6,108,111,99,97,108,115,114,217,0,0, - 0,114,189,0,0,0,114,96,0,0,0,90,8,103,108,111, - 98,97,108,115,95,114,188,0,0,0,90,7,99,117,116,95, - 111,102,102,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,218,10,95,95,105,109,112,111,114,116,95,95,57,4, - 0,0,115,30,0,0,0,0,11,8,1,10,2,16,1,8, - 1,12,1,4,3,8,1,18,1,4,1,4,4,26,3,32, - 1,10,1,12,2,114,224,0,0,0,99,1,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,67, - 0,0,0,115,38,0,0,0,116,0,160,1,124,0,161,1, - 125,1,124,1,100,0,107,8,114,30,116,2,100,1,124,0, - 23,0,131,1,130,1,116,3,124,1,131,1,83,0,41,2, - 78,122,25,110,111,32,98,117,105,108,116,45,105,110,32,109, - 111,100,117,108,101,32,110,97,109,101,100,32,41,4,114,163, - 0,0,0,114,170,0,0,0,114,79,0,0,0,114,162,0, - 0,0,41,2,114,17,0,0,0,114,95,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,218,18,95, - 98,117,105,108,116,105,110,95,102,114,111,109,95,110,97,109, - 101,94,4,0,0,115,8,0,0,0,0,1,10,1,8,1, - 12,1,114,225,0,0,0,99,2,0,0,0,0,0,0,0, - 0,0,0,0,10,0,0,0,5,0,0,0,67,0,0,0, - 115,166,0,0,0,124,1,97,0,124,0,97,1,116,2,116, - 1,131,1,125,2,116,1,106,3,160,4,161,0,68,0,93, - 72,92,2,125,3,125,4,116,5,124,4,124,2,131,2,114, - 26,124,3,116,1,106,6,107,6,114,60,116,7,125,5,110, - 18,116,0,160,8,124,3,161,1,114,26,116,9,125,5,110, - 2,113,26,116,10,124,4,124,5,131,2,125,6,116,11,124, - 6,124,4,131,2,1,0,113,26,116,1,106,3,116,12,25, - 0,125,7,100,1,68,0,93,46,125,8,124,8,116,1,106, - 3,107,7,114,138,116,13,124,8,131,1,125,9,110,10,116, - 1,106,3,124,8,25,0,125,9,116,14,124,7,124,8,124, - 9,131,3,1,0,113,114,100,2,83,0,41,3,122,250,83, - 101,116,117,112,32,105,109,112,111,114,116,108,105,98,32,98, - 121,32,105,109,112,111,114,116,105,110,103,32,110,101,101,100, - 101,100,32,98,117,105,108,116,45,105,110,32,109,111,100,117, - 108,101,115,32,97,110,100,32,105,110,106,101,99,116,105,110, - 103,32,116,104,101,109,10,32,32,32,32,105,110,116,111,32, - 116,104,101,32,103,108,111,98,97,108,32,110,97,109,101,115, - 112,97,99,101,46,10,10,32,32,32,32,65,115,32,115,121, - 115,32,105,115,32,110,101,101,100,101,100,32,102,111,114,32, - 115,121,115,46,109,111,100,117,108,101,115,32,97,99,99,101, - 115,115,32,97,110,100,32,95,105,109,112,32,105,115,32,110, - 101,101,100,101,100,32,116,111,32,108,111,97,100,32,98,117, - 105,108,116,45,105,110,10,32,32,32,32,109,111,100,117,108, - 101,115,44,32,116,104,111,115,101,32,116,119,111,32,109,111, - 100,117,108,101,115,32,109,117,115,116,32,98,101,32,101,120, - 112,108,105,99,105,116,108,121,32,112,97,115,115,101,100,32, - 105,110,46,10,10,32,32,32,32,41,3,114,23,0,0,0, - 114,194,0,0,0,114,63,0,0,0,78,41,15,114,56,0, - 0,0,114,15,0,0,0,114,14,0,0,0,114,92,0,0, - 0,218,5,105,116,101,109,115,114,198,0,0,0,114,78,0, - 0,0,114,163,0,0,0,114,88,0,0,0,114,177,0,0, - 0,114,146,0,0,0,114,152,0,0,0,114,1,0,0,0, - 114,225,0,0,0,114,5,0,0,0,41,10,218,10,115,121, - 115,95,109,111,100,117,108,101,218,11,95,105,109,112,95,109, - 111,100,117,108,101,90,11,109,111,100,117,108,101,95,116,121, - 112,101,114,17,0,0,0,114,96,0,0,0,114,111,0,0, - 0,114,95,0,0,0,90,11,115,101,108,102,95,109,111,100, - 117,108,101,90,12,98,117,105,108,116,105,110,95,110,97,109, - 101,90,14,98,117,105,108,116,105,110,95,109,111,100,117,108, - 101,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 218,6,95,115,101,116,117,112,101,4,0,0,115,36,0,0, - 0,0,9,4,1,4,3,8,1,18,1,10,1,10,1,6, - 1,10,1,6,2,2,1,10,1,12,3,10,1,8,1,10, - 1,10,2,10,1,114,229,0,0,0,99,2,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,67, - 0,0,0,115,38,0,0,0,116,0,124,0,124,1,131,2, - 1,0,116,1,106,2,160,3,116,4,161,1,1,0,116,1, - 106,2,160,3,116,5,161,1,1,0,100,1,83,0,41,2, - 122,48,73,110,115,116,97,108,108,32,105,109,112,111,114,116, - 101,114,115,32,102,111,114,32,98,117,105,108,116,105,110,32, - 97,110,100,32,102,114,111,122,101,110,32,109,111,100,117,108, - 101,115,78,41,6,114,229,0,0,0,114,15,0,0,0,114, - 193,0,0,0,114,123,0,0,0,114,163,0,0,0,114,177, - 0,0,0,41,2,114,227,0,0,0,114,228,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,8, - 95,105,110,115,116,97,108,108,136,4,0,0,115,6,0,0, - 0,0,2,10,2,12,1,114,230,0,0,0,99,0,0,0, - 0,0,0,0,0,0,0,0,0,1,0,0,0,4,0,0, - 0,67,0,0,0,115,32,0,0,0,100,1,100,2,108,0, - 125,0,124,0,97,1,124,0,160,2,116,3,106,4,116,5, - 25,0,161,1,1,0,100,2,83,0,41,3,122,57,73,110, - 115,116,97,108,108,32,105,109,112,111,114,116,101,114,115,32, - 116,104,97,116,32,114,101,113,117,105,114,101,32,101,120,116, - 101,114,110,97,108,32,102,105,108,101,115,121,115,116,101,109, - 32,97,99,99,101,115,115,114,22,0,0,0,78,41,6,218, - 26,95,102,114,111,122,101,110,95,105,109,112,111,114,116,108, - 105,98,95,101,120,116,101,114,110,97,108,114,130,0,0,0, - 114,230,0,0,0,114,15,0,0,0,114,92,0,0,0,114, - 1,0,0,0,41,1,114,231,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,218,27,95,105,110,115, - 116,97,108,108,95,101,120,116,101,114,110,97,108,95,105,109, - 112,111,114,116,101,114,115,144,4,0,0,115,6,0,0,0, - 0,3,8,1,4,1,114,232,0,0,0,41,2,78,78,41, - 1,78,41,2,78,114,22,0,0,0,41,4,78,78,114,10, - 0,0,0,114,22,0,0,0,41,50,114,3,0,0,0,114, - 130,0,0,0,114,12,0,0,0,114,18,0,0,0,114,59, - 0,0,0,114,33,0,0,0,114,42,0,0,0,114,19,0, - 0,0,114,20,0,0,0,114,48,0,0,0,114,49,0,0, - 0,114,52,0,0,0,114,64,0,0,0,114,66,0,0,0, - 114,76,0,0,0,114,86,0,0,0,114,90,0,0,0,114, - 97,0,0,0,114,113,0,0,0,114,114,0,0,0,114,91, - 0,0,0,114,146,0,0,0,114,152,0,0,0,114,156,0, - 0,0,114,109,0,0,0,114,93,0,0,0,114,161,0,0, - 0,114,162,0,0,0,114,94,0,0,0,114,163,0,0,0, - 114,177,0,0,0,114,182,0,0,0,114,190,0,0,0,114, - 192,0,0,0,114,197,0,0,0,114,203,0,0,0,90,15, - 95,69,82,82,95,77,83,71,95,80,82,69,70,73,88,90, - 8,95,69,82,82,95,77,83,71,114,208,0,0,0,218,6, - 111,98,106,101,99,116,114,209,0,0,0,114,210,0,0,0, - 114,211,0,0,0,114,216,0,0,0,114,221,0,0,0,114, - 224,0,0,0,114,225,0,0,0,114,229,0,0,0,114,230, - 0,0,0,114,232,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,218,8,60,109, - 111,100,117,108,101,62,1,0,0,0,115,94,0,0,0,4, - 24,4,2,8,8,8,8,4,2,4,3,16,4,14,68,14, - 21,14,16,8,37,8,17,8,11,14,8,8,11,8,12,8, - 16,8,36,14,100,16,26,10,45,14,72,8,17,8,17,8, - 30,8,37,8,42,8,15,14,75,14,78,14,13,8,9,8, - 9,10,47,8,16,4,1,8,2,8,27,6,3,8,16,10, - 15,14,37,8,27,10,37,8,7,8,35,8,8, -}; diff --git a/Python/instrumentation.c b/Python/instrumentation.c new file mode 100644 index 00000000000000..39a7eaa3bd2d42 --- /dev/null +++ b/Python/instrumentation.c @@ -0,0 +1,2021 @@ + + +#include "Python.h" +#include "pycore_call.h" +#include "pycore_frame.h" +#include "pycore_interp.h" +#include "pycore_long.h" +#include "pycore_namespace.h" +#include "pycore_object.h" +#include "pycore_opcode.h" +#include "pycore_pyerrors.h" +#include "pycore_pystate.h" + +/* Uncomment this to dump debugging output when assertions fail */ +// #define INSTRUMENT_DEBUG 1 + +static PyObject DISABLE = +{ + _PyObject_IMMORTAL_REFCNT, + &PyBaseObject_Type +}; + +PyObject _PyInstrumentation_MISSING = +{ + _PyObject_IMMORTAL_REFCNT, + &PyBaseObject_Type +}; + +static const int8_t EVENT_FOR_OPCODE[256] = { + [RETURN_CONST] = PY_MONITORING_EVENT_PY_RETURN, + [INSTRUMENTED_RETURN_CONST] = PY_MONITORING_EVENT_PY_RETURN, + [RETURN_VALUE] = PY_MONITORING_EVENT_PY_RETURN, + [INSTRUMENTED_RETURN_VALUE] = PY_MONITORING_EVENT_PY_RETURN, + [CALL] = PY_MONITORING_EVENT_CALL, + [INSTRUMENTED_CALL] = PY_MONITORING_EVENT_CALL, + [CALL_FUNCTION_EX] = PY_MONITORING_EVENT_CALL, + [INSTRUMENTED_CALL_FUNCTION_EX] = PY_MONITORING_EVENT_CALL, + [RESUME] = -1, + [YIELD_VALUE] = PY_MONITORING_EVENT_PY_YIELD, + [INSTRUMENTED_YIELD_VALUE] = PY_MONITORING_EVENT_PY_YIELD, + [JUMP_FORWARD] = PY_MONITORING_EVENT_JUMP, + [JUMP_BACKWARD] = PY_MONITORING_EVENT_JUMP, + [POP_JUMP_IF_FALSE] = PY_MONITORING_EVENT_BRANCH, + [POP_JUMP_IF_TRUE] = PY_MONITORING_EVENT_BRANCH, + [POP_JUMP_IF_NONE] = PY_MONITORING_EVENT_BRANCH, + [POP_JUMP_IF_NOT_NONE] = PY_MONITORING_EVENT_BRANCH, + [INSTRUMENTED_JUMP_FORWARD] = PY_MONITORING_EVENT_JUMP, + [INSTRUMENTED_JUMP_BACKWARD] = PY_MONITORING_EVENT_JUMP, + [INSTRUMENTED_POP_JUMP_IF_FALSE] = PY_MONITORING_EVENT_BRANCH, + [INSTRUMENTED_POP_JUMP_IF_TRUE] = PY_MONITORING_EVENT_BRANCH, + [INSTRUMENTED_POP_JUMP_IF_NONE] = PY_MONITORING_EVENT_BRANCH, + [INSTRUMENTED_POP_JUMP_IF_NOT_NONE] = PY_MONITORING_EVENT_BRANCH, + [FOR_ITER] = PY_MONITORING_EVENT_BRANCH, + [INSTRUMENTED_FOR_ITER] = PY_MONITORING_EVENT_BRANCH, + [END_FOR] = PY_MONITORING_EVENT_STOP_ITERATION, + [INSTRUMENTED_END_FOR] = PY_MONITORING_EVENT_STOP_ITERATION, + [END_SEND] = PY_MONITORING_EVENT_STOP_ITERATION, + [INSTRUMENTED_END_SEND] = PY_MONITORING_EVENT_STOP_ITERATION, +}; + +static const uint8_t DE_INSTRUMENT[256] = { + [INSTRUMENTED_RESUME] = RESUME, + [INSTRUMENTED_RETURN_VALUE] = RETURN_VALUE, + [INSTRUMENTED_RETURN_CONST] = RETURN_CONST, + [INSTRUMENTED_CALL] = CALL, + [INSTRUMENTED_CALL_FUNCTION_EX] = CALL_FUNCTION_EX, + [INSTRUMENTED_YIELD_VALUE] = YIELD_VALUE, + [INSTRUMENTED_JUMP_FORWARD] = JUMP_FORWARD, + [INSTRUMENTED_JUMP_BACKWARD] = JUMP_BACKWARD, + [INSTRUMENTED_POP_JUMP_IF_FALSE] = POP_JUMP_IF_FALSE, + [INSTRUMENTED_POP_JUMP_IF_TRUE] = POP_JUMP_IF_TRUE, + [INSTRUMENTED_POP_JUMP_IF_NONE] = POP_JUMP_IF_NONE, + [INSTRUMENTED_POP_JUMP_IF_NOT_NONE] = POP_JUMP_IF_NOT_NONE, + [INSTRUMENTED_FOR_ITER] = FOR_ITER, + [INSTRUMENTED_END_FOR] = END_FOR, + [INSTRUMENTED_END_SEND] = END_SEND, +}; + +static const uint8_t INSTRUMENTED_OPCODES[256] = { + [RETURN_CONST] = INSTRUMENTED_RETURN_CONST, + [INSTRUMENTED_RETURN_CONST] = INSTRUMENTED_RETURN_CONST, + [RETURN_VALUE] = INSTRUMENTED_RETURN_VALUE, + [INSTRUMENTED_RETURN_VALUE] = INSTRUMENTED_RETURN_VALUE, + [CALL] = INSTRUMENTED_CALL, + [INSTRUMENTED_CALL] = INSTRUMENTED_CALL, + [CALL_FUNCTION_EX] = INSTRUMENTED_CALL_FUNCTION_EX, + [INSTRUMENTED_CALL_FUNCTION_EX] = INSTRUMENTED_CALL_FUNCTION_EX, + [YIELD_VALUE] = INSTRUMENTED_YIELD_VALUE, + [INSTRUMENTED_YIELD_VALUE] = INSTRUMENTED_YIELD_VALUE, + [RESUME] = INSTRUMENTED_RESUME, + [INSTRUMENTED_RESUME] = INSTRUMENTED_RESUME, + [JUMP_FORWARD] = INSTRUMENTED_JUMP_FORWARD, + [INSTRUMENTED_JUMP_FORWARD] = INSTRUMENTED_JUMP_FORWARD, + [JUMP_BACKWARD] = INSTRUMENTED_JUMP_BACKWARD, + [INSTRUMENTED_JUMP_BACKWARD] = INSTRUMENTED_JUMP_BACKWARD, + [POP_JUMP_IF_FALSE] = INSTRUMENTED_POP_JUMP_IF_FALSE, + [INSTRUMENTED_POP_JUMP_IF_FALSE] = INSTRUMENTED_POP_JUMP_IF_FALSE, + [POP_JUMP_IF_TRUE] = INSTRUMENTED_POP_JUMP_IF_TRUE, + [INSTRUMENTED_POP_JUMP_IF_TRUE] = INSTRUMENTED_POP_JUMP_IF_TRUE, + [POP_JUMP_IF_NONE] = INSTRUMENTED_POP_JUMP_IF_NONE, + [INSTRUMENTED_POP_JUMP_IF_NONE] = INSTRUMENTED_POP_JUMP_IF_NONE, + [POP_JUMP_IF_NOT_NONE] = INSTRUMENTED_POP_JUMP_IF_NOT_NONE, + [INSTRUMENTED_POP_JUMP_IF_NOT_NONE] = INSTRUMENTED_POP_JUMP_IF_NOT_NONE, + [END_FOR] = INSTRUMENTED_END_FOR, + [INSTRUMENTED_END_FOR] = INSTRUMENTED_END_FOR, + [END_SEND] = INSTRUMENTED_END_SEND, + [INSTRUMENTED_END_SEND] = INSTRUMENTED_END_SEND, + + [INSTRUMENTED_LINE] = INSTRUMENTED_LINE, + [INSTRUMENTED_INSTRUCTION] = INSTRUMENTED_INSTRUCTION, +}; + +static inline bool +opcode_has_event(int opcode) { + return opcode < INSTRUMENTED_LINE && + INSTRUMENTED_OPCODES[opcode] > 0; +} + +static inline bool +is_instrumented(int opcode) { + assert(opcode != 0); + assert(opcode != RESERVED); + return opcode >= MIN_INSTRUMENTED_OPCODE; +} + +static inline bool +monitors_equals(_Py_Monitors a, _Py_Monitors b) +{ + for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + if (a.tools[i] != b.tools[i]) { + return false; + } + } + return true; +} + +static inline _Py_Monitors +monitors_sub(_Py_Monitors a, _Py_Monitors b) +{ + _Py_Monitors res; + for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + res.tools[i] = a.tools[i] & ~b.tools[i]; + } + return res; +} + +static inline _Py_Monitors +monitors_and(_Py_Monitors a, _Py_Monitors b) +{ + _Py_Monitors res; + for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + res.tools[i] = a.tools[i] & b.tools[i]; + } + return res; +} + +static inline _Py_Monitors +monitors_or(_Py_Monitors a, _Py_Monitors b) +{ + _Py_Monitors res; + for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + res.tools[i] = a.tools[i] | b.tools[i]; + } + return res; +} + +static inline bool +monitors_are_empty(_Py_Monitors m) +{ + for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + if (m.tools[i]) { + return false; + } + } + return true; +} + +static inline bool +multiple_tools(_Py_Monitors *m) +{ + for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + if (_Py_popcount32(m->tools[i]) > 1) { + return true; + } + } + return false; +} + +static inline _PyMonitoringEventSet +get_events(_Py_Monitors *m, int tool_id) +{ + _PyMonitoringEventSet result = 0; + for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + if ((m->tools[e] >> tool_id) & 1) { + result |= (1 << e); + } + } + return result; +} + +/* Line delta. + * 8 bit value. + * if line_delta == -128: + * line = None # represented as -1 + * elif line_delta == -127: + * line = PyCode_Addr2Line(code, offset * sizeof(_Py_CODEUNIT)); + * else: + * line = first_line + (offset >> OFFSET_SHIFT) + line_delta; + */ + +#define NO_LINE -128 +#define COMPUTED_LINE -127 + +#define OFFSET_SHIFT 4 + +static int8_t +compute_line_delta(PyCodeObject *code, int offset, int line) +{ + if (line < 0) { + return NO_LINE; + } + int delta = line - code->co_firstlineno - (offset >> OFFSET_SHIFT); + if (delta <= INT8_MAX && delta > COMPUTED_LINE) { + return delta; + } + return COMPUTED_LINE; +} + +static int +compute_line(PyCodeObject *code, int offset, int8_t line_delta) +{ + if (line_delta > COMPUTED_LINE) { + return code->co_firstlineno + (offset >> OFFSET_SHIFT) + line_delta; + } + if (line_delta == NO_LINE) { + + return -1; + } + assert(line_delta == COMPUTED_LINE); + /* Look it up */ + return PyCode_Addr2Line(code, offset * sizeof(_Py_CODEUNIT)); +} + +static int +instruction_length(PyCodeObject *code, int offset) +{ + int opcode = _PyCode_CODE(code)[offset].op.code; + assert(opcode != 0); + assert(opcode != RESERVED); + if (opcode == INSTRUMENTED_LINE) { + opcode = code->_co_monitoring->lines[offset].original_opcode; + } + if (opcode == INSTRUMENTED_INSTRUCTION) { + opcode = code->_co_monitoring->per_instruction_opcodes[offset]; + } + int deinstrumented = DE_INSTRUMENT[opcode]; + if (deinstrumented) { + opcode = deinstrumented; + } + else { + opcode = _PyOpcode_Deopt[opcode]; + } + assert(opcode != 0); + assert(!is_instrumented(opcode)); + assert(opcode == _PyOpcode_Deopt[opcode]); + return 1 + _PyOpcode_Caches[opcode]; +} + +#ifdef INSTRUMENT_DEBUG + +static void +dump_instrumentation_data_tools(PyCodeObject *code, uint8_t *tools, int i, FILE*out) +{ + if (tools == NULL) { + fprintf(out, "tools = NULL"); + } + else { + fprintf(out, "tools = %d", tools[i]); + } +} + +static void +dump_instrumentation_data_lines(PyCodeObject *code, _PyCoLineInstrumentationData *lines, int i, FILE*out) +{ + if (lines == NULL) { + fprintf(out, ", lines = NULL"); + } + else if (lines[i].original_opcode == 0) { + fprintf(out, ", lines = {original_opcode = No LINE (0), line_delta = %d)", lines[i].line_delta); + } + else { + fprintf(out, ", lines = {original_opcode = %s, line_delta = %d)", _PyOpcode_OpName[lines[i].original_opcode], lines[i].line_delta); + } +} + +static void +dump_instrumentation_data_line_tools(PyCodeObject *code, uint8_t *line_tools, int i, FILE*out) +{ + if (line_tools == NULL) { + fprintf(out, ", line_tools = NULL"); + } + else { + fprintf(out, ", line_tools = %d", line_tools[i]); + } +} + +static void +dump_instrumentation_data_per_instruction(PyCodeObject *code, _PyCoMonitoringData *data, int i, FILE*out) +{ + if (data->per_instruction_opcodes == NULL) { + fprintf(out, ", per-inst opcode = NULL"); + } + else { + fprintf(out, ", per-inst opcode = %s", _PyOpcode_OpName[data->per_instruction_opcodes[i]]); + } + if (data->per_instruction_tools == NULL) { + fprintf(out, ", per-inst tools = NULL"); + } + else { + fprintf(out, ", per-inst tools = %d", data->per_instruction_tools[i]); + } +} + +static void +dump_monitors(const char *prefix, _Py_Monitors monitors, FILE*out) +{ + fprintf(out, "%s monitors:\n", prefix); + for (int event = 0; event < PY_MONITORING_UNGROUPED_EVENTS; event++) { + fprintf(out, " Event %d: Tools %x\n", event, monitors.tools[event]); + } +} + +/* Like _Py_GetBaseOpcode but without asserts. + * Does its best to give the right answer, but won't abort + * if something is wrong */ +int get_base_opcode_best_attempt(PyCodeObject *code, int offset) +{ + int opcode = _Py_OPCODE(_PyCode_CODE(code)[offset]); + if (INSTRUMENTED_OPCODES[opcode] != opcode) { + /* Not instrumented */ + return _PyOpcode_Deopt[opcode] == 0 ? opcode : _PyOpcode_Deopt[opcode]; + } + if (opcode == INSTRUMENTED_INSTRUCTION) { + if (code->_co_monitoring->per_instruction_opcodes[offset] == 0) { + return opcode; + } + opcode = code->_co_monitoring->per_instruction_opcodes[offset]; + } + if (opcode == INSTRUMENTED_LINE) { + if (code->_co_monitoring->lines[offset].original_opcode == 0) { + return opcode; + } + opcode = code->_co_monitoring->lines[offset].original_opcode; + } + int deinstrumented = DE_INSTRUMENT[opcode]; + if (deinstrumented) { + return deinstrumented; + } + if (_PyOpcode_Deopt[opcode] == 0) { + return opcode; + } + return _PyOpcode_Deopt[opcode]; +} + +/* No error checking -- Don't use this for anything but experimental debugging */ +static void +dump_instrumentation_data(PyCodeObject *code, int star, FILE*out) +{ + _PyCoMonitoringData *data = code->_co_monitoring; + fprintf(out, "\n"); + PyObject_Print(code->co_name, out, Py_PRINT_RAW); + fprintf(out, "\n"); + if (data == NULL) { + fprintf(out, "NULL\n"); + return; + } + dump_monitors("Global", PyInterpreterState_Get()->monitors, out); + dump_monitors("Code", data->local_monitors, out); + dump_monitors("Active", data->active_monitors, out); + int code_len = (int)Py_SIZE(code); + bool starred = false; + for (int i = 0; i < code_len; i += instruction_length(code, i)) { + _Py_CODEUNIT *instr = &_PyCode_CODE(code)[i]; + int opcode = instr->op.code; + if (i == star) { + fprintf(out, "** "); + starred = true; + } + fprintf(out, "Offset: %d, line: %d %s: ", i, PyCode_Addr2Line(code, i*2), _PyOpcode_OpName[opcode]); + dump_instrumentation_data_tools(code, data->tools, i, out); + dump_instrumentation_data_lines(code, data->lines, i, out); + dump_instrumentation_data_line_tools(code, data->line_tools, i, out); + dump_instrumentation_data_per_instruction(code, data, i, out); + fprintf(out, "\n"); + ; + } + if (!starred && star >= 0) { + fprintf(out, "Error offset not at valid instruction offset: %d\n", star); + fprintf(out, " "); + dump_instrumentation_data_tools(code, data->tools, star, out); + dump_instrumentation_data_lines(code, data->lines, star, out); + dump_instrumentation_data_line_tools(code, data->line_tools, star, out); + dump_instrumentation_data_per_instruction(code, data, star, out); + fprintf(out, "\n"); + } +} + +#define CHECK(test) do { \ + if (!(test)) { \ + dump_instrumentation_data(code, i, stderr); \ + } \ + assert(test); \ +} while (0) + +bool valid_opcode(int opcode) { + if (opcode > 0 && + opcode != RESERVED && + opcode < 255 && + _PyOpcode_OpName[opcode] && + _PyOpcode_OpName[opcode][0] != '<' + ) { + return true; + } + return false; +} + +static void +sanity_check_instrumentation(PyCodeObject *code) +{ + _PyCoMonitoringData *data = code->_co_monitoring; + if (data == NULL) { + return; + } + _Py_Monitors active_monitors = PyInterpreterState_Get()->monitors; + if (code->_co_monitoring) { + _Py_Monitors local_monitors = code->_co_monitoring->local_monitors; + active_monitors = monitors_or(active_monitors, local_monitors); + } + assert(monitors_equals( + code->_co_monitoring->active_monitors, + active_monitors) + ); + int code_len = (int)Py_SIZE(code); + for (int i = 0; i < code_len;) { + int opcode = _PyCode_CODE(code)[i].op.code; + int base_opcode = _Py_GetBaseOpcode(code, i); + CHECK(valid_opcode(opcode)); + CHECK(valid_opcode(base_opcode)); + if (opcode == INSTRUMENTED_INSTRUCTION) { + opcode = data->per_instruction_opcodes[i]; + if (!is_instrumented(opcode)) { + CHECK(_PyOpcode_Deopt[opcode] == opcode); + } + if (data->per_instruction_tools) { + uint8_t tools = active_monitors.tools[PY_MONITORING_EVENT_INSTRUCTION]; + CHECK((tools & data->per_instruction_tools[i]) == data->per_instruction_tools[i]); + } + } + if (opcode == INSTRUMENTED_LINE) { + CHECK(data->lines); + CHECK(valid_opcode(data->lines[i].original_opcode)); + opcode = data->lines[i].original_opcode; + CHECK(opcode != END_FOR); + CHECK(opcode != RESUME); + CHECK(opcode != INSTRUMENTED_RESUME); + if (!is_instrumented(opcode)) { + CHECK(_PyOpcode_Deopt[opcode] == opcode); + } + CHECK(opcode != INSTRUMENTED_LINE); + } + else if (data->lines && !is_instrumented(opcode)) { + CHECK(data->lines[i].original_opcode == 0 || + data->lines[i].original_opcode == base_opcode || + DE_INSTRUMENT[data->lines[i].original_opcode] == base_opcode); + } + if (is_instrumented(opcode)) { + CHECK(DE_INSTRUMENT[opcode] == base_opcode); + int event = EVENT_FOR_OPCODE[DE_INSTRUMENT[opcode]]; + if (event < 0) { + /* RESUME fixup */ + event = _PyCode_CODE(code)[i].op.arg; + } + CHECK(active_monitors.tools[event] != 0); + } + if (data->lines && base_opcode != END_FOR) { + int line1 = compute_line(code, i, data->lines[i].line_delta); + int line2 = PyCode_Addr2Line(code, i*sizeof(_Py_CODEUNIT)); + CHECK(line1 == line2); + } + CHECK(valid_opcode(opcode)); + if (data->tools) { + uint8_t local_tools = data->tools[i]; + if (opcode_has_event(base_opcode)) { + int event = EVENT_FOR_OPCODE[base_opcode]; + if (event == -1) { + /* RESUME fixup */ + event = _PyCode_CODE(code)[i].op.arg; + } + CHECK((active_monitors.tools[event] & local_tools) == local_tools); + } + else { + CHECK(local_tools == 0xff); + } + } + i += instruction_length(code, i); + assert(i <= code_len); + } +} +#else + +#define CHECK(test) assert(test) + +#endif + +/* Get the underlying opcode, stripping instrumentation */ +int _Py_GetBaseOpcode(PyCodeObject *code, int i) +{ + int opcode = _PyCode_CODE(code)[i].op.code; + if (opcode == INSTRUMENTED_LINE) { + opcode = code->_co_monitoring->lines[i].original_opcode; + } + if (opcode == INSTRUMENTED_INSTRUCTION) { + opcode = code->_co_monitoring->per_instruction_opcodes[i]; + } + CHECK(opcode != INSTRUMENTED_INSTRUCTION); + CHECK(opcode != INSTRUMENTED_LINE); + int deinstrumented = DE_INSTRUMENT[opcode]; + if (deinstrumented) { + return deinstrumented; + } + return _PyOpcode_Deopt[opcode]; +} + +static void +de_instrument(PyCodeObject *code, int i, int event) +{ + assert(event != PY_MONITORING_EVENT_INSTRUCTION); + assert(event != PY_MONITORING_EVENT_LINE); + + _Py_CODEUNIT *instr = &_PyCode_CODE(code)[i]; + uint8_t *opcode_ptr = &instr->op.code; + int opcode = *opcode_ptr; + if (opcode == INSTRUMENTED_LINE) { + opcode_ptr = &code->_co_monitoring->lines[i].original_opcode; + opcode = *opcode_ptr; + } + if (opcode == INSTRUMENTED_INSTRUCTION) { + opcode_ptr = &code->_co_monitoring->per_instruction_opcodes[i]; + opcode = *opcode_ptr; + } + int deinstrumented = DE_INSTRUMENT[opcode]; + if (deinstrumented == 0) { + return; + } + CHECK(_PyOpcode_Deopt[deinstrumented] == deinstrumented); + *opcode_ptr = deinstrumented; + if (_PyOpcode_Caches[deinstrumented]) { + instr[1].cache = adaptive_counter_warmup(); + } +} + +static void +de_instrument_line(PyCodeObject *code, int i) +{ + _Py_CODEUNIT *instr = &_PyCode_CODE(code)[i]; + uint8_t *opcode_ptr = &instr->op.code; + int opcode =*opcode_ptr; + if (opcode != INSTRUMENTED_LINE) { + return; + } + _PyCoLineInstrumentationData *lines = &code->_co_monitoring->lines[i]; + int original_opcode = lines->original_opcode; + CHECK(original_opcode != 0); + CHECK(original_opcode == _PyOpcode_Deopt[original_opcode]); + *opcode_ptr = instr->op.code = original_opcode; + if (_PyOpcode_Caches[original_opcode]) { + instr[1].cache = adaptive_counter_warmup(); + } + assert(*opcode_ptr != INSTRUMENTED_LINE); + assert(instr->op.code != INSTRUMENTED_LINE); +} + + +static void +de_instrument_per_instruction(PyCodeObject *code, int i) +{ + _Py_CODEUNIT *instr = &_PyCode_CODE(code)[i]; + uint8_t *opcode_ptr = &instr->op.code; + int opcode =*opcode_ptr; + if (opcode == INSTRUMENTED_LINE) { + opcode_ptr = &code->_co_monitoring->lines[i].original_opcode; + opcode = *opcode_ptr; + } + if (opcode != INSTRUMENTED_INSTRUCTION) { + return; + } + int original_opcode = code->_co_monitoring->per_instruction_opcodes[i]; + CHECK(original_opcode != 0); + CHECK(original_opcode == _PyOpcode_Deopt[original_opcode]); + instr->op.code = original_opcode; + if (_PyOpcode_Caches[original_opcode]) { + instr[1].cache = adaptive_counter_warmup(); + } + assert(instr->op.code != INSTRUMENTED_INSTRUCTION); + /* Keep things clean for sanity check */ + code->_co_monitoring->per_instruction_opcodes[i] = 0; +} + + +static void +instrument(PyCodeObject *code, int i) +{ + _Py_CODEUNIT *instr = &_PyCode_CODE(code)[i]; + uint8_t *opcode_ptr = &instr->op.code; + int opcode =*opcode_ptr; + if (opcode == INSTRUMENTED_LINE) { + _PyCoLineInstrumentationData *lines = &code->_co_monitoring->lines[i]; + opcode_ptr = &lines->original_opcode; + opcode = *opcode_ptr; + } + if (opcode == INSTRUMENTED_INSTRUCTION) { + opcode_ptr = &code->_co_monitoring->per_instruction_opcodes[i]; + opcode = *opcode_ptr; + CHECK(!is_instrumented(opcode)); + CHECK(opcode == _PyOpcode_Deopt[opcode]); + } + CHECK(opcode != 0); + if (!is_instrumented(opcode)) { + int deopt = _PyOpcode_Deopt[opcode]; + int instrumented = INSTRUMENTED_OPCODES[deopt]; + assert(instrumented); + *opcode_ptr = instrumented; + if (_PyOpcode_Caches[deopt]) { + instr[1].cache = adaptive_counter_warmup(); + } + } +} + +static void +instrument_line(PyCodeObject *code, int i) +{ + uint8_t *opcode_ptr = &_PyCode_CODE(code)[i].op.code; + int opcode =*opcode_ptr; + if (opcode == INSTRUMENTED_LINE) { + return; + } + _PyCoLineInstrumentationData *lines = &code->_co_monitoring->lines[i]; + lines->original_opcode = _PyOpcode_Deopt[opcode]; + CHECK(lines->original_opcode > 0); + *opcode_ptr = INSTRUMENTED_LINE; +} + +static void +instrument_per_instruction(PyCodeObject *code, int i) +{ + _Py_CODEUNIT *instr = &_PyCode_CODE(code)[i]; + uint8_t *opcode_ptr = &instr->op.code; + int opcode =*opcode_ptr; + if (opcode == INSTRUMENTED_LINE) { + _PyCoLineInstrumentationData *lines = &code->_co_monitoring->lines[i]; + opcode_ptr = &lines->original_opcode; + opcode = *opcode_ptr; + } + if (opcode == INSTRUMENTED_INSTRUCTION) { + return; + } + CHECK(opcode != 0); + if (is_instrumented(opcode)) { + code->_co_monitoring->per_instruction_opcodes[i] = opcode; + } + else { + assert(opcode != 0); + assert(_PyOpcode_Deopt[opcode] != 0); + assert(_PyOpcode_Deopt[opcode] != RESUME); + code->_co_monitoring->per_instruction_opcodes[i] = _PyOpcode_Deopt[opcode]; + } + assert(code->_co_monitoring->per_instruction_opcodes[i] > 0); + *opcode_ptr = INSTRUMENTED_INSTRUCTION; +} + +#ifndef NDEBUG +static bool +instruction_has_event(PyCodeObject *code, int offset) +{ + _Py_CODEUNIT instr = _PyCode_CODE(code)[offset]; + int opcode = instr.op.code; + if (opcode == INSTRUMENTED_LINE) { + opcode = code->_co_monitoring->lines[offset].original_opcode; + } + if (opcode == INSTRUMENTED_INSTRUCTION) { + opcode = code->_co_monitoring->per_instruction_opcodes[offset]; + } + return opcode_has_event(opcode); +} +#endif + +static void +remove_tools(PyCodeObject * code, int offset, int event, int tools) +{ + assert(event != PY_MONITORING_EVENT_LINE); + assert(event != PY_MONITORING_EVENT_INSTRUCTION); + assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); + assert(instruction_has_event(code, offset)); + _PyCoMonitoringData *monitoring = code->_co_monitoring; + if (monitoring && monitoring->tools) { + monitoring->tools[offset] &= ~tools; + if (monitoring->tools[offset] == 0) { + de_instrument(code, offset, event); + } + } + else { + /* Single tool */ + uint8_t single_tool = code->_co_monitoring->active_monitors.tools[event]; + assert(_Py_popcount32(single_tool) <= 1); + if (((single_tool & tools) == single_tool)) { + de_instrument(code, offset, event); + } + } +} + +#ifndef NDEBUG +static bool +tools_is_subset_for_event(PyCodeObject * code, int event, int tools) +{ + int global_tools = PyInterpreterState_Get()->monitors.tools[event]; + int local_tools = code->_co_monitoring->local_monitors.tools[event]; + return tools == ((global_tools | local_tools) & tools); +} +#endif + +static void +remove_line_tools(PyCodeObject * code, int offset, int tools) +{ + assert(code->_co_monitoring); + if (code->_co_monitoring->line_tools) + { + uint8_t *toolsptr = &code->_co_monitoring->line_tools[offset]; + *toolsptr &= ~tools; + if (*toolsptr == 0 ) { + de_instrument_line(code, offset); + } + } + else { + /* Single tool */ + uint8_t single_tool = code->_co_monitoring->active_monitors.tools[PY_MONITORING_EVENT_LINE]; + assert(_Py_popcount32(single_tool) <= 1); + if (((single_tool & tools) == single_tool)) { + de_instrument_line(code, offset); + } + } +} + +static void +add_tools(PyCodeObject * code, int offset, int event, int tools) +{ + assert(event != PY_MONITORING_EVENT_LINE); + assert(event != PY_MONITORING_EVENT_INSTRUCTION); + assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); + assert(code->_co_monitoring); + if (code->_co_monitoring && + code->_co_monitoring->tools + ) { + code->_co_monitoring->tools[offset] |= tools; + } + else { + /* Single tool */ + assert(_Py_popcount32(tools) == 1); + assert(tools_is_subset_for_event(code, event, tools)); + } + instrument(code, offset); +} + +static void +add_line_tools(PyCodeObject * code, int offset, int tools) +{ + assert(tools_is_subset_for_event(code, PY_MONITORING_EVENT_LINE, tools)); + assert(code->_co_monitoring); + if (code->_co_monitoring->line_tools + ) { + code->_co_monitoring->line_tools[offset] |= tools; + } + else { + /* Single tool */ + assert(_Py_popcount32(tools) == 1); + } + instrument_line(code, offset); +} + + +static void +add_per_instruction_tools(PyCodeObject * code, int offset, int tools) +{ + assert(tools_is_subset_for_event(code, PY_MONITORING_EVENT_INSTRUCTION, tools)); + assert(code->_co_monitoring); + if (code->_co_monitoring->per_instruction_tools + ) { + code->_co_monitoring->per_instruction_tools[offset] |= tools; + } + else { + /* Single tool */ + assert(_Py_popcount32(tools) == 1); + } + instrument_per_instruction(code, offset); +} + + +static void +remove_per_instruction_tools(PyCodeObject * code, int offset, int tools) +{ + assert(code->_co_monitoring); + if (code->_co_monitoring->per_instruction_tools) + { + uint8_t *toolsptr = &code->_co_monitoring->per_instruction_tools[offset]; + *toolsptr &= ~tools; + if (*toolsptr == 0 ) { + de_instrument_per_instruction(code, offset); + } + } + else { + /* Single tool */ + uint8_t single_tool = code->_co_monitoring->active_monitors.tools[PY_MONITORING_EVENT_INSTRUCTION]; + assert(_Py_popcount32(single_tool) <= 1); + if (((single_tool & tools) == single_tool)) { + de_instrument_per_instruction(code, offset); + } + } +} + + +/* Return 1 if DISABLE returned, -1 if error, 0 otherwise */ +static int +call_one_instrument( + PyInterpreterState *interp, PyThreadState *tstate, PyObject **args, + Py_ssize_t nargsf, int8_t tool, int event) +{ + assert(0 <= tool && tool < 8); + assert(tstate->tracing == 0); + PyObject *instrument = interp->monitoring_callables[tool][event]; + if (instrument == NULL) { + return 0; + } + int old_what = tstate->what_event; + tstate->what_event = event; + tstate->tracing++; + PyObject *res = _PyObject_VectorcallTstate(tstate, instrument, args, nargsf, NULL); + tstate->tracing--; + tstate->what_event = old_what; + if (res == NULL) { + return -1; + } + Py_DECREF(res); + return (res == &DISABLE); +} + +static const int8_t MOST_SIGNIFICANT_BITS[16] = { + -1, 0, 1, 1, + 2, 2, 2, 2, + 3, 3, 3, 3, + 3, 3, 3, 3, +}; + +/* We could use _Py_bit_length here, but that is designed for larger (32/64) bit ints, + and can perform relatively poorly on platforms without the necessary intrinsics. */ +static inline int most_significant_bit(uint8_t bits) { + assert(bits != 0); + if (bits > 15) { + return MOST_SIGNIFICANT_BITS[bits>>4]+4; + } + else { + return MOST_SIGNIFICANT_BITS[bits]; + } +} + +static bool +is_version_up_to_date(PyCodeObject *code, PyInterpreterState *interp) +{ + return interp->monitoring_version == code->_co_instrumentation_version; +} + +#ifndef NDEBUG +static bool +instrumentation_cross_checks(PyInterpreterState *interp, PyCodeObject *code) +{ + _Py_Monitors expected = monitors_or( + interp->monitors, + code->_co_monitoring->local_monitors); + return monitors_equals(code->_co_monitoring->active_monitors, expected); +} +#endif + +static inline uint8_t +get_tools_for_instruction(PyCodeObject * code, int i, int event) +{ + uint8_t tools; + assert(event != PY_MONITORING_EVENT_LINE); + assert(event != PY_MONITORING_EVENT_INSTRUCTION); + assert(instrumentation_cross_checks(PyThreadState_GET()->interp, code)); + _PyCoMonitoringData *monitoring = code->_co_monitoring; + if (event >= PY_MONITORING_UNGROUPED_EVENTS) { + assert(event == PY_MONITORING_EVENT_C_RAISE || + event == PY_MONITORING_EVENT_C_RETURN); + event = PY_MONITORING_EVENT_CALL; + } + if (event < PY_MONITORING_INSTRUMENTED_EVENTS && monitoring->tools) { + tools = monitoring->tools[i]; + } + else { + tools = code->_co_monitoring->active_monitors.tools[event]; + } + CHECK(tools_is_subset_for_event(code, event, tools)); + CHECK((tools & code->_co_monitoring->active_monitors.tools[event]) == tools); + return tools; +} + +static int +call_instrumentation_vector( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, Py_ssize_t nargs, PyObject *args[]) +{ + if (tstate->tracing) { + return 0; + } + assert(!_PyErr_Occurred(tstate)); + assert(args[0] == NULL); + PyCodeObject *code = frame->f_code; + assert(code->_co_instrumentation_version == tstate->interp->monitoring_version); + assert(is_version_up_to_date(code, tstate->interp)); + assert(instrumentation_cross_checks(tstate->interp, code)); + assert(args[1] == NULL); + args[1] = (PyObject *)code; + int offset = (int)(instr - _PyCode_CODE(code)); + /* Offset visible to user should be the offset in bytes, as that is the + * convention for APIs involving code offsets. */ + int bytes_offset = offset * (int)sizeof(_Py_CODEUNIT); + PyObject *offset_obj = PyLong_FromSsize_t(bytes_offset); + if (offset_obj == NULL) { + return -1; + } + assert(args[2] == NULL); + args[2] = offset_obj; + uint8_t tools = get_tools_for_instruction(code, offset, event); + Py_ssize_t nargsf = nargs | PY_VECTORCALL_ARGUMENTS_OFFSET; + PyObject **callargs = &args[1]; + int err = 0; + PyInterpreterState *interp = tstate->interp; + while (tools) { + int tool = most_significant_bit(tools); + assert(tool >= 0 && tool < 8); + assert(tools & (1 << tool)); + tools ^= (1 << tool); + int res = call_one_instrument(interp, tstate, callargs, nargsf, tool, event); + if (res == 0) { + /* Nothing to do */ + } + else if (res < 0) { + /* error */ + err = -1; + break; + } + else { + /* DISABLE */ + remove_tools(code, offset, event, 1 << tool); + } + } + Py_DECREF(offset_obj); + return err; +} + +int +_Py_call_instrumentation( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr) +{ + PyObject *args[3] = { NULL, NULL, NULL }; + return call_instrumentation_vector(tstate, event, frame, instr, 2, args); +} + +int +_Py_call_instrumentation_arg( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *arg) +{ + PyObject *args[4] = { NULL, NULL, NULL, arg }; + return call_instrumentation_vector(tstate, event, frame, instr, 3, args); +} + +int +_Py_call_instrumentation_2args( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *arg0, PyObject *arg1) +{ + PyObject *args[5] = { NULL, NULL, NULL, arg0, arg1 }; + return call_instrumentation_vector(tstate, event, frame, instr, 4, args); +} + +int +_Py_call_instrumentation_jump( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, _Py_CODEUNIT *target +) { + assert(event == PY_MONITORING_EVENT_JUMP || + event == PY_MONITORING_EVENT_BRANCH); + assert(frame->prev_instr == instr); + frame->prev_instr = target; + PyCodeObject *code = frame->f_code; + int to = (int)(target - _PyCode_CODE(code)); + PyObject *to_obj = PyLong_FromLong(to * (int)sizeof(_Py_CODEUNIT)); + if (to_obj == NULL) { + return -1; + } + PyObject *args[4] = { NULL, NULL, NULL, to_obj }; + int err = call_instrumentation_vector(tstate, event, frame, instr, 3, args); + Py_DECREF(to_obj); + return err; +} + +static void +call_instrumentation_vector_protected( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, Py_ssize_t nargs, PyObject *args[]) +{ + assert(_PyErr_Occurred(tstate)); + PyObject *exc = _PyErr_GetRaisedException(tstate); + int err = call_instrumentation_vector(tstate, event, frame, instr, nargs, args); + if (err) { + Py_XDECREF(exc); + } + else { + _PyErr_SetRaisedException(tstate, exc); + } + assert(_PyErr_Occurred(tstate)); +} + +void +_Py_call_instrumentation_exc0( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr) +{ + assert(_PyErr_Occurred(tstate)); + PyObject *args[3] = { NULL, NULL, NULL }; + call_instrumentation_vector_protected(tstate, event, frame, instr, 2, args); +} + +void +_Py_call_instrumentation_exc2( + PyThreadState *tstate, int event, + _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *arg0, PyObject *arg1) +{ + assert(_PyErr_Occurred(tstate)); + PyObject *args[5] = { NULL, NULL, NULL, arg0, arg1 }; + call_instrumentation_vector_protected(tstate, event, frame, instr, 4, args); +} + + +int +_Py_Instrumentation_GetLine(PyCodeObject *code, int index) +{ + _PyCoMonitoringData *monitoring = code->_co_monitoring; + assert(monitoring != NULL); + assert(monitoring->lines != NULL); + assert(index >= code->_co_firsttraceable); + assert(index < Py_SIZE(code)); + _PyCoLineInstrumentationData *line_data = &monitoring->lines[index]; + int8_t line_delta = line_data->line_delta; + int line = compute_line(code, index, line_delta); + return line; +} + +int +_Py_call_instrumentation_line(PyThreadState *tstate, _PyInterpreterFrame* frame, _Py_CODEUNIT *instr) +{ + frame->prev_instr = instr; + PyCodeObject *code = frame->f_code; + assert(is_version_up_to_date(code, tstate->interp)); + assert(instrumentation_cross_checks(tstate->interp, code)); + int i = (int)(instr - _PyCode_CODE(code)); + _PyCoMonitoringData *monitoring = code->_co_monitoring; + _PyCoLineInstrumentationData *line_data = &monitoring->lines[i]; + uint8_t original_opcode = line_data->original_opcode; + if (tstate->tracing) { + goto done; + } + PyInterpreterState *interp = tstate->interp; + int8_t line_delta = line_data->line_delta; + int line = compute_line(code, i, line_delta); + uint8_t tools = code->_co_monitoring->line_tools != NULL ? + code->_co_monitoring->line_tools[i] : + (interp->monitors.tools[PY_MONITORING_EVENT_LINE] | + code->_co_monitoring->local_monitors.tools[PY_MONITORING_EVENT_LINE] + ); + PyObject *line_obj = PyLong_FromSsize_t(line); + if (line_obj == NULL) { + return -1; + } + PyObject *args[3] = { NULL, (PyObject *)code, line_obj }; + while (tools) { + int tool = most_significant_bit(tools); + assert(tool >= 0 && tool < 8); + assert(tools & (1 << tool)); + tools &= ~(1 << tool); + int res = call_one_instrument(interp, tstate, &args[1], + 2 | PY_VECTORCALL_ARGUMENTS_OFFSET, + tool, PY_MONITORING_EVENT_LINE); + if (res == 0) { + /* Nothing to do */ + } + else if (res < 0) { + /* error */ + Py_DECREF(line_obj); + return -1; + } + else { + /* DISABLE */ + remove_line_tools(code, i, 1 << tool); + } + } + Py_DECREF(line_obj); +done: + assert(original_opcode != 0); + assert(original_opcode < INSTRUMENTED_LINE); + assert(_PyOpcode_Deopt[original_opcode] == original_opcode); + return original_opcode; +} + +int +_Py_call_instrumentation_instruction(PyThreadState *tstate, _PyInterpreterFrame* frame, _Py_CODEUNIT *instr) +{ + PyCodeObject *code = frame->f_code; + assert(is_version_up_to_date(code, tstate->interp)); + assert(instrumentation_cross_checks(tstate->interp, code)); + int offset = (int)(instr - _PyCode_CODE(code)); + _PyCoMonitoringData *instrumentation_data = code->_co_monitoring; + assert(instrumentation_data->per_instruction_opcodes); + int next_opcode = instrumentation_data->per_instruction_opcodes[offset]; + if (tstate->tracing) { + return next_opcode; + } + PyInterpreterState *interp = tstate->interp; + uint8_t tools = instrumentation_data->per_instruction_tools != NULL ? + instrumentation_data->per_instruction_tools[offset] : + (interp->monitors.tools[PY_MONITORING_EVENT_INSTRUCTION] | + code->_co_monitoring->local_monitors.tools[PY_MONITORING_EVENT_INSTRUCTION] + ); + int bytes_offset = offset * (int)sizeof(_Py_CODEUNIT); + PyObject *offset_obj = PyLong_FromSsize_t(bytes_offset); + if (offset_obj == NULL) { + return -1; + } + PyObject *args[3] = { NULL, (PyObject *)code, offset_obj }; + while (tools) { + int tool = most_significant_bit(tools); + assert(tool >= 0 && tool < 8); + assert(tools & (1 << tool)); + tools &= ~(1 << tool); + int res = call_one_instrument(interp, tstate, &args[1], + 2 | PY_VECTORCALL_ARGUMENTS_OFFSET, + tool, PY_MONITORING_EVENT_INSTRUCTION); + if (res == 0) { + /* Nothing to do */ + } + else if (res < 0) { + /* error */ + Py_DECREF(offset_obj); + return -1; + } + else { + /* DISABLE */ + remove_per_instruction_tools(code, offset, 1 << tool); + } + } + Py_DECREF(offset_obj); + assert(next_opcode != 0); + return next_opcode; +} + + +PyObject * +_PyMonitoring_RegisterCallback(int tool_id, int event_id, PyObject *obj) +{ + PyInterpreterState *is = _PyInterpreterState_Get(); + assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); + assert(0 <= event_id && event_id < PY_MONITORING_EVENTS); + PyObject *callback = is->monitoring_callables[tool_id][event_id]; + is->monitoring_callables[tool_id][event_id] = Py_XNewRef(obj); + return callback; +} + +static void +initialize_tools(PyCodeObject *code) +{ + uint8_t* tools = code->_co_monitoring->tools; + assert(tools != NULL); + int code_len = (int)Py_SIZE(code); + for (int i = 0; i < code_len; i++) { + _Py_CODEUNIT *instr = &_PyCode_CODE(code)[i]; + int opcode = instr->op.code; + if (opcode == INSTRUMENTED_LINE) { + opcode = code->_co_monitoring->lines[i].original_opcode; + } + bool instrumented = is_instrumented(opcode); + if (instrumented) { + opcode = DE_INSTRUMENT[opcode]; + assert(opcode != 0); + } + opcode = _PyOpcode_Deopt[opcode]; + if (opcode_has_event(opcode)) { + if (instrumented) { + int8_t event; + if (opcode == RESUME) { + event = instr->op.arg != 0; + } + else { + event = EVENT_FOR_OPCODE[opcode]; + assert(event > 0); + } + assert(event >= 0); + assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); + tools[i] = code->_co_monitoring->active_monitors.tools[event]; + CHECK(tools[i] != 0); + } + else { + tools[i] = 0; + } + } +#ifdef Py_DEBUG + /* Initialize tools for invalid locations to all ones to try to catch errors */ + else { + tools[i] = 0xff; + } + for (int j = 1; j <= _PyOpcode_Caches[opcode]; j++) { + tools[i+j] = 0xff; + } +#endif + i += _PyOpcode_Caches[opcode]; + } +} + +#define NO_LINE -128 + +static void +initialize_lines(PyCodeObject *code) +{ + _PyCoLineInstrumentationData *line_data = code->_co_monitoring->lines; + assert(line_data != NULL); + int code_len = (int)Py_SIZE(code); + PyCodeAddressRange range; + _PyCode_InitAddressRange(code, &range); + for (int i = 0; i < code->_co_firsttraceable && i < code_len; i++) { + line_data[i].original_opcode = 0; + line_data[i].line_delta = -127; + } + int current_line = -1; + for (int i = code->_co_firsttraceable; i < code_len; ) { + int opcode = _Py_GetBaseOpcode(code, i); + int line = _PyCode_CheckLineNumber(i*(int)sizeof(_Py_CODEUNIT), &range); + line_data[i].line_delta = compute_line_delta(code, i, line); + int length = instruction_length(code, i); + switch (opcode) { + case END_ASYNC_FOR: + case END_FOR: + case END_SEND: + case RESUME: + /* END_FOR cannot start a line, as it is skipped by FOR_ITER + * END_SEND cannot start a line, as it is skipped by SEND + * RESUME must not be instrumented with INSTRUMENT_LINE */ + line_data[i].original_opcode = 0; + break; + default: + if (line != current_line && line >= 0) { + line_data[i].original_opcode = opcode; + } + else { + line_data[i].original_opcode = 0; + } + if (line >= 0) { + current_line = line; + } + } + for (int j = 1; j < length; j++) { + line_data[i+j].original_opcode = 0; + line_data[i+j].line_delta = NO_LINE; + } + switch (opcode) { + case RETURN_VALUE: + case RAISE_VARARGS: + case RERAISE: + /* Blocks of code after these terminators + * should be treated as different lines */ + current_line = -1; + } + i += length; + } +} + +static void +initialize_line_tools(PyCodeObject *code, _Py_Monitors *all_events) +{ + uint8_t *line_tools = code->_co_monitoring->line_tools; + assert(line_tools != NULL); + int code_len = (int)Py_SIZE(code); + for (int i = 0; i < code_len; i++) { + line_tools[i] = all_events->tools[PY_MONITORING_EVENT_LINE]; + } +} + +static +int allocate_instrumentation_data(PyCodeObject *code) +{ + + if (code->_co_monitoring == NULL) { + code->_co_monitoring = PyMem_Malloc(sizeof(_PyCoMonitoringData)); + if (code->_co_monitoring == NULL) { + PyErr_NoMemory(); + return -1; + } + code->_co_monitoring->local_monitors = (_Py_Monitors){ 0 }; + code->_co_monitoring->active_monitors = (_Py_Monitors){ 0 }; + code->_co_monitoring->tools = NULL; + code->_co_monitoring->lines = NULL; + code->_co_monitoring->line_tools = NULL; + code->_co_monitoring->per_instruction_opcodes = NULL; + code->_co_monitoring->per_instruction_tools = NULL; + } + return 0; +} + +static int +update_instrumentation_data(PyCodeObject *code, PyInterpreterState *interp) +{ + int code_len = (int)Py_SIZE(code); + if (allocate_instrumentation_data(code)) { + return -1; + } + _Py_Monitors all_events = monitors_or( + interp->monitors, + code->_co_monitoring->local_monitors); + bool multitools = multiple_tools(&all_events); + if (code->_co_monitoring->tools == NULL && multitools) { + code->_co_monitoring->tools = PyMem_Malloc(code_len); + if (code->_co_monitoring->tools == NULL) { + PyErr_NoMemory(); + return -1; + } + initialize_tools(code); + } + if (all_events.tools[PY_MONITORING_EVENT_LINE]) { + if (code->_co_monitoring->lines == NULL) { + code->_co_monitoring->lines = PyMem_Malloc(code_len * sizeof(_PyCoLineInstrumentationData)); + if (code->_co_monitoring->lines == NULL) { + PyErr_NoMemory(); + return -1; + } + initialize_lines(code); + } + if (multitools && code->_co_monitoring->line_tools == NULL) { + code->_co_monitoring->line_tools = PyMem_Malloc(code_len); + if (code->_co_monitoring->line_tools == NULL) { + PyErr_NoMemory(); + return -1; + } + initialize_line_tools(code, &all_events); + } + } + if (all_events.tools[PY_MONITORING_EVENT_INSTRUCTION]) { + if (code->_co_monitoring->per_instruction_opcodes == NULL) { + code->_co_monitoring->per_instruction_opcodes = PyMem_Malloc(code_len * sizeof(_PyCoLineInstrumentationData)); + if (code->_co_monitoring->per_instruction_opcodes == NULL) { + PyErr_NoMemory(); + return -1; + } + /* This may not be necessary, as we can initialize this memory lazily, but it helps catch errors. */ + for (int i = 0; i < code_len; i++) { + code->_co_monitoring->per_instruction_opcodes[i] = 0; + } + } + if (multitools && code->_co_monitoring->per_instruction_tools == NULL) { + code->_co_monitoring->per_instruction_tools = PyMem_Malloc(code_len); + if (code->_co_monitoring->per_instruction_tools == NULL) { + PyErr_NoMemory(); + return -1; + } + /* This may not be necessary, as we can initialize this memory lazily, but it helps catch errors. */ + for (int i = 0; i < code_len; i++) { + code->_co_monitoring->per_instruction_tools[i] = 0; + } + } + } + return 0; +} + +static const uint8_t super_instructions[256] = { + [LOAD_FAST__LOAD_FAST] = 1, + [LOAD_FAST__LOAD_CONST] = 1, + [STORE_FAST__LOAD_FAST] = 1, + [STORE_FAST__STORE_FAST] = 1, + [LOAD_CONST__LOAD_FAST] = 1, +}; + +/* Should use instruction metadata for this */ +static bool +is_super_instruction(int opcode) { + return super_instructions[opcode] != 0; +} + +int +_Py_Instrument(PyCodeObject *code, PyInterpreterState *interp) +{ + + if (is_version_up_to_date(code, interp)) { + assert( + interp->monitoring_version == 0 || + instrumentation_cross_checks(interp, code) + ); + return 0; + } + int code_len = (int)Py_SIZE(code); + if (update_instrumentation_data(code, interp)) { + return -1; + } + _Py_Monitors active_events = monitors_or( + interp->monitors, + code->_co_monitoring->local_monitors); + _Py_Monitors new_events; + _Py_Monitors removed_events; + + bool restarted = interp->last_restart_version > code->_co_instrumentation_version; + if (restarted) { + removed_events = code->_co_monitoring->active_monitors; + new_events = active_events; + } + else { + removed_events = monitors_sub(code->_co_monitoring->active_monitors, active_events); + new_events = monitors_sub(active_events, code->_co_monitoring->active_monitors); + assert(monitors_are_empty(monitors_and(new_events, removed_events))); + } + code->_co_monitoring->active_monitors = active_events; + code->_co_instrumentation_version = interp->monitoring_version; + if (monitors_are_empty(new_events) && monitors_are_empty(removed_events)) { +#ifdef INSTRUMENT_DEBUG + sanity_check_instrumentation(code); +#endif + return 0; + } + /* Insert instrumentation */ + for (int i = 0; i < code_len; i+= instruction_length(code, i)) { + _Py_CODEUNIT *instr = &_PyCode_CODE(code)[i]; + if (is_super_instruction(instr->op.code)) { + instr->op.code = _PyOpcode_Deopt[instr->op.code]; + } + CHECK(instr->op.code != 0); + int base_opcode = _Py_GetBaseOpcode(code, i); + if (opcode_has_event(base_opcode)) { + int8_t event; + if (base_opcode == RESUME) { + event = instr->op.arg > 0; + } + else { + event = EVENT_FOR_OPCODE[base_opcode]; + assert(event > 0); + } + uint8_t removed_tools = removed_events.tools[event]; + if (removed_tools) { + remove_tools(code, i, event, removed_tools); + } + uint8_t new_tools = new_events.tools[event]; + if (new_tools) { + add_tools(code, i, event, new_tools); + } + } + } + uint8_t new_line_tools = new_events.tools[PY_MONITORING_EVENT_LINE]; + uint8_t removed_line_tools = removed_events.tools[PY_MONITORING_EVENT_LINE]; + if (new_line_tools | removed_line_tools) { + _PyCoLineInstrumentationData *line_data = code->_co_monitoring->lines; + for (int i = code->_co_firsttraceable; i < code_len;) { + if (line_data[i].original_opcode) { + if (removed_line_tools) { + remove_line_tools(code, i, removed_line_tools); + } + if (new_line_tools) { + add_line_tools(code, i, new_line_tools); + } + } + i += instruction_length(code, i); + } + } + uint8_t new_per_instruction_tools = new_events.tools[PY_MONITORING_EVENT_INSTRUCTION]; + uint8_t removed_per_instruction_tools = removed_events.tools[PY_MONITORING_EVENT_INSTRUCTION]; + if (new_per_instruction_tools | removed_per_instruction_tools) { + for (int i = code->_co_firsttraceable; i < code_len;) { + int opcode = _Py_GetBaseOpcode(code, i); + if (opcode == RESUME || opcode == END_FOR) { + i += instruction_length(code, i); + continue; + } + if (removed_per_instruction_tools) { + remove_per_instruction_tools(code, i, removed_per_instruction_tools); + } + if (new_per_instruction_tools) { + add_per_instruction_tools(code, i, new_per_instruction_tools); + } + i += instruction_length(code, i); + } + } +#ifdef INSTRUMENT_DEBUG + sanity_check_instrumentation(code); +#endif + return 0; +} + +#define C_RETURN_EVENTS \ + ((1 << PY_MONITORING_EVENT_C_RETURN) | \ + (1 << PY_MONITORING_EVENT_C_RAISE)) + +#define C_CALL_EVENTS \ + (C_RETURN_EVENTS | (1 << PY_MONITORING_EVENT_CALL)) + + +static int +instrument_all_executing_code_objects(PyInterpreterState *interp) { + _PyRuntimeState *runtime = &_PyRuntime; + HEAD_LOCK(runtime); + PyThreadState* ts = PyInterpreterState_ThreadHead(interp); + HEAD_UNLOCK(runtime); + while (ts) { + _PyInterpreterFrame *frame = ts->cframe->current_frame; + while (frame) { + if (frame->owner != FRAME_OWNED_BY_CSTACK) { + if (_Py_Instrument(frame->f_code, interp)) { + return -1; + } + } + frame = frame->previous; + } + HEAD_LOCK(runtime); + ts = PyThreadState_Next(ts); + HEAD_UNLOCK(runtime); + } + return 0; +} + +static void +set_events(_Py_Monitors *m, int tool_id, _PyMonitoringEventSet events) +{ + assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); + for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + uint8_t *tools = &m->tools[e]; + int val = (events >> e) & 1; + *tools &= ~(1 << tool_id); + *tools |= (val << tool_id); + } +} + +static int +check_tool(PyInterpreterState *interp, int tool_id) +{ + if (tool_id < PY_MONITORING_SYS_PROFILE_ID && + interp->monitoring_tool_names[tool_id] == NULL + ) { + PyErr_Format(PyExc_ValueError, "tool %d is not in use", tool_id); + return -1; + } + return 0; +} + +int +_PyMonitoring_SetEvents(int tool_id, _PyMonitoringEventSet events) +{ + assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); + PyInterpreterState *interp = _PyInterpreterState_Get(); + assert(events < (1 << PY_MONITORING_UNGROUPED_EVENTS)); + if (check_tool(interp, tool_id)) { + return -1; + } + uint32_t existing_events = get_events(&interp->monitors, tool_id); + if (existing_events == events) { + return 0; + } + set_events(&interp->monitors, tool_id, events); + interp->monitoring_version++; + return instrument_all_executing_code_objects(interp); +} + +int +_PyMonitoring_SetLocalEvents(PyCodeObject *code, int tool_id, _PyMonitoringEventSet events) +{ + assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); + PyInterpreterState *interp = _PyInterpreterState_Get(); + assert(events < (1 << PY_MONITORING_UNGROUPED_EVENTS)); + if (check_tool(interp, tool_id)) { + return -1; + } + if (allocate_instrumentation_data(code)) { + return -1; + } + _Py_Monitors *local = &code->_co_monitoring->local_monitors; + uint32_t existing_events = get_events(local, tool_id); + if (existing_events == events) { + return 0; + } + set_events(local, tool_id, events); + if (is_version_up_to_date(code, interp)) { + /* Force instrumentation update */ + code->_co_instrumentation_version = UINT64_MAX; + } + if (_Py_Instrument(code, interp)) { + return -1; + } + return 0; +} + +/*[clinic input] +module monitoring +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=37257f5987a360cf]*/ +/*[clinic end generated code]*/ + +#include "clinic/instrumentation.c.h" + +static int +check_valid_tool(int tool_id) +{ + if (tool_id < 0 || tool_id >= PY_MONITORING_SYS_PROFILE_ID) { + PyErr_Format(PyExc_ValueError, "invalid tool %d (must be between 0 and 5)", tool_id); + return -1; + } + return 0; +} + +/*[clinic input] +monitoring.use_tool_id + + tool_id: int + name: object + / + +[clinic start generated code]*/ + +static PyObject * +monitoring_use_tool_id_impl(PyObject *module, int tool_id, PyObject *name) +/*[clinic end generated code: output=30d76dc92b7cd653 input=ebc453761c621be1]*/ +{ + if (check_valid_tool(tool_id)) { + return NULL; + } + if (!PyUnicode_Check(name)) { + PyErr_SetString(PyExc_ValueError, "tool name must be a str"); + return NULL; + } + PyInterpreterState *interp = _PyInterpreterState_Get(); + if (interp->monitoring_tool_names[tool_id] != NULL) { + PyErr_Format(PyExc_ValueError, "tool %d is already in use", tool_id); + return NULL; + } + interp->monitoring_tool_names[tool_id] = Py_NewRef(name); + Py_RETURN_NONE; +} + +/*[clinic input] +monitoring.free_tool_id + + tool_id: int + / + +[clinic start generated code]*/ + +static PyObject * +monitoring_free_tool_id_impl(PyObject *module, int tool_id) +/*[clinic end generated code: output=86c2d2a1219a8591 input=a23fb6be3a8618e9]*/ +{ + if (check_valid_tool(tool_id)) { + return NULL; + } + PyInterpreterState *interp = _PyInterpreterState_Get(); + Py_CLEAR(interp->monitoring_tool_names[tool_id]); + Py_RETURN_NONE; +} + +/*[clinic input] +monitoring.get_tool + + tool_id: int + / + +[clinic start generated code]*/ + +static PyObject * +monitoring_get_tool_impl(PyObject *module, int tool_id) +/*[clinic end generated code: output=1c05a98b404a9a16 input=eeee9bebd0bcae9d]*/ + +/*[clinic end generated code]*/ +{ + if (check_valid_tool(tool_id)) { + return NULL; + } + PyInterpreterState *interp = _PyInterpreterState_Get(); + PyObject *name = interp->monitoring_tool_names[tool_id]; + if (name == NULL) { + Py_RETURN_NONE; + } + return Py_NewRef(name); +} + +/*[clinic input] +monitoring.register_callback + + + tool_id: int + event: int + func: object + / + +[clinic start generated code]*/ + +static PyObject * +monitoring_register_callback_impl(PyObject *module, int tool_id, int event, + PyObject *func) +/*[clinic end generated code: output=e64daa363004030c input=df6d70ea4cf81007]*/ +{ + if (check_valid_tool(tool_id)) { + return NULL; + } + if (_Py_popcount32(event) != 1) { + PyErr_SetString(PyExc_ValueError, "The callback can only be set for one event at a time"); + return NULL; + } + int event_id = _Py_bit_length(event)-1; + if (event_id < 0 || event_id >= PY_MONITORING_EVENTS) { + PyErr_Format(PyExc_ValueError, "invalid event %d", event); + return NULL; + } + if (func == Py_None) { + func = NULL; + } + func = _PyMonitoring_RegisterCallback(tool_id, event_id, func); + if (func == NULL) { + Py_RETURN_NONE; + } + return func; +} + +/*[clinic input] +monitoring.get_events -> int + + tool_id: int + / + +[clinic start generated code]*/ + +static int +monitoring_get_events_impl(PyObject *module, int tool_id) +/*[clinic end generated code: output=4450cc13f826c8c0 input=a64b238f76c4b2f7]*/ +{ + if (check_valid_tool(tool_id)) { + return -1; + } + _Py_Monitors *m = &_PyInterpreterState_Get()->monitors; + _PyMonitoringEventSet event_set = get_events(m, tool_id); + return event_set; +} + +/*[clinic input] +monitoring.set_events + + tool_id: int + event_set: int + / + +[clinic start generated code]*/ + +static PyObject * +monitoring_set_events_impl(PyObject *module, int tool_id, int event_set) +/*[clinic end generated code: output=1916c1e49cfb5bdb input=a77ba729a242142b]*/ +{ + if (check_valid_tool(tool_id)) { + return NULL; + } + if (event_set < 0 || event_set >= (1 << PY_MONITORING_EVENTS)) { + PyErr_Format(PyExc_ValueError, "invalid event set 0x%x", event_set); + return NULL; + } + if ((event_set & C_RETURN_EVENTS) && (event_set & C_CALL_EVENTS) != C_CALL_EVENTS) { + PyErr_Format(PyExc_ValueError, "cannot set C_RETURN or C_RAISE events independently"); + return NULL; + } + event_set &= ~C_RETURN_EVENTS; + if (_PyMonitoring_SetEvents(tool_id, event_set)) { + return NULL; + } + Py_RETURN_NONE; +} + +/*[clinic input] +monitoring.get_local_events -> int + + tool_id: int + code: object + / + +[clinic start generated code]*/ + +static int +monitoring_get_local_events_impl(PyObject *module, int tool_id, + PyObject *code) +/*[clinic end generated code: output=d3e92c1c9c1de8f9 input=bb0f927530386a94]*/ +{ + if (!PyCode_Check(code)) { + PyErr_Format( + PyExc_TypeError, + "code must be a code object" + ); + return -1; + } + if (check_valid_tool(tool_id)) { + return -1; + } + _PyMonitoringEventSet event_set = 0; + _PyCoMonitoringData *data = ((PyCodeObject *)code)->_co_monitoring; + if (data != NULL) { + for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + if ((data->local_monitors.tools[e] >> tool_id) & 1) { + event_set |= (1 << e); + } + } + } + return event_set; +} + +/*[clinic input] +monitoring.set_local_events + + tool_id: int + code: object + event_set: int + / + +[clinic start generated code]*/ + +static PyObject * +monitoring_set_local_events_impl(PyObject *module, int tool_id, + PyObject *code, int event_set) +/*[clinic end generated code: output=68cc755a65dfea99 input=5655ecd78d937a29]*/ +{ + if (!PyCode_Check(code)) { + PyErr_Format( + PyExc_TypeError, + "code must be a code object" + ); + return NULL; + } + if (check_valid_tool(tool_id)) { + return NULL; + } + if (event_set < 0 || event_set >= (1 << PY_MONITORING_EVENTS)) { + PyErr_Format(PyExc_ValueError, "invalid event set 0x%x", event_set); + return NULL; + } + if ((event_set & C_RETURN_EVENTS) && (event_set & C_CALL_EVENTS) != C_CALL_EVENTS) { + PyErr_Format(PyExc_ValueError, "cannot set C_RETURN or C_RAISE events independently"); + return NULL; + } + event_set &= ~C_RETURN_EVENTS; + if (_PyMonitoring_SetLocalEvents((PyCodeObject*)code, tool_id, event_set)) { + return NULL; + } + Py_RETURN_NONE; +} + +/*[clinic input] +monitoring.restart_events + +[clinic start generated code]*/ + +static PyObject * +monitoring_restart_events_impl(PyObject *module) +/*[clinic end generated code: output=e025dd5ba33314c4 input=add8a855063c8008]*/ +{ + /* We want to ensure that: + * last restart version > instrumented version for all code objects + * last restart version < current version + */ + PyInterpreterState *interp = _PyInterpreterState_Get(); + interp->last_restart_version = interp->monitoring_version + 1; + interp->monitoring_version = interp->last_restart_version + 1; + if (instrument_all_executing_code_objects(interp)) { + return NULL; + } + Py_RETURN_NONE; +} + +static int +add_power2_constant(PyObject *obj, const char *name, int i) +{ + PyObject *val = PyLong_FromLong(1<monitors.tools[e]; + if (tools == 0) { + continue; + } + PyObject *tools_obj = PyLong_FromLong(tools); + assert(tools_obj != NULL); + int err = PyDict_SetItemString(res, event_names[e], tools_obj); + Py_DECREF(tools_obj); + if (err < 0) { + Py_DECREF(res); + return NULL; + } + } + return res; +} + +static PyMethodDef methods[] = { + MONITORING_USE_TOOL_ID_METHODDEF + MONITORING_FREE_TOOL_ID_METHODDEF + MONITORING_GET_TOOL_METHODDEF + MONITORING_REGISTER_CALLBACK_METHODDEF + MONITORING_GET_EVENTS_METHODDEF + MONITORING_SET_EVENTS_METHODDEF + MONITORING_GET_LOCAL_EVENTS_METHODDEF + MONITORING_SET_LOCAL_EVENTS_METHODDEF + MONITORING_RESTART_EVENTS_METHODDEF + MONITORING__ALL_EVENTS_METHODDEF + {NULL, NULL} // sentinel +}; + +static struct PyModuleDef monitoring_module = { + PyModuleDef_HEAD_INIT, + .m_name = "sys.monitoring", + .m_size = -1, /* multiple "initialization" just copies the module dict. */ + .m_methods = methods, +}; + +PyObject *_Py_CreateMonitoringObject(void) +{ + PyObject *mod = _PyModule_CreateInitialized(&monitoring_module, PYTHON_API_VERSION); + if (mod == NULL) { + return NULL; + } + if (PyObject_SetAttrString(mod, "DISABLE", &DISABLE)) { + goto error; + } + if (PyObject_SetAttrString(mod, "MISSING", &_PyInstrumentation_MISSING)) { + goto error; + } + PyObject *events = _PyNamespace_New(NULL); + if (events == NULL) { + goto error; + } + int err = PyObject_SetAttrString(mod, "events", events); + Py_DECREF(events); + if (err) { + goto error; + } + for (int i = 0; i < PY_MONITORING_EVENTS; i++) { + if (add_power2_constant(events, event_names[i], i)) { + goto error; + } + } + err = PyObject_SetAttrString(events, "NO_EVENTS", _PyLong_GetZero()); + if (err) goto error; + PyObject *val = PyLong_FromLong(PY_MONITORING_DEBUGGER_ID); + err = PyObject_SetAttrString(mod, "DEBUGGER_ID", val); + Py_DECREF(val); + if (err) goto error; + val = PyLong_FromLong(PY_MONITORING_COVERAGE_ID); + err = PyObject_SetAttrString(mod, "COVERAGE_ID", val); + Py_DECREF(val); + if (err) goto error; + val = PyLong_FromLong(PY_MONITORING_PROFILER_ID); + err = PyObject_SetAttrString(mod, "PROFILER_ID", val); + Py_DECREF(val); + if (err) goto error; + val = PyLong_FromLong(PY_MONITORING_OPTIMIZER_ID); + err = PyObject_SetAttrString(mod, "OPTIMIZER_ID", val); + Py_DECREF(val); + if (err) goto error; + return mod; +error: + Py_DECREF(mod); + return NULL; +} diff --git a/Python/legacy_tracing.c b/Python/legacy_tracing.c new file mode 100644 index 00000000000000..cf345bddda79b0 --- /dev/null +++ b/Python/legacy_tracing.c @@ -0,0 +1,528 @@ +/* Support for legacy tracing on top of PEP 669 instrumentation + * Provides callables to forward PEP 669 events to legacy events. + */ + +#include +#include "Python.h" +#include "pycore_ceval.h" +#include "pycore_object.h" +#include "pycore_sysmodule.h" + +typedef struct _PyLegacyEventHandler { + PyObject_HEAD + vectorcallfunc vectorcall; + int event; +} _PyLegacyEventHandler; + +/* The Py_tracefunc function expects the following arguments: + * obj: the trace object (PyObject *) + * frame: the current frame (PyFrameObject *) + * kind: the kind of event, see PyTrace_XXX #defines (int) + * arg: The arg (a PyObject *) + */ + +static PyObject * +call_profile_func(_PyLegacyEventHandler *self, PyObject *arg) +{ + PyThreadState *tstate = _PyThreadState_GET(); + if (tstate->c_profilefunc == NULL) { + Py_RETURN_NONE; + } + PyFrameObject *frame = PyEval_GetFrame(); + if (frame == NULL) { + PyErr_SetString(PyExc_SystemError, + "Missing frame when calling profile function."); + return NULL; + } + Py_INCREF(frame); + int err = tstate->c_profilefunc(tstate->c_profileobj, frame, self->event, arg); + Py_DECREF(frame); + if (err) { + return NULL; + } + Py_RETURN_NONE; +} + +static PyObject * +sys_profile_func2( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 2); + return call_profile_func(self, Py_None); +} + +static PyObject * +sys_profile_func3( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 3); + return call_profile_func(self, args[2]); +} + +static PyObject * +sys_profile_call_or_return( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 4); + PyObject *callable = args[2]; + if (PyCFunction_Check(callable)) { + return call_profile_func(self, callable); + } + if (Py_TYPE(callable) == &PyMethodDescr_Type) { + PyObject *self_arg = args[3]; + /* For backwards compatibility need to + * convert to builtin method */ + + /* If no arg, skip */ + if (self_arg == &_PyInstrumentation_MISSING) { + Py_RETURN_NONE; + } + PyObject *meth = Py_TYPE(callable)->tp_descr_get( + callable, self_arg, (PyObject*)Py_TYPE(self_arg)); + if (meth == NULL) { + return NULL; + } + PyObject *res = call_profile_func(self, meth); + Py_DECREF(meth); + return res; + } + Py_RETURN_NONE; +} + +static PyObject * +call_trace_func(_PyLegacyEventHandler *self, PyObject *arg) +{ + PyThreadState *tstate = _PyThreadState_GET(); + if (tstate->c_tracefunc == NULL) { + Py_RETURN_NONE; + } + PyFrameObject *frame = PyEval_GetFrame(); + if (frame == NULL) { + PyErr_SetString(PyExc_SystemError, + "Missing frame when calling trace function."); + return NULL; + } + Py_INCREF(frame); + int err = tstate->c_tracefunc(tstate->c_traceobj, frame, self->event, arg); + Py_DECREF(frame); + if (err) { + return NULL; + } + Py_RETURN_NONE; +} + +static PyObject * +sys_trace_exception_func( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 3); + PyObject *exc = args[2]; + assert(PyExceptionInstance_Check(exc)); + PyObject *type = (PyObject *)Py_TYPE(exc); + PyObject *tb = PyException_GetTraceback(exc); + if (tb == NULL) { + tb = Py_NewRef(Py_None); + } + PyObject *tuple = PyTuple_Pack(3, type, exc, tb); + Py_DECREF(tb); + if (tuple == NULL) { + return NULL; + } + PyObject *res = call_trace_func(self, tuple); + Py_DECREF(tuple); + return res; +} + +static PyObject * +sys_trace_func2( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 2); + return call_trace_func(self, Py_None); +} + +static PyObject * +sys_trace_return( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(!PyErr_Occurred()); + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 3); + assert(PyCode_Check(args[0])); + PyObject *val = args[2]; + PyObject *res = call_trace_func(self, val); + return res; +} + +static PyObject * +sys_trace_yield( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 3); + return call_trace_func(self, args[2]); +} + +static PyObject * +sys_trace_instruction_func( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 2); + PyFrameObject *frame = PyEval_GetFrame(); + if (frame == NULL) { + PyErr_SetString(PyExc_SystemError, + "Missing frame when calling trace function."); + return NULL; + } + if (!frame->f_trace_opcodes) { + Py_RETURN_NONE; + } + Py_INCREF(frame); + PyThreadState *tstate = _PyThreadState_GET(); + int err = tstate->c_tracefunc(tstate->c_traceobj, frame, self->event, Py_None); + frame->f_lineno = 0; + Py_DECREF(frame); + if (err) { + return NULL; + } + Py_RETURN_NONE; +} + +static PyObject * +trace_line( + PyThreadState *tstate, _PyLegacyEventHandler *self, + PyFrameObject *frame, int line +) { + if (!frame->f_trace_lines) { + Py_RETURN_NONE; + } + if (line < 0) { + Py_RETURN_NONE; + } + frame ->f_last_traced_line = line; + Py_INCREF(frame); + frame->f_lineno = line; + int err = tstate->c_tracefunc(tstate->c_traceobj, frame, self->event, Py_None); + frame->f_lineno = 0; + Py_DECREF(frame); + if (err) { + return NULL; + } + Py_RETURN_NONE; +} + +static PyObject * +sys_trace_line_func( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + PyThreadState *tstate = _PyThreadState_GET(); + if (tstate->c_tracefunc == NULL) { + Py_RETURN_NONE; + } + assert(PyVectorcall_NARGS(nargsf) == 2); + int line = _PyLong_AsInt(args[1]); + assert(line >= 0); + PyFrameObject *frame = PyEval_GetFrame(); + if (frame == NULL) { + PyErr_SetString(PyExc_SystemError, + "Missing frame when calling trace function."); + return NULL; + } + assert(args[0] == (PyObject *)frame->f_frame->f_code); + if (frame ->f_last_traced_line == line) { + /* Already traced this line */ + Py_RETURN_NONE; + } + return trace_line(tstate, self, frame, line); +} + + +static PyObject * +sys_trace_jump_func( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + PyThreadState *tstate = _PyThreadState_GET(); + if (tstate->c_tracefunc == NULL) { + Py_RETURN_NONE; + } + assert(PyVectorcall_NARGS(nargsf) == 3); + int from = _PyLong_AsInt(args[1])/sizeof(_Py_CODEUNIT); + assert(from >= 0); + int to = _PyLong_AsInt(args[2])/sizeof(_Py_CODEUNIT); + assert(to >= 0); + PyFrameObject *frame = PyEval_GetFrame(); + if (frame == NULL) { + PyErr_SetString(PyExc_SystemError, + "Missing frame when calling trace function."); + return NULL; + } + if (!frame->f_trace_lines) { + Py_RETURN_NONE; + } + PyCodeObject *code = (PyCodeObject *)args[0]; + assert(PyCode_Check(code)); + assert(code == frame->f_frame->f_code); + /* We can call _Py_Instrumentation_GetLine because we always set + * line events for tracing */ + int to_line = _Py_Instrumentation_GetLine(code, to); + /* Backward jump: Always generate event + * Forward jump: Only generate event if jumping to different line. */ + if (to > from && frame->f_last_traced_line == to_line) { + /* Already traced this line */ + Py_RETURN_NONE; + } + return trace_line(tstate, self, frame, to_line); +} + +/* We don't care about the exception here, + * we just treat it as a possible new line + */ +static PyObject * +sys_trace_exception_handled( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + PyThreadState *tstate = _PyThreadState_GET(); + if (tstate->c_tracefunc == NULL) { + Py_RETURN_NONE; + } + assert(PyVectorcall_NARGS(nargsf) == 3); + PyFrameObject *frame = PyEval_GetFrame(); + PyCodeObject *code = (PyCodeObject *)args[0]; + assert(PyCode_Check(code)); + assert(code == frame->f_frame->f_code); + assert(PyLong_Check(args[1])); + int offset = _PyLong_AsInt(args[1])/sizeof(_Py_CODEUNIT); + /* We can call _Py_Instrumentation_GetLine because we always set + * line events for tracing */ + int line = _Py_Instrumentation_GetLine(code, offset); + if (frame->f_last_traced_line == line) { + /* Already traced this line */ + Py_RETURN_NONE; + } + return trace_line(tstate, self, frame, line); +} + + +PyTypeObject _PyLegacyEventHandler_Type = { + _PyVarObject_IMMORTAL_INIT(&PyType_Type, 0), + "sys.legacy_event_handler", + sizeof(_PyLegacyEventHandler), + .tp_dealloc = (destructor)PyObject_Free, + .tp_vectorcall_offset = offsetof(_PyLegacyEventHandler, vectorcall), + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | + Py_TPFLAGS_HAVE_VECTORCALL | Py_TPFLAGS_DISALLOW_INSTANTIATION, + .tp_call = PyVectorcall_Call, +}; + +static int +set_callbacks(int tool, vectorcallfunc vectorcall, int legacy_event, int event1, int event2) +{ + _PyLegacyEventHandler *callback = + PyObject_NEW(_PyLegacyEventHandler, &_PyLegacyEventHandler_Type); + if (callback == NULL) { + return -1; + } + callback->vectorcall = vectorcall; + callback->event = legacy_event; + Py_XDECREF(_PyMonitoring_RegisterCallback(tool, event1, (PyObject *)callback)); + if (event2 >= 0) { + Py_XDECREF(_PyMonitoring_RegisterCallback(tool, event2, (PyObject *)callback)); + } + Py_DECREF(callback); + return 0; +} + +#ifndef NDEBUG +/* Ensure that tstate is valid: sanity check for PyEval_AcquireThread() and + PyEval_RestoreThread(). Detect if tstate memory was freed. It can happen + when a thread continues to run after Python finalization, especially + daemon threads. */ +static int +is_tstate_valid(PyThreadState *tstate) +{ + assert(!_PyMem_IsPtrFreed(tstate)); + assert(!_PyMem_IsPtrFreed(tstate->interp)); + return 1; +} +#endif + +int +_PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) +{ + assert(is_tstate_valid(tstate)); + /* The caller must hold the GIL */ + assert(PyGILState_Check()); + + /* Call _PySys_Audit() in the context of the current thread state, + even if tstate is not the current thread state. */ + PyThreadState *current_tstate = _PyThreadState_GET(); + if (_PySys_Audit(current_tstate, "sys.setprofile", NULL) < 0) { + return -1; + } + /* Setup PEP 669 monitoring callbacks and events. */ + if (!tstate->interp->sys_profile_initialized) { + tstate->interp->sys_profile_initialized = true; + if (set_callbacks(PY_MONITORING_SYS_PROFILE_ID, + (vectorcallfunc)sys_profile_func2, PyTrace_CALL, + PY_MONITORING_EVENT_PY_START, PY_MONITORING_EVENT_PY_RESUME)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_PROFILE_ID, + (vectorcallfunc)sys_profile_func3, PyTrace_RETURN, + PY_MONITORING_EVENT_PY_RETURN, PY_MONITORING_EVENT_PY_YIELD)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_PROFILE_ID, + (vectorcallfunc)sys_profile_func2, PyTrace_RETURN, + PY_MONITORING_EVENT_PY_UNWIND, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_PROFILE_ID, + (vectorcallfunc)sys_profile_call_or_return, PyTrace_C_CALL, + PY_MONITORING_EVENT_CALL, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_PROFILE_ID, + (vectorcallfunc)sys_profile_call_or_return, PyTrace_C_RETURN, + PY_MONITORING_EVENT_C_RETURN, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_PROFILE_ID, + (vectorcallfunc)sys_profile_call_or_return, PyTrace_C_EXCEPTION, + PY_MONITORING_EVENT_C_RAISE, -1)) { + return -1; + } + } + + int delta = (func != NULL) - (tstate->c_profilefunc != NULL); + tstate->c_profilefunc = func; + PyObject *old_profileobj = tstate->c_profileobj; + tstate->c_profileobj = Py_XNewRef(arg); + Py_XDECREF(old_profileobj); + tstate->interp->sys_profiling_threads += delta; + assert(tstate->interp->sys_profiling_threads >= 0); + + uint32_t events = 0; + if (tstate->interp->sys_profiling_threads) { + events = + (1 << PY_MONITORING_EVENT_PY_START) | (1 << PY_MONITORING_EVENT_PY_RESUME) | + (1 << PY_MONITORING_EVENT_PY_RETURN) | (1 << PY_MONITORING_EVENT_PY_YIELD) | + (1 << PY_MONITORING_EVENT_CALL) | (1 << PY_MONITORING_EVENT_PY_UNWIND); + } + return _PyMonitoring_SetEvents(PY_MONITORING_SYS_PROFILE_ID, events); +} + +int +_PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) +{ + assert(is_tstate_valid(tstate)); + /* The caller must hold the GIL */ + assert(PyGILState_Check()); + + /* Call _PySys_Audit() in the context of the current thread state, + even if tstate is not the current thread state. */ + PyThreadState *current_tstate = _PyThreadState_GET(); + if (_PySys_Audit(current_tstate, "sys.settrace", NULL) < 0) { + return -1; + } + + assert(tstate->interp->sys_tracing_threads >= 0); + /* Setup PEP 669 monitoring callbacks and events. */ + if (!tstate->interp->sys_trace_initialized) { + tstate->interp->sys_trace_initialized = true; + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_func2, PyTrace_CALL, + PY_MONITORING_EVENT_PY_START, PY_MONITORING_EVENT_PY_RESUME)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_func2, PyTrace_CALL, + PY_MONITORING_EVENT_PY_THROW, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_return, PyTrace_RETURN, + PY_MONITORING_EVENT_PY_RETURN, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_yield, PyTrace_RETURN, + PY_MONITORING_EVENT_PY_YIELD, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_exception_func, PyTrace_EXCEPTION, + PY_MONITORING_EVENT_RAISE, PY_MONITORING_EVENT_STOP_ITERATION)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_line_func, PyTrace_LINE, + PY_MONITORING_EVENT_LINE, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_func2, PyTrace_RETURN, + PY_MONITORING_EVENT_PY_UNWIND, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_jump_func, PyTrace_LINE, + PY_MONITORING_EVENT_JUMP, PY_MONITORING_EVENT_BRANCH)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_instruction_func, PyTrace_OPCODE, + PY_MONITORING_EVENT_INSTRUCTION, -1)) { + return -1; + } + if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, + (vectorcallfunc)sys_trace_exception_handled, PyTrace_LINE, + PY_MONITORING_EVENT_EXCEPTION_HANDLED, -1)) { + return -1; + } + } + + int delta = (func != NULL) - (tstate->c_tracefunc != NULL); + tstate->c_tracefunc = func; + PyObject *old_traceobj = tstate->c_traceobj; + tstate->c_traceobj = Py_XNewRef(arg); + Py_XDECREF(old_traceobj); + tstate->interp->sys_tracing_threads += delta; + assert(tstate->interp->sys_tracing_threads >= 0); + + uint32_t events = 0; + if (tstate->interp->sys_tracing_threads) { + events = + (1 << PY_MONITORING_EVENT_PY_START) | (1 << PY_MONITORING_EVENT_PY_RESUME) | + (1 << PY_MONITORING_EVENT_PY_RETURN) | (1 << PY_MONITORING_EVENT_PY_YIELD) | + (1 << PY_MONITORING_EVENT_RAISE) | (1 << PY_MONITORING_EVENT_LINE) | + (1 << PY_MONITORING_EVENT_JUMP) | (1 << PY_MONITORING_EVENT_BRANCH) | + (1 << PY_MONITORING_EVENT_PY_UNWIND) | (1 << PY_MONITORING_EVENT_PY_THROW) | + (1 << PY_MONITORING_EVENT_STOP_ITERATION) | + (1 << PY_MONITORING_EVENT_EXCEPTION_HANDLED); + if (tstate->interp->f_opcode_trace_set) { + events |= (1 << PY_MONITORING_EVENT_INSTRUCTION); + } + } + return _PyMonitoring_SetEvents(PY_MONITORING_SYS_TRACE_ID, events); +} diff --git a/Python/makeopcodetargets.py b/Python/makeopcodetargets.py index 33a4b4a76a1253..5aa31803397ce4 100755 --- a/Python/makeopcodetargets.py +++ b/Python/makeopcodetargets.py @@ -32,7 +32,6 @@ def write_contents(f): """ opcode = find_module('opcode') targets = ['_unknown_opcode'] * 256 - targets[255] = "TARGET_DO_TRACING" for opname, op in opcode.opmap.items(): if not opcode.is_pseudo(op): targets[op] = "TARGET_%s" % opname diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index f57b76aeb31050..4681ed03aff582 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -13,6 +13,8 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 0; case RESUME: return 0; + case INSTRUMENTED_RESUME: + return 0; case LOAD_CLOSURE: return 0; case LOAD_FAST_CHECK: @@ -39,6 +41,12 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 0; case END_FOR: return 1+1; + case INSTRUMENTED_END_FOR: + return 2; + case END_SEND: + return 2; + case INSTRUMENTED_END_SEND: + return 2; case UNARY_NEGATIVE: return 1; case UNARY_NOT: @@ -97,8 +105,12 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 1; case RETURN_VALUE: return 1; + case INSTRUMENTED_RETURN_VALUE: + return 1; case RETURN_CONST: return 0; + case INSTRUMENTED_RETURN_CONST: + return 0; case GET_AITER: return 1; case GET_ANEXT: @@ -109,6 +121,8 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 2; case SEND_GEN: return 2; + case INSTRUMENTED_YIELD_VALUE: + return 1; case YIELD_VALUE: return 1; case POP_EXCEPT: @@ -263,6 +277,8 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 1; case FOR_ITER: return 1; + case INSTRUMENTED_FOR_ITER: + return 0; case FOR_ITER_LIST: return 1; case FOR_ITER_TUPLE: @@ -287,6 +303,8 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 1; case KW_NAMES: return 0; + case INSTRUMENTED_CALL: + return 0; case CALL: return oparg + 2; case CALL_BOUND_METHOD_EXACT_ARGS: @@ -323,6 +341,8 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return oparg + 2; case CALL_NO_KW_METHOD_DESCRIPTOR_FAST: return oparg + 2; + case INSTRUMENTED_CALL_FUNCTION_EX: + return 0; case CALL_FUNCTION_EX: return ((oparg & 1) ? 1 : 0) + 3; case MAKE_FUNCTION: @@ -339,10 +359,28 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 2; case SWAP: return (oparg-2) + 2; + case INSTRUMENTED_LINE: + return 0; + case INSTRUMENTED_INSTRUCTION: + return 0; + case INSTRUMENTED_JUMP_FORWARD: + return 0; + case INSTRUMENTED_JUMP_BACKWARD: + return 0; + case INSTRUMENTED_POP_JUMP_IF_TRUE: + return 0; + case INSTRUMENTED_POP_JUMP_IF_FALSE: + return 0; + case INSTRUMENTED_POP_JUMP_IF_NONE: + return 0; + case INSTRUMENTED_POP_JUMP_IF_NOT_NONE: + return 0; case EXTENDED_ARG: return 0; case CACHE: return 0; + case RESERVED: + return 0; default: return -1; } @@ -359,6 +397,8 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return 0; case RESUME: return 0; + case INSTRUMENTED_RESUME: + return 0; case LOAD_CLOSURE: return 1; case LOAD_FAST_CHECK: @@ -385,6 +425,12 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return 1; case END_FOR: return 0+0; + case INSTRUMENTED_END_FOR: + return 0; + case END_SEND: + return 1; + case INSTRUMENTED_END_SEND: + return 1; case UNARY_NEGATIVE: return 1; case UNARY_NOT: @@ -443,8 +489,12 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return 0; case RETURN_VALUE: return 0; + case INSTRUMENTED_RETURN_VALUE: + return 0; case RETURN_CONST: return 0; + case INSTRUMENTED_RETURN_CONST: + return 0; case GET_AITER: return 1; case GET_ANEXT: @@ -455,6 +505,8 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return 2; case SEND_GEN: return 1; + case INSTRUMENTED_YIELD_VALUE: + return 1; case YIELD_VALUE: return 1; case POP_EXCEPT: @@ -609,6 +661,8 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return 1; case FOR_ITER: return 2; + case INSTRUMENTED_FOR_ITER: + return 0; case FOR_ITER_LIST: return 2; case FOR_ITER_TUPLE: @@ -633,6 +687,8 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return ((oparg & 1) ? 1 : 0) + 1; case KW_NAMES: return 0; + case INSTRUMENTED_CALL: + return 0; case CALL: return 1; case CALL_BOUND_METHOD_EXACT_ARGS: @@ -669,6 +725,8 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return 1; case CALL_NO_KW_METHOD_DESCRIPTOR_FAST: return 1; + case INSTRUMENTED_CALL_FUNCTION_EX: + return 0; case CALL_FUNCTION_EX: return 1; case MAKE_FUNCTION: @@ -685,10 +743,28 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return 1; case SWAP: return (oparg-2) + 2; + case INSTRUMENTED_LINE: + return 0; + case INSTRUMENTED_INSTRUCTION: + return 0; + case INSTRUMENTED_JUMP_FORWARD: + return 0; + case INSTRUMENTED_JUMP_BACKWARD: + return 0; + case INSTRUMENTED_POP_JUMP_IF_TRUE: + return 0; + case INSTRUMENTED_POP_JUMP_IF_FALSE: + return 0; + case INSTRUMENTED_POP_JUMP_IF_NONE: + return 0; + case INSTRUMENTED_POP_JUMP_IF_NOT_NONE: + return 0; case EXTENDED_ARG: return 0; case CACHE: return 0; + case RESERVED: + return 0; default: return -1; } @@ -707,6 +783,7 @@ extern const struct opcode_metadata _PyOpcode_opcode_metadata[256]; const struct opcode_metadata _PyOpcode_opcode_metadata[256] = { [NOP] = { true, INSTR_FMT_IX }, [RESUME] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_RESUME] = { true, INSTR_FMT_IB }, [LOAD_CLOSURE] = { true, INSTR_FMT_IB }, [LOAD_FAST_CHECK] = { true, INSTR_FMT_IB }, [LOAD_FAST] = { true, INSTR_FMT_IB }, @@ -720,6 +797,9 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[256] = { [POP_TOP] = { true, INSTR_FMT_IX }, [PUSH_NULL] = { true, INSTR_FMT_IX }, [END_FOR] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_END_FOR] = { true, INSTR_FMT_IX }, + [END_SEND] = { true, INSTR_FMT_IX }, + [INSTRUMENTED_END_SEND] = { true, INSTR_FMT_IX }, [UNARY_NEGATIVE] = { true, INSTR_FMT_IX }, [UNARY_NOT] = { true, INSTR_FMT_IX }, [UNARY_INVERT] = { true, INSTR_FMT_IX }, @@ -749,12 +829,15 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[256] = { [RAISE_VARARGS] = { true, INSTR_FMT_IB }, [INTERPRETER_EXIT] = { true, INSTR_FMT_IX }, [RETURN_VALUE] = { true, INSTR_FMT_IX }, + [INSTRUMENTED_RETURN_VALUE] = { true, INSTR_FMT_IX }, [RETURN_CONST] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_RETURN_CONST] = { true, INSTR_FMT_IB }, [GET_AITER] = { true, INSTR_FMT_IX }, [GET_ANEXT] = { true, INSTR_FMT_IX }, [GET_AWAITABLE] = { true, INSTR_FMT_IB }, [SEND] = { true, INSTR_FMT_IBC }, [SEND_GEN] = { true, INSTR_FMT_IBC }, + [INSTRUMENTED_YIELD_VALUE] = { true, INSTR_FMT_IX }, [YIELD_VALUE] = { true, INSTR_FMT_IX }, [POP_EXCEPT] = { true, INSTR_FMT_IX }, [RERAISE] = { true, INSTR_FMT_IB }, @@ -832,6 +915,7 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[256] = { [GET_ITER] = { true, INSTR_FMT_IX }, [GET_YIELD_FROM_ITER] = { true, INSTR_FMT_IX }, [FOR_ITER] = { true, INSTR_FMT_IBC }, + [INSTRUMENTED_FOR_ITER] = { true, INSTR_FMT_IB }, [FOR_ITER_LIST] = { true, INSTR_FMT_IBC }, [FOR_ITER_TUPLE] = { true, INSTR_FMT_IBC }, [FOR_ITER_RANGE] = { true, INSTR_FMT_IBC }, @@ -844,6 +928,7 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[256] = { [LOAD_ATTR_METHOD_NO_DICT] = { true, INSTR_FMT_IBC00000000 }, [LOAD_ATTR_METHOD_LAZY_DICT] = { true, INSTR_FMT_IBC00000000 }, [KW_NAMES] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_CALL] = { true, INSTR_FMT_IB }, [CALL] = { true, INSTR_FMT_IBC00 }, [CALL_BOUND_METHOD_EXACT_ARGS] = { true, INSTR_FMT_IBC00 }, [CALL_PY_EXACT_ARGS] = { true, INSTR_FMT_IBC00 }, @@ -862,6 +947,7 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[256] = { [CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS] = { true, INSTR_FMT_IBC00 }, [CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS] = { true, INSTR_FMT_IBC00 }, [CALL_NO_KW_METHOD_DESCRIPTOR_FAST] = { true, INSTR_FMT_IBC00 }, + [INSTRUMENTED_CALL_FUNCTION_EX] = { true, INSTR_FMT_IX }, [CALL_FUNCTION_EX] = { true, INSTR_FMT_IB }, [MAKE_FUNCTION] = { true, INSTR_FMT_IB }, [RETURN_GENERATOR] = { true, INSTR_FMT_IX }, @@ -870,7 +956,16 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[256] = { [COPY] = { true, INSTR_FMT_IB }, [BINARY_OP] = { true, INSTR_FMT_IBC }, [SWAP] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_LINE] = { true, INSTR_FMT_IX }, + [INSTRUMENTED_INSTRUCTION] = { true, INSTR_FMT_IX }, + [INSTRUMENTED_JUMP_FORWARD] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_JUMP_BACKWARD] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_POP_JUMP_IF_TRUE] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_POP_JUMP_IF_FALSE] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_POP_JUMP_IF_NONE] = { true, INSTR_FMT_IB }, + [INSTRUMENTED_POP_JUMP_IF_NOT_NONE] = { true, INSTR_FMT_IB }, [EXTENDED_ARG] = { true, INSTR_FMT_IB }, [CACHE] = { true, INSTR_FMT_IX }, + [RESERVED] = { true, INSTR_FMT_IX }, }; #endif diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index c502471bcd17b6..9d6616666f7ac1 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -4,17 +4,19 @@ static void *opcode_targets[256] = { &&TARGET_PUSH_NULL, &&TARGET_INTERPRETER_EXIT, &&TARGET_END_FOR, + &&TARGET_END_SEND, &&TARGET_BINARY_OP_ADD_FLOAT, &&TARGET_BINARY_OP_ADD_INT, &&TARGET_BINARY_OP_ADD_UNICODE, - &&TARGET_BINARY_OP_INPLACE_ADD_UNICODE, &&TARGET_NOP, - &&TARGET_BINARY_OP_MULTIPLY_FLOAT, + &&TARGET_BINARY_OP_INPLACE_ADD_UNICODE, &&TARGET_UNARY_NEGATIVE, &&TARGET_UNARY_NOT, + &&TARGET_BINARY_OP_MULTIPLY_FLOAT, &&TARGET_BINARY_OP_MULTIPLY_INT, - &&TARGET_BINARY_OP_SUBTRACT_FLOAT, &&TARGET_UNARY_INVERT, + &&TARGET_BINARY_OP_SUBTRACT_FLOAT, + &&TARGET_RESERVED, &&TARGET_BINARY_OP_SUBTRACT_INT, &&TARGET_BINARY_SUBSCR_DICT, &&TARGET_BINARY_SUBSCR_GETITEM, @@ -22,21 +24,21 @@ static void *opcode_targets[256] = { &&TARGET_BINARY_SUBSCR_TUPLE_INT, &&TARGET_CALL_PY_EXACT_ARGS, &&TARGET_CALL_PY_WITH_DEFAULTS, - &&TARGET_CALL_BOUND_METHOD_EXACT_ARGS, - &&TARGET_CALL_BUILTIN_CLASS, &&TARGET_BINARY_SUBSCR, &&TARGET_BINARY_SLICE, &&TARGET_STORE_SLICE, - &&TARGET_CALL_BUILTIN_FAST_WITH_KEYWORDS, - &&TARGET_CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS, + &&TARGET_CALL_BOUND_METHOD_EXACT_ARGS, + &&TARGET_CALL_BUILTIN_CLASS, &&TARGET_GET_LEN, &&TARGET_MATCH_MAPPING, &&TARGET_MATCH_SEQUENCE, &&TARGET_MATCH_KEYS, - &&TARGET_CALL_NO_KW_BUILTIN_FAST, + &&TARGET_CALL_BUILTIN_FAST_WITH_KEYWORDS, &&TARGET_PUSH_EXC_INFO, &&TARGET_CHECK_EXC_MATCH, &&TARGET_CHECK_EG_MATCH, + &&TARGET_CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS, + &&TARGET_CALL_NO_KW_BUILTIN_FAST, &&TARGET_CALL_NO_KW_BUILTIN_O, &&TARGET_CALL_NO_KW_ISINSTANCE, &&TARGET_CALL_NO_KW_LEN, @@ -46,8 +48,6 @@ static void *opcode_targets[256] = { &&TARGET_CALL_NO_KW_METHOD_DESCRIPTOR_O, &&TARGET_CALL_NO_KW_STR_1, &&TARGET_CALL_NO_KW_TUPLE_1, - &&TARGET_CALL_NO_KW_TYPE_1, - &&TARGET_COMPARE_OP_FLOAT, &&TARGET_WITH_EXCEPT_START, &&TARGET_GET_AITER, &&TARGET_GET_ANEXT, @@ -55,39 +55,39 @@ static void *opcode_targets[256] = { &&TARGET_BEFORE_WITH, &&TARGET_END_ASYNC_FOR, &&TARGET_CLEANUP_THROW, + &&TARGET_CALL_NO_KW_TYPE_1, + &&TARGET_COMPARE_OP_FLOAT, &&TARGET_COMPARE_OP_INT, &&TARGET_COMPARE_OP_STR, - &&TARGET_FOR_ITER_LIST, - &&TARGET_FOR_ITER_TUPLE, &&TARGET_STORE_SUBSCR, &&TARGET_DELETE_SUBSCR, + &&TARGET_FOR_ITER_LIST, + &&TARGET_FOR_ITER_TUPLE, &&TARGET_FOR_ITER_RANGE, &&TARGET_FOR_ITER_GEN, &&TARGET_LOAD_ATTR_CLASS, &&TARGET_LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN, - &&TARGET_LOAD_ATTR_INSTANCE_VALUE, - &&TARGET_LOAD_ATTR_MODULE, &&TARGET_GET_ITER, &&TARGET_GET_YIELD_FROM_ITER, - &&TARGET_LOAD_ATTR_PROPERTY, + &&TARGET_LOAD_ATTR_INSTANCE_VALUE, &&TARGET_LOAD_BUILD_CLASS, - &&TARGET_LOAD_ATTR_SLOT, - &&TARGET_LOAD_ATTR_WITH_HINT, + &&TARGET_LOAD_ATTR_MODULE, + &&TARGET_LOAD_ATTR_PROPERTY, &&TARGET_LOAD_ASSERTION_ERROR, &&TARGET_RETURN_GENERATOR, + &&TARGET_LOAD_ATTR_SLOT, + &&TARGET_LOAD_ATTR_WITH_HINT, &&TARGET_LOAD_ATTR_METHOD_LAZY_DICT, &&TARGET_LOAD_ATTR_METHOD_NO_DICT, &&TARGET_LOAD_ATTR_METHOD_WITH_VALUES, &&TARGET_LOAD_CONST__LOAD_FAST, &&TARGET_LOAD_FAST__LOAD_CONST, + &&TARGET_RETURN_VALUE, &&TARGET_LOAD_FAST__LOAD_FAST, + &&TARGET_SETUP_ANNOTATIONS, &&TARGET_LOAD_GLOBAL_BUILTIN, - &&TARGET_RETURN_VALUE, &&TARGET_LOAD_GLOBAL_MODULE, - &&TARGET_SETUP_ANNOTATIONS, &&TARGET_STORE_ATTR_INSTANCE_VALUE, - &&TARGET_STORE_ATTR_SLOT, - &&TARGET_STORE_ATTR_WITH_HINT, &&TARGET_POP_EXCEPT, &&TARGET_STORE_NAME, &&TARGET_DELETE_NAME, @@ -110,9 +110,9 @@ static void *opcode_targets[256] = { &&TARGET_IMPORT_NAME, &&TARGET_IMPORT_FROM, &&TARGET_JUMP_FORWARD, + &&TARGET_STORE_ATTR_SLOT, + &&TARGET_STORE_ATTR_WITH_HINT, &&TARGET_STORE_FAST__LOAD_FAST, - &&TARGET_STORE_FAST__STORE_FAST, - &&TARGET_STORE_SUBSCR_DICT, &&TARGET_POP_JUMP_IF_FALSE, &&TARGET_POP_JUMP_IF_TRUE, &&TARGET_LOAD_GLOBAL, @@ -140,9 +140,9 @@ static void *opcode_targets[256] = { &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, &&TARGET_JUMP_BACKWARD, - &&TARGET_STORE_SUBSCR_LIST_INT, + &&TARGET_STORE_FAST__STORE_FAST, &&TARGET_CALL_FUNCTION_EX, - &&TARGET_UNPACK_SEQUENCE_LIST, + &&TARGET_STORE_SUBSCR_DICT, &&TARGET_EXTENDED_ARG, &&TARGET_LIST_APPEND, &&TARGET_SET_ADD, @@ -152,15 +152,15 @@ static void *opcode_targets[256] = { &&TARGET_YIELD_VALUE, &&TARGET_RESUME, &&TARGET_MATCH_CLASS, - &&TARGET_UNPACK_SEQUENCE_TUPLE, - &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, + &&TARGET_STORE_SUBSCR_LIST_INT, + &&TARGET_UNPACK_SEQUENCE_LIST, &&TARGET_FORMAT_VALUE, &&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_STRING, + &&TARGET_UNPACK_SEQUENCE_TUPLE, + &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, &&TARGET_SEND_GEN, &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, &&TARGET_LIST_EXTEND, &&TARGET_SET_UPDATE, &&TARGET_DICT_MERGE, @@ -237,22 +237,22 @@ static void *opcode_targets[256] = { &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, - &&TARGET_DO_TRACING + &&TARGET_INSTRUMENTED_POP_JUMP_IF_NONE, + &&TARGET_INSTRUMENTED_POP_JUMP_IF_NOT_NONE, + &&TARGET_INSTRUMENTED_RESUME, + &&TARGET_INSTRUMENTED_CALL, + &&TARGET_INSTRUMENTED_RETURN_VALUE, + &&TARGET_INSTRUMENTED_YIELD_VALUE, + &&TARGET_INSTRUMENTED_CALL_FUNCTION_EX, + &&TARGET_INSTRUMENTED_JUMP_FORWARD, + &&TARGET_INSTRUMENTED_JUMP_BACKWARD, + &&TARGET_INSTRUMENTED_RETURN_CONST, + &&TARGET_INSTRUMENTED_FOR_ITER, + &&TARGET_INSTRUMENTED_POP_JUMP_IF_FALSE, + &&TARGET_INSTRUMENTED_POP_JUMP_IF_TRUE, + &&TARGET_INSTRUMENTED_END_FOR, + &&TARGET_INSTRUMENTED_END_SEND, + &&TARGET_INSTRUMENTED_INSTRUCTION, + &&TARGET_INSTRUMENTED_LINE, + &&_unknown_opcode }; diff --git a/Python/pystate.c b/Python/pystate.c index d09c1d5743a4c6..1e04887ef04a2c 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -625,7 +625,6 @@ free_interpreter(PyInterpreterState *interp) main interpreter. We fix those fields here, in addition to the other dynamically initialized fields. */ - static void init_interpreter(PyInterpreterState *interp, _PyRuntimeState *runtime, int64_t id, @@ -650,12 +649,22 @@ init_interpreter(PyInterpreterState *interp, _PyGC_InitState(&interp->gc); PyConfig_InitPythonConfig(&interp->config); _PyType_InitCache(interp); + for(int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + interp->monitors.tools[i] = 0; + } + for (int t = 0; t < PY_MONITORING_TOOL_IDS; t++) { + for(int e = 0; e < PY_MONITORING_EVENTS; e++) { + interp->monitoring_callables[t][e] = NULL; + } + } + interp->sys_profile_initialized = false; + interp->sys_trace_initialized = false; if (interp != &runtime->_main_interpreter) { /* Fix the self-referential, statically initialized fields. */ interp->dtoa = (struct _dtoa_state)_dtoa_state_INIT(interp); } - + interp->f_opcode_trace_set = false; interp->_initialized = 1; } @@ -788,6 +797,20 @@ interpreter_clear(PyInterpreterState *interp, PyThreadState *tstate) Py_CLEAR(interp->audit_hooks); + for(int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + interp->monitors.tools[i] = 0; + } + for (int t = 0; t < PY_MONITORING_TOOL_IDS; t++) { + for(int e = 0; e < PY_MONITORING_EVENTS; e++) { + Py_CLEAR(interp->monitoring_callables[t][e]); + } + } + interp->sys_profile_initialized = false; + interp->sys_trace_initialized = false; + for (int t = 0; t < PY_MONITORING_TOOL_IDS; t++) { + Py_CLEAR(interp->monitoring_tool_names[t]); + } + PyConfig_Clear(&interp->config); Py_CLEAR(interp->codec_search_path); Py_CLEAR(interp->codec_search_cache); @@ -845,7 +868,7 @@ interpreter_clear(PyInterpreterState *interp, PyThreadState *tstate) interp->code_watchers[i] = NULL; } interp->active_code_watchers = 0; - + interp->f_opcode_trace_set = false; // XXX Once we have one allocator per interpreter (i.e. // per-interpreter GC) we must ensure that all of the interpreter's // objects have been cleaned up at the point. @@ -1237,6 +1260,7 @@ init_threadstate(PyThreadState *tstate, tstate->datastack_chunk = NULL; tstate->datastack_top = NULL; tstate->datastack_limit = NULL; + tstate->what_event = -1; tstate->_status.initialized = 1; } @@ -1412,8 +1436,14 @@ PyThreadState_Clear(PyThreadState *tstate) "PyThreadState_Clear: warning: thread still has a generator\n"); } - tstate->c_profilefunc = NULL; - tstate->c_tracefunc = NULL; + if (tstate->c_profilefunc != NULL) { + tstate->interp->sys_profiling_threads--; + tstate->c_profilefunc = NULL; + } + if (tstate->c_tracefunc != NULL) { + tstate->interp->sys_tracing_threads--; + tstate->c_tracefunc = NULL; + } Py_CLEAR(tstate->c_profileobj); Py_CLEAR(tstate->c_traceobj); @@ -1986,14 +2016,13 @@ _PyThread_CurrentExceptions(void) if (id == NULL) { goto fail; } - PyObject *exc_info = _PyErr_StackItemToExcInfoTuple(err_info); - if (exc_info == NULL) { - Py_DECREF(id); - goto fail; - } - int stat = PyDict_SetItem(result, id, exc_info); + PyObject *exc = err_info->exc_value; + assert(exc == NULL || + exc == Py_None || + PyExceptionInstance_Check(exc)); + + int stat = PyDict_SetItem(result, id, exc == NULL ? Py_None : exc); Py_DECREF(id); - Py_DECREF(exc_info); if (stat < 0) { goto fail; } diff --git a/Python/specialize.c b/Python/specialize.c index a9d3226ee39f5f..3fa28f409892dc 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -273,7 +273,8 @@ _PyCode_Quicken(PyCodeObject *code) _Py_CODEUNIT *instructions = _PyCode_CODE(code); for (int i = 0; i < Py_SIZE(code); i++) { int previous_opcode = opcode; - opcode = _PyOpcode_Deopt[instructions[i].op.code]; + opcode = _Py_GetBaseOpcode(code, i); + assert(opcode < MIN_INSTRUMENTED_OPCODE); int caches = _PyOpcode_Caches[opcode]; if (caches) { instructions[i + 1].cache = adaptive_counter_warmup(); @@ -1737,6 +1738,7 @@ _Py_Specialize_Call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, { assert(ENABLE_SPECIALIZATION); assert(_PyOpcode_Caches[CALL] == INLINE_CACHE_ENTRIES_CALL); + assert(_Py_OPCODE(*instr) != INSTRUMENTED_CALL); _PyCallCache *cache = (_PyCallCache *)(instr + 1); int fail; if (PyCFunction_CheckExact(callable)) { @@ -2149,7 +2151,9 @@ _Py_Specialize_ForIter(PyObject *iter, _Py_CODEUNIT *instr, int oparg) goto success; } else if (tp == &PyGen_Type && oparg <= SHRT_MAX) { - assert(instr[oparg + INLINE_CACHE_ENTRIES_FOR_ITER + 1].op.code == END_FOR); + assert(instr[oparg + INLINE_CACHE_ENTRIES_FOR_ITER + 1].op.code == END_FOR || + instr[oparg + INLINE_CACHE_ENTRIES_FOR_ITER + 1].op.code == INSTRUMENTED_END_FOR + ); instr->op.code = FOR_ITER_GEN; goto success; } diff --git a/Python/sysmodule.c b/Python/sysmodule.c index f1a294de598420..4d693a1be1f89e 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -3409,6 +3409,7 @@ _PySys_SetPreliminaryStderr(PyObject *sysdict) return _PyStatus_ERR("can't set preliminary stderr"); } +PyObject *_Py_CreateMonitoringObject(void); /* Create sys module without all attributes. _PySys_UpdateConfig() should be called later to add remaining attributes. */ @@ -3458,6 +3459,16 @@ _PySys_Create(PyThreadState *tstate, PyObject **sysmod_p) goto error; } + PyObject *monitoring = _Py_CreateMonitoringObject(); + if (monitoring == NULL) { + goto error; + } + int err = PyDict_SetItemString(sysdict, "monitoring", monitoring); + Py_DECREF(monitoring); + if (err < 0) { + goto error; + } + assert(!_PyErr_Occurred(tstate)); *sysmod_p = sysmod; diff --git a/Tools/build/deepfreeze.py b/Tools/build/deepfreeze.py index ec808526b7bbb7..aba5fecd8b1a99 100644 --- a/Tools/build/deepfreeze.py +++ b/Tools/build/deepfreeze.py @@ -255,7 +255,6 @@ def generate_code(self, name: str, code: types.CodeType) -> str: self.write(f".co_names = {co_names},") self.write(f".co_exceptiontable = {co_exceptiontable},") self.field(code, "co_flags") - self.write("._co_linearray_entry_size = 0,") self.field(code, "co_argcount") self.field(code, "co_posonlyargcount") self.field(code, "co_kwonlyargcount") @@ -276,7 +275,6 @@ def generate_code(self, name: str, code: types.CodeType) -> str: self.write(f".co_qualname = {co_qualname},") self.write(f".co_linetable = {co_linetable},") self.write(f"._co_cached = NULL,") - self.write("._co_linearray = NULL,") self.write(f".co_code_adaptive = {co_code_adaptive},") for i, op in enumerate(code.co_code[::2]): if op == RESUME: diff --git a/Tools/build/generate_opcode_h.py b/Tools/build/generate_opcode_h.py index 614b17df740929..645b9f1de1170b 100644 --- a/Tools/build/generate_opcode_h.py +++ b/Tools/build/generate_opcode_h.py @@ -89,6 +89,7 @@ def main(opcode_py, outfile='Include/opcode.h', internaloutfile='Include/interna HAVE_ARGUMENT = opcode["HAVE_ARGUMENT"] MIN_PSEUDO_OPCODE = opcode["MIN_PSEUDO_OPCODE"] MAX_PSEUDO_OPCODE = opcode["MAX_PSEUDO_OPCODE"] + MIN_INSTRUMENTED_OPCODE = opcode["MIN_INSTRUMENTED_OPCODE"] NUM_OPCODES = len(opname) used = [ False ] * len(opname) @@ -105,9 +106,6 @@ def main(opcode_py, outfile='Include/opcode.h', internaloutfile='Include/interna specialized_opmap[name] = next_op opname_including_specialized[next_op] = name used[next_op] = True - specialized_opmap['DO_TRACING'] = 255 - opname_including_specialized[255] = 'DO_TRACING' - used[255] = True with open(outfile, 'w') as fobj, open(internaloutfile, 'w') as iobj: fobj.write(header) @@ -120,6 +118,8 @@ def main(opcode_py, outfile='Include/opcode.h', internaloutfile='Include/interna fobj.write(DEFINE.format("HAVE_ARGUMENT", HAVE_ARGUMENT)) if op == MIN_PSEUDO_OPCODE: fobj.write(DEFINE.format("MIN_PSEUDO_OPCODE", MIN_PSEUDO_OPCODE)) + if op == MIN_INSTRUMENTED_OPCODE: + fobj.write(DEFINE.format("MIN_INSTRUMENTED_OPCODE", MIN_INSTRUMENTED_OPCODE)) fobj.write(DEFINE.format(name, op)) @@ -130,12 +130,10 @@ def main(opcode_py, outfile='Include/opcode.h', internaloutfile='Include/interna for name, op in specialized_opmap.items(): fobj.write(DEFINE.format(name, op)) - iobj.write("\nextern const uint32_t _PyOpcode_RelativeJump[9];\n") iobj.write("\nextern const uint32_t _PyOpcode_Jump[9];\n") iobj.write("\nextern const uint8_t _PyOpcode_Caches[256];\n") iobj.write("\nextern const uint8_t _PyOpcode_Deopt[256];\n") iobj.write("\n#ifdef NEED_OPCODE_TABLES\n") - write_int_array_from_ops("_PyOpcode_RelativeJump", opcode['hasjrel'], iobj) write_int_array_from_ops("_PyOpcode_Jump", opcode['hasjrel'] + opcode['hasjabs'], iobj) iobj.write("\nconst uint8_t _PyOpcode_Caches[256] = {\n") diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index d82fb967297cae..5924ab7860d8d5 100644 --- a/Tools/c-analyzer/cpython/_parser.py +++ b/Tools/c-analyzer/cpython/_parser.py @@ -328,7 +328,6 @@ def clean_lines(text): _abs('Python/frozen_modules/*.h'): (20_000, 500), _abs('Python/opcode_targets.h'): (10_000, 500), _abs('Python/stdlib_module_names.h'): (5_000, 500), - _abs('Python/importlib.h'): (200_000, 5000), # These large files are currently ignored (see above). _abs('Modules/_ssl_data.h'): (80_000, 10_000), diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index e36024ad16c271..5c173b1041e3e4 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -135,6 +135,9 @@ Objects/stringlib/unicode_format.h - PyFieldNameIter_Type - Objects/unicodeobject.c - EncodingMapType - #Objects/unicodeobject.c - PyFieldNameIter_Type - #Objects/unicodeobject.c - PyFormatterIter_Type - +Python/legacy_tracing.c - _PyLegacyEventHandler_Type - +Objects/object.c - _PyLegacyEventHandler_Type - + ##----------------------- ## static builtin structseq @@ -297,6 +300,8 @@ Objects/object.c - _Py_NotImplementedStruct - Objects/setobject.c - _dummy_struct - Objects/setobject.c - _PySet_Dummy - Objects/sliceobject.c - _Py_EllipsisObject - +Python/instrumentation.c - DISABLE - +Python/instrumentation.c - _PyInstrumentation_MISSING - ################################## @@ -311,11 +316,6 @@ Objects/sliceobject.c - _Py_EllipsisObject - ##----------------------- ## static types -Modules/_collectionsmodule.c - defdict_type - -Modules/_collectionsmodule.c - deque_type - -Modules/_collectionsmodule.c - dequeiter_type - -Modules/_collectionsmodule.c - dequereviter_type - -Modules/_collectionsmodule.c - tuplegetter_type - Modules/_io/bufferedio.c - PyBufferedIOBase_Type - Modules/_io/bytesio.c - _PyBytesIOBuffer_Type - Modules/_io/iobase.c - PyIOBase_Type - diff --git a/Tools/cases_generator/interpreter_definition.md b/Tools/cases_generator/interpreter_definition.md index c7bd38d32ff411..6f902f60c68ee7 100644 --- a/Tools/cases_generator/interpreter_definition.md +++ b/Tools/cases_generator/interpreter_definition.md @@ -137,9 +137,9 @@ The following definitions may occur: `foo_1` is legal. `$` is not legal, nor is `struct` or `class`. The optional `type` in an `object` is the C type. It defaults to `PyObject *`. -The objects before the "--" are the objects on top of the the stack at the start -of the instruction. Those after the "--" are the objects on top of the the stack -at the end of the instruction. +The objects before the "--" are the objects on top of the stack at the start of +the instruction. Those after the "--" are the objects on top of the stack at the +end of the instruction. An `inst` without `stack_effect` is a transitional form to allow the original C code definitions to be copied. It lacks information to generate anything other than the diff --git a/configure b/configure index 9e99352f589f21..4ae8258438e620 100755 --- a/configure +++ b/configure @@ -3104,7 +3104,6 @@ if test "$srcdir" != . -a "$srcdir" != "$(pwd)"; then # resources get picked up before their $srcdir counterparts. # Objects/ -> typeslots.inc # Include/ -> Python.h - # Python/ -> importlib.h # (A side effect of this is that these resources will automatically be # regenerated when building out-of-tree, regardless of whether or not # the $srcdir counterpart is up-to-date. This is an acceptable trade diff --git a/configure.ac b/configure.ac index 31b7a2157a2bcc..4d9eb46f5ce7d8 100644 --- a/configure.ac +++ b/configure.ac @@ -97,7 +97,6 @@ if test "$srcdir" != . -a "$srcdir" != "$(pwd)"; then # resources get picked up before their $srcdir counterparts. # Objects/ -> typeslots.inc # Include/ -> Python.h - # Python/ -> importlib.h # (A side effect of this is that these resources will automatically be # regenerated when building out-of-tree, regardless of whether or not # the $srcdir counterpart is up-to-date. This is an acceptable trade