Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schedule for a beta release of greenlet 2.0? #303

Closed
vstinner opened this issue May 23, 2022 · 3 comments
Closed

Schedule for a beta release of greenlet 2.0? #303

vstinner opened this issue May 23, 2022 · 3 comments

Comments

@vstinner
Copy link
Contributor

Hi,

The stable branch of greenlet (greenlet 1.x) is currently not compatible with Python 3.11. The compatibility was added in the master branch, in the released greenlet 2.0.0 alpha 2. Problem: it's an alpha version, I would prefer to not use it in production.

In Fedora, we are working on replacing Python 3.10 with Python 3.11, and we would like to use greenlet 2.0. It would make me more comfortable if this branch would reach at least the beta phase. Is there a schedule for that? Can I help you?

It seems like gevent and eventlet just works with Python 3.11 beta1 and greenlet 2.0.0a2. There are compiler warnings #301 but Python 3.11 got fixed and the fix will be part of Python 3.11 beta2 scheduled next week.

Links about past issues.

Both greenlet commtis are part of released greenlet 2.0.0a2.

@hroncok
Copy link

hroncok commented Jun 2, 2022

The stable branch of greenlet (greenlet 1.x) is currently not compatible with Python 3.11.

I have a patch for greenlet 1.x that appears to be working. Since there is no branch I could send PR to, I've added it here, in case other distributors find it useful. Unfortunately, it includes internal/pycore_frame.h.

diff --git a/src/greenlet/greenlet.c b/src/greenlet/greenlet.c
index f47bbf8..999989d 100644
--- a/src/greenlet/greenlet.c
+++ b/src/greenlet/greenlet.c
@@ -170,9 +170,11 @@ green_clear_exc(PyGreenlet* g)
 {
 #if GREENLET_PY37
     g->exc_info = NULL;
-    g->exc_state.exc_type = NULL;
     g->exc_state.exc_value = NULL;
+#if !GREENLET_PY311
+    g->exc_state.exc_type = NULL;
     g->exc_state.exc_traceback = NULL;
+#endif
     g->exc_state.previous_item = NULL;
 #else
     g->exc_type = NULL;
@@ -525,8 +527,12 @@ g_switchstack(void)
     { /* save state */
         PyGreenlet* current = ts_current;
         PyThreadState* tstate = PyThreadState_GET();
+#if GREENLET_PY311
+        current->recursion_depth = tstate->recursion_limit - tstate->recursion_remaining;
+#else
         current->recursion_depth = tstate->recursion_depth;
         current->top_frame = tstate->frame;
+#endif
 #if GREENLET_PY37
         current->context = tstate->context;
 #endif
@@ -551,6 +557,15 @@ g_switchstack(void)
          */
         current->cframe = tstate->cframe;
         ts__g_switchstack_use_tracing = tstate->cframe->use_tracing;
+#if GREENLET_PY311
+        current->current_frame = tstate->cframe->current_frame;
+        current->datastack_chunk = tstate->datastack_chunk;
+        current->datastack_top = tstate->datastack_top;
+        current->datastack_limit = tstate->datastack_limit;
+        PyFrameObject *frame = PyThreadState_GetFrame((PyThreadState *)tstate);
+        Py_XDECREF(frame); /* PyThreadState_GetFrame gives us a new reference. */
+        current->top_frame = frame;
+#endif
 #endif
     }
 
@@ -574,8 +589,6 @@ g_switchstack(void)
         PyGreenlet* target = ts_target;
         PyGreenlet* origin = ts_current;
         PyThreadState* tstate = PyThreadState_GET();
-        tstate->recursion_depth = target->recursion_depth;
-        tstate->frame = target->top_frame;
         target->top_frame = NULL;
 
 #if GREENLET_PY37
@@ -607,7 +620,16 @@ g_switchstack(void)
         */
         tstate->cframe->use_tracing = ts__g_switchstack_use_tracing;
 #endif
-
+#if GREENLET_PY311
+        tstate->recursion_remaining = tstate->recursion_limit - target->recursion_depth;
+        tstate->cframe->current_frame = target->current_frame;
+        tstate->datastack_chunk = target->datastack_chunk;
+        tstate->datastack_top = target->datastack_top;
+        tstate->datastack_limit = target->datastack_limit;
+#else
+        tstate->recursion_depth = target->recursion_depth;
+        tstate->frame = target->top_frame;
+#endif
         assert(ts_origin == NULL);
         Py_INCREF(target);
         ts_current = target;
@@ -810,7 +832,7 @@ static int GREENLET_NOINLINE(g_initialstub)(void* mark)
       We want to defer copying the state info until we're sure
       we need it and are in a stable place to do so.
     */
-    CFrame trace_info;
+    _PyCFrame trace_info;
 #endif
     /* save exception in case getattr clears it */
     PyErr_Fetch(&exc, &val, &tb);
@@ -875,7 +897,11 @@ static int GREENLET_NOINLINE(g_initialstub)(void* mark)
     }
     self->top_frame = NULL;
     green_clear_exc(self);
+#if GREENLET_PY311
+    self->recursion_depth = PyThreadState_GET()->recursion_limit - PyThreadState_GET()->recursion_remaining;
+#else
     self->recursion_depth = PyThreadState_GET()->recursion_depth;
+#endif
 
     /* restore arguments in case they are clobbered */
     ts_target = self;
@@ -1006,13 +1032,13 @@ green_new(PyTypeObject* type, PyObject* args, PyObject* kwds)
           it uses the ``root_cframe`` just to have something to put there.
           However, once the greenlet is actually switched to for the first
           time, ``g_initialstub`` (which doesn't actually "return" while the
-          greenlet is running) stores a new CFrame on its local stack, and
+          greenlet is running) stores a new _PyCFrame on its local stack, and
           copies the appropriate values from the currently running CFrame;
-          this is then made the CFrame for the newly-minted greenlet.
+          this is then made the _PyCFrame for the newly-minted greenlet.
           ``g_initialstub`` then proceeds to call ``glet.run()``, which
-          results in ``PyEval_...`` adding the CFrame to the list. Switches
+          results in ``PyEval_...`` adding the _PyCFrame to the list. Switches
           continue as normal. Finally, when the greenlet finishes, the call to
-          ``glet.run()`` returns and the CFrame is taken out of the linked
+          ``glet.run()`` returns and the _PyCFrame is taken out of the linked
           list and the stack value is now unused and free to expire.
         */
         ((PyGreenlet*)o)->cframe = &PyThreadState_GET()->root_cframe;
@@ -1121,9 +1147,11 @@ green_traverse(PyGreenlet* self, visitproc visit, void* arg)
     Py_VISIT(self->context);
 #endif
 #if GREENLET_PY37
-    Py_VISIT(self->exc_state.exc_type);
     Py_VISIT(self->exc_state.exc_value);
+#if !GREENLET_PY311
+    Py_VISIT(self->exc_state.exc_type);
     Py_VISIT(self->exc_state.exc_traceback);
+#endif
 #else
     Py_VISIT(self->exc_type);
     Py_VISIT(self->exc_value);
@@ -1159,9 +1187,11 @@ green_clear(PyGreenlet* self)
     Py_CLEAR(self->context);
 #endif
 #if GREENLET_PY37
-    Py_CLEAR(self->exc_state.exc_type);
     Py_CLEAR(self->exc_state.exc_value);
+#if !GREENLET_PY311
+    Py_CLEAR(self->exc_state.exc_type);
     Py_CLEAR(self->exc_state.exc_traceback);
+#endif
 #else
     Py_CLEAR(self->exc_type);
     Py_CLEAR(self->exc_value);
@@ -1253,9 +1283,11 @@ green_dealloc(PyGreenlet* self)
     Py_CLEAR(self->context);
 #endif
 #if GREENLET_PY37
-    Py_CLEAR(self->exc_state.exc_type);
     Py_CLEAR(self->exc_state.exc_value);
+#if !GREENLET_PY311
+    Py_CLEAR(self->exc_state.exc_type);
     Py_CLEAR(self->exc_state.exc_traceback);
+#endif
 #else
     Py_CLEAR(self->exc_type);
     Py_CLEAR(self->exc_value);
diff --git a/src/greenlet/greenlet.h b/src/greenlet/greenlet.h
index 830bef8..4a26ace 100644
--- a/src/greenlet/greenlet.h
+++ b/src/greenlet/greenlet.h
@@ -14,6 +14,15 @@ extern "C" {
 /* This is deprecated and undocumented. It does not change. */
 #define GREENLET_VERSION "1.0.0"
 
+#if PY_VERSION_HEX >= 0x30B00A6
+# define GREENLET_PY311 1
+/* We need this for _PyInterpreterFrame apparently */
+#include <internal/pycore_frame.h>
+#else
+#define GREENLET_PY311 0
+#define _PyCFrame CFrame
+#endif
+
 typedef struct _greenlet {
     PyObject_HEAD
     char* stack_start;
@@ -25,6 +34,12 @@ typedef struct _greenlet {
     PyObject* run_info;
     struct _frame* top_frame;
     int recursion_depth;
+#if GREENLET_PY311
+    _PyInterpreterFrame *current_frame;
+    _PyStackChunk *datastack_chunk;
+    PyObject **datastack_top;
+    PyObject **datastack_limit;
+#endif
     PyObject* weakreflist;
 #if PY_VERSION_HEX >= 0x030700A3
     _PyErr_StackItem* exc_info;
@@ -39,7 +54,7 @@ typedef struct _greenlet {
     PyObject* context;
 #endif
 #if PY_VERSION_HEX >= 0x30A00B1
-    CFrame* cframe;
+    _PyCFrame* cframe;
 #endif
 } PyGreenlet;
 

@vstinner
Copy link
Contributor Author

vstinner commented Jun 6, 2022

I have a patch for greenlet 1.x that appears to be working.

For that, I created:

@jamadden
Copy link
Contributor

Thanks to everyone for their help. Version 1.1.3 and version 2.0.0 with Python 3.11 support have been released. (gevent 21.10.2 is required for greenlet 2.0.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants