Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-104787: use managed dict in _asyncio #104795

Merged
merged 1 commit into from
May 26, 2023
Merged

GH-104787: use managed dict in _asyncio #104795

merged 1 commit into from
May 26, 2023

Conversation

kumaraditya303
Copy link
Contributor

@kumaraditya303 kumaraditya303 commented May 23, 2023 •
edited by bedevere-bot
Loading

@kumaraditya303 kumaraditya303 added performance Performance or resource usage topic-asyncio labels May 23, 2023
@bedevere-bot bedevere-bot mentioned this pull request May 23, 2023
Closed
@kumaraditya303 kumaraditya303 added skip news 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section labels May 23, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit d1cee85 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label May 23, 2023
@kumaraditya303 kumaraditya303 marked this pull request as ready for review May 23, 2023 10:33
@itamaro
Copy link
Contributor

itamaro commented May 23, 2023

is this a runtime optimization? memory optimization? what's the impact?

@gvanrossum
Copy link
Member

From reading the 3.12 docs this seems to be primarily meant as a "better" way to support __dict__ , letting the standard object infrastructure manage the dict instead of managing it "by hand". (The dict is still present, so it doesn't really save memory, but there's a potential for future optimizations once a type adopts this.)

@kumaraditya303 Shouldn't we do this for weakrefs too? There's a corresponding Py_TPFLAGS_MANAGED_WEAKREF flag.

@kumaraditya303
Copy link
Contributor Author

From reading the 3.12 docs this seems to be primarily meant as a "better" way to support dict , letting the standard object infrastructure manage the dict instead of managing it "by hand". (The dict is still present, so it doesn't really save memory, but there's a potential for future optimizations once a type adopts this.)

See #95707, it does save memory as the dict isn't really allocated, the values pointer is used directly so it saves memory.

Shouldn't we do this for weakrefs too? There's a corresponding Py_TPFLAGS_MANAGED_WEAKREF flag.

Haven't thought about using that yet, will check it in another PR for now.

@gvanrossum gvanrossum mentioned this pull request May 23, 2023
Open
3 tasks
@gvanrossum
Copy link
Member

I see. I've forgotten how this works, I've added a ping asking for more documentation.

@kumaraditya303
Copy link
Contributor Author

I see. I've forgotten how this works, I've added a ping asking for more documentation.

Ok, I'll merge this now.

@kumaraditya303 kumaraditya303 merged commit 72c3d2e into python:main May 26, 2023
@kumaraditya303 kumaraditya303 deleted the managed-dict branch May 26, 2023 05:00
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x has failed when building commit 72c3d2e.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/345/builds/4934) and take a look at the build logs.
  4. Check if the failure is related to this commit (72c3d2e) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/345/builds/4934

Failed tests:

  • test__xxsubinterpreters

Summary of the results of the build (if available):

==

Click to see traceback logs 
remote: Enumerating objects: 7, done.        
remote: Counting objects:  14% (1/7)        
remote: Counting objects:  28% (2/7)        
remote: Counting objects:  42% (3/7)        
remote: Counting objects:  57% (4/7)        
remote: Counting objects:  71% (5/7)        
remote: Counting objects:  85% (6/7)        
remote: Counting objects: 100% (7/7)        
remote: Counting objects: 100% (7/7), done.        
remote: Compressing objects:  25% (1/4)        
remote: Compressing objects:  50% (2/4)        
remote: Compressing objects:  75% (3/4)        
remote: Compressing objects: 100% (4/4)        
remote: Compressing objects: 100% (4/4), done.        
remote: Total 4 (delta 3), reused 0 (delta 0), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '72c3d2e105f120f6f2bce410699b34dac4e948fd'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 72c3d2e105 GH-104787: use managed dict in `_asyncio` (#104795)
Switched to and reset branch 'main'

configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

Fatal Python error: drop_gil: PyMUTEX_LOCK(gil->switch_mutex) failed
Python runtime state: initialized

Thread 0x00007fa8c3e13280 (most recent call first):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test__xxsubinterpreters.py", line 80 in clean_up_interpreters
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test__xxsubinterpreters.py", line 88 in tearDown
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 594 in _callTearDown
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 637 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 690 in __call__
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/runner.py", line 240 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 1115 in _run_suite
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 1241 in run_unittest
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 294 in _test_module
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 330 in _runtest_inner2
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 373 in _runtest_inner
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 248 in _runtest
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 278 in runtest
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 365 in rerun_failed_tests
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 783 in _main
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 738 in main
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 802 in main
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/__main__.py", line 2 in <module>
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 88 in _run_code
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 198 in _run_module_as_main
Fatal Python error: Aborted

Thread 0x00007fa8c3e13280 (most recent call first):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test__xxsubinterpreters.py", line 80 in clean_up_interpreters
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test__xxsubinterpreters.py", line 88 in tearDown
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 594 in _callTearDown
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 637 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 690 in __call__
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/runner.py", line 240 in run
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 1115 in _run_suite
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 1241 in run_unittest
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 294 in _test_module
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 330 in _runtest_inner2
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 373 in _runtest_inner
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 248 in _runtest
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 278 in runtest
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 365 in rerun_failed_tests
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 783 in _main
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 738 in main
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 802 in main
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/__main__.py", line 2 in <module>
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 88 in _run_code
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 198 in _run_module_as_main

Extension modules: _testcapi, _xxsubinterpreters (total: 2)
Aborted
make: *** [Makefile:1998: buildbottest] Error 134

Cannot open file '/root/buildarea/3.x.angelico-debian-amd64/build/test-results.xml' for upload

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PixelCode01 PixelCode01 PixelCode01 approved these changes

@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner

@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner

@gvanrossum gvanrossum Awaiting requested review from gvanrossum

@willingc willingc Awaiting requested review from willingc willingc is a code owner

Assignees
No one assigned
Labels
performance Performance or resource usage skip news topic-asyncio
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@kumaraditya303 @bedevere-bot @itamaro @gvanrossum @PixelCode01