Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools.target: use slots for Channel and User types #2233

Merged
merged 1 commit into from
Jan 8, 2022
Merged

Conversation

dgw
Copy link
Member

@dgw dgw commented Dec 31, 2021

Description

Tin, really. Python has this feature where you can tell it to store instance attributes in a "slot", and using it can make small objects more time- and space-efficient by eliminating the need to look up attributes in the instance's __dict__.

This can save some significant memory when Sopel is in many and/or large channels. For example, without this patch (in bytes):

  • Size of empty channel: 1200
  • Size of channel with 100 users: 84560
  • Size of channel with 1000 users: 817944

And with this patch (also in bytes):

  • Size of empty channel: 680
  • Size of channel with 100 users: 72608
  • Size of channel with 1000 users: 705304

An empty channel is reduced by nearly half, and channels with some significant user count can be reduced by some 10-15%.

I have not checked the impact of this patch on execution time, but would be glad to do so if requested.

Notes

These values were calculated using Pympler.asizeof (basically a recursive version of sys.getsizeof) in a simple test script:

from sopel.tools import Identifier, target
from pympler.asizeof import asizeof

c = target.Channel(Identifier('#Sopel'))

print('Size of empty channel:', asizeof(c))

for n in range(1, 100):
    name = 'user{}'.format(n)
    c.add_user(
        target.User(
            Identifier(name)
            name,
            'some.example.org'
        )
    )

print('Size of channel with 100 users:', asizeof(c))

for n in range(101, 1000):
    name = 'user{}'.format(n)
    c.add_user(
        target.User(
            Identifier(name),
            name,
            'some.example.org'
        )
    )

print('Size of channel with 1000 users:', asizeof(c))

Checklist

  • I have read CONTRIBUTING.md
  • I can and do license this contribution under the EFLv2
  • No issues are reported by make qa (runs make quality and make test)
    • flake8 flags some errors in my target_sizes.py test script, which I didn't bother to make PEP8-compliant.
  • I have tested the functionality of the things this change touches

@dgw dgw added the Tweak label Dec 31, 2021
@dgw dgw added this to the 8.0.0 milestone Dec 31, 2021
@dgw dgw requested a review from a team December 31, 2021 21:45
Copy link
Contributor

@Exirel Exirel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking of the year.

sopel/tools/target.py Outdated Show resolved Hide resolved
sopel/tools/target.py Outdated Show resolved Hide resolved
This can save some significant memory when Sopel is in many and/or large
channels. For example, without this patch (in bytes):

- Size of empty channel: 1200
- Size of channel with 100 users: 84560
- Size of channel with 1000 users: 817944

And with this patch (also in bytes):

- Size of empty channel: 680
- Size of channel with 100 users: 72608
- Size of channel with 1000 users: 705304

An empty channel is reduced by nearly half, and channels with some
significant user count can be reduced by some 10-15%.

These values were calculated using `Pympler.asizeof.asizeof()`, in a
simple test script:

    from sopel.tools import Identifier, target
    from pympler.asizeof import asizeof

    c = target.Channel(Identifier('#Sopel'))

    print('Size of empty channel:', asizeof(c))

    for n in range(1, 100):
        name = 'user{}'.format(n)
        c.add_user(
            target.User(
                Identifier(name)
                name,
                'some.example.org'
            )
        )

    print('Size of channel with 100 users:', asizeof(c))

    for n in range(101, 1000):
        name = 'user{}'.format(n)
        c.add_user(
            target.User(
                Identifier(name),
                name,
                'some.example.org'
            )
        )

    print('Size of channel with 1000 users:', asizeof(c))
@dgw dgw requested a review from Exirel December 31, 2021 23:19
Copy link
Contributor

@Exirel Exirel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge that asap.

@dgw dgw merged commit 1195e78 into master Jan 8, 2022
@dgw dgw deleted the target-slots branch January 8, 2022 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants