Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Objects that do a database call in their init #484

Open
lfdversluis opened this issue Jun 2, 2016 · 2 comments
Open

Objects that do a database call in their init #484

lfdversluis opened this issue Jun 2, 2016 · 2 comments

Comments

@lfdversluis
Copy link
Contributor

lfdversluis commented Jun 2, 2016

I have run the unit tests of dispersy and profiled them with yappi. I exported the output to callgrind format and exported this into a dotfile using gprof2dot.

Using this code I parsed the dotfile to find inits that call the database eventually.

I noticed that a lot of database functions are not covered by the tests such as executemany. So this list is most definitely incomplete.

Below the results from this run:
dispersy/database.py:database:250:DispersyDatabase.execute
dispersy/crypto.py:crypto:253:M2CryptoPK.__init__
dispersy/util.py:util:146:wrapper

edit: I think the code can be improved. Doing that now

@lfdversluis
Copy link
Contributor Author

lfdversluis commented Jun 2, 2016

Just realized that this whole approach isn't going to work at all.
If a method uses a decorator that is also used for database operations then it will think that it (may, and thus will when doing a dfs) do database operations eventually while this may not be the case.

In any case here is the output that lead me to this conclusion:

dispersy/crypto.py:crypto:253:M2CryptoPK.__init__ -> dispersy/util.py:util:146:wrapper -> dispersy/database.py:database:250:DispersyDatabase.execute
dispersy/crypto.py:crypto:328:M2CryptoSK.__init__ -> dispersy/util.py:util:146:wrapper -> dispersy/database.py:database:250:DispersyDatabase.execute
debugcommunity/node.py:node:33:DebugNode.__init__ -> dispersy/member.py:member:73:Member.__init__ -> dispersy/crypto.py:crypto:146:ECCrypto.key_to_bin -> dispersy/util.py:util:146:wrapper -> dispersy/database.py:database:250:DispersyDatabase.execute
dispersy/member.py:member:73:Member.__init__ -> dispersy/crypto.py:crypto:146:ECCrypto.key_to_bin -> dispersy/util.py:util:146:wrapper -> dispersy/database.py:database:250:DispersyDatabase.execute
dispersy/message.py:message:321:Implementation.__init__ -> dispersy/util.py:util:146:wrapper -> dispersy/database.py:database:250:DispersyDatabase.execute

Everywhere is the wrapper involved, which is the attach_runtime_statistics decorator method.

@lfdversluis
Copy link
Contributor Author

lfdversluis commented Jun 2, 2016

Plan 2:
Analyze the call stack at the database function level, if it contains an __init__ throw an exception and fix the case. Rinse and repeat. (Thread switching kills this approach though)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant