Prune unneeded transitive dependencies #3395
Conversation
|
In terms of performance improvement seen, I've tested this in combination with #3394. That commit reduces my meson runtime from over 10 to just below 9. Adding in this commit reduces the meson runtime on the same test (building dpdk project with all examples) to BELOW 3 seconds! That's approx 3x increase in speed. |
mesonbuild/build.py
Outdated
| @@ -803,12 +803,13 @@ def get_outputs(self): | |||
| def get_extra_args(self, language): | |||
| return self.extra_args.get(language, []) | |||
|
|
|||
| def get_dependencies(self): | |||
| def get_dependencies(self, exclude=[]): | |||
There was a problem hiding this comment.
Don't initialize kwargs with mutable objects: http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments
mesonbuild/build.py
Outdated
| if t not in transitive_deps and t not in exclude: | ||
| transitive_deps.append(t) | ||
| if isinstance(t, StaticLibrary): | ||
| transitive_deps += t.get_dependencies(transitive_deps + exclude) |
There was a problem hiding this comment.
Nitpick: it's easier to read if this is written as:
if t in transitive_deps or t in exclude:
continue
transitive_deps.append(t)
...There was a problem hiding this comment.
ok, thanks. Let me rework. My python is rather rusty.
When getting dependencies, we don't need to get the same dependencies and dependency chains multiple times. If library a depends on x, y and z, and library b depends on a, then we should not have to iterate through x, y and z multiple times. Pruning at the stage of scanning the dependencies leads to significant time savings when running meson
bruce-richardson
force-pushed
the
dependency_optimization
branch
from
2601ceb to
f9a594f
Compare
|
reworked following feedback. Performance improvement is now only ~2.5x, which I'm still happy enough about! :-) |
Interesting that the kwarg change reduced the performance. Can you try other versions such as using tuples (which are immutable) instead of lists? |
|
hmmm, maybe it didn't. Now that I think about it, I think I may have dropped change #3394 from the tree when I did the later test. It doesn't matter much anyway, it's fast enough for me with this change. |
When getting dependencies, we don't need to get the same dependencies and
dependency chains multiple times. If library a depends on x, y and z, and
library b depends on a, then we should not have to iterate through x, y and
z multiple times. Pruning at the stage of scanning the dependencies leads
to significant time savings when running meson