Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up creation of HoloViews objects #29

Closed
philippjfr opened this issue Mar 29, 2016 · 1 comment
Closed

Speed up creation of HoloViews objects #29

philippjfr opened this issue Mar 29, 2016 · 1 comment
Assignees
Milestone

Comments

@philippjfr
Copy link
Contributor

In the past HoloViews has generally been used for reasonably small datasets, where loading or generating the data is many times more expensive than creating param.Parameterized objects. However in this project we'll be working with iris cubes, which can potentially be very large and can easily be grouped into tens or even hundreds of thousands of individual Images, Contours or other Element types. At that point creating this many objects, which may each also create other parameterized objects, can become a serious bottleneck.

param was designed with the idea that the user should be able to tweak each parameter on each object, when declaring many thousands of objects at once however it is clear that the user will not tweak each individual parameter and it might indeed be desirable to share certain parameters across all these objects. Therefore there is this tension between being able to tweak parameters and quickly declaring thousands of identical objects.

This issue is meant as a place to brainstorm how we can speed up creation of objects in HoloViews, by providing better control over when Parameterized objects make deep copies of their parameters. This could take the form of a context manager, which could be used in any place when a lot of identical objects will be generated, an option on Parameterized objects or something else entirely.

I've already opened holoviz/param#122, which speeds up object creation by ~50%, such that we can create 20k Image objects in about 5 seconds. Hopefully we can reduce that further by coming up with a good proposal here.

@marqh marqh added this to the v0.1.0 milestone Apr 7, 2016
@marqh
Copy link
Member

marqh commented Apr 18, 2016

param optimisation will address this, closing for this project

@marqh marqh closed this as completed Apr 18, 2016
@marqh marqh removed the ready label Apr 18, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants