You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the past HoloViews has generally been used for reasonably small datasets, where loading or generating the data is many times more expensive than creating param.Parameterized objects. However in this project we'll be working with iris cubes, which can potentially be very large and can easily be grouped into tens or even hundreds of thousands of individual Images, Contours or other Element types. At that point creating this many objects, which may each also create other parameterized objects, can become a serious bottleneck.
param was designed with the idea that the user should be able to tweak each parameter on each object, when declaring many thousands of objects at once however it is clear that the user will not tweak each individual parameter and it might indeed be desirable to share certain parameters across all these objects. Therefore there is this tension between being able to tweak parameters and quickly declaring thousands of identical objects.
This issue is meant as a place to brainstorm how we can speed up creation of objects in HoloViews, by providing better control over when Parameterized objects make deep copies of their parameters. This could take the form of a context manager, which could be used in any place when a lot of identical objects will be generated, an option on Parameterized objects or something else entirely.
I've already opened holoviz/param#122, which speeds up object creation by ~50%, such that we can create 20k Image objects in about 5 seconds. Hopefully we can reduce that further by coming up with a good proposal here.
The text was updated successfully, but these errors were encountered:
In the past HoloViews has generally been used for reasonably small datasets, where loading or generating the data is many times more expensive than creating param.Parameterized objects. However in this project we'll be working with iris cubes, which can potentially be very large and can easily be grouped into tens or even hundreds of thousands of individual Images, Contours or other Element types. At that point creating this many objects, which may each also create other parameterized objects, can become a serious bottleneck.
param was designed with the idea that the user should be able to tweak each parameter on each object, when declaring many thousands of objects at once however it is clear that the user will not tweak each individual parameter and it might indeed be desirable to share certain parameters across all these objects. Therefore there is this tension between being able to tweak parameters and quickly declaring thousands of identical objects.
This issue is meant as a place to brainstorm how we can speed up creation of objects in HoloViews, by providing better control over when Parameterized objects make deep copies of their parameters. This could take the form of a context manager, which could be used in any place when a lot of identical objects will be generated, an option on Parameterized objects or something else entirely.
I've already opened holoviz/param#122, which speeds up object creation by ~50%, such that we can create 20k Image objects in about 5 seconds. Hopefully we can reduce that further by coming up with a good proposal here.
The text was updated successfully, but these errors were encountered: