You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently under pinot.server.instance.dataDir we can only configure single directory path with that we cannot utilize the all the disks present in the system. Because of this limitation we need to add more nodes to launch servers and launch servers per disk to accommodate the data.
By having simple of map of segment vs directory path we can use multiple directories under data directories. It would be better to introduce the new configuration pinot.server.instance.dataDirs and while allocating directory to a segment we can have round robin based or size availability based approaches. During the start up also we can build the map by looking at the segments present in the directories configured and update when ever required.
Working on the design document to support the same and will share in few days.
The text was updated successfully, but these errors were encountered:
Currently under pinot.server.instance.dataDir we can only configure single directory path with that we cannot utilize the all the disks present in the system. Because of this limitation we need to add more nodes to launch servers and launch servers per disk to accommodate the data.
By having simple of map of segment vs directory path we can use multiple directories under data directories. It would be better to introduce the new configuration pinot.server.instance.dataDirs and while allocating directory to a segment we can have round robin based or size availability based approaches. During the start up also we can build the map by looking at the segments present in the directories configured and update when ever required.
Working on the design document to support the same and will share in few days.
The text was updated successfully, but these errors were encountered: