Skip to content

Commit

Permalink
updating user documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
manishamde committed Apr 24, 2014
1 parent fecf89a commit 719d009
Showing 1 changed file with 2 additions and 5 deletions.
7 changes: 2 additions & 5 deletions docs/mllib-classification-regression.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,12 +294,9 @@ The recursive tree construction is stopped at a node when one of the two conditi
1. The node depth is equal to the `maxDepth` training paramemter
2. No split candidate leads to an information gain at the node.

### Practical Limitations

The tree implementation stores an Array[Double] of size *O(#features \* #splits \* 2^maxDepth)* in memory for aggregating histograms over partitions. The current implementation might not scale to very deep trees since the memory requirement grows exponentially with tree depth.

Please drop us a line if you encounter any issues. We are planning to solve this problem in the near future and real-world examples will be great.
### Implementation Details

The tree implementation stores an Array[Double] of size *O(#features \* #splits \* 2^maxDepth)* in memory for aggregating histograms over partitions. Based upon the 'maxMemory' parameter set during training (default is 128 MB), the task is broken down into smaller groups to avoid out-of-memory errors during computation.

## Implementation in MLlib

Expand Down

0 comments on commit 719d009

Please sign in to comment.