Condensed-Gradient Boosting - Examples
This project has five main purposes:
- Provides additional comparisons
- Fixes the compared models' bugs
- provides real examples to use `C-GB`
- Provide the wrapper of different compared models
- provides codes for reproduction the paper experiments
Moreover, in this project, one may find additional experiments which they are not in the paper.
First, the following packages should be installed.
- Condensed Gradient Boosting Decision Tree
- GradientBoostingClassifier
- GradientBoostingRegressor
- BoostedTreesEstimator
- Gradient Boosted Decision Tree for Multiple Outputs
For some of the experiments, it would be easier to use related wrappers. For this purpose, the wrappers have designed as the following;
- C-GB - Last version
- GB-Classifier - Last version
- GB-Regressor - Last version
- GBDT-MO - Version 0.0.1
- TFBT - Version 2.4.1
In the latest updates of the
TensorFlow
, theTFBT
was marked as deprecated and replaced with theRF
model. But you can find the base repository in the following link.
There are related codes for different experiments. The experiments are as follows;
- Precision analysis
- Measuring the RMSE
- Measuring the accuracy
- Training the C-GB model
- Measuring the loss curve
- Training time calculations
- Measuring the usage of the memory
- Optimization class for all compared models
- Decision boundaries (For the ensembles and the base learner)
- Related scatter plots for regression (including different experiments)
- Drawing base classifiers (Decision Tree regressors) for different ensembles
In the following, two samples of the included experiments are revealed. Of course, the Decision Boudry includes more samples.
- Decision boundary example
- Regression example
In the following, you will find an example of decision boundary for three studied models, including the Condensed-Gradient Boosting (C-GB
) model.
Here, the C-GB
model is trained at the same time for a multi-output regression problem with two outputs in one training procedure. As the plots show, the model works perfectly for all of the outputs.
To run the related experiments of the paper, the following libraries would be required. These libraries are only for related experiments. For the C-GB
model, you do not need to install any library as it handles the dependencies.
- SciPy
- ctypes
- Pandas
- Numba
- Numpy
- Matplotlib
- tracemalloc
- memory_profiler