C_GB-EX

Condensed-Gradient Boosting - Examples

About

This project has five main purposes:

• Provides additional comparisons
• Fixes the compared models' bugs
• provides real examples to use `C-GB`
• Provide the wrapper of different compared models
• provides codes for reproduction the paper experiments

Moreover, in this project, one may find additional experiments which they are not in the paper.

Usage

First, the following packages should be installed.

• Condensed Gradient Boosting Decision Tree
• GradientBoostingClassifier
• GradientBoostingRegressor
• BoostedTreesEstimator
• Gradient Boosted Decision Tree for Multiple Outputs

For some of the experiments, it would be easier to use related wrappers. For this purpose, the wrappers have designed as the following;

• GBDTM-O-wrapper
• TFBT-wrapper

Models used for comparison

• C-GB - Last version
• GB-Classifier - Last version
• GB-Regressor - Last version
• GBDT-MO - Version 0.0.1
• TFBT - Version 2.4.1

In the latest updates of the TensorFlow, the TFBT was marked as deprecated and replaced with the RF model. But you can find the base repository in the following link.

The included examples

There are related codes for different experiments. The experiments are as follows;

• Precision analysis
• Measuring the RMSE
• Measuring the accuracy
• Training the C-GB model
• Measuring the loss curve
• Training time calculations
• Measuring the usage of the memory
• Optimization class for all compared models
• Decision boundaries (For the ensembles and the base learner)
• Related scatter plots for regression (including different experiments)
• Drawing base classifiers (Decision Tree regressors) for different ensembles

Sample of some of the experiments

In the following, two samples of the included experiments are revealed. Of course, the Decision Boudry includes more samples.

• Decision boundary example
• Regression example

Visualization

Decision boundary

In the following, you will find an example of decision boundary for three studied models, including the Condensed-Gradient Boosting (C-GB) model.

Regression example

Here, the C-GB model is trained at the same time for a multi-output regression problem with two outputs in one training procedure. As the plots show, the model works perfectly for all of the outputs.

Requirements

To run the related experiments of the paper, the following libraries would be required. These libraries are only for related experiments. For the C-GB model, you do not need to install any library as it handles the dependencies.

• SciPy
• ctypes
• Pandas
• Numba
• Numpy
• Matplotlib
• tracemalloc
• memory_profiler