Model Analysis

How to perform an analysis of the model weights

Introduction

It is often useful to know which features are have the most impact on model decisions and Adaboost models are rather obscure on that matter.

Expected feature weights

One way of rating features is to compute the expected weight vector at the example level over a set of examples (for instance, the test set). Here is a small script to perform this calculation using the WEAKC lines from icsiboost output with the -V option.

icsiboost -S stem -W 3 -C -V < stem.data 2>&1 \
  | grep WEAKC \
  | sed 's/[<>=]/ /' \
  | awk '
        function abs(x){return x<0 ? -x : x}
        {t[$2]+=$7}
        END{
            for(i in t){
                total+=abs(t[i])
            }
            for(i in t){
                print i,100*abs(t[i])/total
            }
        }'