Vasilis/orf speed

[vsyrgkanis]

• Speed up prediction time of ORF. Replaced in-class pointwise_effect function with static method that takes the subset of arrays for each point. Also using processes instead of threading. Leads to at around a 10x speed-up.
• Enabled global residualization in continuous treatment orthoforest. This then replicates exactly the functionality of grf and brings parity with grf. Added class causal_forest.CausalForest, that essentially is a continuous treatment ortho forest with global residualization. Global residualization is computationally much faster, but might lose statistical power especially with W not none, as it fits a global residualizer and not a local residualizer for each target test point. This now allows us to do sth like ForestDML but with many treatments. Moreover, unlike ForestDML this uses exactly the splitting criterion of the causal forest.
• Enabled _inference in all ortho forests by adding inference methods in the BLBInference class in orthoforest.

Speed up example

For instance, here is a code example with 1000 trees and 1000 prediction points:

np.random.seed(123)
n = 2000
p = 10
X = np.random.normal(size=(n, p))
def true_propensity(x): return .4 + .2 * (x[:, 0] > 0)
def true_effect(x): return (x[:, 0] * (x[:, 0] > 0))
def true_conf(x): return x[:, 1] + np.clip(x[:, 2], - np.inf, 0)

T = np.random.binomial(1, true_propensity(X))
Y = true_effect(X) * T + true_conf(X) + np.random.normal(size=(n,))

X_test = np.zeros((1000, p))
X_test[:, 0] = np.linspace(-2, 2, 1000)


est3 = DiscreteTreatmentOrthoForest(model_Y=Lasso(alpha=0.01),
                                    propensity_model=LogisticRegression(C=1),
                                    model_Y_final=WeightedLassoCV(cv=3),
                                    propensity_model_final=LogisticRegressionCV(cv=3),
                                    n_trees=1000, min_leaf_size=10)
est3.fit(Y, T, X)
lb, ub = est3.effect_interval(X_test)
pred3 = est3.effect(X_test)

Here is the running time of the new code:

and here is the running time of the current master code:

Moreover, now there is a big benefit for parallelism as with 8 cores we get around 5x speed-up, compared to non-parallel.