Starting the change for XGBoost integration into EVADb.

[jineetd]

No description provided.

[jineetd]

Unit test added is failing with the following error:

ERROR    evadb.utils.logging_manager:plan_executor.py:186 `best_iteration` is only defined when early stopping is used.
Traceback (most recent call last):
  File "/Users/jineetdesai/evadb/evadb/executor/plan_executor.py", line 182, in execute_plan
    yield from output
  File "/Users/jineetdesai/evadb/evadb/executor/create_function_executor.py", line 447, in exec
    ) = self.handle_xgboost_function()
  File "/Users/jineetdesai/evadb/evadb/executor/create_function_executor.py", line 196, in handle_xgboost_function
    model.fit(
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/automl.py", line 1928, in fit
    self._search()
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/automl.py", line 2482, in _search
    self._search_sequential()
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/automl.py", line 2318, in _search_sequential
    analysis = tune.run(
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/tune/tune.py", line 808, in run
    result = evaluation_function(trial_to_run.config)
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/state.py", line 302, in _compute_with_config_base
    ) = compute_estimator(
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/ml.py", line 369, in compute_estimator
    val_loss, metric_for_logging, train_time, pred_time = task.evaluate_model_CV(
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/task/generic_task.py", line 737, in evaluate_model_CV
    val_loss_i, metric_i, train_time_i, pred_time_i = get_val_loss(
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/ml.py", line 494, in get_val_loss
    estimator.fit(X_train, y_train, budget=budget, free_mem_ratio=free_mem_ratio, **fit_kwargs)
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/model.py", line 1652, in fit
    return super().fit(X_train, y_train, budget, free_mem_ratio, **kwargs)
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/flaml/automl/model.py", line 1415, in fit
    self._model.get_booster().best_iteration
  File "/Users/jineetdesai/evadb/test_evadb_venv/lib/python3.8/site-packages/xgboost/core.py", line 2602, in best_iteration
    raise AttributeError(
AttributeError: `best_iteration` is only defined when early stopping is used.

Will need to check this further.

[xzdandy]

Please merge the latest staging please. Thanks!

[xzdandy]

Thanks @jineetd. I will add the dependency requirements.

[xzdandy]

Hi @jineetd, please update the documentation. You can use https://evadb.readthedocs.io/en/stable/source/reference/ai/model-train-sklearn.html as the reference.

[jineetd]

Sure @xzdandy , shall update the documentation for XGBoost.

[xzdandy]

There is no guarantee that last the column is the value to predict I think in this case. We need to store the column to predict in this case. You can again check the ludwig for example.

[jineetd]

But it seems that the auto_train methods for Ludwig and XGBoost are different. In Ludwig, you provide the entire dataset (X + Y) to the auto train method and then specify the column which is supposed to act as Y. Whereas XGBoost auto train specifies the feature matrix X and prediction variable Y.

[xzdandy]

Could you elaborate what is the difference?

What I meant originally is that the following query will not work, because the predict column is not the last one.

CREATE FUNCTION IF NOT EXISTS PredictRent FROM
( SELECT number_of_rooms, number_of_bathrooms, days_on_market, rental_price FROM HomeRentals )
TYPE XGBoost
PREDICT 'number_of_rooms';

[jineetd]

As discussed, we now pass the prediction column to the .py model files

[xzdandy]

We need add documentation on all the parameters XGBoost support. time_limit and metric are the two parameters we support now.

[xzdandy]

Thanks! Could you fix the merge conflicts?