[QUESTION] Train and inference TFTModel with past_covariates values only

[nhthanh0809]

Hi,
I'm struggling with problem of training and testing with TFTModel within past_covariates data only.
More detail I have a data includes 6 columns (1 target, and 5 features).
My model need to forecast target value based on information from 5 features.
My training code is below running without any error:

train_target, train_covariates = create_timeseries(train_data, "target", ["ft_1", "ft_2", "ft_3", "ft_4", "ft_5"])
val_target, val_covariates = create_timeseries(val_data, "target", ["ft_1", "ft_2", "ft_3", "ft_4", "ft_5"])

    
scaler_target = Scaler()
scaler_covariates = Scaler()

train_target = scaler_target.fit_transform(train_target)
train_covariates = scaler_covariates.fit_transform(train_covariates)

val_target = scaler_target.transform(val_target)
val_covariates = scaler_covariates.transform(val_covariates)

model = TFTModel(
        input_chunk_length=30,  #   forecast based on 30 past days
        output_chunk_length=7,  # forecast next 7 days
        hidden_size=64,
        lstm_layers=1,
        num_attention_heads=4,
        # hidden_continuous_size=16,
        dropout=0.1,
        batch_size=16,
        n_epochs=100,
        # learning_rate=1e-4,
        add_encoders={"cyclic": {"future": ["day"]}},
        add_relative_index=False,
        likelihood=None,
        random_state=42,
        model_name="tft_model",
        save_checkpoints=True,
        force_reset=True,
        optimizer_kwargs={"lr": 1e-4},
)

model.fit(
        train_target,
        past_covariates=train_covariates,
        val_series=val_target,
        val_past_covariates=val_covariates,
        verbose=True,
    )

But when i load trained model then run inference with new data:

test_target, test_covariates = create_timeseries(test_data, "target", list_of_covariates)
scaler_target = Scaler()
scaler_covariates = Scaler()

test_target = scaler_target.fit_transform(test_target)
test_covariates = scaler_covariates.fit_transform(test_covariates)

model = TFTModel.load_from_checkpoint(model_name, work_dir=checkpoint_dir)
forecast = model.predict(7, past_covariates=test_covariates)

I got error in model.predict line with below infomation:

ValueError: For the given forecasting case, the provided past covariates at dataset index `0` do not extend far enough into the past. The past covariates must start at time step `2023-06-22`, whereas now they start at time step `2024-02-28`

So, I have 2 questions:

1. Is it possible to force the TFTModel to train with only past_covariates as the way I did in above?
2. If possible, how to correct my inference source code (also train source code if need) in order that the TFTModel can run train and inference with past_covariates only as I expect?

[dennisbader]

Hi @nhthanh0809. Just quickly to verify some Darts vocabulary:

• the target series is the one which we want to forecast
• past covariates are other features (not the target variable itself) which are known (only) in the past (e.g. measurements, ...).

A) So if you ask whether it's possible to only use the past covariates (not the past of the target variable itself) to forecast the future of the target, then that is not possible with TFTModel.

Covariates-only forecasting is currently only supported by our RegressionModel (and all models deriving from it).

B) If you do want to use the past of the target variable and the past of the past covariates, then that is possible.

However, to be able to forecast the target over the time frame of your test_covariates, you must know and supply the target values and past covariates values over the last input_chunk_length=30 time steps before the start time of test_covariates.

If your test set begins right after the end of the validation set, then you would have to call predict as follows:

forecast_test = model.predict(n=len(test_covariates), series=val_target, past_covariates=val_covariates)

This generates a forecast over the entire test set using the target and covariates from before the test set start.

If your test set is longer than output_chunk_length=7, and you want to know how it would have performed if repeatedly predicting with n=7, then you should consider model.historical_forecasts().

from darts import concatenate

# concatenate val and test sets so we can start at the beginning of test
val_test_target = concatenate([val_target, test_target], axis=0)
val_test_covariates = concatenate([val_covariates, test_covariates], axis=0)

hfc = model.historical_forecasts(
    n=7,
    series=val_test_target,
    past_covariates=val_test_covariates,
    start=test_target.start_time(),  # start at test start
    retrain=False,
    last_points_only=False,  # return the full forecasts
    stride=1,  # optionally, you could also advance 7 steps after each prediction to get contiguous forecasts (in time)
)

[nhthanh0809]

Hi @dennisbader ,
Thank for your support.

B) If you do want to use the past of the target variable and the past of the past covariates, then that is possible.

How can i do like that, could you please correct my code for this purpose?
By the way, after training how can i run inference on train set and val set to check how model predict on that dataset?
I've tried your suggestion but it can not run with n=len(test_covariates)