Add support for Neuralforecast #1115

americast · 2023-09-14T01:05:50Z

Adding support for neuralforecast. Fixes #1112.

DROP TABLE IF EXISTS AirData;

CREATE TABLE AirData (
    unique_id TEXT(30),
    ds TEXT(30),
    y INTEGER);

LOAD CSV 'data/forecasting/air-passengers.csv' INTO AirData;

DROP FUNCTION IF EXISTS Forecast;

CREATE FUNCTION Forecast FROM
(SELECT unique_id, ds, y FROM AirData)
TYPE Forecasting
PREDICT 'y'
HORIZON 12
LIBRARY 'neuralforecast';

SELECT Forecast(12);

One quick issue here is that neuralforecast needs horizon as a parameter while training, unlike statsforecast. Thus, a better way to call the UDF would be simply SELECT Forecast();, which is currently unsupported. @xzdandy Please let me know your thoughts.

List of stuff yet to be done:

Incorporate neuralforecast
Fix HORIZON redundancy (UPDATE: Being fixed in Support for calling functions without any arguments #1121)
Reuse model with lower horizon no
Add support for ~~multivariate forecasting~~ exogenous variables
Add tests
Add docs

cases

xzdandy · 2023-09-14T05:06:31Z

What is the error message when you try SELECT Forecast()? On the other hand, if the Forecast does not need any parameters , why we are not doing SELECT FIT_PREDICT(unique_id, ds, y, 12) FROM Air_DATA; What is the advantage of separating training and inference?

americast · 2023-09-14T05:31:18Z

What is the error message when you try SELECT Forecast()?

    raise UnexpectedToken(token, e.allowed, state=parser_state, token_history=[last_token], terminals_by_name=self.root_lexer.terminals_by_name)
lark.exceptions.UnexpectedToken: Unexpected token Token('RR_BRACKET', ')') at line 1, column 17.
Expected one of: 
        * MAX
        * AVG
        * DECIMAL_LITERAL
        * SUM
        * ONE_DECIMAL
        * MINUS
        * LR_SQ_BRACKET
        * LR_BRACKET
        * COUNT
        * BIT_NOT_OP
        * TRUE
        * FALSE
        * LOCAL_ID
        * STRING_LITERAL
        * TWO_DECIMAL
        * SEGMENT
        * STAR
        * REVERSE_QUOTE_ID
        * PLUS
        * ID
        * MIN
        * FIRST
        * REAL_LITERAL
        * ANYDIM
        * NOT
        * LAST
        * EXCLAMATION_SYMBOL
        * ZERO_DECIMAL
Previous tokens: [Token('LR_BRACKET', '(')]

jarulraj · 2023-09-16T04:12:34Z

You can copy paste the grammar file and the query into this IDE: https://www.lark-parser.org/ide/

jarulraj · 2023-09-26T03:07:04Z

Nice job, @americast :)

evadb/binder/statement_binder.py

evadb/executor/create_function_executor.py

test/unit_tests/binder/test_statement_binder.py

xzdandy · 2023-09-29T04:03:27Z

Great work. Looks perfect. I will also update the notebook and push to this PR. Leave some minors comments to fix. Thanks.

docs/source/reference/ai/model-forecasting.rst

review-notebook-app · 2023-09-29T04:35:50Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

evadb/functions/forecast.py

test/integration_tests/long/test_model_forecasting.py

xzdandy · 2023-09-29T04:43:19Z

Updated the forecast notebook. Found out some new issues. Please check. We'd also like to try the neuralforecast in the notebook to see if it addresses several limitations we found with statsforecast. Create #1236 for the follow up work.

…o neuralforecast

evadb/functions/forecast.py

xzdandy · 2023-09-30T09:08:24Z

The PR looks good. Thanks for the good work! I will update the notebook.

@xzdandy

Adding support for `neuralforecast`. Fixes georgia-tech-db#1112. ```sql DROP TABLE IF EXISTS AirData; CREATE TABLE AirData ( unique_id TEXT(30), ds TEXT(30), y INTEGER); LOAD CSV 'data/forecasting/air-passengers.csv' INTO AirData; DROP FUNCTION IF EXISTS Forecast; CREATE FUNCTION Forecast FROM (SELECT unique_id, ds, y FROM AirData) TYPE Forecasting PREDICT 'y' HORIZON 12 LIBRARY 'neuralforecast'; SELECT Forecast(12); ``` One quick issue here is that `neuralforecast` needs `horizon` as a parameter while training, unlike `statsforecast`. Thus, a better way to call the UDF would be simply `SELECT Forecast();`, which is currently unsupported. @xzdandy Please let me know your thoughts. List of stuff yet to be done: - [x] Incorporate `neuralforecast` - [x] Fix `HORIZON` redundancy (UPDATE: Being fixed in georgia-tech-db#1121) - [x] Reuse model with lower horizon no - [x] Add support for ~multivariate forecasting~ exogenous variables - [x] Add tests - [x] Add docs --------- Co-authored-by: xzdandy <[email protected]>

@xzdandy

Adding support for `neuralforecast`. Fixes georgia-tech-db#1112. ```sql DROP TABLE IF EXISTS AirData; CREATE TABLE AirData ( unique_id TEXT(30), ds TEXT(30), y INTEGER); LOAD CSV 'data/forecasting/air-passengers.csv' INTO AirData; DROP FUNCTION IF EXISTS Forecast; CREATE FUNCTION Forecast FROM (SELECT unique_id, ds, y FROM AirData) TYPE Forecasting PREDICT 'y' HORIZON 12 LIBRARY 'neuralforecast'; SELECT Forecast(12); ``` One quick issue here is that `neuralforecast` needs `horizon` as a parameter while training, unlike `statsforecast`. Thus, a better way to call the UDF would be simply `SELECT Forecast();`, which is currently unsupported. @xzdandy Please let me know your thoughts. List of stuff yet to be done: - [x] Incorporate `neuralforecast` - [x] Fix `HORIZON` redundancy (UPDATE: Being fixed in georgia-tech-db#1121) - [x] Reuse model with lower horizon no - [x] Add support for ~multivariate forecasting~ exogenous variables - [x] Add tests - [x] Add docs --------- Co-authored-by: xzdandy <[email protected]>

@xzdandy

Adding support for `neuralforecast`. Fixes georgia-tech-db#1112. ```sql DROP TABLE IF EXISTS AirData; CREATE TABLE AirData ( unique_id TEXT(30), ds TEXT(30), y INTEGER); LOAD CSV 'data/forecasting/air-passengers.csv' INTO AirData; DROP FUNCTION IF EXISTS Forecast; CREATE FUNCTION Forecast FROM (SELECT unique_id, ds, y FROM AirData) TYPE Forecasting PREDICT 'y' HORIZON 12 LIBRARY 'neuralforecast'; SELECT Forecast(12); ``` One quick issue here is that `neuralforecast` needs `horizon` as a parameter while training, unlike `statsforecast`. Thus, a better way to call the UDF would be simply `SELECT Forecast();`, which is currently unsupported. @xzdandy Please let me know your thoughts. List of stuff yet to be done: - [x] Incorporate `neuralforecast` - [x] Fix `HORIZON` redundancy (UPDATE: Being fixed in georgia-tech-db#1121) - [x] Reuse model with lower horizon no - [x] Add support for ~multivariate forecasting~ exogenous variables - [x] Add tests - [x] Add docs --------- Co-authored-by: xzdandy <[email protected]>

@xzdandy

Adding support for `neuralforecast`. Fixes georgia-tech-db#1112. ```sql DROP TABLE IF EXISTS AirData; CREATE TABLE AirData ( unique_id TEXT(30), ds TEXT(30), y INTEGER); LOAD CSV 'data/forecasting/air-passengers.csv' INTO AirData; DROP FUNCTION IF EXISTS Forecast; CREATE FUNCTION Forecast FROM (SELECT unique_id, ds, y FROM AirData) TYPE Forecasting PREDICT 'y' HORIZON 12 LIBRARY 'neuralforecast'; SELECT Forecast(12); ``` One quick issue here is that `neuralforecast` needs `horizon` as a parameter while training, unlike `statsforecast`. Thus, a better way to call the UDF would be simply `SELECT Forecast();`, which is currently unsupported. @xzdandy Please let me know your thoughts. List of stuff yet to be done: - [x] Incorporate `neuralforecast` - [x] Fix `HORIZON` redundancy (UPDATE: Being fixed in georgia-tech-db#1121) - [x] Reuse model with lower horizon no - [x] Add support for ~multivariate forecasting~ exogenous variables - [x] Add tests - [x] Add docs --------- Co-authored-by: xzdandy <[email protected]>

xzdandy and others added 9 commits September 13, 2023 03:02

Set the write output column type for forecast functions

8f49397

Fix forecast integration test

043d671

Move the generic utils test

0977c1f

Fix ludwig unittest cases and add unittestcase for normal forecasting

092c03f

cases

Add unitest cases for forecast with rename in binder.

96e40db

Add unittest when an expected column is passed to forecasting

5648371

Add unittest when required columns are missing in binder

8692ff1

Merge branch 'staging' into neuralforecast

0679200

Add neuralforecast support

1fd3c02

americast added this to the v0.3.5 milestone Sep 14, 2023

americast requested a review from xzdandy September 14, 2023 01:05

less horizon no retrain

65ed6e1

xzdandy removed this from the v0.3.5 milestone Sep 20, 2023

xzdandy added AI Engines Features, Bugs, related to AI Engines Integrations 🧩 Pull requests that update an integration labels Sep 22, 2023

americast added 4 commits September 24, 2023 19:24

Merge branch 'staging' into neuralforecast

5fd8af7

add support for exogenous variables

be242ee

Fix exogenous support; add tests

583e778

add tests

52c563e

americast marked this pull request as ready for review September 25, 2023 15:01

americast force-pushed the neuralforecast branch from b7e70f2 to c1ff289 Compare September 25, 2023 18:13

wip: fix test

84a159e

americast force-pushed the neuralforecast branch from c1ff289 to 84a159e Compare September 25, 2023 18:15

americast requested review from jarulraj and gaurav274 September 25, 2023 18:35

remove strict column check in test

06a7db0

americast added 2 commits September 28, 2023 14:05

Merge remote-tracking branch 'origin/staging' into neuralforecast

fda2b40

added auto support; updated docs

736d9e0

americast force-pushed the neuralforecast branch from 8181ba8 to 736d9e0 Compare September 29, 2023 01:06