Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batched fetch ohlcv #37

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

jacopomaroli
Copy link

TL;DR: fixes (notice timestamp)
wrong:

2021-02-20 12:32:14.933770 - fetch_ohlcv - Attempt 0
Fetching: BTC/USDT, TF: 30m, Since: 1612024334932, Limit: 9999
2021-02-20 12:32:16.110623 - fetch_ohlcv - Attempt 0
Fetching: BTC/USDT, TF: 30m, Since: 1612024334932, Limit: 9999
***LIVE***

correct:

2021-02-20 12:37:13.208514 - fetch_ohlcv - Attempt 0
Fetching: BTC/USDT, TF: 30m, Since: 1607824633206, Limit: 9999
2021-02-20 12:37:13.983441 - fetch_ohlcv - Attempt 0
Fetching: BTC/USDT, TF: 30m, Since: 1609639200000, Limit: 9999
2021-02-20 12:37:14.750390 - fetch_ohlcv - Attempt 0
Fetching: BTC/USDT, TF: 30m, Since: 1611437400000, Limit: 9999
2021-02-20 12:37:15.520330 - fetch_ohlcv - Attempt 0
Fetching: BTC/USDT, TF: 30m, Since: 1613239200000, Limit: 9999
2021-02-20 12:37:16.267332 - fetch_ohlcv - Attempt 0
Fetching: BTC/USDT, TF: 30m, Since: 1613824200000, Limit: 9999
***LIVE***

Longer explanation: in some broker such as binance you can't fetch more than 1000 OHLCV records at the same time. You therefore need to fetch them in batches.
Without this fix, the second time the while True loop in _fetch_ohlcv will fetch again the same batch, not add it to the main data pool and break the loop thinking no new data has been added.
This results in old data delivered in the strategy as live ones.

This fix allows you to walk through the data until the current point in time resulting in correct LIVE data delivered to the strategy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant