data leakage

[PietroAmin]

Please note that there is the possibility of data leakage. The way data are standardize is very dangerous because you shift back in time future information. Just try to delete (.shift(-num_historical_days)) in your scaling method and you will see how results will get worser.

[PietroAmin]

There is the same problem in numerous github codes that try to forecast stock's future prices with GAN.

[nupurdeshpande11]

There is the same problem in numerous github codes that try to forecast stock's future prices with GAN.

There is the same problem in numerous github codes that try to forecast stock's future prices with GAN.

Cool...I'll check out what you are talking about...what kinda data leakage exactly? Plus the gans are experimental since they haven't been used extensively for time series...

[PietroAmin]

Just try to visualize the data. When you calculate the moving average, min and max at time (t) and then moving back those informations to "num_historical_days" times before you are anticipating those information. Indeed, if you visualize you will see the moving average always predictig the path of the real time series

[PietroAmin]

I'm trying to construct the GAN with LSTM as generator and CCN as generator :)

[nupurdeshpande11]

The ma has a weight added to it for this reason

…

On Thu, Mar 26, 2020, 3:51 PM PietroAmin ***@***.***> wrote: Just try to visualize the data. When you calculate the moving average, min and max at time (t) and then moving back those informations to "num_historical_days" times before you are anticipating those information. Indeed, if you visualize you will see the moving average always predictig the path of the real time series — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHUXKH5SRGBGBMPXJS4YUJDRJOW5FANCNFSM4LUO2R3Q> .

[PietroAmin]

Linear Scaling
x ′ = ( x − x (mean)) / ( x m a x − x m i n )
Just give me some link where they explain why the shift is needed

[nupurdeshpande11]

Sure...I'll have to look at my refs and let u know

…

On Thu, Mar 26, 2020, 7:40 PM PietroAmin ***@***.***> wrote: Linear Scaling x ′ = ( x − x (mean)) / ( x m a x − x m i n ) Just give me some link where they explain why the shift is needed — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHUXKH7Z33VOPZDDMLPKMXTRJPRW5ANCNFSM4LUO2R3Q> .

[yanbigong2]

I'm trying to construct the GAN with LSTM as generator and CCN as generator :)

Have you finished this code? Will you open it?

[deshpandenu]

Please note that there is the possibility of data leakage. The way data are standardize is very dangerous because you shift back in time future information. Just try to delete (.shift(-num_historical_days)) in your scaling method and you will see how results will get worser.

Can you please explain this problem by typing in the equation and code you are referring to. Thank you