Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alway profit 0 when evaluating a model #7

Open
Kalelv45 opened this issue Aug 28, 2018 · 16 comments
Open

Alway profit 0 when evaluating a model #7

Kalelv45 opened this issue Aug 28, 2018 · 16 comments

Comments

@Kalelv45
Copy link

It always stays at profit 0.00 after I evaluate some model.
Why can this be?

@kkuette
Copy link

kkuette commented Sep 12, 2018

Your agent did not learn well, i think it's just overfitting.

@iorobot
Copy link

iorobot commented Sep 15, 2018

Look at #1.
I also have that problem but even with that solution my results using almost the same model(model_1000 or model_1010) are extremely volatile.

@kkuette
Copy link

kkuette commented Sep 25, 2018

Try to tweak your settings.
if you want something with more functionalities you can look at this

@satinder147
Copy link

satinder147 commented Dec 31, 2018

@Kalelv45 @kkuette @iorobot @edwardhdlu.I tried it myself and found that tweaking the code a little bit yields very good results. I changed the way reward was provided. If there were more that 20 consecutive buys or 50 consecutive "no action", then I gave the agent a big negative reward like -500. Apart from this I removed the max(in the buy action) and multiplied the reward by 100. I trained the model for 220 episodes and the 220th showed a profit of 25000 dollars on the last three year google stock price data.

@iorobot
Copy link

iorobot commented Jan 9, 2019

Nice!Can you pull the code?

@chmbrs
Copy link

chmbrs commented Jan 17, 2019

@Kalelv45 @kkuette @iorobot @edwardhdlu.I tried it myself and found that tweaking the code a little bit yields very good results. I changed the way reward was provided. If there were more that 20 consecutive buys or 50 consecutive "no action", then I gave the agent a big negative reward like -500. Apart from this I removed the max(in the buy action) and multiplied the reward by 100. I trained the model for 220 episodes and the 220th showed a profit of 25000 dollars on the last three year google stock price data.

You mean that you removed the max function in the sell action, right?

@kkuette
Copy link

kkuette commented Jan 18, 2019

Are you sure that your model isnt overfitted ?

@satinder147
Copy link

satinder147 commented Jan 18, 2019

I removed the max function because otherwise the model is not penalized for wrong decisions. Then you can increase the size of the penalty(not everytime) by multiplying the reward it gets after selling by 100.
As far as overfitting is concerned, I have tried it with a bunch of other stocks and it is working really good.

@gucciwang
Copy link

What's the window size you're using? @satinder147

@satinder147
Copy link

@calvin-is-seksy 10

@satinder147
Copy link

satinder147 commented Feb 19, 2019

I just made an pull request. You all can have a look

@iorobot
Copy link

iorobot commented Mar 4, 2019

Even with your code i still have 0 profit with the GSPC dataset. Any ideas why?

@gucciwang
Copy link

I used a window size of 20, and found that my model converges and profits the most on my 30th episode. All models after 100 seems to diverge and just never buy anything. Any ideas on how to tweak the model to further converge?

image

@iorobot
Copy link

iorobot commented Mar 5, 2019

well, it seems to me that is only a coincidence, have you try to use different data?Do you find the same behaviour?

@GarfieldHuang
Copy link

GarfieldHuang commented Jul 29, 2019

I used a window size of 20, and found that my model converges and profits the most on my 30th episode. All models after 100 seems to diverge and just never buy anything. Any ideas on how to tweak the model to further converge?

image

hello, I want to know how to plot this graph??

@rui-ren
Copy link

rui-ren commented Oct 4, 2019

I used a window size of 20, and found that my model converges and profits the most on my 30th episode. All models after 100 seems to diverge and just never buy anything. Any ideas on how to tweak the model to further converge?

image

@ How about your running time? It is super slow on my laptop, I have RTX 2060 , But still 10 episodes took 1 hour. THX

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants