Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generalize to sgd with mini batches #93

Open
dustinvtran opened this issue Feb 2, 2017 · 0 comments
Open

generalize to sgd with mini batches #93

dustinvtran opened this issue Feb 2, 2017 · 0 comments

Comments

@dustinvtran
Copy link
Member

dustinvtran commented Feb 2, 2017

From an e-mail exchange with Kamal Hamidieh:

First of all, thank you for the SGD package. Just a quick question: Is there a setting in the command sgd where I can force sgd to perform batch gradient descent? I know the purpose of the package is to do the stochastic version but I like to know if this is possible.

gradient always does data subsampling with one data point, e.g., see below:

https://github.com/airoldilab/sgd/blob/master/src/model/glm_model.h#L44

the gradient function can be generalized to do mini batches. alternatively, another way to write it is to write the mini-batch loop in the sgd methods:

https://github.com/airoldilab/sgd/blob/master/src/sgd/explicit_sgd.h#L25

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant