Part 1 : Implementation of first order and second order methods including Hessian calculations; all done for multiclass logistic Regression. Gradient Descent, Newton Raphson, Stoichastic Gradient Descent, Minibatch SGD, minbatch GD, SVRG
For Hessian calculations and derivation for this setting, please refer to this excellent blog. http://fourier.eng.hmc.edu/e176/lectures/ch7/node14.html
Part 2: Implementation of Subgradients and Proximal gradients. Looked at both of them for data denoising task.