Final Report Outline

[izaakniksan]

• Go over rubric

• Explain that sometimes cache size < sum of all the totals because tensors come in and out of existence at different times. So all won't exist at once.

• Explain why in resnet, the current cache is much smaller than the max cache. Also, CUDA driver takes up hundreds of MB when initialized (Memory Usage is higher than other Pytorch implementation? facebookresearch/maskrcnn-benchmark#182)

• Explain stack trace of building from source, show how I searched

• Acknowledgements : how to format it? Put it right after abstract?

• make sure all listings are labelled

• Conclusion

• In section 4.5, change activation to feature map

• Put all gathered data tables. Remove "iteration #, epoch#' from the table titles, and just explain that I waited until steady state.

• Future work (profile more models, extend to multi-gpu [say that I couldn't do this because I don't have access to such a setup])

• Talk about why gradients increase in mem usage (what are intermediate gradients?)

• Cite the resnet paper/implementation/imagenet dataset in appendix

• Explain memory profiler (step by step, how hooks work, etc.)

• Change tables to isolate intermediate grads on their own, since i didn't include them in the bar graphs

• Get more recommendation data

• System diagram on powerpoint

• Add official cover page

[tygamvrelis]

Looks good man