Co-creators: Vinay Uday Prabhu and Ryan Teehan
Citation:
@inproceedings{prabhu2020spices,
title={SPICES: Survey Papers As Interactive Cheatsheet Embeddings},
author={Prabhu, Uday Vinay and
McAteer, Matthew and Teehan, Ryan },
booktitle={Rethinking ML Papers - ICLR 2021 Workshop},
howpublished = {\url{https://openreview.net/pdf?id=1sysg9hi3KS}},
month = {April},
year = {2020},
note = {(Accessed on 04/03/2021)}
year={2020}
}
The success enjoyed by the Attention is all you need spurred an avalanche of papers whose title or content banked off of the X-is all you need quip 😅.
In Figure below, we present a survey of all these papers that allows the reader to summarize where and why these quips occur.
| Authors | Title | Publication | Volume | Number | Pages | Year | Publisher |
|---|---|---|---|---|---|---|---|
| Kidambi, Rahul; Chang, Jonathan; Sun, Wen; | Optimism is All You Need: Model-Based Imitation Learning From Observation Alone | arXiv preprint arXiv:2102.10769 | 2021.0 | ||||
| Shen, Chao; Wu, Hau-Tieng; | Scalability and robustness of spectral embedding: landmark diffusion is all you need | arXiv preprint arXiv:2001.00801 | 2020.0 | ||||
| Bertasius, Gedas; Wang, Heng; Torresani, Lorenzo; | Is Space-Time Attention All You Need for Video Understanding? | arXiv preprint arXiv:2102.05095 | 2021.0 | ||||
| Bach, Francis; | The sum of a geometric series is all you need! | ||||||
| de Witt, Christian Schroeder; Gupta, Tarun; Makoviichuk, Denys; Makoviychuk, Viktor; Torr, Philip HS; Sun, Mingfei; Whiteson, Shimon; | Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge? | arXiv preprint arXiv:2011.09533 | 2020.0 | ||||
| Prinzing, Michael; | Friendly superintelligent AI: all you need is love | 3rd Conference on" Philosophy and Theory of Artificial Intelligence | 288-301 | 2017.0 | Springer | ||
| Javed, Khurram; Yao, Hengshuai; White, Martha; | Is Fast Adaptation All You Need? | arXiv preprint arXiv:1910.01705 | 2019.0 | ||||
| Abdu-Aguye, Mubarak G; Gomaa, Walid; Makihara, Yasushi; Yagi, Yasushi; | Adaptive Pooling Is All You Need: An Empirical Study on Hyperparameter-insensitive Human Action Recognition Using Wearable Sensors | 2020 International Joint Conference on Neural Networks (IJCNN) | 1-6 | 2020.0 | IEEE | ||
| Cui, Xuefeng; | Attention is all you need for general-purpose protein structure embedding | bioRxiv | 2021.0 | Cold Spring Harbor Laboratory | |||
| Xiao, Wencong; | All You Need to Know about Scheduling Deep Learning Jobs | ||||||
| Roberts, Nicholas; Prabhu, Vinay Uday; McAteer, Matthew; | Model weight theft with just noise inputs: The curious case of the petulant attacker | arXiv preprint arXiv:1912.08987 | 2019.0 | ||||
| Gao, Yang; Meyer, Christian M; Mesgar, Mohsen; Gurevych, Iryna; | Good Rewards Are All You Need: Reward Learning for Efficient Reinforcement Learning in Document Summarisation | ||||||
| Ramsey, Norman; | All you need is the monad.. what monad was that again | PPS Workshop | 2016.0 | ||||
| Le, James; | Meta-Learning Is All You Need MATH 789-Mathematics of Deep Learning | 2020.0 | |||||
| Disambiguation, Cross-lingual Visual Verb Sense; | Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need! | ||||||
| Isola, Phillip; | Rethinking Few-Shot Image Classification: A Good Embedding is All You Need? | Springer | |||||
| Silva, Francisco Alejandro Gallegos; Alvaracín-Paula, Mario Eduardo; González, Hugo Nicolás Acosta; | Estimation of the causal effect of tourism advertising campaign All You Need is Ecuador on digital exposure | Communications in Statistics: Case Studies, Data Analysis and Applications | 6.0 | 2.0 | 247-269 | 2020.0 | Taylor & Francis |
| Pedregosa, Fabian; | Sufficient decrease is all you need | ||||||
| Meinert, Judith; | All you need is a (heuristic) cue? | ||||||
| Joshi, Anirudh; | QANet: Convolutions and Attention are All you Need | ||||||
| Ivanov, Andrei; Dryden, Nikoli; Ben-Nun, Tal; Li, Shigang; Hoefler, Torsten; | Data Movement Is All You Need: A Case Study of Transformer Networks | arXiv preprint arXiv:2007.00072 | 2020.0 | ||||
| Namazifar, Mahdi; Papangelis, Alexandros; Tur, Gokhan; Hakkani-Tür, Dilek; | Language Model is All You Need: Natural Language Understanding as Question Answering | arXiv preprint arXiv:2011.03023 | 2020.0 | ||||
| Kostrikov, Ilya; Yarats, Denis; Fergus, Rob; | Image augmentation is all you need: Regularizing deep reinforcement learning from pixels | arXiv preprint arXiv:2004.13649 | 2020.0 | ||||
| Eikema, Bryan; Aziz, Wilker; | Is map decoding all you need? the inadequacy of the mode in neural machine translation | arXiv preprint arXiv:2005.10283 | 2020.0 | ||||
| Chen, Weijie; Xie, Di; Zhang, Yuan; Pu, Shiliang; | All you need is a few shifts: Designing efficient convolutional neural networks for image classification | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition | 7241-7250 | 2019.0 | |||
| Träuble, Frederik; Creager, Elliot; Kilbertus, Niki; Goyal, Anirudh; Locatello, Francesco; Schölkopf, Bernhard; Bauer, Stefan; | Is independence all you need? on the generalization of representations learned from correlated data | arXiv preprint arXiv:2006.07886 | 2020.0 | ||||
| Tamborrino, Alexandre; Pellicano, Nicola; Pannier, Baptiste; Voitot, Pascal; Naudin, Louise; | Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning | arXiv preprint arXiv:2004.14074 | 2020.0 | ||||
| Barnes, Jeff; | Azure machine learning | Microsoft Azure Essentials. 1st ed, Microsoft | 2015.0 | ||||
| Uhlich, Stefan; Mauch, Lukas; Cardinaux, Fabien; Yoshiyama, Kazuki; Garcia, Javier Alonso; Tiedemann, Stephen; Kemp, Thomas; Nakamura, Akira; | Mixed precision dnns: All you need is a good parametrization | arXiv preprint arXiv:1905.11452 | 2019.0 | ||||
| Kalavri, Vasiliki; Liagouris, John; Hoffmann, Moritz; Dimitrova, Desislava; Forshaw, Matthew; Roscoe, Timothy; | Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows | 13th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 18) | 783-798 | 2018.0 | |||
| Le, James; | Meta-Learning Is All You Need | 2020.0 | |||||
| Wu, Junru; Dai, Xiyang; Chen, Dongdong; Chen, Yinpeng; Liu, Mengchen; Yu, Ye; Wang, Zhangyang; Liu, Zicheng; Chen, Mei; Yuan, Lu; | Weak NAS Predictors Are All You Need | arXiv preprint arXiv:2102.10490 | 2021.0 | ||||
| Zhu, Jing; Lu, Xingyu; Heimann, Mark; Koutra, Danai; | Node Proximity Is All You Need: Unified Structural and Positional Node and Graph Embedding | arXiv preprint arXiv:2102.13582 | 2021.0 | ||||
| Bilkhu, Manjot; Wang, Siyang; Dobhal, Tushar; | Attention is all you need for Videos: Self-attention based Video Summarization using Universal Transformers | arXiv preprint arXiv:1906.02792 | 2019.0 | ||||
| Prasad, Adarsh; Balakrishnan, Sivaraman; Ravikumar, Pradeep; | A Robust Univariate Mean Estimator is All You Need | International Conference on Artificial Intelligence and Statistics | 4034-4044 | 2020.0 | PMLR | ||
| Kumar, Saurabh; Kumar, Aviral; Levine, Sergey; Finn, Chelsea; | One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL | arXiv preprint arXiv:2010.14484 | 2020.0 | ||||
| Ivanov, Andrei; Dryden, Nikoli; Ben-Nun, Tal; Li, Shigang; Hoefler, Torsten; | Data Movement Is All You Need: A Case Study on Optimizing Transformers | arXiv e-prints | arXiv: 2007.00072 | 2020.0 | |||
| Ren, Xuanchi; Yang, Tao; Wang, Yuwang; Zeng, Wenjun; | Do Generative Models Know Disentanglement? Contrastive Learning is All You Need | arXiv preprint arXiv:2102.10543 | 2021.0 | ||||
| Boudiaf, Malik; Kervadec, Hoel; Masud, Ziko Imtiaz; Piantanida, Pablo; Ayed, Ismail Ben; Dolz, Jose; | Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need? | arXiv preprint arXiv:2012.06166 | 2020.0 | ||||
| Blondé, Lionel; Strasser, Pablo; Kalousis, Alexandros; | Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning | arXiv preprint arXiv:2006.16785 | 2020.0 | ||||
| Bai, Shaojie; Kolter, J Zico; Koltun, Vladlen; | Deep equilibrium models | arXiv preprint arXiv:1909.01377 | 2019.0 | ||||
| Mishkin, Dmytro; Matas, Jiri; | All you need is a good init | arXiv preprint arXiv:1511.06422 | 2015.0 | ||||
| Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Lukasz; Polosukhin, Illia; | Attention is all you need | arXiv preprint arXiv:1706.03762 | 2017.0 | ||||
| Chollet, Francois; | The limitations of deep learning | Deep Learning With Python | 2017.0 | Manning Publications | |||
| Tran, Ba-Hien; Rossi, Simone; Milios, Dimitrios; Filippone, Maurizio; | All You Need is a Good Functional Prior for Bayesian Deep Learning | arXiv preprint arXiv:2011.12829 | 2020.0 | ||||
| Chen, Qiming; Wu, Ren; | CNN is all you need | arXiv preprint arXiv:1712.09662 | 2017.0 | ||||
| Bachlechner, Thomas; Majumder, Bodhisattwa Prasad; Mao, Huanru Henry; Cottrell, Garrison W; McAuley, Julian; | Rezero is all you need: Fast convergence at large depth | arXiv preprint arXiv:2003.04887 | 2020.0 | ||||
| Xie, Di; Xiong, Jiang; Pu, Shiliang; | All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation | Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition | 6176-6185 | 2017.0 | |||
| Guo, Yunhui; Li, Yandong; Wang, Liqiang; Rosing, Tajana; | Depthwise convolution is all you need for learning multiple visual domains | Proceedings of the AAAI Conference on Artificial Intelligence | 33.0 | 1.0 | 8368-8375 | 2019.0 | |
| Li, Guohao; Xiong, Chenxin; Thabet, Ali; Ghanem, Bernard; | Deepergcn: All you need to train deeper gcns | arXiv preprint arXiv:2006.07739 | 2020.0 | ||||
| Jeddi, Ahmadreza; Shafiee, Mohammad Javad; Wong, Alexander; | A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning | arXiv preprint arXiv:2012.13628 | 2020.0 | ||||
| Gröndahl, Tommi; Pajola, Luca; Juuti, Mika; Conti, Mauro; Asokan, N; | All You Need is" Love" Evading Hate Speech Detection | Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security | 2-12 | 2018.0 | |||
| Hu, Ronghang; Singh, Amanpreet; | Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer | arXiv preprint arXiv:2102.10772 | 2021.0 | ||||
| Wang, Hao; Lu, Pu; Zhang, Hui; Yang, Mingkun; Bai, Xiang; Xu, Yongchao; He, Mengchao; Wang, Yongpan; Liu, Wenyu; | All you need is boundary: Toward arbitrary-shaped text spotting | Proceedings of the AAAI Conference on Artificial Intelligence | 34.0 | 7.0 | 12160-12167 | 2020.0 | |
| Subakan, Cem; Ravanelli, Mirco; Cornell, Samuele; Bronzi, Mirko; Zhong, Jianyuan; | Attention is All You Need in Speech Separation | arXiv preprint arXiv:2010.13154 | 2020.0 | ||||
| Sharp, Nicholas; Attaiki, Souhaib; Crane, Keenan; Ovsjanikov, Maks; | Diffusion is All You Need for Learning on Surfaces | arXiv preprint arXiv:2012.00888 | 2020.0 | ||||
| O’Mahony, Niall; Campbell, Sean; Carvalho, Anderson; Harapanahalli, Suman; Hernandez, Gustavo Velasco; Krpalkova, Lenka; Riordan, Daniel; Walsh, Joseph; | Deep learning vs. traditional computer vision | Science and Information Conference | 128-144 | 2019.0 | Springer | ||
| Chaudhury, Ayan; Boudon, Frédéric; Godin, Christophe; | 3D Plant Phenotyping: All You Need is Labelled Point Cloud Data | European Conference on Computer Vision | 244-260 | 2020.0 | Springer | ||
| Lu, Amy X; Lu, Alex X; Moses, Alan; | Evolution Is All You Need: Phylogenetic Augmentation for Contrastive Learning | arXiv preprint arXiv:2012.13475 | 2020.0 | ||||
| Kato, Natsumi; Osone, Hiroyuki; Oomori, Kotaro; Ooi, Chun Wei; Ochiai, Yoichi; | Gans-based clothes design: Pattern maker is all you need to design clothing | Proceedings of the 10th Augmented Human International Conference 2019 | 1-7 | 2019.0 | |||
| Klein, Tassilo; Nabi, Moin; | Attention is (not) all you need for commonsense reasoning | arXiv preprint arXiv:1905.13497 | 2019.0 | ||||
| Huang, Shaoli; Tao, Dacheng; | All you need is a good representation: A multi-level and classifier-centric representation for few-shot learning | arXiv preprint arXiv:1911.12476 | 2019.0 | ||||
| Wood-Doughty, Zach; Andrews, Nicholas; Dredze, Mark; | Convolutions are all you need (for classifying character sequences) | Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text | 208-213 | 2018.0 | |||
| Malach, Eran; Yehudai, Gilad; Shalev-Schwartz, Shai; Shamir, Ohad; | Proving the lottery ticket hypothesis: Pruning is all you need | International Conference on Machine Learning | 6682-6691 | 2020.0 | PMLR | ||
| Tian, Yonglong; Wang, Yue; Krishnan, Dilip; Tenenbaum, Joshua B; Isola, Phillip; | Rethinking few-shot image classification: a good embedding is all you need? | arXiv preprint arXiv:2003.11539 | 2020.0 | ||||
| Bojchevski, Aleksandar; Klicpera, Johannes; Perozzi, Bryan; Blais, Martin; Kapoor, Amol; Lukasik, Michal; Günnemann, Stephan; | Is pagerank all you need for scalable graph neural networks? | ACM KDD, MLG Workshop | 2019.0 | ||||
| Zhao, Chen; Ghanem, Bernard; | ThumbNet: One Thumbnail Image Contains All You Need for Recognition | Proceedings of the 28th ACM International Conference on Multimedia | 1506-1514 | 2020.0 | |||
| Zisserman, AP; Afouras, T; Chung, JS; | ASR is all you need: cross-modal distillation for lip reading | Proceedings of the International Conference of Acoustics, Speech, and Signal Processing (ICASSP) | 2020.0 | IEEE | |||
| Nizan, Ori; Tal, Ayellet; | Breaking the cycle-colleagues are all you need | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition | 7860-7869 | 2020.0 | |||
| Prajwal, KR; Mukhopadhyay, Rudrabha; Namboodiri, Vinay P; Jawahar, CV; | A lip sync expert is all you need for speech to lip generation in the wild | Proceedings of the 28th ACM International Conference on Multimedia | 484-492 | 2020.0 | |||
| Wu, Tsung-Han; Hsieh, Chun-Chen; Chen, Yen-Hao; Chi, Po-Han; Lee, Hung-yi; | Hand-crafted Attention is All You Need? A Study of Attention on Self-supervised Audio Transformer | arXiv preprint arXiv:2006.05174 | 2020.0 | ||||
| Cao, Meng; Zou, Yuexian; | All you need is a second look: Towards Tighter Arbitrary shape text detection | ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | 2228-2232 | 2020.0 | IEEE | ||
| Ayala, Daniel; Borrego, Agustín; Hernández, Inma; Rivero, Carlos R; Ruiz, David; | AYNEC: all you need for evaluating completion techniques in knowledge graphs | European Semantic Web Conference | 397-411 | 2019.0 | Springer | ||
| Snower, Michael; Kadav, Asim; Lai, Farley; Graf, Hans Peter; | 15 Keypoints Is All You Need | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition | 6738-6748 | 2020.0 | |||
| Cheng, Zehua; Wu, Yuxiang; Xu, Zhenghua; Lukasiewicz, Thomas; Wang, Weiyang; | Segmentation is All You Need | arXiv preprint arXiv:1904.13300 | 2019.0 | ||||
| Afouras, Triantafyllos; Chung, Joon Son; Zisserman, Andrew; | Asr is all you need: Cross-modal distillation for lip reading | ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | 2143-2147 | 2020.0 | IEEE | ||
| Al-Dujaili, Abdullah; O'Reilly, Una-May; | Sign bits are all you need for black-box attacks | International Conference on Learning Representations | 2019.0 | ||||
| Duan, Sufeng; Zhao, Hai; | Attention Is All You Need for Chinese Word Segmentation | arXiv preprint arXiv:1910.14537 | 2019.0 | ||||
| Gelly, Sylvain; Kurach, Karol; Michalski, Marcin; Zhai, Xiaohua; | MemGEN: Memory is All You Need | arXiv preprint arXiv:1803.11203 | 2018.0 | ||||
| Orseau, Laurent; Hutter, Marcus; Rivasplata, Omar; | Logarithmic pruning is all you need | Advances in Neural Information Processing Systems | 33.0 | 2020.0 | |||
| Komatsuzaki, Aran; | One epoch is all you need | arXiv preprint arXiv:1906.06669 | 2019.0 | ||||
| Entezari, Negin; Al-Sayouri, Saba A; Darvishzadeh, Amirali; Papalexakis, Evangelos E; | All you need is low (rank) defending against adversarial attacks on graphs | Proceedings of the 13th International Conference on Web Search and Data Mining | 169-177 | 2020.0 | |||
| Ramsauer, Hubert; Schäfl, Bernhard; Lehner, Johannes; Seidl, Philipp; Widrich, Michael; Gruber, Lukas; Holzleitner, Markus; Pavlović, Milena; Sandve, Geir Kjetil; Greiff, Victor; | Hopfield networks is all you need | arXiv preprint arXiv:2008.02217 | 2020.0 | ||||