Notes introducing two sets of theorems known as No Free Lunch Theorem:
- Wolpert's original ones, from his seminal 1996 paper
- One found under this name in Shalev-Shwartz & Ben-David's 2014 textbook on theoretical machine learning (related to lower-bounds on VC theory)
Main goals are:
- see the theorems as related to the philosophical problem of induction
- understand what these theorems, which are commonly thrown around in online discussions, do and do not imply
The theorems are stated and discussed, but proofs are omitted.