I was trying to find a consolidated list of papers in machine learning (ICML, NIPS, AAAI, SIGIR) and natural language processing (ACL, EMNLP, NAACL) published after 2000, which are held in some regard, perhaps by winning prizes such as Test-of-time paper at these major conferences. However, there seems to be no such list, or if it is, it’s hidden too deep and it may just be quicker to prepare a similar list of my own. I will add the papers in reverse chronological order of their publication year.











Some random observations:

  1. NLP venues didn’t really have a classic paper section until this year’s NAACL, which is probably why so many papers were nominated.
  2. 2001 seems to have been a dismal year for NLP, with no good papers in the long run. By contrast, the community appears to have bounced back next year, with all 3 NAACL 2018 test-of-time awards given to papers from 2002.
  3. I have no idea why BLEU won. It was supposed to be an “understudy,” which is pretty clear from its name. The fact that it is still being used as an evaluation metric speaks more of a general failure to construct better metrics than of its strength.
  4. Since the papers are from before 2010, deep learning is conspicuous by its absence. In fact, Collobert and Weston’s ICML’08 paper on a unified architecture for language is the only such paper.
  5. Ali Rahimi’s “ML is alchemy” talk at NIPS’17 got a lot of attention, probably much more than his paper on random features.

Other similar lists