Deep Learning for NLP

Understanding Classic Deep Learning NLP Methods and their Implementations. Do Something Useful in a Long Time Feature. Slow but Valuable.

Terms

• NLI: Natural Language Inference
• NLU: Natural Language Understanding
• NLG: Natural Language Generation
• QA: Question Answering
• MRC: Machine Reading Comprehension

Code Reading

• graykode/nlp-tutorial

Valuable Blogs

• Chris McCormick
• 苏剑林

Important Tools

• stanfordnlp/stanza

  • The Stanford NLP Group's official Python NLP library. It contains support for running various accurate natural language processing tools on 60+ languages and for accessing the Java Stanford CoreNLP software from Python. website
  • pytorch, 2020
  • 66 languages(including Chinese); from raw text and pipeline without error progation; high perfermance;
  • language-agnostic

• spaCy

  • 10 languages(not including Chinese);
  • python

• coreNLP

  • 6 languages(including Chinese);
  • Java, 2014

Important Board

1. MS MARCO

   • KeyPhrase Extraction(10/18/2019) ranked by F1 @3 on Eval
     • OpenKP (OpenKeyPhrase) is a large scale, open-domain keyphrase extraction dataset,
     • which was first released in the paper Open Domain Web Keyphrase Extraction Beyond Language Modeling at EMNLP-IJCNLP 2019.
   • Passage Retrieval(10/26/2018-Present) ranked by MRR on Eval
   • Q&A Task(03/01/2018-Present)
   • Q&A + Natural Language Generation Task(03/01/2018-Present)
   • MS MARCO V1 Leaderboard(12/01/2016-03/31/2018)

2. GLUE: General Language Understanding Evaluation benchmark

   • GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
   • Wang et al. 2018
   • glue-benchmark

Important Links

1. thunlp/RCPapers
2. Overview of Modern Deep Learning Techniques Applied to Natural Language Processing
3. Hankcs

Important Agencies

1. [Google AI]
   • Transformer
   • Bert
2. Open AI
   • GPT1.0/2.0
3. MSRA
   • MT-DNN

Courses

1. 2019 Stanford, CS224U: Natural Language Understanding

   • website

2. 2019 Standfor, CS224N: Natural Language Processing with Deep Learning

   • website

Problems

• language modeling

• sequence labeling

  • POS tagging (part-of-speech tagging)
  • Chunking
  • NER ( named entity recognition)
  • Semantic role labeling

• machine translation

• sentence matching/paraphrase

• text classification

Research dire

Services

• ShannonAI/service-streamer
  • ServiceStreamer is a middleware for web service of machine learning applications. Queue requests from users are sampled into mini-batches.
  • ServiceStreamer can significantly enhance the overall performance of the system by improving GPU utilization.
  • A good and fast deep learning deploy frames.

NLP Corpus

• SophonPlus/ChineseNlpCorpus