Can Transformer Language Models Learn $n$-gram Language Models?

Getting started with the code

Clone the repository:

$ git clone https://github.com/rycolab/learning-ngrams
$ cd learning-ngrams

It may be beneficial to create a new Python virtual environment (e.g., using conda). We aim at Python 3.10 version and above.

Then install the package (in the required package) in editable mode:

$ pip install -e .

We use pytest to unit test the code.