This repository contains code for the paper End-to-end symbolic regression with transformers. An interactive demonstration of the paper may be found here.
The code is based on the repository Deep Learning for Symbolic Mathematics.
Most of the code specific to recurrent sequences lies in the folder src/envs.
Using conda and the environment.yml file:
conda env create --name symbolic regression --file=environment.yml
Also manually install a fork of sympytorch:
pip install git+https://github.com/pakamienny/sympytorch
To launch a model training use with additional arguments (arg1,val1), (arg2,val2):
python train.py --arg1 val1 --arg2 --val2
All hyper-parameters related to training are specified in parsers.py, and environment HPs are in envs/environment.py
To launch evaluation, please use the flag reload_checkpoint to specify in which folder the saved model is located.
python evaluate.py --reload_checkpoint XXX
We include a small notebook that loads a pre-trained model you can play with in Example.ipynb
You can also check the demo website where you can play with the model without a single line of code here.
Distributed training is available via Slurm and submitit with grid-search:
pip install submitit
To launch a run on 2 nodes with 8 GPU each, use the submit.py script.
If you want to reuse this material, please considering citing the following:
@article{kamienny2022end,
title={End-to-end symbolic regression with transformers},
author={Kamienny, Pierre-Alexandre and d'Ascoli, St{\'e}phane and Lample, Guillaume and Charton, Fran{\c{c}}ois},
journal={arXiv preprint arXiv:2204.10532},
year={2022}
}
The majority of this repository is released under the Apache 2.0 license as found in the LICENSE file.