Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P. Xing, Liang Lin. “GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning”. Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2021). [PDF]
Download GeoQA benchmark: [Google Drive]
01/10/2022 Update: We find some minor errors in data annotation. These errors have been fixed, please download the latest GeoQA benchmark. Besides, both arXiv paper and this project have also been updated.
python=3.6
allennlp==0.9.0
Document for allennlp
git clone https://github.com/chen-judge/GeoQA.git
cd GeoQA
pip install -r requirements.txt
Download the data.zip, move it to GeoQA path, and unzip it.
allennlp train config/NGS_Aux.json --include-package NGS_Aux -s save/test
Evaluate your trained model:
allennlp evaluate save/test data/GeoQA3/test.pk --include-package NGS_Aux_test --cuda-device 0
Or, you can use our checkpoint NGS_Aux_CKPT.zip, move it to save path, unzip it, and run:
allennlp evaluate save/NGS_Aux_CKPT data/GeoQA3/test.pk --include-package NGS_Aux_test --cuda-device 0
The result of our checkpoint should be:
| Method | Acc | Angle | Length | Other |
|---|---|---|---|---|
| NGS-Auxiliary | 60.0 | 71.5 | 48.8 | 29.6 |