This is the github repo for IJCAI 2020 paper "TransOMCS: From Linguistic Graphs to Commonsense Knowledge".
Python 3.6, Pytorch 1.0
If you only want to use TransOMCS, you can download it from TransOMCS.
Without any further filtering, TransOMCS contains 20 commonsense relations, 101 thousand unique words, and 18.48 million triplets.
Here are the statistics and examples of different commonsense relations.
| Relation Name | Number of triplets | Reasonable Ratio | Example |
|---|---|---|---|
| CapableOf | 6,145,829 | 58.4% | (government, CapableOf, protect) |
| UsedFor | 3,475,254 | 50.8% | (kitchen, UsedFor, eat in) |
| HasProperty | 2,127,824 | 59.1% | (account, HasProperty, established) |
| AtLocation | 1,969,298 | 51.3% | (dryer, AtLocation, dishwasher) |
| HasA | 1,562,961 | 68.9% | (forest, HasA, pool) |
| ReceivesAction | 1,492,915 | 53.7% | (news, ReceivesAction, misattribute) |
| InstanceOf | 777,688 | 52.2% | (atlanta, InstanceOf, city) |
| PartOf | 357,486 | 62.8% | (player, PartOf, team) |
| CausesDesire | 249,755 | 52.0% | (music, CausesDesire, listen) |
| MadeOf | 114,111 | 55.3% | (world, MadeOf, country) |
| CreatedBy | 52,957 | 64.6% | (film, CreatedBy, director) |
| Causes | 50,439 | 53.4% | (misinterpret, Causes, apologize) |
| HasPrerequisite | 43,141 | 62.7% | (doubt, HasPrerequisite, bring proof) |
| HasSubevent | 18,904 | 56.1% | (be sure, HasSubevent, ask) |
| MotivatedByGoal | 15,322 | 55.8% | (come, MotivatedByGoal, fun) |
| HasLastSubevent | 14,048 | 58.9% | (hungry, HasLastSubevent, eat) |
| Desires | 10,668 | 56.4% | (dog, Desires, play) |
| HasFirstSubevent | 2,962 | 58.4% | (talk to, HasFirstSubevent, call) |
| DefinedAs | 36 | 37.5% | (door, DefinedAs, entrance) |
| LocatedNear | 19 | 85.7% | (shoe, LocatedNear, foot) |
The reasonable ratio scores are annotated on the random sample over all of the extracted knowledge (no knowledge ranking).
In general, TransOMCS is still quite noisy because TransOMCS is extracted from raw data with patterns. However, as shown in the paper, a careful use of the data in the downstream applications helps. We will keep working on improving its quality.
If you want to repeat the process of creating TransOMCS with OMCS and ASER, please follow the following steps.
- Download the core version of ASER from ASER Homepage and install ASER 0.1 following the guideline.
- Download the selected Commonsense OMCS Tuples and associated ASER graphs from OMCS and ASER matches.
- Download the randomly split knowledge ranking dataset from Ranking Dataset.
- Unzip the downloaded matched OMCS tuple and ASER graphs in the same folder.
- Extract patterns:
python Pattern_Extraction.py. - Apply the extracted patterns to extract knowledge from ASER (You need to modify the location of your .db file):
python Knowledge_Extraction.py. - Train a ranking model to rank extracted knowledge:
python Train_and_Predict.py.
Please use the code in reading comprehension model and replace the external knowledge with different subsets of TransOMCS based on your need.
Please use the code in dialog model and replace the external knowledge with different subsets of TransOMCS based on your need.
- Filter the current TransOMCS to further improve the quality (e.g., merge pronouns like 'he' and 'she' to human).
@inproceedings{zhang2020TransOMCS,
author = {Hongming Zhang and Daniel Khashabi and Yangqiu Song and Dan Roth},
title = {TransOMCS: From Linguistic Graphs to Commonsense Knowledge},
booktitle = {Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) 2020},
year = {2020}
}
If you have any other questions about this repo, you are welcome to open an issue or send me an email, I will respond to that as soon as possible.