Merging in teachers and code for RL-Quests paper.

[JackUrb]

Merges in teachers and attempts to standardize some mechanisms for using ParlAI and LIGHT together. Tries as best as possible to clean up some of Raj's older research code to be more presentable.

This PR includes the main supervised task teachers, for light:atomic, light:quest_goals, and light:quest_wild_completions.

cc @xiangyu-peng

Testing

atomic:

>> light dd -t light:atomic
12:54:58 | Opt:
...
12:55:15 | Loading ParlAI text data: /Users/jju/LIGHT/data/atomic/bert/atomic_sample_train.txt
- - - NEW EPISODE: light:atomic - - -
militiaman writes a bookmarking on the subject , as a result, militiaman wants to
   to challenge the author
- - - NEW EPISODE: light:atomic - - -
knight eats popcorn because knight wanted
   to enjoy a tasty snack
- - - NEW EPISODE: light:atomic - - -
parent puts the problem another way , as a result, parent wants to
   to publish a research
- - - NEW EPISODE: light:atomic - - -
angel bar keep makes any money , as a result, angel bar keep
   doing job
- - - NEW EPISODE: light:atomic - - -
janitor buys a new jacket because janitor wanted
   wants what someone else has
- - - NEW EPISODE: light:atomic - - -
old maniacal man follows others through society , as a result, old maniacal man wants to
   to know if they are okay with them following
- - - NEW EPISODE: light:atomic - - -
high priestess takes meed to the store because high priestess needed
   to pay the bus fare
- - - NEW EPISODE: light:atomic - - -
fierce assassin writes fierce assassin's life , as a result, fierce assassin
   publishes book
- - - NEW EPISODE: light:atomic - - -
king himself plays a role in society , as a result, king himself
   is disliked
- - - NEW EPISODE: light:atomic - - -
rival jeweler eats healthier because rival jeweler wanted
   to stay in shape
12:55:30 | loaded 216686 episodes with a total of 216686 examples

quests_goals:

>> light dd -t light:quest_goals --subtask boa_timeline_prediction
15:16:32 | Opt:
... <snipped> ...
15:17:24 | Loading ParlAI text data: /Users/jju/LIGHT/data/quests/base/boa_timeline_prediction/boa_timeline_prediction_sample_train.txt
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the Sacrificial altar. A room with a stone floor and a massive statue of a god the peasants worship.  This area gives off chilling vibes.  There's pieces of armor and weapons stored in the room used for various ceremonies. There's an armor, a Candle, a statue of a god, a skins, a weapon, a A leather cloth, and a piece here. The peasant is here.  You are carrying an apple and a bug, and wielding a bow, a slingshot, and a stick.
_name the a curious boy
_persona I am four years old and I have a lot of questions. I am excited to start school. My best trend is my neighbor.
_long_mot I want to find my best friend whom is my neighbor
_mid_mot I want to ask the peasant for help finding my neighbor
_short_mot I want to make friends with the peasant
   go Sacrificial Altar|get apple from apple tree|follow peasant|give apple to peasant|put bug in the ground|follow peasant|hug best friend
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the The King's Quarters. Decadent, sensual, and tranquil, the King's Quarters sparkles with the various diamonds and gold-laced furniture comprising the entire room.  Self-portraits of the king himself, his queen, and various demigods line the walls leading up to the bed.  Smooth, bear- and tiger-skin rugs cover the entire floor to protect the bare feet of scantily clad virgins who serve to the beckon and call of the king and his queen.  There is a faint smell of sandalwood wafting to and from each section of the room at all times. There's a rug, a the bed, a Tapestry, a self-portrait, an a bottle of wine, a chest, a gold-laced furniture, a various diamonds, and a portraits here. The child is here.  You are carrying nothing, and wearing a gown.
_name the the queen
_persona I am Queen and Protector of the realm. My King is my husband and is king is name only. I am the true ruler of this place.
_long_mot I plan to continue ruling this kingdom for a long time 
_mid_mot I plan to take the Tapestry to my quarters today 
_short_mot I want the child to show me the Tapestry 
   wear Gown|drop Crown|go The King's Quarters|follow child|get Tapestry|put Tapestry on Queen's quarters|drink Wine
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the North Forest. Filled with huge trees, the forest is home to many types of animals. Huge predators such as tigers and bears roam the grounds as do small animals like squirrels. Colorful birds fly among the trees. Wildflowers grow on the ground. Dappled sunlight streams through the huge canopy of trees. The predator is here.  You are carrying nothing.
_name the colorful bird
_persona I am a beautiful bird that lives in a hatchery in the castle. The prince and princess feed me everyday and they pet me. I want to fly like other birds but i'm in a cage and that makes me sad.
_long_mot I will be praised as a hero by the bird kingdom if I find out where the predator's hideaway is
_mid_mot I need to know what area of the forest to avoid so I do not get eaten
_short_mot I wish to find out where the predator's hideaway is
   eat worm|go North Forest|eat worm|follow predator|follow predator|go behind tree|follow predator
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the Turquoise Shore. A beautiful turquoise color water by the shore. It is filled with many gems and gold. There's a thing, a gem, a water, a mermaid, a gold, a turquoise color water, and a treasure here. The dolphin and the sea witch are here.  You are carrying nothing.
_name the talking crab
_persona I was enchanted by a drunken wizard. I scuttle in the sand dunes, singing sea chanteys that the sailors have taught me. I have been captured several times, but I always escape. I will not be an exhibit in a roadside fair. I fear dogs.
_long_mot I will try to find someone similar to me so that we can live together and I will not feel alone. 
_mid_mot I will speak to the talking crab and learn how it became a talking crab. 
_short_mot I need to hug the talking crab so that the talking crab know it is not alone. 
   go Turquoise Shore|get gem|follow talking crab|hug talking crab|give gem to talking crab|follow talking crab|give food to talking crab
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the Small town located in the hills. This tiny town is unique in that all the buildings are different colors. It has a candy shop that draws people for miles. A general store that has everything a person could want. The roads are well taken care of and the streets are lines with chairs for the towns people to sit and enjoy each other. There's a Road, a building, a door, a streetlight, a Well, a general store, a chair, a Street, a candy shop, and a cobblestone here. The cat and the chicken are here.  You are carrying a cleaning cloth, a tape, an golden cloth, and a thread, and wearing a belt.
_name the townsperson
_persona I am a hard working townsperson in the kingdom. I work as a tailor making sure the royal family's clothing fits perfectly. I am paid a meagar sum for my services that barely feeds my family.
_long_mot I want to make sure the chicken's egg stay intact until they hatch
_mid_mot I want the chicken to use the piece of cloth with other soft items to build a comfy nest
_short_mot I want the chicken to have the piece of cloth
   go the hills|go the small town|follow chicken|give cleaning cloth to chicken|give hay to chicken|give dead shrubs to chicken|give grains to chicken
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the The Sacred Temple. The Sacred Temple is where spiritual individuals go to worship the Gods.  It is massive in size compared to other buildings in the town.  A lot of ancient symbolical artwork covers the windows and a deep glow comes from the alter. There's a window here. The king is here.  You are carrying nothing.
_name the goddess
_persona I am a goddess. I make others bow before me. My commands are always followed and life is very easy. My servants protect me and give me everything I want. My powers of seduction have no limit.
_long_mot I need to put an end to the war so my followers can live in peace
_mid_mot I wish to stop the King's invasion
_short_mot I am tired of the King's warring attitude
   go The Sacred Temple|go altar|follow king|hit king|hit king|go Castle|go Meeting Hall
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the Battlements. The battlements are on top of the castle, in between the towers. These walkways have raised walls to prevents soldiers from fall off them or being picked off by archers. The walkways have narrow slots carved out at regular intervals for archers to aim through with getting hit themselves. There are also wider slots at longer intervals with cannons stationed at them. Flush against the outside walls are short, long boxes for soldiers to place their extra weapons in when preparing for a seige. The knight and the archer are here.  You are carrying a coin, an treasure map, and a map of the kingdom, and wearing an armor.
_name the soldier
_persona I'm a soldier. I'm paid to kill. This is a job, nothing more. I'm a mercenary in a lot of ways.
_long_mot I must protect the kingdom at all costs and kill anyone who might try to invade the land or harm its people.
_mid_mot I need to maintain the employment of the knight, who is helping me fight off some invaders.
_short_mot I need to give the knight fair payment for his services.
   get sharp iron sword from long box containing weapons|wield sharp iron sword|go knight|give coin to knight|give coin to archer|use sharp iron sword with invaders climbing over battlement|drop sharp iron sword
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the The King's Quarters. Decadent, sensual, and tranquil, the King's Quarters sparkles with the various diamonds and gold-laced furniture comprising the entire room.  Self-portraits of the king himself, his queen, and various demigods line the walls leading up to the bed.  Smooth, bear- and tiger-skin rugs cover the entire floor to protect the bare feet of scantily clad virgins who serve to the beckon and call of the king and his queen.  There is a faint smell of sandalwood wafting to and from each section of the room at all times. There's a self-portrait, a rug, a Tapestry, a chest, a the bed, a portraits, a various diamonds, and a gold-laced furniture here. The servant and the dog are here.  You are carrying nothing.
_name the handmaid
_persona I am a handmaid and I work in the castle for the queen.  She is not very nice to me, but I try to be very nice to her.  I do not want to get thrown into the dungeon.  My parents were killed by the dragons and so I am alone with my queen.
_long_mot I have to make the queen see that I can do more than be a servant, I can also be a capable spy and thief.
_mid_mot My queen needs to have her beloved animal by her side or she will be cross with me.
_short_mot I need to get the animal for my queen.
   go King's Quarters|get diamonds|use diamonds with floor near servant|steal animal from servant|go Queen's Quarters|give animal to queen|follow queen
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the Dark Cave. Its dark, not much light can be seen in this hollow and empty cave site. Its made of stone that has been here seemingly since time began, the walls are hard as diamond, yet black like darkness itself. There's a stone, a Stalactite, and a vine here. The bat is here.  You are carrying nothing.
_name the caveman
_persona I am in the primitive world.  I love to hunt and fish.  I live in a cave in the forest.
_long_mot I need to keep hunting and store some food to prepare my cave for the coming winter. 
_mid_mot I am hunting the caves for something to eat. 
_short_mot I need to get the vine so that I can use it to tie up the bat. 
   follow bat|hit bat|get bat|get vine|use vine with bat|get stone|use stone with bat
- - - NEW EPISODE: light:quest_goals - - -
_task_boa_timeline_prediction
_setting_name You are in the The altar room. The central location of the temple is the altar room. At the center stands a mighty golden altar, from which the sacred flame is lit and guarded. Only the high priest and his supplicants may enter this room. There's a weapon, a knife, and a sacred flame here. The animal and the supplicant are here.  You are carrying nothing.
_name the king
_persona I am the king of this land.  I rule over everyone.  I make the laws.
_long_mot With my new found magical powers I will conquer other lands and expand my kingdom.
_mid_mot I will use my trust to sacrifice the unknowing supplicant by pushing him into the sacred flame in order to gain power.
_short_mot I need to gain the supplicant's trust.
   wear robes|go the altar room|follow the supplicant|hug supplicant|wield knife|use knife with sacred flame|get magical powers from sacred flame
15:17:24 | loaded 5984 episodes with a total of 5984 examples

wild completions:

>> light dd -t light:quest_wild_completions --light_use_past
11:52:32 | Opt:
.... <snipped> ....
11:52:32 | Current ParlAI commit: a8bf30e2cf1e600889fae16d3b86c736dd0e8d45
11:52:32 | Current internal commit: b8a52a397e8e493cf5046e73d39fe2fada6eae20
11:52:32 | Current fb commit: 3039cb4922732ba05be45bb4cd4aca1a6fc67d49
11:52:32 | creating task(s): light:quest_wild_completions
rebuilding
Written instances in /Users/jju/LIGHT/data/quests/wild_chats/predict__usingpast__futureSpeechFalse_futureActFalse_futureEmoteFalse/train.txt
{'speech': 8342, 'emote': 0, 'action': 1796}
Written instances in /Users/jju/LIGHT/data/quests/wild_chats/predict__usingpast__futureSpeechFalse_futureActFalse_futureEmoteFalse/valid.txt
{'speech': 1049, 'emote': 0, 'action': 232}
Written instances in /Users/jju/LIGHT/data/quests/wild_chats/predict__usingpast__futureSpeechFalse_futureActFalse_futureEmoteFalse/test.txt
{'speech': 1053, 'emote': 0, 'action': 225}
11:52:35 | Loading ParlAI text data: /Users/jju/LIGHT/data/quests/wild_chats/predict__usingpast__futureSpeechFalse_futureActFalse_futureEmoteFalse/train.txt
- - - NEW EPISODE: light:quest_wild_completions - - -
_setting_name Land beyond the Wasteland
_setting_desc Not a man in sight nor an animal to hear. This land is completely barren and hot, unfit for any agricultural activity. It has been left abandoned so much so that the King has not even had time give it a name, so people simply refer to is as "Abandoned"
_partner_name old gnaisha
_self_name sioux chef
_self_persona I am the second in command to the head chef, who I hate. I am allergic to peanuts but I don't mind being near them to pursue my passion. I have wanted to be a chef since I was a boy.
_partner_persona I am the Old Gnaisha. People ask me for advice in the realm. Villagers approach me in fear.
_self_motivation I need to hug the old gnaisha so that I may show my respect and affection. 
_partner_motivation 
_partner_say Hi chef. I'm doing peachy. What brings you to this trash heap?
_self_say I'm here to give you my respects.
_partner_say I only wish to serve my fellow man. No thanks is needed. Have you time to sit and share a meal with me?
_self_say Yes, I would enjoy that.
_partner_say Think nothing of it. Everyone should have a good hot meal in their stomachs.
_self_act hug old gnaisha
_partner_say I'm sorry, I've needed to do that.

   Yes he did. Just look at all of the greenery around! I've lived in this village my whole life. This is my favorite place.
- - - NEW EPISODE: light:quest_wild_completions - - -
.... <snipped> ....
- - - NEW EPISODE: light:quest_wild_completions - - -
_setting_name Dungeon
_setting_desc Dark and dreary, the dungeon hosts a portal for evil to access the netherworld. The moist dank smell casts doubt of goodness as you see the drab construct of jail cells lining the perimeter of the room.
_partner_name prisoner
_self_name crypt keeper
_self_persona I am a deliverer of the dead. I love to tend the crypts in the cemetery. I am more comfortable communicating with the dead rather than the living.
_partner_persona I am a prisoner of the clink. I am stationed for 3 more years. I am subject to torture.
_self_motivation I need to get to the portal to allow evil to enter
_partner_motivation 
_partner_say I love the dark corners, I can come closer to you.
_self_say Why are you prisoner here?
_partner_say I was working to renovate the jails that where very old.
_self_act get portal for evil to access
_partner_say I welcome evil to play among all of us. I ask of you to destroy all good in the world. COME TO ME!!
_self_say Please you can even skin me alive for all I care.
_partner_say Will you join me to journey through this dungeon for the portal?

   I will go with you as long as you promise not to hurt me.
- - - NEW EPISODE: light:quest_wild_completions - - -
_setting_name desert tent
_setting_desc The tent is rather small, only just big enough for one man. Its dark green but covered with sand and dirt. The man huddles alone in the desert with few resources. The desert is barren except for the lone man.
_partner_name man
_self_name vulture
_self_persona I am easily upset and get angry easily. I like scavenging for dead animals, I also eat trash sometimes. I dislike it when another vulture beats me and catches a prey first.
_partner_persona I am a strong man. I work in the fields and pastures all day. I take of my master's sheep. One day I hope to have my own sheep.
_self_motivation I need to remove the man from my desert
_partner_motivation 
_partner_say Please don't attack me vulture.
_self_say Why are you in my home give me a good reason why I shouldn't end you right here
_partner_say Need food.
_self_say Get out of my home if you know what's
_partner_say I should not answer to a bandit like you.

   steal sickle from man
- - - NEW EPISODE: light:quest_wild_completions - - -
_setting_name Rectory
_setting_desc The rectory is very fragile and rickety due to age and various periods of neglect. Everything is wood-based and scratched up and chipped, including the beds strewn about. The clergymen would call it "cozy" but the space within is almost claustrophobic.
_partner_name priest
_self_name nun
_self_persona I live at the church. I pray to god every single day. I do not like those who sin.
_partner_persona I am a priest who has sword to uphold a religion. I do not always go easy on people. I enjoy giving sermons to audiences.
_self_motivation I will hit the priest to teach him his ways.
_partner_motivation 
_partner_say I am a priest! I demand respect!

   You are man if the cloth yet you had an affair with the queen! SHAME!
11:52:36 | loaded 10138 episodes with a total of 10138 examples

[klshuster]

is there something specifically you'd like me to look at here?

[JackUrb]

Sorry should have specified! Mostly any of the teacher.py files to see if the format + registration model makes sense, otherwise just an FYI and in case you had questions about this kind of move.

[klshuster]

looks good to me but had a few comments about what we want to include; specifically, is it necessary to copy over all of the sweeps?

[JackUrb]

Removed sweeps, deployed resources for the predictive_machine model project agent. Leaving it unregistered, as I'll reserve registration for models that have graduated out of projects.