This project has the following contributions:
1- Human tracking using Pepper robot.
2- Procedure of turning on the awareness in Pepper robot of its physical surroundings.
3- Reading and storing images from 3D sensor and VGA forehead camera of the Pepper robot.
4- Pepper Interact with people with four set of actions
1- Doing nothing (also called waiting in social terms)
2- Looking towards human through tracking human with head motion.
3- Waving hand and saying hello
4- Handshaking: The robot also grabs the hand of the person if the FSR touch sensor on the robot's hand detects the touch.
Requirements:
1- Pepper Robot (https://www.ald.softbankrobotics.com/en/cool-robots/pepper)
2- [Optional] Mbed microcontroller (https://developer.mbed.org/platforms/mbed-LPC1768/)
3- [Optional] FSR Touch sensor (https://www.sparkfun.com/products/9376)
Procedure:
1- Bring FSR into action through integration with mbed or any microcontroller (please burn the code 'uC.cpp' on mbed).
2- Paste FSR on Pepper's right hand.
3- Run mbed_usb.py and then main.py on two seperate terminals
We utilized this project together with a deep neural network based policy network to enable a robot to learn social norms through interaction with people in public places. The network tells the robot which action to choose (Waiting, look towards human, wave hand and handshake) in any given scenario.
Please refer to the following paper for more details:
Qureshi AH, Nakamura Y, Yoshikawa Y, Ishiguro H. Robot gains social intelligence through multimodal deep reinforcement learning. InHumanoid Robots (Humanoids), 2016 IEEE-RAS 16th International Conference on 2016 Nov 15 (pp. 745-751). IEEE.