Polling is an activity that is common in classrooms to foster student engagement and promote learning. Traditionally, polling is conducted as a pre-planned activity using online polling websites like Poll Everywhere or physical clickers such as iClicker and Turning. These technologies have constraints of cost and/or time, requiring the instructor to prepare the polls in advance. However, there are numerous instances where impromptu polling in the classroom can provide instructors with valuable student feedback to help guide their instruction. Such impromptu polls are typically conducted by announcing the poll activity and gauging the visual cues, e.g., raising of hands and performing other hand gestures, that are provided by the students in response. However, this can be a time consuming or an error prone process in large classrooms.
This project introduces a vision-based system to quickly and accurately conduct impromptu polls in classrooms. The system relies on a camera commonly found in many classrooms to gather visual data and process it using Machine Learning-based computer vision tools. The system provides a simple interface to the instructor to conduct the impromptu polls and display the counts for various gestures that are gathered in response. The system also provides the added advantage of conducting impromptu polls that involve questions with more nuanced responses than binary gestures (e.g., thumbs up/down or raising of hands) that can be quickly and accurately evaluated by the instructor.
- Jung In Koh, Samantha Ray, Josh Cherian, Paul Taele, and Tracy Hammond. 2022. Show of Hands: Leveraging Hand Gestural Cues in Virtual Meetings for Intelligent Impromptu Polling Interactions. In 27th International Conference on Intelligent User Interfaces (Helsinki, Finland) (IUI ’22). Association for Computing Machinery, New York, NY, USA, 292–309. https://doi.org/10.1145/3490099.3511153
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
- MediaPipe Hand Landmark Detection
- MediaPipe Gesture Recognition
- YOLO-NAS Pose
- ONNX Python Runtime
- Hand Recognition Sample Repo