A Robot Photographer Uses Humor to Capture Genuine Smiles, Interact with Users

A team from the Computer Science Department from Yale University have created a system for improving photos of people — by making a robot which makes them laugh before capturing the image.

Anyone who has posed for a photo as a child will remember "smile for the birdie," and even through adulthood "say cheese" remains a constant refrain in an effort to produce a natural-looking smile in the finished image. The Yale researchers have taken a different approach: A robot photographer which surprises the subject with amusing imagery before capturing their photo, eliciting a hopefully-genuine smile response.

"Our human-centered design process involved professional and amateur photographers to better understand the social dimensions of subject-photographer interactions. This exploration then guided our design of a robot photographer, which employs humor to elicit spontaneous smiles during photography events," the researchers explain. "In a laboratory evaluation of our robot prototype, we found that the majority of the subjects considered the robot’s humor to be comical and appreciated it. More spontaneous smiles were elicited by the robot when it delivered humorous content to its subjects than when it was not humorous."

While robot photographers aren't new — the team highlight commercial products designed for portraiture at weddings and other social events — the team's focus was on both autonomous operation and the use of humor. "The robot would operate in the context of an indoor, semi-public university space without strict noise constraints, e.g., the entrance of a university’s publicly accessible library." the team writes. "Such a space would provide protection from the weather and vandalism. Also, it would allow for interactions with a diverse array of people, including campus visitors, students, staff, and faculty.

"As visitors come to explore the campus, the robot would serve as a local photographer who can capture pictures of them for free. It would provide them with both a fun experience and with a photo souvenir of their visit. For the regular members of the community, the robot would serve as a tool to release stress and document relevant moments of their life on campus. Our robot would thus inhabit the social role of a fun, comfortable, and helpful service supplier and entertainer."

The robot itself is a tabletop unit based on the commercial WidowX Robot Arm Kit from Trossen Robotics, modified to house a robot head with RealSense D435i RGB-D camera, capable of capturing both the finished photo and tracking users, and screen "face" for interaction. Having initially experimented with a laptop, the team eventually settled on a more powerful desktop computer with microphone array and stereo speakers running the Robot Operating System (ROS), OpenCV face recognition, and text-to-speech and speech-to-text services from the Google Cloud platform.

The results were promising: "[We] found that our robot was able to leverage humor to elicit spontaneous smiles from subjects," the team concludes. "The more people smiled naturally as a result of the robot’s humor, the higher they rated their photography experience with the robot."

More information is available in the team's paper, published as part of the proceedings of the ACM/IEEE International Conference on Human-Robot Interaction 2020 (HRI'20) and made available via GitLab.