Monthly Archives: November 2016

Contest for her work fighting bias in machine

When Joy Buolamwini, an MIT master’s candidate in media arts and sciences, sits in front a mirror, she sees a black woman in her 20s. But when her photo is run through recognition software, it does not recognize her face. A seemingly neutral machine programmed with algorithms-codified processes simply fails to detect her features. Buolamwini is, she says, “on the wrong side of computational decisions” that can lead to exclusionary and discriminatory practices and behaviors in society.

That phenomenon, which Buolamwini calls the “coded gaze,”  is what motivated her late last year to launch the Algorithmic Justice League (AJL) to highlight such bias through provocative media and interactive exhibitions; to provide space for people to voice concerns and experiences with coded discrimination; and to develop practices for accountability during the design, development, and deployment phases of coded systems.

That work is what contributed to the Media Lab student earning the grand prize in the professional category of The Search for Hidden Figures. The nationwide contest, created by PepsiCo and 21st Century Fox in partnership with the New York Academy of Sciences, is named for a recently released film that tells the real-life story of three African-American women at NASA whose math brilliance helped launch the United States into the space race in the early 1960s.

“I’m honored to receive this recognition, and I’ll use the prize to continue my mission to show compassion through computation,” says Buolamwini, who was born in Canada, then lived in Ghana and, at the age of four, moved to Oxford, Mississippi. She’s a two-time recipient of an Astronaut Scholarship in a program established by NASA’s Mercury 7 crew members, including late astronaut John Glenn, who are depicted in the film “Hidden Figures.”

The film had a big impact on Buolamwini when she saw a special MIT sneak preview in early December: “I witnessed the power of storytelling to change cultural perceptions by highlighting hidden truths. After the screening where I met Margot Lee Shetterly, who wrote the book on which the film is based, I left inspired to tell my story, and applied for the contest. Being selected as a grand prize winner provides affirmation that pursuing STEM is worth celebrating. And it’s an important reminder to share the stories of discriminatory experiences that necessitate the Algorithmic Justice League as well as the uplifting stories of people who come together to create a world where technology can work for all of us and drive social change.”

The Search for Hidden Figures contest attracted 7,300 submissions from students across the United States. As one of two grand prize winners, Buolamwini receives a $50,000 scholarship, a trip to the Kennedy Space Center in Florida, plus access to New York Academy of Sciences training materials and programs in STEM. She plans to use the prize resources to develop what she calls “bias busting” tools to help defeat bias in machine learning.

That is the focus of her current research at the MIT Media Lab, where Buolamwini is in the Civic Media group pursuing a master’s degree with an eye toward a PhD. “The Media Lab serves as a unifying thread in my journey in STEM. Until I saw the lab on TV, I didn’t realize there was a place dedicated to exploring the future of humanity and technology by allowing us to indulge our imaginations by continuously asking, ‘What if?'”

Learns to recognize sounds

In recent years, computers have gotten remarkably good at recognizing speech and images: Think of the dictation software on most cellphones, or the algorithms that automatically identify people in photos posted to Facebook.

But recognition of natural sounds — such as crowds cheering or waves crashing — has lagged behind. That’s because most automated recognition systems, whether they process audio or visual information, are the result of machine learning, in which computers search for patterns in huge compendia of training data. Usually, the training data has to be first annotated by hand, which is prohibitively expensive for all but the highest-demand applications.

Sound recognition may be catching up, however, thanks to researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). At the Neural Information Processing Systems conference next week, they will present a sound-recognition system that outperforms its predecessors but didn’t require hand-annotated data during training.

Instead, the researchers trained the system on video. First, existing computer vision systems that recognize scenes and objects categorized the images in the video. The new system then found correlations between those visual categories and natural sounds.

“Computer vision has gotten so good that we can transfer it to other domains,” says Carl Vondrick, an MIT graduate student in electrical engineering and computer science and one of the paper’s two first authors. “We’re capitalizing on the natural synchronization between vision and sound. We scale up with tons of unlabeled video to learn to understand sound.”

The researchers tested their system on two standard databases of annotated sound recordings, and it was between 13 and 15 percent more accurate than the best-performing previous system. On a data set with 10 different sound categories, it could categorize sounds with 92 percent accuracy, and on a data set with 50 categories it performed with 74 percent accuracy. On those same data sets, humans are 96 percent and 81 percent accurate, respectively.

“Even humans are ambiguous,” says Yusuf Aytar, the paper’s other first author and a postdoc in the lab of MIT professor of electrical engineering and computer science Antonio Torralba. Torralba is the final co-author on the paper.

“We did an experiment with Carl,” Aytar says. “Carl was looking at the computer monitor, and I couldn’t see it. He would play a recording and I would try to guess what it was. It turns out this is really, really hard. I could tell indoor from outdoor, basic guesses, but when it comes to the details — ‘Is it a restaurant?’ — those details are missing. Even for annotation purposes, the task is really hard.”

Complementary modalities

Because it takes far less power to collect and process audio data than it does to collect and process visual data, the researchers envision that a sound-recognition system could be used to improve the context sensitivity of mobile devices.

When coupled with GPS data, for instance, a sound-recognition system could determine that a cellphone user is in a movie theater and that the movie has started, and the phone could automatically route calls to a prerecorded outgoing message. Similarly, sound recognition could improve the situational awareness of autonomous robots.

“For instance, think of a self-driving car,” Aytar says. “There’s an ambulance coming, and the car doesn’t see it. If it hears it, it can make future predictions for the ambulance — which path it’s going to take — just purely based on sound.”

The intersection of policy and technology

“When you’re part of a community, you want to leave it better than you found it,” says Keertan Kini, an MEng student in the Department of Electrical Engineering, or Course 6. That philosophy has guided Kini throughout his years at MIT, as he works to improve policy both inside and out of MIT.

As a member of the Undergraduate Student Advisory Group, former chair of the Course 6 Underground Guide Committee, member of the Internet Policy Research Initiative (IPRI), and of the Advanced Network Architecture group, Kini’s research focus has been in finding ways that technology and policy can work together. As Kini puts it, “there can be unintended consequences when you don’t have technology makers who are talking to policymakers and you don’t have policymakers talking to technologists.” His goal is to allow them to talk to each other.

At 14, Kini first started to get interested in politics. He volunteered for President Obama’s 2008 campaign, making calls and putting up posters. “That was the point I became civically engaged,” says Kini. After that, he was campaigning for a ballot initiative to raise more funding for his high school, and he hasn’t stopped being interested in public policy since.

High school was also where Kini became interested in computer science. He took a computer science class in high school on the recommendation of his sister, and in his senior year, he started watching computer science lectures on MIT OpenCourseWare (OCW) by Hal Abelson, a professor in MIT’s Department of Electrical Engineering and Computer Science.

“That lecture reframed what computer science was. I loved it,” Kini recalls. “The professor said ‘it’s not about computers, and it’s not about science’. It might be an art or engineering, but it’s not science, because what we’re working with are idealized components, and ultimately the power of what we can actually achieve with them is not based so much on physical limitations so much as the limitations of the mind.”

In part thanks to Abelson’s OCW lectures, Kini came to MIT to study electrical engineering and computer science. Kini is currently pursuing an MEng in electrical engineering and computer science, a fifth-year master’s program following his undergraduate studies in electrical engineering and computer science.

Combining two disciplines

Kini set his policy interest to the side his freshman year, until he took 6.805J (Foundations of Information Policy), with Abelson, the same professor who inspired Kini to study computer science. After taking Abelson’s course, Kini joined him and Daniel Weitzner, a principal research scientist in the Computer Science and Artificial Intelligence Laboratory, in putting together a big data and privacy workshop for the White House in the wake of the Edward Snowden leak of classified information from the National Security Agency. Four years later, Kini is now a teaching assistant for 6.805J.

With Weitzner as his advisor, Kini went on to work on a SuperUROP, an advanced version of the Undergraduate Research Opportunities Program in which students take on their own research project for a full year. Kini’s project focused on making it easier for organizations that had experienced a cybersecurity breach to share how the breach happened with other organizations, without accidentally sharing private or confidential information as well.