“AlphaGo,” the artificial intelligence program that beat the world’s best Go player this week, had some of its roots in minds at the University of Massachusetts Amherst in the 1970s.
Go is an ancient board game played with black and white stones on a grid. The game is so complex that artificial intelligence experts did not think a computer would be able to beat the top human players for decades.
Google DeepMind, which developed AlphaGo, proved them wrong this week when their program bested Lee Se-dol, the world champion, four games to one.
“This is a huge breakthrough for AI and machine learning,” said Sridhar Mahadevan, co-director of the Autonomous Learning Laboratory at the College of Information and Computer Sciences at UMass Amherst. “AlphaGo was not programmed explicitly to play Go, but learned to play using an approach called reinforcement learning, or RL, pioneered here at UMass Amherst several decades ago by Andy Barto, considered the father of RL, and his students, principally Rich Sutton. Many of the top researchers in the field today are his students. So this achievement was made possible by research done at UMass Amherst computer science, and it’s a source of special pride for us all.”
Barto and Sutton’s book, “Reinforcement Learning,” is considered the bible in the field, and was cited in DeepMind’s January paper in “Nature” laying out AlphaGo’s strategy, according to Mahadevan.
Reinforcement learning is inspired by trial-and-error learning that humans and other animals do. It was popular when developed in 1977 at UMass, but has since fallen out of favor, according to Barto,who co-directs the Autonomous Learning Laboratory with Mahadevan.
“I came to think it had been prematurely dismissed,” he said.
According to Barto, the concept dates back to the 19th century in psychology and was first used by pioneer computer scientist Alan Turing in 1948 in a computer science context.
Using that practice in the context of Go meant coming up with a good evaluation of which move the computer should take when matched against the astronomical possibilities.
“Short of planning moves by doing a lot of lookahead searches, you need a shortcut to predict, a very quick assessment without doing all that lookahead,” Barto said. “That’s what AlphaGo has come up with and what many people didn’t think was possible.”
Barto and Sutton are now revising their 1998 textbook, which has been cited 20,000 times, and will include a description of AlphaGo’s success as a case study.
Dave Eisenstadter can be reached at deisen@gazettenet.com.