Xuedong Huang, technical fellow in charge of Microsoft’s speech, natural language and machine translation efforts. (Photo by Scott Eklund/Red Box Pictures)
Microsoft researchers announced that they believe they have created the first machine translation system that can translate sentences of news articles from Chinese to English with the same quality and accuracy as a person.
Researchers in Microsoft’s Asia and U.S. labs said that their system achieved human parity on a commonly used test set of news stories developed by a group of industry and academic partners. To ensure the results were both accurate and on par with what people would have done, the team hired external bilingual human evaluators, who compared Microsoft’s results to two independently produced human reference translations.
Machine translation is a problem that researchers have worked on for decades. And for much of that time, many believed human parity could never be achieved. One of the major reasons is because translation is one of the most challenging, complex and nuanced natural language processing tasks.
To reach the human parity milestone, three research teams in Microsoft’s Beijing and Redmond, Washington research labs worked together to use several training methods that made the system more fluent and accurate. In many cases, these new methods mimic how people improve their own work iteratively, by going over it again and again until they get it right. These approaches include dual learning, deliberation networks and joint training.
This milestone is especially significant not only because it removes language barriers to help people communicate and understand each other better; the methods and techniques used will also be useful for improving machine translation in other languages and for making other AI breakthroughs, such as reaching human parity in speech-to-speech translation.