Voicebox streamlines data collection for NLP using AI

Voicebox Advanced Technologies Team announced a significant innovation that reduces the burden of data collection while producing impressive, industry-leading results. More importantly, the innovation helps overcome the challenge of parsing multi-language utterances, known in linguistic circles as “code-switching.”

AI techniques such as semantic parsing hold the promise of revolutionizing natural language processing. However acquiring training data for each domain in each supported language remains cumbersome and expensive. This limitation has become increasingly significant as companies demand intelligent, conversational voice applications to support their global product strategies.

“To date, voice navigation systems, for example, have struggled with multi-lingual use cases. This has been a serious usability issue in multi-language regions, such as Europe or Asia where code-switching is common. For example, a French person driving in Germany may say, “Wie viele Universitäten ont Paris?” (How many universities does Paris have?) Similarly, it is very common for a person in China to ask, “Let It Be.” (Play the song Let It Be)”, said Dr. Phil Cohen, Chief Scientist, AI at Voicebox.

Voicebox’s approach applies the ‘learning’ of one language to another. The team evaluated utterances in German, English and a mix of both in a single utterance. They developed a neural network model, trained on both English- and German-only sentences. As part of a single semantic parsing process, this model transfers information from one language to the other, thus leveraging English data to reduce the amount of data needed for German.

As a result, performance on each language improved, yielding state-of-the-art accuracy comparable to the likes of Google. The team also evaluated utterances that could contain a mix of both languages and weren’t in any of the training data. Results were similarly impressive.

Source: Voicebox

Banner space