In a recent series of experiments led by researcher Lin and his colleagues, the brain-to-text translation system known as DeWave demonstrated a notable improvement, achieving just over 40 percent accuracy in one set of metrics. While this marks a 3 percent enhancement over the previous standard for translating thoughts from EEG recordings, the researchers aim to further refine DeWave’s accuracy to approximately 90 percent, aligning it with conventional language translation methods and speech recognition software.
Unlike alternative approaches that involve invasive surgeries or cumbersome MRI machines, DeWave offers a non-intrusive solution for daily use. Previous methods often required eye-tracking technology to convert brain signals into word-level chunks, assuming that the brain pauses briefly between processing each word during eye movement. However, translating raw EEG waves into words without relying on eye tracking presents a more challenging task.
DeWave employs an innovative encoder that, after extensive training, translates EEG waves into a code. This code is then matched to specific words based on their proximity to entries in DeWave’s ‘codebook,’ representing a discrete encoding technique in the brain-to-text translation process. According to Lin, this approach marks a significant milestone in neural decoding and expands the intersection of neuroscience and AI.
The researchers utilized a combination of language models, incorporating BERT with GPT, to train DeWave. Testing involved existing datasets with recorded eye tracking and brain activity during text reading. The system learned to associate brain wave patterns with words and underwent additional training with an open-source large language model, forming coherent sentences.
While DeWave excelled in translating verbs, translating nouns proved to be challenging, often resulting in pairs of words with similar meanings rather than exact translations. The researchers attribute this to the semantic similarity of words producing analogous brain wave patterns during processing.
The study’s significance lies in its relatively large sample size, addressing the variability in EEG wave distributions among individuals. Despite this progress, challenges remain, particularly concerning the noise in EEG signals received through caps instead of electrodes implanted in the brain.
Acknowledging the value and complexity of directly translating thoughts from the brain, the researchers emphasize the need for sustained efforts in this challenging field. With the rapid advancement of Large Language Models, they advocate for increased attention to encoding methods that bridge brain activity with natural language, paving the way for future breakthroughs in brain-to-text technology.
Related topics:
What is Computer Intelligence & How does Computer Intelligence work