Improving brain-to-text models utilizing recurrent neural networks and transfer learning for neuroprosthetic speech
Loading...
Date
2023
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This PhD thesis focuses on the decoding of silent speech from electroencephalogram (EEG) signals using Brain-Computer Interfaces (BCIs) and transfer learning tech- niques. Silent speech, also known as imagined speech, involves the generation of neural patterns related to speech without audible sound production. Decoding silent speech from EEG signals is challenging due to the noisy and low-resolution nature of the signals. While previous studies have achieved varying levels of success in classify- ing individual words or phonemes from EEG signals, the translation of full sentences remains intricate.
The motivation for this research stems from the potential of BCIs to restore com- munication abilities for individuals with speech impairments caused by neuromuscular disorders. Existing speech decoding methods, particularly for silent speech, heavily rely on neural networks and require large amounts of data, which is both costly and time-intensive to record. Transfer learning is proposed as a solution to address this data limitation in silent speech decoding.
The research encompasses three main studies: subject-independent sentence de- coding, decoding individual words from continuous imagined speech, and applying Connectionist Temporal Classification (CTC) loss for EEG-based speech decoding. The findings highlight the potential of Deep Neural Networks (DNNs) in decoding speech signals across individuals, while also shedding light on the challenges inherent in silent speech production. Additionally, this thesis introduces the “Large Spanish EEG” dataset, facilitating further advancements in the field.
Overall, this thesis demonstrates the feasibility and potential of deep learning techniques and transfer learning in EEG-based speech decoding. The research not only paves the way for future exploration but also brings us closer to the development of robust and real-world applicable tools for silent speech decoding. Ultimately, these advancements hold the promise of enhancing the quality of life for individuals with speech and language impairments.
Description
Tesis (Ph.D. in Biological and Medical Engineering)--Pontificia Universidad Católica de Chile, 2023