Research Interests & Projects
Current Projects Previous Projects
Current Projects
Analysis of child-adult interactions in pre-school environment (PI: [Dr. John H.L. Hansen, Co-PI: Dr. Dwight Irvin): Details coming soon !
Speech activity detection for prompted pre-school children speech (PI: Dr. Thomas F. Campbell, Co-PI: Dr. John H.L. Hansen): Speech sound disorder (SSD), a developmental disorder which affects children’s ability to produce the sounds (and words) within their native language, has a prevalence rate of 3-16% among children in USA. Screening for SSDs generally requires recording, evaluation, and decision-making by a certified speech-language pathologist (SLP). Automating part or all of this process could significantly reduce the amount of time and effort in the screening process. However, in order for this process and especially the final ‘pass’/‘fail’ screening decision to be automated, children’s speech content must be extracted from within a collected audio sample and therefore requires speech/silence activity detection. For this study, an iOS application for field use was developed to collect speech word productions from children, with algorithmic processing of all participants assigned a Percentage of Consonants Correct-Revised (PCC-R) score by a certified SLP. An unsupervised speech-activity-detection (SAD) algorithm is explored. COMBO-SAD, originally developed during the DARPA-RATS program, was modified to for use on child speech. Model evaluation was performed on a diverse collected child corpus based on their PCC-R score. Finally, a duration “shoulder” extension of SAD boundary labels was also analyzed to benchmark potential system impact on model performance.
Previous Projects
Sequential pattern learning in children with Developmental Language Disorder (PI: Dr. Lisa Goffman, Co-PIs: Dr. Sebastien Hélie, Dr. Jun Wang):
Developmental language disorder (DLD; aka specific language impairment) affects approximately 7% of children at the time they enter kindergarten, with longstanding adverse academic, social, and communicative consequences. While there is no question that language deficits feature prominently in children diagnosed with DLD, there are other cognitive and motor capacities affected—such as pattern induction, rhythmic grouping, and sequential organization. Previous findings show that the capacity for deploying sequentially patterned information mediates language and motor production. Sequential patterning is a central component of phonology and morphosyntax–domains of language difficulty presented in children with DLD. The central aim of this project is to implement a novel framework for applying domain general cognitive mechanisms to learning and generalization; specifically all of the proposed experiments have in common the hypothesis that children with DLD will learn more effectively and generalize more broadly when targets are selected to emphasize the regularity of sequential patterns.Articulatory kinematic analysis of subjects with Amyotrophic Lateral Sclerosis (PI: Dr. Jun Wang):
Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease that severely impairs voluntary motor function. As ALS progresses, patients will experience loss of motor control including difficulties to maintain speech and swallowing abilities. Accurate detection and monitoring of ALS are critical for effective therapeutic intervention. Recent studies discovered that the articulatory subsystem was most sensitive to ALS for monitoring speech performance decline and predicted that the speed of the tongue might be particularly sensitive to early changes in speech motor performance. Research suggests that articulatory kinematics may be beneficial for earlier diagnosis of ALS. This project was motivated by the need to develop more sensitive markers of disease progression for subjects with ALS which will improve current clinical practise. [Journal in review]Articulatory kinematic analysis of subjects with Laryngectomy vs. Silent Speech (PI: Dr. Jun Wang):
Laryngectomy is the surgical removal of a larynx due to oral or laryngeal cancer. After the larynx is removed, the articulatory control system cannot modify the length of the vocal tract based on larynx height and the patients lose the ability to control pitch. To overcome limitations of available voice prosthesis options, studies have focused on the development of silent speech interface (SSI) technology which allows for communication without the vibration of the vocal folds. Specifically, SSIs convert articulatory motion data to an acoustic output. A better understanding of alaryngeal speech is needed to advance scientific knowledge of motor control and for the development of SSI technology. The aim of this study was to characterize articulatory movements during the production of phrases obtained from healthy (voiced, silent) speakers and laryngectomees. In addition, a support vector machine (SVM) model was used to classify the three speech modes based on tongue and lip motion patterns. [Poster]Real-time voice activity detection from non-invasive neuromagnetic (MEG) signals (PI: Dr. Jun Wang):
Neural speech decoding-driven brain-computer interface (BCI) or speech-BCI is a novel paradigm for exploring communication restoration for locked-in (fully paralyzed but aware) patients. Speech-BCIs aim to map a direct transformation from neural signals to text or speech, which has the potential for a higher communication rate than the current BCIs. Although recent progress has demonstrated the potential of speech-BCIs from either invasive or non-invasive neural signals, the majority of the systems developed so far still assume knowing the onset and offset of the speech utterances within the continuous neural recordings. To address this issue, in this study, an attempt was made to automatically detect the voice/speech activity directly from the neural signals recorded using magnetoencephalography (MEG) using long short-term memory-recurrent neural networks (LSTM-RNN). [Journal]