Personal tools

Skip to content. | Skip to navigation

You are here: Home HAIL Abstracts 2008 Automatic Cognitive Load Detection Based on Speech Features

Automatic Cognitive Load Detection Based on Speech Features


Automatic Cognitive Load Detection Based on Speech Features

Dr Fang Chen
Senior Principal Researcher/Research Group Manager
> Making Sense of Data Theme<br> NICTA

Tuesday 11 March 2008 at 11am



Monitoring cognitive load is important for the prevention of faulty errors in task-critical operations, and the development of adaptive user interfaces, to maintain productivity and efficiency in work performance. Cognitive load variations have been found to impact multimodal behaviour, in particular, features of spoken input. Our research goal is the implicit, objective, automated and real-time estimation of a user's cognitive load based on human's natural speech features, suitable for high-complexity, real-time deployment.

In this talk, we focus on potential speech feature indices. We present our attempt to induce controlled levels of load and solicit natural speech, and the use of machine learning in the development of a speech classifier that is able to detect different levels of load in the speech signal. The user studies confirm a significant variation of speech features in different cognitive load levels.

To capture the variation information of speech features and output the cognitive load level as discrete level ranges, we propose a novel, fully automatic, speaker-independent classification approach to monitor cognitive load level by using speech features. In this approach, a Gaussian Mixture Model (GMM) based classifier is created with unsupervised training. Channel and speaker normalization are deployed for improving robustness. Different delta techniques are investigated for capturing temporal information. And a background model is introduced to reduce the impact of insufficient training data. The proposed automatic cognitive load detection system achieves 71.1% and 77.5% accuracy in two tasks respectively.

To validate this proposed approach in a real-world scenario, we investigate a real life bush fire management situation. In this scenario, incident commander exchange information with and give orders to other roles mainly by speech. The proposed system is tested in this scenario and receives reasonable results considering the much more background noises and lower speech quality. This experiment shows a great potential in real-world applications.

Short resume

Dr. Chen was employed with Beijing Jiaotong University in China from 1995-1999. She was appointed Associate Professor of the Faculty of Electronic and Information Engineering in 1995, the Deputy Director of the Institute of Information Science in 1996 and then Dean of Faculty of Electronic and Information Engineering in 1997. She began her career in industry in 1999 as senior researcher and Team Leader of Text-to-Speech Group in Intel China Research Centre. She joined Motorola in 2000 as a Principal Researcher, and was the founding research manager of the Speech and Language Generation Research Lab of Motorola China Research Centre, where she also acted as the account manager of business relationships for the Centre.

Dr Chen moved to Australia in 2002 to work for the Motorola Australian Research Centre, where she chaired the Patent and Publication Committees. She jointed NICTA in 2004 and is currently Senior Principal Researcher and ATP research group manager in the theme of making sense of data. Dr.Chen has received visiting Professor and Honorary Associate positions with the University of New South Wales and the University of Sydney.

Back to HAIL Home Page

Document Actions
Museum Robot

School students and others living in all corners of the country will soon have greater access to Australia's history, thanks to a CSIRO robot and high speed broadband connectivity. Read more...

Colin Griffith Apps Day spotlight

The Australian Centre for Broadband Innovation's first industry event demonstrated there is strong interest in driving innovation in broadband-enabled services. Read more...

Wifi Team

The CSIRO team that invented one of the foundation technologies of Wi-Fi has been named as a finalist in the European Inventor Award 2012. Read more

HAIL enquiries

Jessie Yin