Visualization and Intelligent Systems Laboratory



Contact Information

Winston Chung Hall Room 216
University of California, Riverside
900 University Avenue
Riverside, CA 92521-0425

Tel: (951)-827-3954

Bourns College of Engineering
NSF IGERT on Video Bioinformatics

UCR Collaborators:

Other Collaborators:
Keio University

Other Activities:
IEEE Biometrics Workshop 2019
IEEE Biometrics Workshop 2018
Worshop on DVSN 2009
Multibiometrics Book

Webmaster Contact Information:
Alex Shin

Last updated: July 1, 2017



***This webpage is under construction***

Understanding Subtle Non-Social Facial Expressivity to Boost Learning and Computer Interaction

NSF Project ID: IIS -

Principal Investigator

Bir Bhanu
Center for Research in Intelligent Systems
Room 216, Winston Chung Hall, University of California at Riverside,
Riverside, CA 92521
Tel. (951)827-3954, Fax. (951)827-2425

Co-Principal Investigator

Aaron Seitz
Department of Psychology
University of California - Riverside
900 University Avenue
Riverside, CA 92521
Tel. (951) 827-6422, Fax. (951) 827-3985



Facial expressions play a significant role in everyday communication among humans. Computer understanding of these complex and subtle expressions will lead to highly capable interactive cyber-human systems with proactive computers that make more appropriate responses to human interactions. This award brings together an interdisciplinary team of investigators to address key challenges associated with spontaneous microexpression recognition in non-social scenarios. The project concentrates on generating bio-feedback from humans while learning skills, such as game playing and online learning, and being recorded and analyzed in continuous color and depth video streams. It will develop computer algorithms for human-machine synergy and test how this information can provide for superior learning when training applications are augmented with expression-informed bio-feedback in near real-time. This represents a significant step forward in training machines to recognize and classify facial microexpressions and maximizing the synergy of cyber-human systems that will improve the quality of life experiences. Understanding complex and subtle human facial expressions as captured in continuous video streams will have a profound impact on human-computer interaction. It will provide a computing environment within the reach of common people in which the interests or even the health of people can be detected and predicted, with significant impacts on skill learning, education and information retrieval.

The project develops a transformative approach to the understanding of complex and subtle facial microexpressions and bio-feedback where the synergy between cyber and human systems can be fully exploited. It addresses key challenges associated with computational understanding and modeling of intelligence in challenging, realistic contexts. It uses assessment and intervention based on facial microexpressions to maximize synergy of cyber and human systems for skill learning. First, it considers deep learning and closed-loop video analysis for optimized skill learning in a reinforcement learning framework. Second, it develops novel representation of facial microexpressions from color and depth video streams and use them for person independent emotion recognition as well as person-specific emotions recognition when a game play is adapted. Third, it exploits not only the color camera but also the integrated depth camera for precise measurements, which has not been used for microexpressions. The focus is to determine the extent to which real-time classification of microexpressions can provide for more appropriate interactivity that will facilitate human learning in real applications. The results will be broadly disseminated through a website that will have regular releases of databases and software tools by offering tutorials, workshops and demos at major professional meetings.