MY RESEARCH INTERESTS

Basic Research Activities

My training is interdisciplinary, combining a strong background in mathematical psychology and cognitive experimental training at UC Irvine with postdoctoral training in emotion and the measurement of facial behavior, self-report, categorization of emotion, and the emotion lexicon. Expressive behavior and the importance of affect in interpersonal regulation is an important subfield of social psychology. The existence and universality of certain basic expressions of emotion has been key evidence supporting evolutionary biological theories of emotion such as Ekman's “basic emotions” theory. Thus, facial expressions have been at the heart of an ongoing debate about the universality versus social construction of emotional experience. My earliest research sought to place Ekman's basic emotions theory on a more sound empirical foundation by addressing certain challenges to it. My first contribution to that debate was to produce Ekman's results without using a flawed multiple-choice format, and without matching emotion terms to expressions. I was able to demonstrate that people did attribute emotional meanings to the basic expressions, using an arguably more rigorous methodology, thus supporting basic emotions theory. However, I also challenged whether Ekman's contempt expression was properly labeled, suggesting that it might indicate annoyance instead.

One influential theory proposed by critics is that combinations of component movements, each related to some element of the context in which an emotion arises, produce an endless variety of interpretable expressions. I have tested that theory and shown empirically that component movements of facial expressions are not reliably interpreted and do not influence attributional judgments independent of the gestalt formed by an overall facial expression. The inability of observers to reliably interpret components raises serious questions for those proposing component theories of appraisal-related expression and cross-cultural or in-group/out-group dialect theories of facial expression. My use of consensus modeling to test agreement among observers permits such theories to be evaluated empirically and has resulted in disconfirming evidence for proposals that have high intuitive appeal but little evidence supporting them. This research has been important to formulation of my own theory of decoding, drawing upon ideas about categorization and the use of maximally discriminable features to disambiguate visual stimuli.

Dr. Christine Harris and I conducted a comparison of tickle responses with pain (induced by placing a hand in ice water) and humor. This research was the first to study tickling by measuring facial activity and comparing it to self-reported affect. We found that tickling includes movements common to both pain and humor, and that Duchenne smiles, commonly associated with enjoyment, appear unrelated to self-reported pleasure when they occur during tickling. This is important due to the knee-jerk assumption that anyone displaying a Duchenne smile must be feeling pleasure or enjoyment. By administering a coping styles measure to subjects, we used discriminant analysis to demonstrate the impact of display rules and facial control as an aspect of coping.

Prior research at UCSD included my collaboration on studies of color naming and categorization with Dr. Kimberly Jameson. Our work on color naming has been important in challenging previous too simplistic models of color naming linking basic color terms directly to underlying color neurophysiology. We also challenged the more extreme relativist theories of color naming by hypothesizing a set of universal cognitive principles of naming and categorization applied to specific cultural experiences. Our work accounts for complexities unexplained by Berlin & Kay's theories or Kay and colleague's more recent proposals. Dr. Jameson and I have conducted cross-cultural studies of the emotion lexicon, as a comparison to color naming. This comparison between the domains of color and emotion naming has been useful in figuring out which aspects of behavior arise from general cognitive processes and which are specific to each domain. This has given me ideas about how a language's naming practices (especially use of intensifiers) contribute to the emergence of basic emotion terms in a culture.

Applied Research Activities

Cultural misunderstandings about expressivity have important implications for interpersonal interactions in everyday life. Undertreatment of pain among ethnic minorities is a serious, acknowledged problem in the medical literature. Stereotypes, combined with actual differences in expressive and self report behavior may contribute to the undertreatment. My current research, funded by an MBRS/SCORE grant, explores differences in pain facial expression and self-report to determine whether membership in an ethnic or racial subculture influences: (1) expression and self report of pain; (2) beliefs about appropriate display of pain; and (3) evaluation of observed pain in another person of the same or different subculture. This research will contribute to the understanding of reasons for the under-treatment of pain for minorities and may have strong implications for training of medical personnel to recognize pain in minority clients. I am using a cold-pressor task to elicit pain in subjects who are videotaped during the experience. Their expressions will later be presented as stimuli to observers. Four groups of healthy students are being compared: (1) African-American students; (2) Hispanic students (Mexican-American, Mexican and Latin American immigrants); (3) Asian students (Asian-American and Asian immigrants, predominantly Chinese, Vietnamese, Japanese, Korean); (4) European-American students. An additional group of trained health care providers will be compared to untrained students as observers of pain expressions. This research is testing the hypothesis that both stereotypes and cultural beliefs about appropriateness of pain expression differentially affect both pain expression and observer responses to such expressions for members of different subcultures, resulting in greater pain underestimation for minority subcultures.

A second area of applied research for me has been portrayal of expression and recognition of affect by computer systems. In 2001, I assisted Dr. Rosalind Picard at the MIT Media Lab in a project to train a computer to recognize facial expressions and nonverbal behavior indicating frustration. While they were able to successfully train the computer, the Media Lab did not systematically catalog the facial behavior involved. Their focus was on the technical design of the computer system, not generalizations about expressions indicating frustration. At present, I am collaborating with Dr. Heather Lench, formerly at UC Irvine and now at Texas A&M University , to more explicitly study facial expressions of frustration (evoked using unsolvable anagrams).

I spent 2000-2002 as a Visiting Scientist at IBM working on the following projects:

1. Project Joshua -- artificial intelligence. We were simulating capacities of the human mind using a spreading activation semantic network incorporating a novel type of knowledge representation. As part of a four-person team, my contribution was to design and integrate the emotion and motivation system and to guide design of other cognitive functions based on my knowledge of how the human mind works. I have also designed and implemented tests of the first model of the software in a limited virtual environment. We have written several papers describing this work.
2. Emotion avatar -- a facial display that changes with the internal state of the software driving it. I specified the facial expressions in terms of muscle movements and linked affective states to the defined expressions. One potential application of such a system is automation of the expression generation process in a game avatar or animated character in a user interface. Another is to present a continuously updated monitor that summarizes the internal states of a complex system in terms that can be instantly grasped by a human observer.
3. Learning and usability studies -- testing of novel user interfaces. I used my knowledge of experimental design and data analysis to suggest ways of collecting and analyzing data describing user experiences and measuring learning. One project was a new multidimensional navigation tool using multiple sliders, for use on the website of composer Philip Glass. Another was a New York Museum of Modern Art kiosk teaching users about elements of style and aesthetics through similarity judgments about images of art displayed on the screen.
4. Everywhere displays – a projection of computer information onto objects in the environment which can then be interacted with by touching the display on the object. I helped design a series of experiments to investigate whether the nature of the object interacted with perception of the information, and whether functional fixedness interfered with that perception.

While at IBM Research, I also collaborated with Dr. Rosalind Picard at MIT's Media Lab, assisting graduate students with their projects, including: (1) training a computer to recognize the frustration of the user via facial expression and body posture, and (2) a virtual agent that acted as an exercise coach, displaying appropriate facial expressions and gestures to motivate the user. I also lectured on facial expression at the Media Lab, and participated in a working group to develop voice samples for training machine recognition of affect in the voice. This experience has given me specific knowledge to help guide psychology students whose career aspirations lie outside academia.

Better understanding of the dynamics of expression is needed in order to realistically portray animated faces, avatars, and robot expressions. There is a fine line between natural looking and creepy expressions. At this point, the industry feels that it is better to display entirely non-realistic figures (cartoons, caricatures, anthropomorphized animals) than to evoke the highly negative responses that arise from slightly wrong human-like displays. However, financial institutions, medical facilities, and other serious contexts may find stylized and cartoonish avatars inappropriate or counterproductive. Further, miscommunication can arise when facial expression dynamics are not well understood. In 2002-2004, Dr. Christine Harris and I submitted an R01 grant application to NIMH to perform a series of studies investigating the dynamics of facial expression. This was not funded but we have noticed that the dynamics of expression have now become an emerging area of strong interest in facial expression research.