Emote Project

Emote (2017) is an ongoing series of real-time experimental animation that resulted from an interdisciplinary collaboration between Media Art Nexus and the NTU School of Computer Science and Engineering, (SCSE) and NTU School of Social Sciences (SSS) Psychology. The Emote Project is about feelings, brain waves and real-time experimental animation.

Using EEG, the brain signals were recorded and analyzed for recognizing emotional patterns and used for generating 20 animated chapters in real time after the non-lyrical music clips of the known emotional class (such as happy, sad, exciting, scary, etc). The soundtracks were chosen from predominantly Japanese television dramas, anime, and films and were tested for the emotion responds they evoke. The same audio was used for generating animated paintings. A study observes if the visuals done after the music would evoke the same feelings.

In this work, music language is a means for learning about the emotions in the brain and is used as an external stimulus in response to the emotional state of the brain.

Two sets of visuals were completed for this project using the Derivative TouchDesigner’s reactive audio tool-sets. The final collection of videos are numbered 1–20 and in 4 sets consisting of the emotions Happy, Exciting, and Frightening & Melancholy. In testing, a graphics and music clip related to the recognized emotion are selected and played to the subject, and their emotional neuro-feedback is recorded. Then, the resulting data is classified and analyzed. The future goal of these studies would be to create public arts to induce emotions that help the subject in cognitive functions such as learning, concentration, relaxation, etc.

Emote, by Mark Chavez, experimental animation, 50 minutes, Derivative TouchDesigner

Emote Credits: animation by Mark Chavez, collaborator Ina Conradi.
Brain data analysis by Annabel Chen, Psychology NTU School of Social Sciences.
EEG data collection and analysis by Quek Hiok Chai NTU School of Computer Science and Engineering, College of Engineering.

Related research

Emotion Study was part of the earlier research done with the Wee Kim Wee School of Communications where we created an online questionnaire loaded with images designed to produce the feelings: SAD, HAPPY, ANGRY, DISGUST, MELANCHOLY. About 50 people did the test. The images were abstract paintings, photographs, and computer-generated imagery. We used the results of the test to select imagery that corresponds to peoples evaluation and our particular vision of the abstract film Emotion Study.

Emotion Study experimental animation, Dir Ina Conradi, Mark Chavez (2010)

Mark Chavez, Yun Ke Chang, “Cinematics and Narratives: Movie Authoring & Design Focused Interaction.” Leonardo Electronic Almanac, Published 2013-07-15, http://journals.gold.ac.uk/index.php/lea/article/view/87

Chang, Yun-Ke, Mark J. Chavez, Miguel A. Morales-Arroyo, and Jaime Jimenez-Guzman. “An Active Cinema Experience: A Study on User Expectation and Perceived Gratifications of a Real-time Animated Film System.” 2012 Ninth International Conference on Information Technology – New Generations (2012): 674-679.
https://www.semanticscholar.org/paper/An-Active-Cinema-Experience%3A-A-Study-on-User-and-of-Chang-Chavez/bbf305de00aa2fd98a5c8a293aeb079ee9365e4b.