Task code Project

DC130

Project Name

Smart Microphone

Project Introduction

Diagram SciFi Karaoke is an installation art project focusing on the human utterance. Basically, the karaoke installation consists of a "smart" microphone and a videoprojector that projects texts on the walls. The microphone should invite the audience to participate in the performance (Audience participation), inviting them to react to the projected materials or to other participants in whatever way they feel like (and are inspired to). The microphone is connected to hardware/software that takes the voice signal of the participants as input, processes it in various manners, and gives the processed signal back to the environment so that it becomes part of the performance. The audience will start to explore the response of the system, and the (sometimes unexpected) system response is supposed to trigger the communication between the attendants. The installation is planned to take place at the end of 2004.

Project and Objectives

The central objective of the project is to explore ways by which a system can react to properties of the human voice, most notably the emotional aspects of the voice and the (emotional and semantic) contents of utterances that are triggered by the context, and through its reactions enable forms of communication. Examples: With respect to the emotional aspects, the system might extract acoustic (pitch, volume, speaking rate) correlates of emotion, manipulate the speech signal in particular ways and synthesize utterances that react to the input utterances. With respect to the emotional and semantic content, the system might listen for particular words and generate relevant messages that act as triggers for further interaction.

Learning objectives concerning main Competencies (additional Competencies - Social and Cultural Awareness; Market orientation - may be addressed where applicable): 1. Emphasis in P0302 will be on Idea and Concpet Generation, more specifically on Brainstorming: students will explore several brainstorming techniques and learn to make a motivated choice of techniques give an particular purpose 2. With respect to Competency 3 (User Focus and Perspective), students will get an overview of methods for the identification of Requirements and experience how the Requirements constrain the concept generation stage. In addition, the research and design process makes the students familiar with particular aspects of communication. 3. With respect to Competency 2 (Integrating Technology), students will get insight into properties of human speech and of the human voice in relation to emotion, and apply signal processing techniques relating to speech (automatic speech recognition) and the human voice (signal manipulation techniques) 4. With respect to Competency 6 (Visual Language), students will learn to propose motivated solutions fitting the requirements: choosing expressive elements of a multimedia performance that implement the central objectives of the performance.

Project steps

• Orientation stage: what the project is about. Outcome: initial scenarios
• Analysis/Research stage: Investigating basic aspects of communication and emotion; Identifying requirements; Exploring technology: getting acquainted with relevant aspects of technologies relating to the human speech and voice
• Design Stage: Developing concepts for the interaction scenarios; Identifying key technologies (Automatic speech recognition; speech manipulation techniques; software architecture); Refining the initial concepts
• Realisation stage: Building demonstrations of the concepts that have been selected for the scenarios
• Delivery stage: Provide a business model, identifying appropriate applications

Deliverables

• Working plan including definition of team roles and time schedule (Due end of first week)
• Presentation containing initial scenarios, concepts fitting the scenario and enabling technologies (Due: interim client meeting)
• Final demonstrator with backing information (embedding framework, motivation for choices) (Due: final client meeting
• A0 poster summarizing theproject, printed A4 report documenting the project, Digital CD contianing the presentations, final report and A0 poster (Due: end of period)

Client Information

Ingeborg Houwen has a Master's degree Theaterwetenschap from the University of Amsterdam, with minors in Taalfilosofie, Filosofische Antropologie, Esthetica and Literatuurwetenschap. She is author, cultural entrepreneur, performer with a strong interest for new media, and the initiator of Diagram (see http://www.iice.nl/diagram-site/karaoke.html). She organized cultural exchanges and was one of the organizers of Cultural Center UI in 1997. Houwen wrote texts for the theatre, published in journals (including a serial in Folia Civitates) and is preparing a second novel. Contact: ingeborg@iice.nl

List of Available Resources/Experts

Resources

• http://www.iice.nl/

• http://www.mediamatic.net/cwolk/view/7491

• R.. Cowie, E. Douglas-Cowie et al. "Emotion Recognition in Human-Computer Interaction", IEEE Signal processing magazine Vol. 18, January 2001, pp. 32-80 (available through URL http://ieeexplore.ieee.org/xpl/tocresult.jsp?isNumber=19669)

• Suzuki.N., Y.Takeuchi and M.Okada: Psychological Effects Derived from Mimicry Voice Using Inarticulate Sounds, in R.Mizoguchi and J.Slaney(Eds.): PRICAI2000, Lecture Notes in Artificial Inteligence 1886, pp.647-656, Springer (2000)

• Reference to similar projects: http://www.mis.atr.co.jp/~christa/NEWWORKS/Artworks.html[[BR]]

Project coach/Experts:

• Richard Appleby (TUE/Richard Appleby Design): design
• Jacques Terken (TUE): Communication/Technology/Human Computer Interaction
• Christoph Bartneck/Kees van Overbeeke (TUE): Emotional Design
• Panos Markopoulos: Requirements engineering