Active Music Listening Interfaces and Singing Synthesis Systems
National Institute of Advanced Industrial Science and Technology (AIST)
Japan
COM2 Level 4
Executive Classroom, COM2-04-02
Abstract:
In this talk, I introduce our research activities on active music listening interfaces and singing synthesis systems.
Active music listening interfaces demonstrate how end users can benefit from music understanding technologies. Active music listening aims at allowing the user to understand better the music he or she listens to and to actively influence the listening experience. For example, a web service for active music listening, "Songle" (http://songle.jp), facilitates deeper understanding of music by visualizing its music scene descriptions estimated automatically, such as music structure, hierarchical beat structure, melody line, and chords. The user can actively browse a song by jumping to a chorus or repeated section during playback.
Singing synthesis systems, "VocaListener" and "VocaWatcher", imitate singing expressions of the voice and face of a human singer. VocaListener can synthesize natural singing voices by analyzing and imitating the pitch and dynamics of the human singing. VocaWatcher can generate realistic facial motions of a humanoid robot by analyzing and imitating facial motions of a human singing that are recorded by a single video camera. These systems that focus on "imitation" are not only promising for representing human-like naturalness, but also useful for providing intuitive control means.
Related URLs:
http://songle.jp
http://staff.aist.go.jp/t.nakano/VocaListener/
http://staff.aist.go.jp/t.nakano/VocaWatcher/
http://staff.aist.go.jp/m.goto/PAPER/ICASSP2007goto.pdf
http://staff.aist.go.jp/m.goto/PAPER/ISMIR2011goto.pdf
http://staff.aist.go.jp/m.goto/PAPER/ICASSP2012goto.pdf
https://staff.aist.go.jp/m.goto/PAPER/ICSP2014KEYNOTEgoto.pdf
Biodata:
Masataka Goto received the Doctor of Engineering degree from Waseda University in 1998. He is currently a Prime Senior Researcher and the Leader of the Media Interaction Group at the National Institute of Advanced Industrial Science and Technology (AIST), Japan. In 1992 he was one of the first to start work on automatic music understanding, and has since been at the forefront of research in music technologies and music interfaces based on those technologies. Over the past 23 years, he has published more than 220 papers in refereed journals and international conferences and has received 40 awards, including several best paper awards, best presentation awards, the Tenth Japan Academy Medal, the Tenth JSPS PRIZE, and the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology (Young Scientists' Prize). He has served as a committee member of over 90 scientific societies and conferences, including the General Chair of the 10th and 15th International Society for Music Information Retrieval Conferences (ISMIR 2009 and 2014). In 2011, as the Research Director he began a 5-year research project (OngaCREST Project) on music technologies, a project funded by the Japan Science and Technology Agency (CREST, JST).